Remote job
Senior Data Architect - Databricks Platform
Job details
About this role
Role overview Design and harden enterprise-scale Databricks data platforms inside client engagements, building the guardrails, automation, and standards that let multiple business teams safely extend a shared platform. The architect owns primary depth in either Data Platform Development or Data Migration & Modernization while staying fluent across both, with Databricks expertise expected regardless of focus. This is a client-facing role that translates real business priorities into governed platform capabilities.
Responsibilities - Lead architectural design for Databricks-based platforms and modernization programs, including workspace consolidation, permissioning and persona models, and configuration-as-code patterns aligned to AWS architecture practices. - Design high-volume batch and real-time streaming pipelines using PySpark and Structured Streaming on Apache Spark. - Design Unity Catalog integrations with third-party metadata repositories such as AWS Glue Catalog and Snowflake Horizon. - Build automation, CI/CD, and guardrail patterns that let business teams safely extend the platform without compromising governance. - Develop certified data products, a semantic layer, and lineage and metadata exposure to support self-service and future agent-driven use cases. - Communicate architectural decisions and trade-offs to both technical and business stakeholders, producing documentation, assessments, and roadmaps.
Requirements - 6+ years in data engineering, database engineering, or architecture roles, including 3+ years designing cloud-based data solutions and 3+ years in the AWS data landscape. - 3+ years hands-on Databricks experience, including workspace and persona architecture, Unity Catalog, lakehouse patterns, job orchestration, governance tooling, and Spark, cluster, and Delta Lake tuning. - Strong grasp of platform security, networking, automation, cost estimation, and performance optimization. - Deep experience with OLTP/OLAP design, schema evolution, performance tuning, HA/DR, security patterns, and cloud-native database architecture for migration-focused candidates. - Skilled with infrastructure-as-code and automation strategy, ideally Terraform, plus FinOps-aligned design and the ability to audit and optimize cloud spend. - Advanced English (C1) and high business acumen, with the ability to work directly with client stakeholders and mentor delivery teams.
Benefits and work setup - 100% remote work. - Medical insurance for the employee and eligible dependents. - Generous holidays and flexible PTO. - Competitive phantom equity. - Paid exams and certifications, plus an annual learning and development stipend. - Peer bonus awards, modern laptop and tools, and an equipment and office stipend. - Individual professional development plan and up to roughly 25% travel depending on business needs.