Databricks Engineer

Role overview

A Databricks Engineer is needed to design, build, and operate scalable data pipelines powering a research-focused data enclave where sensitive datasets must remain secure, governed, and analytically useful. The role sits at the intersection of cloud data engineering and mission-driven analytics, partnering with data scientists, researchers, and platform teams to transform complex data into reliable analytical resources. Strong emphasis is placed on data quality, security, scalability, and reliability across the platform.

Responsibilities

- Design, build, and optimize ETL/ELT pipelines using Databricks, PySpark, SQL, and Delta Lake - Develop and maintain Databricks notebooks, jobs, and workflows for high-volume analytical workloads - Migrate and modernize existing data workloads to improve scalability, performance, and maintainability - Tune Spark workloads and implement automated testing, data quality checks, monitoring, and operational processes - Support CI/CD and infrastructure automation with Git and tools such as Azure DevOps or GitHub Actions - Implement data governance and access controls in secure, role-based cloud environments and troubleshoot production issues

Requirements

- Bachelor's degree in computer science, software engineering, data engineering, or a related technical field - 5+ years of data engineering experience with significant hands-on Databricks work - Strong background in PySpark, SQL, Apache Spark, Delta Lake, and the Databricks platform - Experience with cloud storage services such as AWS S3, Azure Data Lake Storage, or Google Cloud Storage - Familiarity with Git, CI/CD practices, distributed data processing, and pipeline performance optimization - Understanding of cloud security concepts including role-based access control and least-privilege access, plus strong communication skills

Nice to have

- Experience with Databricks Unity Catalog and enterprise data governance - Background in FISMA Moderate/High or other regulated environments - Multi-cloud experience across AWS, Azure, and/or GCP - Infrastructure-as-code experience with Terraform or similar tools - Familiarity with Databricks Lakeflow Declarative Pipelines or Delta Live Tables - Experience implementing data lineage, metadata management, and audit-ready logging

Benefits and work setup

- Multiple medical plan options (some with Health Savings Accounts), plus dental and vision coverage - 401(k) plan with pre- and post-tax contributions and employer matching up to IRS limits - Flex work weeks where possible, plus paid vacation, sick, personal, parental, military, bereavement, and jury duty leave - Approximately 15 days of paid leave per calendar year for new hires, prorated, plus 10 paid holidays - Paid family leave providing up to 160 hours in a rolling 12-month period - Short- and long-term disability, life, accidental death and dismemberment, personal accident, critical illness, and travel insurance - Remote-friendly arrangement with the United States as a primary work location