Lead Data Engineer

Role overview Set the technical direction for moving legacy data warehouse workloads and pipelines to an Azure and Databricks lakehouse. You’ll guide migration teams on reusable engineering patterns, data validation, and quality controls while tackling complex pipeline issues and developing other engineers. The position depends on the associated contract being awarded.

Responsibilities - Define migration architecture and reusable pipeline approaches for moving data and workloads to a lakehouse. - Guide engineers on migration patterns, reconciliation methods, and quality gates. - Design migration pipelines using Azure Data Factory and Databricks. - Establish how source and destination data will be compared to verify completeness and accuracy. - Resolve challenging pipeline, data, and performance problems. - Review engineering work, mentor team members, and coordinate with product, quality assurance, and platform teams.

Requirements - Bachelor’s degree. - Leadership experience on large data migration or modernization programs. - Deep experience with Databricks or Spark and Azure Data Factory. - Strong SQL and Python skills, with experience moving legacy ETL or warehouse workloads to cloud platforms. - Understanding of lakehouse and medallion architectures, including Delta. - Clear communication and ability to lead engineers and collaborate with customers in a remote Agile team. - Authorization to work in the United States; some assignments may require citizenship and eligibility for a federal background investigation or security clearance.

Nice to have - Experience in federal or regulated data environments.

Benefits and work setup The role is fully remote and includes a technology and education stipend, a benefits package, employer-matched retirement savings, flexible paid time off, and paid holidays.