Lead Data Engineer

Role overview Lead data engineering work that turns complex national defense requirements into scalable data pipelines and useful insights. The role combines hands-on platform development with technical direction, stakeholder collaboration, and support for modernizing critical systems.

Responsibilities - Translate stakeholder needs into technical specifications, interface designs, and scalable data models. - Build and improve ETL/ELT workflows in Databricks, maintaining Python and SQL code for performance, reliability, and security. - Diagnose pipeline bottlenecks and data issues, then implement improvements to strengthen reliability and data integrity. - Set technical direction, prioritize work, review code, and mentor engineers in Python development and SQL optimization. - Coordinate with product owners, analysts, data scientists, and engineering peers; explain design decisions to technical and nontechnical audiences. - Create and maintain documentation for pipeline architecture, code, and configurations.

Requirements - Demonstrated experience leading data engineering work and building enterprise-scale platforms and pipelines. - Experience mentoring engineers, reviewing code, and providing day-to-day technical oversight. - Hands-on experience designing and optimizing data transformation workflows in Databricks. - Advanced Python or PySpark skills, including modular, object-oriented development. - Extensive experience writing and optimizing complex SQL queries. - Experience creating user-focused reports or dashboards with data visualization tools.

Nice to have - Experience with Apache Spark or other distributed computing frameworks. - Familiarity with agile and DevOps tools, including GitLab or Jira. - Exposure to defense or intelligence data environments and their compliance requirements. - Knowledge of data governance, metadata management, or data cataloging. - A relevant bachelor’s or master’s degree.