Senior Data Engineer

Role overview This position supports a federal government contracting team by designing, building, and maintaining scalable data pipelines within a Databricks environment hosted on AWS. The work operates under FISMA High and multi-tenant security and compliance standards, supporting ingestion, transformation, and delivery of analytics-ready datasets across the full data lifecycle.

Responsibilities - Analyze and collect data from relational databases, APIs, external providers, and real-time streaming sources - Design and implement scalable pipelines that cleanse, transform, and aggregate data for downstream reporting and analytics - Build and maintain Databricks solutions using the medallion architecture across Bronze, Silver, and Gold layers - Produce source-to-target mapping documentation and conduct unit testing to validate pipeline logic and accuracy - Implement real-time and near-real-time ingestion using AWS Database Migration Service and other AWS tools - Develop transformation logic with PySpark and Pandas, manage versioning and CI/CD through GitLab, and align practices with FISMA High and multi-tenant governance requirements

Requirements - Proven hands-on experience building production-grade data pipelines as a Data Engineer or Databricks Developer - Strong working knowledge of Databricks on AWS, including E2 architecture, cluster configuration, job orchestration, and workspace management - Hands-on experience with Apache Spark, PySpark, and Pandas for distributed data processing - Solid programming skills in Python and advanced SQL including complex joins, window functions, and aggregations - Practical experience with AWS DMS for change data capture and replication, plus Delta Lake techniques such as OPTIMIZE, Z-ORDER, and VACUUM - Experience with GitLab CI/CD pipelines and working in Agile or Scrum environments

Nice to have - Familiarity with additional AWS services such as S3, Glue, Lambda, Step Functions, CloudWatch - Experience operating within FISMA High, FedRAMP, or other regulated multi-tenant environments - Exposure to infrastructure-as-code tooling like Terraform or CloudFormation, plus streaming technologies such as Kafka, Kinesis, or Spark Structured Streaming

Benefits and work setup - Comprehensive benefits including medical, dental, vision, life, and disability coverage, voluntary programs, health savings and flexible spending accounts, and a 401(k) plan - Eligibility for full-time employees working at least 30 hours per week