Senior Data Engineer

Role overview

Own production data workflows that turn raw external files into dependable datasets for product features, analytics, machine learning, and operational decisions. The role includes customer data onboarding, data modeling, pipeline reliability, and ongoing improvements to a growing data platform.

Responsibilities

- Build and operate production pipelines with Python, Spark SQL, and Airflow to process large CSV, Parquet, and JSON datasets. - Map varied source data, including claims, EHR, partner feeds, and flat files, into consistent internal models. - Manage pipeline workflows from ingestion through validation and queryable outputs, with attention to reliability and performance. - Integrate new customer data, troubleshoot format and implementation issues, and coordinate with product and subject matter experts. - Add idempotent transformations, quality checks, schema enforcement, versioning, and observability. - Improve existing pipelines, tune distributed Spark processing, monitor production health, and take part in on-call support. - Develop streaming workflows where needed and contribute to shared data engineering and modeling practices.

Requirements

- Six or more years in data engineering or a comparable role building production pipelines. - Strong experience with Python, Spark SQL, and Airflow, plus large-scale file processing. - Experience standardizing external data into canonical models and integrating customer data with varied formats. - Familiarity with a cloud platform, including storage and distributed computing concepts. - Ability to own complex workflows, balance priorities, and work across technical and non-technical teams. - Clear written and verbal communication, including explaining technical decisions to non-engineering partners.

Nice to have

Experience with healthcare data such as 837 or 835 claims, EHR, UB04, or claims normalization. Familiarity with Kafka or SQS is useful; willingness to learn streaming systems is also considered.

Benefits and work setup

The position can be performed from anywhere in the US. Listed offerings include medical, dental, and vision coverage, flexible time off, equity, and a 401(k) with employer match. The stated base salary range is $180,000–$220,000.