Senior Data Engineer

Role overview

Build and operate a near real-time streaming data platform that brings operational data into reliable, governed forms for business use. This hands-on senior role owns the full lifecycle, from architecture and pipeline implementation through production monitoring, incident response, recovery, and cost management in a regulated environment.

Responsibilities - Own platform architecture, implementation, and production operations, including latency and throughput targets, monitoring, alerts, and runbooks. - Build pipelines that ingest change events from databases and SaaS systems, validate data contracts, and transform and join streams into canonical records. - Manage replay, backfills, dead-letter queues, connector failures, late events, backpressure, and uneven load. - Apply governance and access controls to sensitive health information as data is served to consumers. - Provision cloud data infrastructure as code, maintain delivery pipelines, and balance service targets with cost. - Coordinate source changes and data needs with producers, consumers, and other stakeholders.

Requirements - 6+ years building and operating production data systems, including ownership of streaming or event-driven systems and operational support. - Deep production experience with Kafka, including partitioning, consumer groups, schema management, troubleshooting, and replay strategies. - Strong hands-on Spark and Python experience for production streaming and batch transformations; expert SQL skills. - Experience with cloud data services, infrastructure as code, CI/CD, and observability tools. - Ability to debug distributed pipeline issues and design for idempotency, data quality, and recovery. - Familiarity with CDC, open table formats, schema governance, and access controls for protected health data; strong communication and collaboration skills.

Nice to have Experience with Flink, Java or Kotlin, AI-assisted development, data access for automated agents, or a venture-backed startup. A relevant quantitative or computing degree and comfort with Unix command-line tools are also sought.

Benefits and work setup Benefits include health and wellness coverage, flexible paid time off, up to 16 weeks of paid parental leave, holidays, retirement savings, education reimbursement, transportation support, and paid pet leave.