Senior Software Engineer (Data Platform)

Role overview

Join as a Senior Engineer focused on a large-scale, AI-native Data Platform that powers hundreds of internal services and downstream product workflows. This role sits at the intersection of platform engineering and data engineering, designing reusable frameworks, self-service tooling, and high-performance processing engines rather than one-off pipelines. The position is suited to someone who enjoys bridging architectural strategy and hands-on implementation in a remote-first, distributed engineering environment.

Responsibilities

- Lead the design and rollout of internal SDKs and self-service frameworks that let distributed engineering teams ingest and transform data independently. - Build reusable patterns for both batch and real-time event processing, shifting the team's mindset from pipeline construction to platform engineering. - Own the cost-effectiveness of the Databricks ecosystem, tuning Spark execution plans, optimizing shuffle partitions, and implementing auto-scaling strategies to manage DBU consumption and cloud spend. - Implement Schema-on-Write validation and Data Contracts so that incoming data from hundreds of services meets strict quality standards before entering the Bronze layer. - Partner with data architects and stewards to enforce PII handling, security standards, and metadata lineage across the platform. - Champion AI-assisted development tools to accelerate the SDLC and mentor engineers through code reviews emphasizing maintainability and scalability.

Requirements

- 5+ years of experience building and operating production-grade data systems at massive scale. - Deep, hands-on mastery of the Databricks and Spark ecosystem, including Delta Lake, DLT, Spark UI debugging, and performance tuning. - Proven experience designing real-time or streaming architectures (such as Spark Structured Streaming, Kafka, or Kinesis) in production. - Demonstrated ability to manage and optimize cloud costs in a high-growth environment. - Prior experience building APIs, tools, or frameworks consumed by other internal engineering teams.

Nice to have

- Background in Lakehouse architecture patterns and medallion data modeling. - Familiarity with AI-assisted coding tools and integrating them into team workflows. - Experience mentoring engineers on distributed computing best practices.

Benefits and work setup

- Remote-first working model with flexibility and autonomy over scheduling. - Generous paid parental leave and flexible time off policies. - Medical, dental, and vision insurance, plus spending accounts. - Sabbatical eligibility after five years of service. - Annual bonus opportunity for non-sales roles, with compensation benchmarked against market data using gender-neutral criteria.