Software Engineer (lead)

Role overview

Lead the architecture and implementation of major components in an open-source data and AI infrastructure platform and its commercial products. This is a senior technical leadership role spanning distributed systems, metadata, storage, compute engines, and cloud environments. The position combines deep hands-on engineering with architectural direction, mentoring, customer engagement, and open-source participation.

Responsibilities

- Design and implement substantial platform components across metadata management, storage, distributed compute, and cloud integrations. - Improve the performance, reliability, scalability, and operational characteristics of large-scale systems. - Contribute to and help shape relevant open-source projects and technical communities. - Set technical direction, mentor engineers, and influence architecture decisions across teams. - Engage with major technology customers and engineering leaders to understand complex infrastructure needs. - Share technical knowledge with the broader community, including through industry presentations.

Requirements

- At least 7 years of experience building large-scale distributed systems, including experience as a technical lead. - Strong computer science, systems design, and programming fundamentals. - Experience with data infrastructure, AI infrastructure, and open-source systems in production or similarly demanding environments. - Proficiency in at least one systems-oriented language such as Java, Go, C++, or Rust. - Solid understanding of Linux and operating-system fundamentals. - A record of meaningful open-source contributions and the ability to lead through technical influence.

Nice to have

- Experience with database or data-engine internals, including query execution, transactions, or storage. - Familiarity with lakehouse, metadata, and governance systems. - Exposure to modern AI infrastructure, large language models, or agent systems. - Background in an early-stage infrastructure company, a developer-first organization, or a major technology company. - Experience with technical writing, developer advocacy, or community engagement.

Benefits and work setup

- Based in the San Francisco Bay Area, with remote work available. - Requires up to approximately 30% travel, primarily to the Bay Area. - Published compensation is $180,000–$250,000 in base salary, plus equity and benefits.