Remote job
Staff Software Engineer, Search & Retrieval Infrastructure
Job details
About this role
Role overview Own the architecture and performance of search, indexing, and large-scale data scanning systems. The work spans fast retrieval for interactive use and efficient processing of vast cold-storage datasets, balancing availability, latency, and infrastructure cost.
Responsibilities - Evolve hybrid search across lexical and vector retrieval to improve throughput and query performance. - Design data-tiering approaches for hot, warm, and cold data, and distributed pipelines for asynchronous bulk scans. - Find and address bottlenecks in indexing, querying, compute, storage, and networking. - Set technical direction for the search and indexing domain and advise engineering teams on data modeling and tuning. - Maintain fault tolerance and high availability during heavy ingestion and concurrent query workloads.
Requirements - 8+ years of software engineering experience, including Staff- or Principal-level ownership of distributed systems at petabyte scale. - Production expertise scaling Lucene-based search systems and vector indexing; strong understanding of index internals, chunking, and embedding retrieval. - Experience balancing compute and storage costs, including designing cold-storage scans and performant hot indexes. - Background with large data pipelines, event streaming such as Kafka or Kinesis, and distributed processing of billions of records. - Strong cloud and Kubernetes skills, with infrastructure-as-code experience; GCP is preferred. - Expert proficiency in a backend or systems language such as Go, Rust, Python, Java, or C++.
Benefits and work setup - Remote role in the United States. - Listed salary range: $190,000–$230,000, plus equity and health insurance.