Principal Rust Engineer

Role overview Set technical direction for a distributed search and indexing engine that supports retrieval across vector, full-text, and knowledge-graph data. The role spans low-level Rust implementation and distributed system design, with responsibility for performance, reliability, and safe production evolution. You will work with machine learning, product, and platform colleagues and contribute to related Python services and open-source development.

Responsibilities - Own the architecture and roadmap for the search and indexing engine, delivering improvements incrementally. - Design and implement vector, full-text, and graph indexes, along with query and filtering capabilities. - Evolve index building, merging, sharding, and replication across object storage, PostgreSQL, and messaging infrastructure. - Lead migrations and performance work using profiling and benchmarks while avoiding customer downtime. - Maintain Python services, SDKs, and Rust-to-Python bindings while keeping public APIs stable. - Improve observability, reliability, and security across Kubernetes deployments and CI/CD; write design proposals, review code, mentor engineers, and contribute to the open-source community.

Requirements - Extensive experience building and operating high-performance production systems, including technical leadership. - Expert Rust skills, including asynchronous programming, concurrency, memory-mapped I/O, profiling, and low-level optimization. - Deep understanding of search internals such as approximate nearest-neighbor search, vector quantization, inverted indexes, and ranking. - Experience designing distributed data systems and making storage, sharding, replication, and consistency trade-offs. - Strong experience with PostgreSQL, cloud object storage, and messaging systems such as NATS JetStream or Kafka. - Strong Python skills and experience with Kubernetes, observability, and CI/CD; clear English communication and a willingness to mentor in a remote team.

Nice to have - Experience with Tantivy, Lucene, vector databases, knowledge graphs, or graph-based retrieval. - Familiarity with embedding models such as multi-vector models, and with PyO3, asyncio, FastAPI, or Pydantic. - Open-source maintenance experience, including dependency and licensing review.

Benefits and work setup - Remote-first position with a global team. - Benefits include an employee stock purchase plan, 23 annual vacation days, a birthday day off, community service time off, life and medical insurance, and additional family and health leave.