Remote job
Senior Staff Software Engineer, ML Training Infrastructure
Job details
About this role
Role overview Provide technical direction and hands-on engineering for a high-performance, multi-tenant machine learning training platform. The platform supports the full model workflow, from processing data and preparing datasets through training, validation, profiling, and deployment. You will work across infrastructure, machine learning, autonomy, and safety teams to improve development speed and reliability.
Responsibilities - Design and evolve platform components for orchestration, scheduling, training abstractions, control plane, observability, and performance tuning. - Build end-to-end model pipelines covering log processing, feature extraction, dataset schemas and storage, model configuration, training, and profiling. - Measure infrastructure performance and diagnose bottlenecks across the stack. - Improve distributed training throughput using topology-aware software and hardware approaches. - Develop abstractions and tools that help machine learning engineers iterate independently and efficiently. - Promote effective use of open-source profiling and optimization tools, and maintain strong engineering standards.
Requirements - Bachelor’s or master’s degree in computer science, engineering, or a related field. - Eight or more years of experience with machine learning platforms and machine learning applications. - Demonstrated success optimizing training with distributed methods and building production data pipelines and loaders for large workloads. - Strong Python skills and experience with frameworks or tools such as PyTorch, Ray, Spark, or equivalent technologies. - Track record of building scalable, reliable infrastructure while supporting multiple modeling teams. - Ability to explain technical trade-offs and align partners across functions, alongside strong analytical, problem-solving, and communication skills.
Nice to have Experience with autonomous vehicles or model development. Familiarity with high-speed interconnects and distributed training hardware is relevant to the platform’s performance goals.