Remote job
Member of Technical Staff - Training Platform
Job details
About this role
Role overview Build a hosted platform that lets developers launch LoRA and full fine-tuning jobs on managed GPU clusters through an API or web interface. The work spans Kubernetes-based training infrastructure, backend control-plane services, and the developer-facing tools used to configure and monitor runs.
Responsibilities - Operate training and inference orchestration across multiple Kubernetes clusters and cloud environments. - Create Helm charts that package trainers, inference servers, environment services, and related components into repeatable training stacks. - Build Python agents that monitor pods, report job state to the platform, and keep clusters synchronized. - Develop scheduling and autoscaling for varied GPU hardware, using mechanisms such as KEDA, gang scheduling, and node placement rules. - Improve the platform’s APIs, dashboards, and live run-monitoring experience, working across backend and frontend surfaces. - Use observability and GitOps practices to debug production clusters and keep deployments reliable.
Requirements - Strong Python backend development skills, including experience with APIs and asynchronous services. - Practical Kubernetes operations experience, including Helm, operators or CRDs, scheduling, and production troubleshooting. - Familiarity with GPU infrastructure and distributed training concepts, including parallelism, multi-node scheduling, and interconnects. - Knowledge of the modern model development stack, including fine-tuning approaches and inference engines. - Experience with cloud platforms and infrastructure automation; GCP is preferred. - Ability to work across platform tooling and product surfaces, including TypeScript and React, and to build developer tools or monitoring interfaces.
Nice to have - Experience with open model families, reinforcement-learning-based tuning, or GPU types such as H100, H200, and B200. - Familiarity with Prometheus, Grafana, Loki, OpenTelemetry, or DCGM, plus Linux networking and performance tuning. - Experience with Terraform or Ansible and a GitOps workflow.
Benefits and work setup - Remote work or an office option in San Francisco, with visa sponsorship and relocation support. - The stated cash range is $150,000–$300,000, with equity incentives, a professional development budget, and team off-sites and conference attendance.