ML Infrastructure Engineer

Role overview Build and operate the systems that serve machine learning models at scale. The work spans GPU cluster operations, inference performance, low-latency request handling, and the tools used to evaluate model quality and speed. The position is remote across the Americas or EMEA and full-time.

Responsibilities - Operate and scale GPU inference clusters on AWS, including p4d and p5 instances. - Tune speculative decoding, continuous batching, and KV-cache policies to improve serving performance. - Reduce time to first byte for tool-use requests across multiple model-serving paths. - Build and maintain evaluation and benchmarking infrastructure. - Create observability pipelines for latency percentiles, token throughput, and error rates.

Requirements - At least four years of experience in ML infrastructure or distributed systems. - Practical experience with vLLM, TensorRT-LLM, or a comparable inference framework. - Strong CUDA skills, including GPU profiling and memory optimization. - Experience running containerized workloads at scale with Kubernetes, ECS, or similar. - Familiarity with transformer architecture and attention optimization techniques.