Founding Inference Systems Engineer

Role overview Own the systems path that serves AI inference workloads, from request and model behavior through runtime, hardware, and operating cost. You will use representative workloads and explicit quality targets to guide performance and reliability decisions, then turn measured results into deployable configurations and operating practices.

Responsibilities - Define workload distributions, service objectives, quality constraints, and cost limits before tuning. - Measure latency, throughput, queueing, memory and cache behavior, retries, and recovery. - Compare models, providers, runtimes, routing, batching, quantization, and hardware under a consistent acceptance test. - Trace bottlenecks across preprocessing, scheduling, networks, storage, accelerators, and memory movement. - Build and operate appropriate serving paths, using systems such as vLLM, SGLang, LMCache, or TensorRT-LLM where they fit. - Convert benchmark and trace findings into capacity plans, alerts, rollback thresholds, and runbooks.

Requirements - Strong systems fundamentals and experience diagnosing performance or reliability issues. - Practical knowledge of model serving, concurrency, scheduling, memory, and distributed systems. - Ability to build representative benchmarks and distinguish fixture results from live runtime evidence. - Experience with profiling, telemetry, capacity analysis, and production investigations. - Python plus a systems language, or deep specialization in a runtime. - Ability to connect low-level measurements to user-visible quality and operating cost.

Nice to have GPU performance modeling; KV-cache or memory-tier systems; distributed inference and networking; quantization and numerical evaluation; or serving infrastructure and open-source work.