Member of Engineering (Evaluations / Engineering)

Role overview Join a frontier AI research organization building the evaluation backbone for large-scale model training. The team needs an engineer who can design and ship a self-serve evaluation platform that turns insights into research direction. The work sits at the intersection of infrastructure, developer tooling, and machine learning workflows.

Responsibilities - Architect and build a scalable evaluation platform that researchers and engineers can run independently - Design APIs and tooling that make it easy to construct, execute, and analyze evaluations at scale - Develop pipelines for orchestrating eval workloads, ingesting results, and surfacing actionable insights - Collaborate closely with researchers and ML engineers to identify high-leverage opportunities - Drive instrumentation, monitoring, and reliability practices across the eval stack - Establish engineering standards, documentation, and developer-facing patterns for the platform

Requirements - Strong background in building scalable, distributed backend systems - Production experience with container orchestration, ideally Kubernetes and Docker - Familiarity with event-driven architectures such as Kafka or managed pub/sub systems - Hands-on work with at least one major cloud provider (GCP, AWS, or Azure), including managed services and storage - Experience with observability tooling like Grafana, Prometheus, or Datadog - Track record of designing frameworks or tooling used by other developers

Nice to have - Experience with ML operations platforms and data visualization tooling - A product-oriented mindset for developer-facing software

Benefits and work setup - Fully remote position with flexible hours (EMEA / East Coast) - 37 days of vacation and holidays per year - Health insurance allowance covering the employee and dependents - Company-provided equipment - Allowances for wellbeing, learning, and home office needs - Regular in-person team gatherings