Field Engineer, AI Labs

Role overview

Serve as the technical partner to AI research teams evaluating new models. You’ll connect model endpoints to evaluation workflows, run benchmarks on curated datasets, interpret results, and present findings that help customers identify capability gaps. The work combines Python-based engineering, data handling, customer conversations, and careful preparation of deliverables.

Responsibilities

- Connect model APIs and inference endpoints to evaluation systems and run benchmarks across reasoning, coding, multimodal, and agent tasks. - Analyze scores and failure cases, investigate changes in model behavior, and identify data that may address observed gaps. - Present evaluation findings in clear reports and customer discussions, and respond to technical questions about datasets, task design, difficulty, and scoring. - Use internal data systems to query, prepare, and curate subsets based on benchmark needs and evaluation criteria. - Maintain dataset documentation and check provenance, versions, and specifications. - Coordinate with engineering and quality teams to align schemas and validate data before customer delivery.

Requirements

- One to five years of experience in data science, AI or machine learning engineering, solutions engineering, or analytics, with exposure to generative AI. - Strong Python skills, including working with APIs, data analysis libraries, and SQL. - Practical experience with model evaluation, benchmark design, or automated language-model judging workflows. - Ability to use command-line tools and internal data workflows comfortably. - Clear written and verbal communication, including explaining technical metrics to research and commercial audiences. - Proven ownership of projects and ability to deliver accurately under tight deadlines.

Nice to have

- Experience at an early-stage AI or high-growth technology company. - Familiarity with agent evaluation frameworks, agent workflows, model fine-tuning, open-weight models, human review, or synthetic data generation.

Benefits and work setup

This is a full-time position in the United States, with preference for the San Francisco Bay Area; remote candidates elsewhere in the country may be considered. The stated base salary range is $175,000–$232,000 USD, with eligibility for equity, performance incentives, and benefits.