Senior Software Engineer: Agentic Coding RL Environments

Role overview

Build and evaluate reinforcement-learning environments and benchmarks for coding agents. You will turn realistic software engineering work into rigorous tasks that reveal where automated coding systems succeed, fail, or require expert supervision. The role is suited to an experienced software engineer who enjoys independent technical work, experimentation, critical review, and setting a high standard for production-quality code.

Responsibilities

- Design coding tasks based on realistic engineering scenarios and production experience. - Create tests, evaluation rubrics, and measurable success criteria for agent-generated solutions. - Review and red-team model outputs to identify brittle changes, missed edge cases, and other failure modes. - Provide structured technical feedback that helps improve future model behavior. - Pressure-test coding systems before release and contribute benchmarks used to compare their capabilities. - Develop a detailed understanding of how coding agents perform and how experienced engineers can use them effectively.

Requirements

- Professional software engineering experience shipping real, scaled production systems. - Strong judgment about code quality, engineering trade-offs, and what constitutes a meaningful model failure. - Ability to work independently and communicate precise technical feedback. - Expertise in at least one common programming stack, such as TypeScript, Python, Ruby, PHP, Java, Rust, Go, or C++. - Careful attention to detail and the ability to create rigorous, reproducible evaluations. - Prior AI or machine-learning experience is not required; strong software engineering judgment is the primary qualification.

Benefits and work setup

- Fully remote and flexible working arrangement. - Choose when, where, and how to complete projects, with regular feedback on submitted work. - Work is organized around expert-created environments and benchmarks rather than traditional sprint ceremonies, office requirements, or routine management meetings. - The expected commitment is approximately 20–40 or more hours per week. The selection process includes a coding capability screen, identity verification, and a paid first task.