Senior / Staff Data Scientist, Applied AI

Role overview A senior-level Applied AI role focused on the reliability and quality of an AI copilot embedded within a sophisticated, multi-asset trading platform. The position owns evaluation, benchmarking, and continuous improvement of AI-driven workflows that support market analysis, portfolio and risk reasoning, and order placement. It sits at the intersection of ML systems, financial product safety, and high-stakes user experience.

Responsibilities - Design and operate evaluation frameworks that measure correctness, safety, latency, and regression risk across AI-driven market analysis, risk reasoning, and trading workflows. - Build and maintain benchmarks, including curated golden sets, scenario suites, adversarial stress cases, and market/regime-refreshed corpora. - Implement automated quality gates and regression workflows that prevent releases when key metrics degrade. - Define safe tool and action contracts with deterministic previews, confirmations, and auditability, partnering with engineering and product. - Own model improvement loops tied to evals, including data collection, labeling strategy, error taxonomy, prompt and tooling changes, and where appropriate, fine-tuning or preference optimization. - Stand up monitoring, telemetry, alerting, and incident response processes for AI systems, including root-cause analysis and fix-forward procedures.

Requirements - At least eight years of experience shipping production software, with strong proficiency in at least one programming language. - Solid computer science fundamentals, including systems design and testing methodology. - Hands-on experience building evaluation frameworks, test harnesses, or benchmark suites for complex systems such as LLMs, agents, search, retrieval, ranking, or recommenders. - Track record of running model improvement cycles: dataset curation, labeling and QA, offline experimentation, and shipping changes that move benchmark metrics. - Ability to define metrics, build measurement pipelines, and drive engineering and product decisions from data. - Comfort debugging model and tooling failures, instrumenting services, and partnering with frontend and product teams on UX patterns that improve safety and trust.

Nice to have - Fine-tuning, preference optimization, distillation, or prompt/compiler-style techniques for tool-use reliability. - Creating domain-specific benchmarks and adversarial suites, including known-bad scenarios for high-stakes applications. - Deep trading experience across asset classes and margin types. - Rust and performance-sensitive service experience. - Designing incident response and SLOs for ML or AI systems.

Benefits and work setup - Base salary range of $200,000 to $500,000, depending on experience, skills, and location, plus bonus and equity eligibility. - Competitive compensation package with company equity, 401k matching, gender-neutral parental leave, and full medical, dental, and vision insurance. - Remote-friendly designation with in-office perks such as lunch stipends, fully stocked kitchens, happy hours, and a central location. - Culture emphasizing collaboration, mutual support, and a diverse workforce across ideas, backgrounds, and experiences.