ML / AI Engineer · LLM Probe Infrastructure

Role overview Build the measurement infrastructure used to assess how consumer AI systems respond to prompts about customer websites. This role owns the end-to-end probing workflow, from designing prompt sets and parsing responses to producing statistically sound results and managing the cost of repeated measurements.

Responsibilities - Design prompt batteries focused on buyer intent and run them across multiple tracked AI engines. - Schedule recurring and repeatable probes, handling thousands of measurements per site. - Parse model responses to identify citations and flag citations that appear unsupported or nonexistent. - Calibrate repeat runs and aggregate results with appropriate statistical care. - Develop caching and execution strategies that keep per-site operating costs manageable.

Requirements - Python engineering experience. - Hands-on experience working with large language model APIs. - Statistical intuition and the ability to aggregate measurements responsibly. - Experience with distributed job scheduling.

Nice to have - Published research related to generative engine optimization or answer engine optimization. - Experience applying prompt engineering at scale. - Background in natural language processing.

Benefits and work setup - Full-time remote work available globally, with an asynchronous approach.