AI Evaluation Engineer

Role overview Define how teams determine whether probabilistic AI systems meet their requirements. You will build evaluation methods and tooling across client projects and an AI product, making quality measurable throughout development and after release.

Responsibilities - Translate client requirements into measurable evaluation sets, including challenging and adversarial cases. - Build evaluation harnesses and supporting tools that engineers can run and interpret. - Monitor production behavior for drift and surface problems before clients report them. - Contribute evaluation capabilities to a product where worker activity is reviewed and runs are metered and logged. - Identify requirements or project scope that cannot be measured, and raise the issue early enough to revise them.

Requirements - Experience building evaluation for a machine learning or large language model system. - Ability to turn qualitative requirements into concrete evaluation criteria and test cases. - Skills in developing reusable harnesses and tools for engineering teams. - Understanding of production monitoring and how system behavior can change over time.

Benefits and work setup - Permanent, full-time role based in Lahore or remote. - Mid-to-senior level; compensation is discussed in the first conversation.