Software Development Test Engineer (SDET), AI & Automation

Role overview

A senior Software Development Engineer in Test (SDET) role partnering with an AI engineering team at a cybersecurity software company to shape quality engineering for AI-powered products. The work blends traditional software testing with non-deterministic AI/ML validation, building automated frameworks and shifting quality left of release cycles. The position is fully remote, with a hybrid schedule available for candidates based in the El Dorado Hills, CA or Chicago, IL metro areas.

Responsibilities

- Collaborate with AI engineers, software engineers, and product managers during design to define test strategy, acceptance criteria, and automation requirements - Design and maintain automated test frameworks for AI-powered applications, backend services, and APIs - Develop evaluation strategies that account for both deterministic software behavior and variable AI/ML outputs - Define measurable quality thresholds, tolerances, and evaluation criteria for AI and LLM-driven functionality - Build regression, integration, API, and end-to-end coverage alongside feature development - Integrate automated validation and quality gates into CI/CD pipelines

Requirements

- 5+ years in an SDET, test automation, software development, or other code-intensive quality engineering role - Hands-on experience testing AI/ML, LLM-powered, or model-integrated software systems - Strong programming skills in Python, Java, JavaScript/TypeScript, Rust, or another general-purpose language - Demonstrated experience designing, building, and maintaining automated test frameworks - Experience building automated functional, integration, API, and end-to-end coverage - Experience integrating automated tests into CI/CD pipelines using tools such as GitHub Actions, Jenkins, or GitLab CI

Nice to have

- Experience testing LLMs, agentic systems, retrieval-augmented generation, tool calling, or prompt-driven workflows - Familiarity with AI evaluation metrics for measuring quality across non-deterministic outputs - Experience with Docker, Kubernetes, or cloud-based testing environments - Experience testing cybersecurity, identity, privileged access, or other security-sensitive software - Experience building shared automation frameworks or internal testing tools used across teams

Benefits and work setup

- 100% remote position; hybrid schedule available in El Dorado Hills, CA and Chicago, IL metro areas - Medical, dental, and vision coverage inclusive of domestic partnerships - Employer-paid life insurance plus supplemental coverage for employee, spouse, and children - Voluntary short- and long-term disability insurance - 401(k) plan with both Roth and Traditional options - Generous paid time off that reflects commitment and seniority, including paid bereavement and jury duty leave