AI Quality Engineer

📋 Description

• Define testing strategies for AI agents and conversational workflows.
• Design behavioral test suites for non-deterministic outputs.
• Build evaluation frameworks for groundedness, factuality, relevance, and task completion.
• Identify AI-specific failure modes: hallucinations, prompt injection, drift.
• Develop testing approaches for RAG, function calling, and autonomous workflows.
• Create automated evaluation pipelines and test harnesses for LLMs.

🎯 Requirements

• 3–7 years of software or quality engineering experience.
• Strong Python and/or TypeScript; write test code.
• Experience designing and building automated test frameworks.
• Understanding AI/ML systems; testing LLM outputs differ from deterministic code.
• Experience with CI/CD pipelines and integrating automated tests.
• Experience testing AI/ML apps or LLM evaluation (preferred).

🎁 Benefits

• Career growth in AI quality engineering with no ceiling.
• Depth: become authority on AI evaluation methodology.
• Breadth: move into Senior or Lead AI Solutions Engineer.
• Specialization: LLM security testing or evaluation research.