Senior Quality Engineer (AI)

📋 Description

• Own AI quality strategies for non-deterministic systems.
• Define AI validation for LLMs, RAG systems, and AI agent workflows.
• Build automated validation pipelines integrated into CI/CD.
• Establish production monitoring and quality thresholds for AI outputs.
• Collaborate with engineering, AI/ML, and data teams to ensure accuracy and reliability.
• Lead AI guardrails, confidence scoring, and validation checkpoints.

🎯 Requirements

• 7+ years in Quality Engineering, Software Development, or related field.
• Hands-on experience with AI/ML systems, LLMs, or intelligent apps.
• Experience designing validation frameworks for probabilistic/non-deterministic systems.
• Knowledge of RAG architectures, agent-based workflows, LLM evaluation (hallucination, relevance
• LLM prompting, design, optimization, and regression validation.
• Experience with Node.js, MongoDB, CI/CD integrations for testing/validation.

🎁 Benefits

• Fully remote work opportunity in Canada.
• Flexible hours and competitive compensation with stock options.
• Benefits package and professional development support.
• Opportunity to work on innovative AI solutions.
• Collaborative environment with experienced professionals.