Sr. Software Development Engineer in Test (Agentic)

📋 Description

• Own end-to-end quality for agentic features and workflows, including strategy, development
• Design and build automation tooling and frameworks for AI/LLM-driven systems, including prompt
• Develop and maintain evaluation frameworks (evals) to measure response quality, accuracy, and
• Drive automation coverage (80%+ for critical AI workflows) using deterministic + probabilistic
• Integrate AI quality checks into CI/CD pipelines with fast feedback cycles (<15 minutes for PR
• Build tooling for LLM observability and debugging, including prompt tracing and response analysis.

🎯 Requirements

• 6+ years of experience in software engineering or SDET roles with an emphasis on software
• Strong programming skills in Python (preferred), Java, or JavaScript.
• Experience testing distributed, cloud-native SaaS systems and APIs.
• Demonstrated proficiency in coding with AI agents to accelerate development and improve code
• Hands-on exposure to LLMs or AI/ML systems (e.g., OpenAI, Claude, Gemini, or similar platforms).
• Understanding of non-deterministic systems and probabilistic testing approaches.

🎁 Benefits

• British Columbia, Canada Salary Range: $150,500 CAD — $175,250 CAD