Senior Software Engineer in Test (AI Agentic Systems)

📋 Description

• Own QA for an LLM-based, multi-agent claims adjudication pipeline
• Build a Three-Tier Evaluation Framework for agent reasoning and tools
• Develop semantic validation and grounding data testing libraries
• Audit tool calls with Vertex AI traces and correct arguments
• Validate orchestration across data, knowledge, reasoning, and tooling

🎯 Requirements

• Python SDET Expertise: Python and pytest; custom mocking for Vertex AI/ADK
• AI/LLM Observability: Vertex AI Experiments, Auto-SxS, Cloud Logging
• Data Literacy: SQL (BigQuery) and Pandas; diff datasets for adjudication
• Prompt Engineering for QA: refine prompts based on failed tests
• Architectural Testing: test multi-layer systems with RAG, Vertex AI Search, LangGraph

🎁 Benefits

• Hybrid in-office schedule at Lehi and Plano, requiring two days in the office
• Stock options: 115,000; health insurance, 401k, and PTO
• Professional development and internal mobility opportunities
• Flexible work arrangements and supportive culture