Staff Product Manager - AI Evals

📋 Description

• Own or support the product roadmap for agentic AI evaluations, translating product-area goals into
• Partner closely with engineering and AI teams to understand evaluation needs, technical
• Define how agent performance and product quality should be assessed, including useful measures of
• Turn technical evaluation needs into clear product requirements, workflows, and acceptance criteria.
• Help operationalize evaluation workflows and success metrics so teams can consistently assess agent
• Use evaluation results and feedback to identify gaps, inform product decisions, and improve the

🎯 Requirements

• Strong product experience with AI or agentic systems, including experience building or managing
• Experience directly building evaluations for agents or building the evaluation product, tooling, or
• Ability to translate technical evaluation needs into product requirements, priorities, and
• Ability to define or operationalize meaningful measures of agent performance and product quality.
• Strong communication, collaboration, and product judgment in an evolving, technically complex

🎁 Benefits

• Medical, dental, and vision plan options. Life / AD&D, disability coverage options.
• Paid parental leave: 12 weeks for birthing parents, 4 for non-birthing parents, 6 weeks for
• Paid holidays and flexible PTO.
• 401(k) with pre-tax and Roth options. HSA/FSA options, dependent care FSA.