Software Engineering Director, Agentic Evaluations

📋 Description

• Lead the development and continuous improvement of evaluations that run against live software
• Own the full technology stack supporting the evaluation platform, including administrative tooling
• Design reusable evaluation primitives and architecture that can be applied across multiple software
• Turn recurring integration, onboarding, and maintenance processes into reusable AI skills, agents
• Monitor emerging frameworks, methodologies, and industry practices for agent evaluation
• Partner closely with data science teams to develop proprietary benchmarks and richer approaches to

🎯 Requirements

• 10+ years of professional software development experience, primarily in backend or full-stack
• 2+ years of direct engineering management experience, including team leadership, mentoring, and
• Expert-level backend development skills in languages such as Python, Java/Kotlin
• Hands-on experience building evaluations for customer-facing AI agents, including the use of agent
• Direct experience using frontier models from providers such as OpenAI, Anthropic, or Google in
• Regular experience using coding-agent tools such as Claude Code, Codex, OpenCode, or Pi as part of

🎁 Benefits

• Total earnings of approximately $260,000–$320,000, combining base salary and bonus.
• Equity participation.
• Performance-based bonus opportunities.
• Fully remote position within the United States.
• Flexible working environment designed to support distributed teams.
• Unlimited paid time off.