Software Engineer, Agent Optimization & Evaluation

📋 Description

• Design and build Reddit's agent optimization harness for evaluating, benchmarking, and improving AI
• Develop Reddit-specific evaluation environments that accurately measure agent performance against
• Build systems for offline replay, automated evaluation, synthetic task generation, regression
• Design model routing and orchestration strategies that optimize for quality, latency, and cost.
• Define metrics and experimentation frameworks that measure developer productivity and engineering
• Build tooling to analyze agent behavior, investigate failures, and accelerate iteration.

🎯 Requirements

• 5-12+ years of engineering experience
• Extensive experience building and shipping large-scale production software systems.
• Experience designing evaluation, experimentation, benchmarking, or measurement systems for AI
• Strong understanding of modern LLMs, AI agents, and the engineering systems required to deploy them
• Experience building systems that optimize across multiple dimensions, including quality, latency
• Excellent software engineering fundamentals and a passion for building production-quality systems.

🎁 Benefits

• Comprehensive Healthcare Benefits and Income Replacement Programs
• 401k with Employer Match
• Global Benefit programs that fit your lifestyle, from workspace to professional development to
• Family Planning Support
• Gender-Affirming Care
• Mental Health & Coaching Benefits