Senior Software Engineer, Agent Oversight

📋 Description

• Design and build core platform capabilities for deploying, monitoring, and evaluating agentic apps
• Build reliable APIs and data pipelines capturing telemetry and metrics at scale
• Collaborate with ML engineers to integrate platform work with evaluation loops
• Own reliability, scalability, and observability of platform components for many customers
• Work cross-functionally to translate deployment needs into platform features
• Build features end-to-end: design, implementation, debugging, testing

🎯 Requirements

• 4+ years in professional software engineering with backend/distributed systems, APIs, and data pipelines
• Hands-on experience building ML/LLM-powered production software such as evaluation systems or observability/monitoring tooling
• Understanding how ML/LLM systems behave in production: signals, failures, prompts, experiments, data quality, and offline vs. live tradeoffs
• Experience partnering with ML engineers to turn prototypes and eval loops into reliable platform capabilities
• Experience building internal platforms or developer tools used by other teams
• Track record owning features end-to-end within a larger platform

🎁 Benefits

• Comprehensive health, dental, and vision coverage
• Retirement benefits
• Learning and development stipend
• Generous PTO
• Commuter stipend