Senior Machine Learning Engineer, Agent Oversight

📋 Description

• Build observability into production agent behavior with signals and instrumentation
• Design evaluation metrics and automation for agentic apps
• Build and operate ML systems that detect drift, anomalies, or misalignment
• Design and run experiments to validate improvements before shipping
• Collaborate with platform engineers to take ideas to production
• Collaborate with product, customers and teams to translate requirements into platform capabilities

🎯 Requirements

• 5+ years of experience as ML engineer or applied scientist on production ML/LLM systems
• Strong grounding in at least two: evaluation/monitoring infra; agent design; training methods
• Hands-on with LLMs and agent architectures: tool use and planning
• Comfortable productionizing research with software engineers
• Rigorous experimentation with clear hypotheses and stats
• Cross-functional collaboration to production and mentoring

🎁 Benefits

• Healthcare, dental, and vision coverage
• Retirement benefits
• Learning and development stipend
• Generous PTO
• Commuter stipend eligibility