Data Scientist, Agent

📋 Description

• Own agent quality metrics and drive them up.
• Build eval systems and experiment framework for AI agent changes.
• Turn agent traces into concrete fixes with the engineering team.
• Develop tooling and agents to continuously evaluate the agent as it evolves.
• Define success and error metrics for agent behavior.

🎯 Requirements

• Experience or strong interest in LLM evaluation and observability
• Strong SQL and Python, applied statistics, experimentation
• Designing A/B tests for noisy outcomes
• Ability to work closely with engineers and operate in ambiguity

🎁 Benefits

• Note: Company-provided perks not listed in posting