Staff+ Software Engineer, Safeguards Evals

Added
7 days ago
Type
Full time
Salary
Upgrade to Premium to se...

Related skills

software engineering data analysis data pipelines evaluation llm

πŸ“‹ Description

  • Design and run experiments to improve evaluation quality, developing methods to generate
  • Research how different factors (multi-turn conversations, tools, long context, user diversity)
  • Analyze evaluation coverage to identify measurement gaps, and evolve evals so they remain
  • Build and own the evaluation harness for an agentic investigation system, defining metrics, test
  • Measure agent performance end-to-end (detection precision and recall, investigation quality
  • Construct RL environments to improve Claude's safety investigation capabilities.

🎯 Requirements

  • Experience building and maintaining data pipelines
  • Experience working with LLMs and a working understanding of their capabilities and failure modes
  • Strong data analysis skills, able to draw reliable insights from large datasets
  • Ability to move fluidly between research prototyping and production-quality code
  • Ability to translate ambiguous problems into concrete, testable experiments
  • Care deeply about AI safety and want your work to have real impact

🎁 Benefits

  • Competitive compensation and benefits
  • Optional equity donation matching
  • Generous vacation and parental leave
  • Flexible working hours
  • Lovely office space in which to collaborate with colleagues
Get alerts for new remote engineering jobs
A weekly email with new matches. Unsubscribe anytime.
Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Engineering Jobs. Just set your preferences and Job Copilot will do the rest β€” finding, filtering, and applying while you focus on what matters.

Related Engineering Jobs

See more Engineering jobs β†’