Software Engineer

📋 Description

• Design and ship features end to end: data model, API, background workers, and the UI when the
• Build and improve how agents and models are evaluated: running trials in sandboxed environments
• Improve how benchmark tasks are quality-checked: verifiers, rubric-based and LLM-as-a-judge
• Integrate new models, agents, and benchmarks in a way that generalizes, without a special case for
• Keep the system fast and reliable as usage grows: efficient queries, safe migrations, and
• Review teammates’ pull requests against our engineering guidelines and help keep the codebase

🎯 Requirements

• 5+ years building and operating production backend systems, with ownership of at least one service
• Bachelor’s degree or higher in Software Engineering, Computer Science, or a related field.
• Strong Python and experience with Django (or a similar framework), PostgreSQL, and a task queue
• Hands-on experience in LLM evaluation, beyond calling a model API. Understanding of benchmark task
• Judgment about task quality: spot ambiguous instructions, weak verifiers, leaked answers, checks
• Good tradeoffs: reach for the smallest design, extend existing solutions, explain what you chose

🎁 Benefits

• Work at the frontier of AI, helping leading AI labs improve models by building expert datasets, RL
• Contribute to leading-edge AI research and showcase work at top conferences such as ICLR, ICML, and
• Bring frontier AI innovation to the enterprise, applying lessons from AI labs to solve real-world
• Collaborate with and learn from exceptional colleagues with deep AI experience from Google, Meta
• Move at the pace of AI innovation with startup speed, ownership, and impact.