Director, Research - AI Evals

📋 Description

• Own AI evaluation methods for Figma's AI-powered experiences.
• Build evaluation frameworks, rubrics, golden datasets, and quality bars.
• Combine human evaluation with automated/model-based approaches.
• Partner with engineering to create repeatable evaluation pipelines.
• Produce dashboards for go/no-go and prioritization decisions.
• Socialize a shared definition of quality across teams.

🎯 Requirements

• 10+ years in product, research, or related field; 2+ years in mgmt.
• Hands-on ownership of AI/LLM evaluation for products.
• Expertise designing rubrics, benchmarks, and golden datasets.
• Blend human evaluation with automated/model-based methods.
• Strong data/metrics mindset; reason about model behavior.
• Proven ability to align execs and cross-functional teams.
• Bonus: experience with eval tooling (Braintrust, LangSmith, DeepEval).

🎁 Benefits

• Equity and competitive benefits package.
• Health, dental, and vision; retirement contributions.
• Parental leave and family planning support.
• Mental health and wellness programs; generous PTO.
• Work-from-home stipend and cell phone reimbursement.