Principal Research Scientist

📋 Description

• Own the evaluation methodology: Set the standard for how G2 evaluates AI agents, defend it to
• Contribute to today's product: Work closely with engineers and product managers to build and
• Design realistic evaluations: Build environments and task sets that reflect the work businesses
• Develop sound measurement: Create scoring approaches, including automated checks, LLM-based judges
• Validate the evaluations themselves: Audit tasks and graders, quantify noise, and establish when
• Represent the results: Present and defend evaluation findings to buyers, customers and partners

🎯 Requirements

• Experience working with evaluation design or dataset curation in multi-agentic systems
• 5+ years building or evaluating ML systems, including hands-on work with LLMs or LLM-based agents.
• 2+ years experience managing small to medium teams preferred.
• A graduate degree (MS or PhD) in machine learning, NLP, computer science, statistics or a related
• A track record of original ideas in machine learning, or clear improvements on existing ones, shown
• The ability to own a research agenda: pick the problems worth solving, and carry long-running

🎁 Benefits

• Offers Equity
• AI-powered technology to enhance candidate evaluation process