Senior AI Evals Engineer

📋 Description

• Design and build agent evaluation frameworks — create end-to-end infrastructure for evaluating
• Build synthetic testing environments and simulation strategies to stress-test AI agents under
• Build golden datasets and test harnesses — curate domain-specific evaluation sets for Rubrik's
• Establish eval-as-a-service — build and enhance centralized evaluation platform that any IT team
• Automate regression and drift detection — implement CI/CD-integrated eval pipelines that catch
• Define eval taxonomies and rubrics — establish structured scoring criteria for agentic workflows

🎯 Requirements

• Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or equivalent
• 8+ years of overall software Test engineering experience, with 5+ years focused on test
• 2+ years working with ML/AI systems, building, testing, or evaluating models in production
• 2+ years as a senior/lead, owning evaluation strategy, quality frameworks, and technical decisions
• 1+ years in designing or operating evaluation frameworks specifically for generative AI / LLM
• Deep hands-on experience with LLM evaluation — familiarity with frameworks and libraries such as

🎁 Benefits

• Inclusion @ Rubrik: fostering a culture where people from all backgrounds are valued, feel they
• Equal Opportunity Employer/Veterans/Disabled
• Reasonable accommodations for qualified applicants and employees with disabilities