Senior AI/ML Test and Evaluation Engineer

📋 Description

• Design, implement, and operate benchmark execution and evaluation harnesses for AI models and
• Develop evaluation methodologies that combine automated metrics with structured human subject
• Curate and recommend candidate benchmarks based on mission needs and document the provenance of
• Produce defensible evaluation reports comparing candidate capabilities with current mission
• Define and contribute to common standards for benchmark expression, ingestion, and reporting
• Support partner organizations and vendors as they integrate their capabilities with shared

🎯 Requirements

• U.S. citizenship and eligibility to obtain and maintain a U.S. security clearance
• 6+ years of software engineering or machine learning engineering experience, including 3+ years
• Strong Python proficiency in a machine learning or data science context
• Hands-on experience with common ML frameworks and tooling, such as PyTorch and the Hugging Face
• Experience developing or using model evaluation harnesses, benchmark suites, or test and evaluation
• Experience designing evaluation metrics and applying appropriate statistical rigor when

🎁 Benefits

• Medical, Dental & Vision – 100% paid for employees, 75% for dependents
• 401(k) Match – Up to 5% with full vesting after 2 years
• Unlimited PTO – With a required minimum of 15 days off annually
• Fully Remote Setup – Includes up to $3,000 equipment reimbursement
• Continuous Education – Includes up to $500 reimbursement
• Disability & Life Insurance – 100% employer-paid