Define evaluation methodologies for models across STT, TTS, and multimodal.
Design, build, and maintain automated evaluation pipelines for batch and streaming (WER, latency).
Build scalable evaluation infra — harnesses, orchestration, result-aggregation, GPU clusters.
Translate benchmarks into automated pass/fail gates.
Build and operate canaries and continuous monitoring to detect regressions.
Integrate evaluation gates into CI/CD for continuous verification.

🎯 Requirements

BS/MS/PhD in CS/AI/Applied Math or equivalent experience.
5+ years in software or QA with test infra or evaluation systems.
Backend/scripting in Python, Rust, or Go.
Experience designing automated test pipelines and evaluation frameworks.
Strong analytical skills; distinguish real regressions from noise.
Lead ambiguous challenges and communicate across teams.

Apply on employer's website

This employer gathers applications via their own applicant tracking system.

You will be redirected to an external application form.

Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Engineering Jobs. Just set your preferences and Job Copilot will do the rest — finding, filtering, and applying while you focus on what matters.

Activate JobCopilot