Lead design and implementation of Harvey's Model Infrastructure platform.
Build high-availability, low-latency AI inference systems.
Design and improve Unified Model Controller and Model Selector for routing.
Develop provisioning, capacity management, failover, and traffic engineering across providers.
Integrate new model providers and maintain APIs/SDKs for frontier models.
Improve observability with dashboards, alerts, token usage analytics, and cost reporting.

🎯 Requirements

7+ years building large-scale distributed systems.
Design/operate highly available production services.
Proficient in Go, Java, Python, Rust, or C++.
Deep understanding of distributed systems and cloud infrastructure.
Experience leading multi-team technical projects.
Strong communication and collaboration skills.

🎁 Benefits

Generous equity compensation
Bonus opportunities
Work on frontier AI platform
Collaborative, fast-moving team culture
On-site in San Francisco

Apply on employer's website

This employer gathers applications via their own applicant tracking system.

You will be redirected to an external application form.

Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Engineering Jobs. Just set your preferences and Job Copilot will do the rest — finding, filtering, and applying while you focus on what matters.

Activate JobCopilot