Staff Software Engineer, Model Infrastructure

📋 Description

• Lead the design and implementation of Harvey's Model Infrastructure platform.
• Build systems to ensure high availability, low latency, and operational excellence for AI inference.
• Design and improve Harvey's Unified Model Controller (UMC) and Model Selector platform to
• Develop systems for model provisioning, capacity management, failover, and traffic engineering
• Integrate new model providers and maintain provider APIs and SDKs, enabling Harvey to rapidly adopt
• Improve observability through health dashboards, alerting, token usage analytics, cost reporting

🎯 Requirements

• 7+ years of software engineering experience building large-scale distributed systems.
• Experience designing and operating highly available production services.
• Strong programming skills in Go, Java, Python, Rust, or C++.
• Deep understanding of distributed systems, cloud infrastructure, networking, and observability.
• Experience leading technical projects across multiple engineering teams.
• Ability to balance long-term architecture with pragmatic execution.

🎁 Benefits

• $231,000 - $340,000 USD base salary
• Equity
• Bonus