Staff+ Software Engineer, ML Sampling Path

📋 Description

• Design, build, and operate the backend systems that process every token on the generation path for
• Own latency and reliability end to end: define and maintain SLOs and error budgets for added
• Ship changes to the hot path rapidly but safely — canaried and gradual rollouts, error budget and
• Set technical direction for the sampling path: lead design reviews, make latency, reliability, and

🎯 Requirements

• Have designed, built, and operated high QPS systems at global scale, and were accountable for them
• Have a strong foundation in distributed systems: replication, consistency tradeoffs, failure modes
• Design systems for graceful degradation: you plan for a slow dependency, a dropped stream, or a
• Have successfully shipped broad or all-encompassing changes to mission critical systems (e.g.
• 8+ years of industry software engineering experience.
• Familiarity with LLM inference systems and transformer-based models (not required, but a plus).

🎁 Benefits

• Competitive compensation and benefits
• Optional equity donation matching
• Generous vacation and parental leave
• Flexible working hours
• Lovely office space in which to collaborate with colleagues