Performance Engineer, Inference Engine

📋 Description

• Work on building and optimizing the inference engine at Anthropic scale
• Improve throughput, cost, reliability, and latency across all accelerator and cloud platforms
• Keep device utilization high so accelerators are not waiting due to overheads
• Reuse model state cache instead of recomputing when cheaper
• Measure, model, then change performance to add observability and improve efficiency
• Ensure model quality and robustness

🎯 Requirements

• Working mental model of LLM inference: how prefill and decode land on an accelerator's compute
• Understanding of host responsibilities during inference
• Proven quick learner in deep, unfamiliar systems
• Strong systems programming skills in languages like C++, Rust
• Analytical about performance: observe and profile first
• Low ego and willingness to ask questions

🎁 Benefits

• Competitive compensation and benefits
• Optional equity donation matching
• Generous vacation and parental leave
• Flexible working hours