Software Engineer- Inference Platform

📋 Description

• Build the infrastructure and orchestration systems that deploy and run large-scale distributed LLM
• Design, build, and operate Model APIs, with a focus on advanced inference capabilities: structured
• Implement platform fundamentals such as API versioning, validation, usage metering, quotas, and
• Instrument deep observability (metrics, traces, logs) and build repeatable benchmarks for speed
• Debug and harden complex production systems spanning Kubernetes, distributed runtimes, networking
• Partner with Inference Performance engineers and other teams to make new optimizations broadly

🎯 Requirements

• Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or a related field, or equivalent
• 3+ years building and operating distributed systems, backend infrastructure, or large-scale APIs
• A proven track record of owning low-latency, reliable backend services, including rate limiting
• Infrastructure instincts with a feel for performance: profiling, tracing, capacity planning, and
• Comfort debugging performance and reliability issues across multiple layers of the stack, from
• A strong sense of developer experience. You think about how systems are used, not just how they

🎁 Benefits

• Competitive compensation, including meaningful equity
• (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents
• Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve
• Paid parental leave
• Fertility and family-building stipend through Carrot
• (U.S. only) Company-facilitated 401(k)