Software Engineer- Inference Performance

📋 Description

• Implement and productionize cutting-edge inference techniques, working deep in runtime internals.
• Profile and optimize inference end to end, from kernel launch overhead and memory layout up to
• Turn performance into cost savings. Improve tokens per GPU-hour, raise utilization, and give
• Bring up and tune new model architectures on new hardware quickly, often in the same week they're
• Build benchmarking frameworks that measure real-world performance across model architectures, batch
• Contribute upstream to open-source inference engines (vLLM, SGLang, TensorRT-LLM), and partner

🎯 Requirements

• Bachelor's, Master's, or Ph.D. degree in Computer Science, Engineering, Mathematics, or related
• Experience with one or more general-purpose programming languages, such as Python or C++.
• Familiarity with LLM optimization techniques (e.g., quantization, speculative decoding, continuous
• Strong familiarity with ML libraries, especially PyTorch, TensorRT, or TensorRT-LLM.
• Demonstrated interest and experience in LLMs.
• Deep understanding of GPU architecture.

🎁 Benefits

• Competitive compensation, including meaningful equity
• (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents
• Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve
• Paid parental leave
• Fertility and family-building stipend through Carrot
• (U.S. only) Company-facilitated 401(k)