Senior Machine Learning Engineer, Inference

📋 Description

• Optimize LLM serving frameworks like vLLM or TRT-LLM.
• Implement model acceleration: quantization, distillation, caching, continuous batching.
• Develop high-performance systems in C++, CUDA, or Rust.
• Handle distributed inference scaling with Kubernetes and Ray.
• Own models from research to containerized, optimized, reliable production.
• Profile code and maximize NVIDIA GPU performance.

🎯 Requirements

• Deep understanding of serving frameworks (vLLM, TRT-LLM).
• Experience with quantization, distillation, speculative decoding.
• Proficiency in C++, CUDA, Rust, or optimized Python.
• Experience with Kubernetes, Ray, and multi-GPU inference.
• Strong track record of systems programming or open-source contributions.
• PhD in CS, Physics, Math, or equivalent practical experience.

🎁 Benefits

• Relocation assistance.
• Bonus and equity.
• Flat structure, fast iterations.
• Support for open-source contributions.
• Impact-driven engineering culture.