Senior Machine Learning Engineer, Inference

📋 Description

• Own the optimization process from research models to production deployment.
• Improve latency and throughput for real-time voice and LLM inference at consumer scale.
• Work directly with inference engines such as vLLM, TensorRT-LLM (TRT-LLM), and SGLang to enhance
• Implement model acceleration techniques, including quantization, distillation, speculative
• Develop high-performance GPU workloads using C++, CUDA, Rust, or optimized Python.
• Build and scale distributed multi-GPU and multi-node inference systems using Kubernetes, Ray, and

🎯 Requirements

• At least 5 years of experience in ML inference or systems engineering, at a senior level equivalent
• Practical experience optimizing inference engines such as vLLM or TRT-LLM at the engine level
• A proven background in developing backend and machine learning systems and serving models in
• Strong systems programming and GPU optimization skills using C++, CUDA, Rust, or optimized Python.
• Hands-on knowledge of model acceleration techniques, including quantization, distillation
• Experience with distributed systems, Kubernetes, Ray, load balancing, multi-GPU and multi-node

🎁 Benefits

• Salary: $250,000–$300,000 annual base salary, plus competitive equity.
• Employment: Full-time, permanent position.
• Work arrangement: Onsite in Mountain View, California, preferred; remote opportunities available in
• Visa support: H-1B and OPT transfers supported.
• Relocation assistance: Available for candidates relocating to the San Francisco Bay Area.
• Language: English-speaking working environment.