Engineering Manager - Inference Performance

📋 Description

• Lead, mentor and grow a team of inference performance engineers through regular 1:1s, clear
• Hire top GPU and inference engineering talent, and build a strong, collaborative team culture as
• Own the technical roadmap and execution for runtime performance work, balancing customer needs, new
• Stay close to the technical work. Review designs, guide profiling and optimization efforts, and
• Drive the productionization of inference techniques such as quantization, speculative decoding
• Turn performance wins into measurable outcomes: tokens per GPU-hour, utilization, latency and cost.

🎯 Requirements

• Bachelor's, Master's, or Ph.D. degree in Computer Science, Engineering, Mathematics, or a related
• Experience managing engineers, including hiring, mentoring, giving feedback and running performance
• Experience leading or closely supporting GPU optimization teams in training, inference or
• Strong technical depth in GPU workloads, with a solid understanding of GPU architecture and
• Familiarity with ML libraries such as PyTorch, TensorRT or TensorRT-LLM.
• A track record of driving roadmaps and shipping complex technical projects with a team.

🎁 Benefits

• Competitive compensation, including meaningful equity
• (U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependents
• Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve
• Paid parental leave
• Fertility and family-building stipend through Carrot
• (U.S. only) Company-facilitated 401(k)