ML Platform Engineer, Distributed Training

📋 Description

• Optimize the training platform: Partner with infrastructure teams to improve GKE-based training
• Diagnose platform failures: Investigate underlying causes of infrastructure and training issues
• Tune GPU networking: Optimize multi-node GPU communication and training throughput using
• Run distributed training workloads: Execute, scale, profile, and debug multi-node training jobs
• Identify performance bottlenecks: Analyze compute utilization, memory consumption, input pipelines
• Design performance benchmarks: Develop and run benchmarks that measure throughput, scaling

🎯 Requirements

• Professional experience: At least 5 years of engineering experience, including recent hands-on work
• Python engineering: Strong production-level Python skills, including writing clean, maintainable
• Kubernetes and GKE: Practical experience with Kubernetes on Google Cloud, including GPU node pools
• Distributed training expertise: Direct experience running and debugging multi-node PyTorch training
• GPU networking and interconnects: Strong understanding of RDMA, GPUDirect, NCCL tuning, and
• Performance analysis: Demonstrated ability to design and execute training benchmarks, evaluate

🎁 Benefits

• Annual salary: $159,300–$219,245, with the final salary range determined by the assigned level and
• Comprehensive health insurance: Healthcare coverage to support employee well-being.
• Paid time off: Vacation and general paid leave.
• Paid holidays: Paid time off for designated holidays.
• Sick leave: Leave to support health and recovery.
• Parental leave: Leave to support employees and their families.