AI Infrastructure Engineer

📋 Description

• Design, deploy and operate GPU clusters across AWS, GCP, and other
• Build scheduling that keeps accelerators busy: gang scheduling for distributed jobs, bin-packing to
• Run distributed training infrastructure on PyTorch Distributed, Ray, or Megatron, with fast
• Deploy and tune high-throughput LLM serving with vLLM or similar: tensor and model/expert
• Build benchmarking and traffic-replay tools that measure TTFT, TPOT and latency percentiles, and
• Build health checks across hardware, network and software layers, and automated remediation that

🎯 Requirements

• 5+ years building and operating production distributed systems or ML infrastructure (7+ for Staff).
• Production-quality Python plus one of Go or C++.
• Shell scripting for system administration.
• Deep hands-on Kubernetes and containers. Working knowledge of Slurm or another HPC resource manager.
• Experience running GPU workloads at scale, including scheduling, orchestration and troubleshooting.
• Hands-on with at least one distributed training or serving stack: PyTorch Distributed, Ray, vLLM

🎁 Benefits

• Bi-annual performance bonus structure
• Generous equity grant vested over 4 years
• Up to $15k Relocation bonus
• $10K housing bonus (if you live within 0.5 miles of our office)
• $1.5K monthly stipend for meals
• Free Equinox membership