AI Infrastructure Engineer, Serving Platform

📋 Description

• Build fault-tolerant, high-performance systems for serving LLMs at scale.
• Build an internal platform to enable LLM capability discovery.
• Collaborate with researchers and engineers to productionize models.
• Conduct architecture and design reviews to uphold scalability.
• Develop monitoring and observability to ensure system health and performance.
• Lead projects end-to-end from requirements to implementation.

🎯 Requirements

• 4+ years building large-scale, high-performance backend systems.
• Strong programming in Python/Go/Rust/C++.
• Experience with LLM serving and routing fundamentals (rate limiting, token streaming, load balancing).
• Experience with LLM capabilities: reasoning, tool calling, prompt templates, etc.
• Experience with containers and orchestration: Docker, Kubernetes.
• Familiar with cloud infra (AWS, GCP) and IaC (Terraform).
• Proven ability to solve complex problems and work independently.