Senior Software Engineer, Machine Learning Inference Platform

📋 Description

• Own design and delivery of high-throughput, low-latency inference subsystems.
• Develop robust API layers (gRPC/WebSockets/REST) and SDKs.
• Build multi-tenant control plane for metering, quotas, and isolation.
• Optimize inference performance across the stack, including model engine.
• Build observability and SLOs for economics, cache hits, and GPU usage.
• Collaborate on onboarding, capacity planning, API contracts, and adoption.

🎯 Requirements

• Bachelor's or Master's in CS/Engineering or related field.
• 4+ years building and operating backend distributed systems.
• Data/ML fundamentals: data-intensive distributed systems, concurrency, networking.
• GPU inference experience: KV caches, prefill/decode, throughput/latency trade-offs.
• Experience with inference engines (TensorRT, vLLM) or serving frameworks (Triton).
• Programming: C++, Go, Rust, or Python; PyTorch familiarity.