Senior Software Engineer I - AI Inference Data Plane

📋 Description

• Technical leadership on end-to-end data plane components
• Design for high-scale, multi-tenant AI inference
• Optimize distributed inference with tensor parallelism and KV cache
• Collaborate cross-functionally with PMs and engineers
• Build Kubernetes-native distributed inference frameworks
• Flow control and load balancing for LLMS

🎯 Requirements

• AI/ML domain hosting large LLMs with vLLM/SGLang/TensorRT
• Distributed inference frameworks llm-d, Dynamo, Ray Serve
• Hands-on with vLLM or alternatives (batching, prefix caching)
• KV-cache locality and fast cross-pod KV transfers
• Upstream contributions to vLLM/llm-d/SGLang
• Go or Python proficiency and gRPC familiarity

🎁 Benefits

• Remote work with flexible options
• Career development resources and training
• Competitive benefits and paid time off
• Equity compensation and stock purchase plan
• Inclusive, equal-opportunity employer
• Access to LinkedIn Learning and conferences