Senior Software Engineer I - AI Inference Data Plane

📋 Description

• Technical leadership: drive end-to-end design and delivery of data plane components.
• System design: architect high-scale, multi-tenant AI inference cloud with availability and resiliency.
• Performance optimization: distributed inference hosting with tensor/data parallelism and KV cache.
• Collaboration: cross-functional collaboration to align roadmaps with customer needs.
• Mentorship: coach junior engineers and foster technical excellence.
• Operational excellence: maintain high-scale services with observability and SLOs.

🎯 Requirements

• Distributed systems: microservices, messaging, databases, IaC.
• AI/ML: hosting LLM/multimodal models with engines like vLLM, SGLang, Modular.
• Inference frameworks: llm-d, NVIDIA Dynamo, Ray Serve.
• Hardware/interconnects: GPU optimization, NVLink, XGMI, RoCE.
• LLM architectures and optimizations: batching, quantization.
• Software: Go or Python; gRPC.
• Cloud operations: shipping customer-facing software in high-scale environments.
• Open source mindset: experience with open-source software.

🎁 Benefits

• Innovate with purpose and own impact.
• Career development resources and training.
• Well-being programs and flexible time off.
• Equity compensation and Employee Stock Purchase Program.
• Equal-opportunity employer with inclusive culture.
• Access to LinkedIn Learning and development resources.