Principal Engineer, Inference Memory and Storage Systems

📋 Description

• Define and evolve a unified memory layer for LLM inference
• Architect integrations with serving engines (vLLM, SGLang, TensorRT-LLM)
• Design interfaces for disaggregated prefill/decode and KV cache transfer
• Own eviction and admission policies across tiers
• Partner with infrastructure teams to exploit RDMA and GPU technologies
• Drive unit economics and modeling for cache hit ratio and tiering

🎯 Requirements

• 15+ years building large-scale distributed systems, high-performance storage, or ML systems
• Deep fluency in memory hierarchies (GPU HBM, host DRAM, NVMe, remote/object storage)
• Experience with distributed caching or key-value systems
• Hands-on work with RDMA / NVMe-oF / NVLink-class technologies
• Strong systems programming in C/C++, Go, Rust, or Python
• Excellent written and verbal communication

🎁 Benefits

• Competitive benefits package
• Flexible time off
• Employee Assistance Program
• Local Employee Meetups
• Equity compensation and Employee Stock Purchase Program
• Reimbursement for conferences, training, and education