Senior Software Engineer, ML Infrastructure

📋 Description

• Own fulfilment-ranking pipelines from training orchestration and quality gates through deployment
• Build and operate an offline-evaluation platform, combining LLM-as-judge harnesses with
• Develop the Roku Entertainment Assistant's LLM agent, including tool routing, retrieval, guardrails
• Design caching as an intentional latency and cost lever for high-volume services.
• Build observability for ML and LLM systems, including latency attribution, quality metrics, tracing
• Improve the reliability and operability of distributed systems, and lead the diagnosis of difficult

🎯 Requirements

• Strong production software-engineering experience, including designing, testing, operating and
• Meaningful ownership of ML infrastructure or production model delivery, with clear evidence of your
• A practical understanding of the ML lifecycle: training, evaluation, versioning, deployment, online
• Experience designing for latency, resilience, observability and cost in production systems.
• Experience with LLMs, embeddings, semantic search, retrieval or agent systems, including how you
• Experience with caching systems and the trade-offs involved in using caching for low-latency

🎁 Benefits

• Global access to mental health and financial wellness support and resources.
• Local benefits include healthcare (medical, dental, and vision), life, accident, disability
• Support for time off and evolving work and life needs.