Senior/Principal Local LLM & Generative AI Platform Engineer

📋 Description

• Own architecture and roadmap for a secure local LLM platform.
• Collaborate with eng, product, security, and legal to turn use cases into requirements.
• Build modular inference layer with stable APIs, model routing, streaming, and quotas.
• Design and operate RAG and enterprise-search pipelines with embeddings and provenance.
• Optimize serving across CPU/GPU with batching and context controls.
• Establish versioned evaluation datasets and automated tests for quality and safety.

🎯 Requirements

• BSc/MSc in CS/CE/Data Science or equivalent.
• 7+ years in production software/ML platform.
• Strong Python; Go/Java/C++ a plus.
• Experience shipping LLMs on-prem/private-cloud.
• Docker/Kubernetes, CI/CD, APIs, and observability.
• GPU-backed model serving with CUDA, TensorRT, or ROCm.

🎁 Benefits

• Experience building production LLMs in on-prem/private-cloud environments.
• Strong Python; Go/Java/C++ advantageous.
• Experience with GPUs, CUDA, profiling.
• Familiarity with model governance, security, and data privacy.