Senior AI Engineer

📋 Description

• Local LLM Deployment & Hardware Setup: Design and build local LLM serving environments on GPU
• Inference Optimization & Model Compression: Optimize LLM models for efficient GPU serving
• Serving Engine & Performance Engineering: Deploy and tune high throughput serving engines
• Production Serving Infrastructure: Deploy quantized models with autoscaling, load balancing, and
• Research & Propose Innovative Solutions: exploring and implementing novel inference

🎯 Requirements

• Bachelor's degree in Computer Science, Software Engineering, or a related field.
• 5+ years of experience in software engineering, with a focus on ML infrastructure, LLM inference
• Hands on experience deploying and serving LLMs on GPU hardware in production.
• Strong understanding of quantization and model compression (FP8/FP4/INT8/INT4, GPTQ, AWQ
• Experience with high throughput inference engines (vLLM, TensorRT-LLM, TGI, llama.cpp).
• Solid understanding of GPU architecture, CUDA, and the memory bandwidth bound nature of LLM