Gen AI Contractor

📋 Description

• Help build and run evaluations for our LLM-powered chatbot systems, contributing to our goal of
• Support the design and calibration of LLM-as-judge scorers, working closely with subject matter
• Contribute to conversation simulation and replay infrastructure — a core building block of our
• Assist with real-time observability tooling to detect chatbot performance degradation before
• Help analyze production conversations and surface insights that drive chatbot improvements
• Work alongside ML engineers and analysts to iterate on prompt mutations, model changes, and

🎯 Requirements

• Currently pursuing a Bachelor's or Master's degree (B.E., B.Tech, M.Tech, or equivalent) in
• Genuine curiosity about large language models, AI evaluation, and building reliable Gen AI systems
• Familiarity with Python and comfort working with APIs, data pipelines, or scripting
• Basic understanding of NLP concepts or hands-on experience with LLMs (even through personal
• Strong understanding of probability theory and stochastic systems is a significant plus
• Strong analytical thinking — you ask "how do we know this works?" before shipping