Data Scientist

📋 Description

• Design and own evaluation harnesses for LLM and agentic outputs.
• Build and calibrate LLM-as-judge pipelines; validate against human labels and bias.
• Define and track response-quality metrics: fidelity, grounding, hallucination.
• Curate, version, and grow evaluation datasets as the product surfaces evolve.
• Benchmark models to allocate tasks and quantify quality vs compute.
• Build production forecasting, anomaly detection, and early-warning signals from telemetry.

🎯 Requirements

• Bachelor's or Master's in Data Science, CS, Statistics, Math, or related field or equivalent experience.
• 3+ years in data science, ML, or applied quantitative analysis.
• Strong applied statistics with experimental design and significance testing for noisy outputs.
• Experience building, deploying, and monitoring predictive or time-series models in production.
• Demonstrated work evaluating, analyzing, or improving LLM/NLP systems: eval design, retrieval eval, or agent analysis.
• Proficiency in Python and SQL; fluency with foundation models and LLM tooling.

🎁 Benefits

• Comprehensive medical, dental and vision plans.
• 401(k) plan with employer match.
• Flexible PTO and Volunteer Time Off (VTO).
• 5-year Service Milestone Sabbatical.
• Paid parental leave.
• Generous employee referral bonus program.