Added
16 days ago
Type
Full time
Salary
Upgrade to Premium to se...

Related skills

docker aws python kubernetes go

๐Ÿ“‹ Description

  • Design, build, and own ClickUp's AI evaluation system, including golden datasets, LLM-as-judge and
  • Build and evolve the core agent harness, and the AI that performs well on those evals, partnering
  • Build self-improvement loops where agents modify other agents or prompts, rerun evals, and iterate
  • Benchmark new models as they're released and own model selection, routing, and fallback strategies
  • Architect, design, and implement scalable AI platform services that support the deployment
  • Build and maintain robust APIs and backend systems that integrate AI-powered features into

๐ŸŽฏ Requirements

  • 5+ years of software engineering experience, including 2+ years building production LLM or ML
  • Hands-on experience building LLM evaluation systems in production (golden datasets, LLM-as-judge
  • Strong experiment design and statistical rigor, including A/B testing, offline vs. online metrics
  • Proven track record of applying LLMs and AI models to real-world, customer-facing product features.
  • Proven experience integrating and managing multiple LLMs, including benchmarking, routing, and
  • Deep expertise in backend engineering, distributed systems, and cloud-native technologies (e.g.
Get alerts for new remote engineering jobs
A weekly email with new matches. Unsubscribe anytime.
Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Engineering Jobs. Just set your preferences and Job Copilot will do the rest โ€” finding, filtering, and applying while you focus on what matters.

Related Engineering Jobs

See more Engineering jobs โ†’