Senior AI Engineer - Observability

📋 Description

• Partner with product engineering teams to design, build, and enhance AI-powered features using
• Contribute directly to production codebases using Python, C#/.NET, or related technologies, while
• Evaluate architecture and technology trade-offs across AI quality, latency, cost, privacy
• Design and implement AI evaluation pipelines, scoring methodologies, regression frameworks, and
• Build and maintain versioned golden datasets covering real-world use cases, edge cases, failure
• Apply LLM-as-judge, heuristic, human-feedback, and task-specific evaluation approaches, while

🎯 Requirements

• 5+ years of professional software engineering experience building and supporting production systems
• Hands-on experience developing or operating LLM-powered features, RAG systems, AI workflows, or
• Strong programming capabilities in Python, C#/.NET, or both, with the ability to contribute
• Practical understanding of prompts, embeddings, vector search, retrieval quality, orchestration
• Demonstrated experience designing evaluations, quality metrics, regression frameworks, or
• Strong observability fundamentals covering tracing, logging, metrics, alerting, production

🎁 Benefits

• Fully remote position within the United States
• Company-provided equipment, including laptop and required software
• Comprehensive medical and prescription drug plan options, along with dental and vision coverage
• Employer contribution to a Health Savings Account (HSA) for eligible employees enrolled in a
• Medical and dependent-care Flexible Spending Accounts
• Basic life insurance valued at $50,000 or one times annual salary, whichever is greater