Content Specialist III | AI Evaluation & Prompting

📋 Description

• Test new AI model versions across a broad range of topics, use cases, and conversation scenarios.
• Evaluate AI-generated responses against established rubrics, guidelines, and quality standards.
• Identify, document, and communicate examples of both successful and unsuccessful model behavior.
• Write, test, and refine system prompts designed to influence model personality, tone, responses
• Assess whether evaluation rubrics and quality frameworks accurately measure model performance and
• Conduct quality reviews of human- and agent-based evaluations to ensure accuracy, consistency, and

🎯 Requirements

• 5+ years of professional experience in writing, editing, journalism, production, linguistics, STEM
• Bachelor’s degree or equivalent professional experience.
• At least 1 year of hands-on AI experience is preferred, including prompting, annotation, model
• Strong experience evaluating AI-generated content, large language model outputs, or other complex
• Deep expertise in at least one subject area, with the ability to evaluate content accurately as a
• Strong writing, editing, fact-checking, and research capabilities.

🎁 Benefits

• 6-month remote contract based in the United States.
• Full-time schedule of approximately 40 hours per week.
• Standard schedule of Monday through Friday with 8-hour shifts.
• Health insurance.
• Dental insurance.
• Vision insurance.