SWE-Bench AI Task Auditor - Freelance AI Trainer Project

📋 Description

• Evaluate software engineering tasks for technical accuracy, realism, solvability, reproducibility
• Review task codebases, integrations, tests, and evaluation criteria to identify potential technical
• Rigorously test and troubleshoot complex technical scenarios to determine whether tasks function as
• Investigate codebase integration problems, test failures, logic errors, and other implementation
• Provide clear, precise, and actionable feedback that enables task creators to correct identified
• Apply professional software engineering judgment to assess whether tasks reflect realistic

🎯 Requirements

• Demonstrable professional experience in software engineering, including experience navigating
• Strong knowledge of software engineering principles and familiarity with SWE-Bench-style tasks and
• Strong analytical and problem-solving abilities, with the capacity to investigate complex technical
• Experience troubleshooting code, diagnosing test failures, and identifying underlying logic or
• Ability to evaluate technical work objectively and communicate findings clearly and constructively.
• Strong attention to detail and a rigorous approach to technical validation.

🎁 Benefits

• Fully remote freelance contract.
• Flexible project-based working environment.
• Opportunity to contribute directly to the training and evaluation of AI systems.
• Ability to apply real-world software engineering expertise to challenging technical tasks.
• Compensation of $60/hour, with the final rate determined based on experience, expertise, and
• No company-sponsored health insurance, PTO, or other employee benefits, as this is a freelance