AI Task Auditor

📋 Description

• Evaluate developer workflow tasks for technical accuracy, realism, solvability, reproducibility
• Assess whether tasks are supported by reliable tests and appropriate evaluation criteria.
• Audit AI-assisted developer workflows and identify issues that could affect the quality or validity
• Review IDE integrations and identify technical faults, inconsistencies, or workflow issues.
• Analyze AI-assisted coding processes and identify inefficiencies, logic errors, or technical
• Test and troubleshoot complex technical scenarios to determine whether tasks function as intended.

🎯 Requirements

• Demonstrable professional experience and deep technical knowledge in modern developer tooling.
• Experience with AI-assisted coding tools, developer workflow ecosystems, telemetry, trace analysis
• Strong analytical and problem-solving skills, with the ability to rigorously test and troubleshoot
• Ability to assess technical tasks objectively for accuracy, practicality, reproducibility, and
• Strong understanding of software development workflows and the technical challenges involved in
• Ability to identify IDE integration issues, coding inefficiencies, logic errors, and other

🎁 Benefits

• Fully remote freelance contract.
• Pay rate of $60 per hour, with the exact rate determined based on experience, expertise, and
• Opportunity to contribute technical expertise to the development and evaluation of advanced AI
• Work focused on modern AI-assisted developer workflows and emerging software development
• Opportunity to influence the technical quality, reliability, and practical relevance of AI training
• Flexible remote working environment.