AI Benchmark Engineer | Native Language Specialist

📋 Description

• Design and engineer challenging, realistic benchmark tasks that evaluate coding agents and large
• Create authentic task environments using datasets, files, prompts, and other assets written in your
• Identify model failure points related to native-language prompting, translation gaps, multilingual
• Develop robust reference implementations and highly reliable, deterministic verifier scripts, using
• Analyze execution logs and calibrate task difficulty from Easy to Very Hard using standardized
• Participate in a rigorous multi-layer quality process covering task creation, human review

🎯 Requirements

• At least 1 year of professional experience in software engineering, prompt engineering, or a
• Demonstrated technical experience through work at established technology organizations and/or
• Native or near-native fluency in the target language, with a sophisticated understanding of
• Strong English proficiency for technical communication and collaboration.
• Strong Python skills, along with proficiency in standard shell scripting and data processing
• Extensive experience working with Terminal/CLI-based development environments.

🎁 Benefits

• Fully remote, flexible freelance work that allows you to choose when and how much you work, without
• Competitive project-based compensation with prompt payments and a streamlined invoicing process.
• Opportunity to contribute to cutting-edge AI evaluation and multilingual language technology with
• Access to diverse technical and language-focused projects that can broaden your portfolio and
• Collaboration with a global community of software engineers, linguists, and AI experts.
• Flexible supplemental work that can fit alongside other professional commitments, with project