AI Benchmark Engineer | Native Language Specialist

📋 Description

• Design and engineer high-signal Terminal-Bench tasks that evaluate the ability of coding agents and
• Build authentic task environments using datasets, files, prompts, and other assets written in
• Identify meaningful failure points in AI systems by developing French-language prompts and
• Develop and validate reference implementations that provide robust solutions against which
• Write reliable, deterministic verifier scripts and use rubric-based evaluation only when
• Analyze execution logs and calibrate task difficulty across levels ranging from Easy to Very Hard

🎯 Requirements

• Native or near-native French proficiency with deep knowledge of grammar, register, phrasing, and
• Strong English proficiency for collaboration and documentation.
• At least 5 years' experience in software engineering delivering technically demanding projects.
• Proven success at leading technology companies or graduation from a top engineering program.
• Expertise in Python, Unix shell scripting, and data processing workflows.
• Extensive CLI development experience and familiarity with modern coding agents or AI-assisted

🎁 Benefits

• Fully remote freelance work from France.
• Flexible scheduling without fixed hours.
• Competitive freelance rates with prompt payments and no invoice chasing.
• Opportunities to impact multilingual AI evaluation.
• Exposure to diverse AI projects enhancing your portfolio.
• Access to a global community of linguists and engineers.