Overview

High-impact opportunity for machine learning specialists to design evaluation tasks pushing AI systems to their limits.

Responsibilities

• Design complex, original machine learning problems rooted in your specialized domain of expertise
• Craft evaluation tasks that demand advanced knowledge far beyond standard ML pipelines
• Draw from your own research to create challenges that would stump even the most capable LLMs
• Define precise problem statements, gold-standard solutions, and robust evaluation criteria
• Assess AI-generated ML solutions for correctness, creativity, and methodological rigor
• Document expected failure modes, difficulty levels, and the domain knowledge required to solve each problem
• Collaborate asynchronously with a global network of top researchers and engineers

Qualifications

• You hold a graduate degree (MS or PhD) in a scientific or technical field that intersects with machine learning
• You have strong working knowledge of ML fundamentals — model selection, feature engineering, evaluation metrics, and pipeline design
• You're deeply embedded in active research problems in your field and know where the hard edges are
• You can identify precisely where general ML knowledge falls short and specialized domain expertise becomes essential
• You have experience publishing or conducting original research (highly valued)
• You communicate complex ideas with clarity and precision in writing
• You're self-directed, intellectually curious, and thrive on genuinely challenging work
• Computational biology, genomics, or bioinformatics
• Climate science and environmental modeling
• Medical imaging and healthcare ML
• Materials science and computational chemistry
• Astrophysics and signal processing
• NLP for low-resource or specialized corpora
• Robotics, control theory, or reinforcement learning
• Financial modeling and quantitative analysis
• Work at the frontier — contribute directly to AI evaluation and safety research alongside top research labs
• Make your expertise count — put your specialized knowledge to use in a way that creates real, measurable impact on how AI develops
• Full autonomy — set your own schedule and work entirely remotely on your terms
• Intellectually stimulating work — no routine tasks; every challenge requires genuine deep thinking
• Global collaboration — connect and work asynchronously with a world-class community of researchers and engineers
• Ongoing opportunity — strong performers gain access to extended contracts and deeper research involvement
• Build your profile — become a recognized contributor to cutting-edge AI development

Nice to have

• Experience publishing research
• Communicate complex ideas clearly

Education

• MS or PhD in a relevant field

Benefits

• Full autonomy
• Intellectually stimulating work
• Global collaboration