Research Scientist, Benchmarks & Evals

Skills

About the Role

You will develop public benchmarks and private test sets for image, video, and document models. You will run data-value experiments, design human evaluation studies and automated metrics, publish trusted leaderboards and reports, identify model gaps, and translate results into data deals with lab buyers.

Requirements

• 3+ years of ML research or research engineering experience
• A benchmark or evaluation that other people have used
• Hands-on experience training and fine-tuning image or video models with PyTorch
• Understanding of experimental design and statistics, including controls, ablations, and human studies
• Experience running experiments end to end, from GPUs to final report
• Clear writing for researchers and buyers

Responsibilities

• Design and ship public benchmarks for image, video, and document models
• Build private test sets from real opt-in data that no model has trained on
• Run data-value experiments by training models with and without Kled data and measuring changes
• Design human evaluation studies and automated metrics
• Publish leaderboards and reports trusted by frontier-lab researchers
• Identify model shortcomings and recommend data to collect next
• Work with founders and lab buyers to turn results into data deals