Research Engineer / Scientist, Data & Evals

Skills

About the Role

You will build the data and measurement systems powering post-training, benchmarking and agent capability efforts. You'll own synthetic data pipelines, evaluation design and grading, and work with researchers and legal experts to turn model failures into measurable improvements.

Requirements

• Experience building data pipelines, ML evaluation systems or experimentation tooling
• Strong Python skills and production-quality code
• Rigorous approach to measurement, statistical reasoning and experimental design
• Ability to translate ambiguous goals into structured datasets, rubrics and criteria
• Plus: synthetic data, LLM evaluations, automated raters or preference data experience

Responsibilities

• Build synthetic data pipelines to generate, filter and curate training and evaluation samples
• Design evaluations and verifiers calibrated against expert judgment
• Build and maintain capability benchmarks like the Legal Agent Benchmark and product-focused evals
• Work with legal experts to define quality metrics and rubrics
• Own training and evaluation data quality including provenance, coverage and contamination safeguards
• Close the loop from product usage and customer feedback to new evals and training data
• Partner with researchers to investigate model failures and guide training and release decisions