Research Engineer Scientist Data and Evals
HarveyVisit Harvey website
Harvey is an AI platform for legal and professional-services organizations.
Series F+Recently funded
Distributed
About Harvey
Harvey provides agents and collaborative tools for legal work, including legal research, document analysis, secure workspaces, contract intelligence, and organization-level analytics.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will build synthetic-data pipelines, evaluation systems, benchmarks, grading mechanisms, and training environments for legal reasoning and agent workflows. You will establish data quality safeguards, translate expert judgment into measurable criteria, use customer feedback to improve training data and evaluations, and investigate model failures with researchers and legal experts.
Requirements
- Experience building data pipelines, ML evaluation systems, or experimentation tooling.
- Strong Python skills and ownership of production-quality code.
- Statistical reasoning and experimental design skills.
- Understanding of when metrics can mislead.
- Ability to translate ambiguous goals or expert judgment into structured datasets, rubrics, and measurable criteria.
Responsibilities
- Build synthetic-data pipelines for training and evaluation samples.
- Design evaluations and verifiers for legal work and calibrate them against expert judgment.
- Build and maintain capability benchmarks and product-focused evaluations.
- Define quality metrics, rubrics, and grading mechanisms with legal domain experts.
- Own training and evaluation data quality, provenance, coverage, and contamination safeguards.
- Turn product usage and customer feedback into evaluations and training data.
- Investigate model failures and use findings to guide training and release decisions.
Benefits
- Equity