Member of Technical Staff - Data Quality Engineer Post-training
Reflection is an AI research lab building open frontier models and a full AI stack for developers, enterprises, and public-sector users.
Funding history
About Reflection
Reflection develops open-weight AI models, open-source software for customizing and running agents, AI-factory infrastructure, and related solutions. Its current research emphasizes large language models, reinforcement learning, and agentic reasoning.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will define and operationalize measurable data-quality standards for LLM post-training and evaluation. You will analyze datasets, build and validate automated QA methods, create reusable quality pipelines, provide feedback to vendors, and monitor quality trends to improve acceptance criteria.
Requirements
- Experience building data pipelines, QA systems, or evaluation workflows for post-training data and agentic environments
- Understanding of SFT, RL, training, and evaluation
- Experience designing automated quality checks
- Proficiency in Python
- Experience building ML or LLM workflows
- Experience with large datasets and automated evaluation or quality-checking systems
- Familiarity with LLM training and evaluation
- Communication skills
Responsibilities
- Own upstream data quality for LLM post-training and evaluation
- Translate requirements into measurable quality signals
- Provide actionable feedback to external data vendors
- Design, validate, and scale automated QA methods
- Build reusable QA pipelines for model training and evaluation
- Monitor and report data quality over time
- Improve quality standards, processes, and acceptance criteria
Benefits
- Stock options
- Medical, dental, vision, and life insurance
- Annual wellness allowance
- Daily office lunch and dinner
- 22 weeks of paid parental leave
- Unlimited paid time off in the U.S.
- 30 vacation days in the U.K.
- Visa sponsorship support
- Regular off-sites, happy hours, and team celebrations