Research Post Training Evaluations
Artificial-intelligence research and product company building customizable AI systems, including the Tinker training API and Inkling open-weight models.
Funding history
About Thinking Machines Lab
Thinking Machines Lab develops AI products that let researchers and developers fine-tune and use models, while also releasing open-weight multimodal models.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will create internal evaluations and research signals for model capabilities and behaviors. You will develop usability evaluations, improve evaluation robustness, build benchmark-auditing methods, create agentic evaluation environments and user simulators, and evaluate personalized preferences, biases, values, and nuanced model behavior.
Requirements
- Bachelor’s degree or equivalent experience in Computer Science, Machine Learning, Physics, Mathematics, or a related discipline
- Experience designing, building, or analyzing evaluations, benchmarks, datasets, graders, or measurement systems
- Strong written and verbal communication skills
- Ability to collaborate across research and engineering teams
Responsibilities
- Create internal evaluations and research signals
- Develop usability evaluations for research and product workflows
- Improve evaluation robustness and grader reliability
- Build benchmark-auditing methodologies
- Develop agentic evaluation environments, user simulators, and supporting harnesses
- Develop evaluations for personalized preferences, biases, values, and nuanced model behavior
Benefits
- Health benefits
- Dental benefits
- Vision benefits
- Unlimited PTO
- Paid parental leave
- Relocation support