Software Engineer Evaluation Platform Infra
Artificial-intelligence research and product company building customizable AI systems, including the Tinker training API and Inkling open-weight models.
Funding history
About Thinking Machines Lab
Thinking Machines Lab develops AI products that let researchers and developers fine-tune and use models, while also releasing open-weight multimodal models.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will design, build, and maintain a platform for authoring, running, tracking, and analyzing model evaluations. You will work across evaluation libraries, distributed backend systems, data pipelines, APIs, and user-facing applications. You will make results reproducible, partner with researchers, and turn bespoke workflows into self-serve systems.
Requirements
- Bachelor’s degree or equivalent practical experience in computer science, engineering, machine learning, or a related field.
- Two years of post-graduate software engineering or ML engineering experience, excluding internships.
- Experience building or maintaining evaluations, benchmarks, graders, or model-quality systems for large language or multimodal models.
- Software engineering fundamentals and experience building reliable, maintainable systems.
- Proficiency in a backend programming language.
- Experience with databases, data pipelines, distributed systems, or data-intensive infrastructure.
- Ability to work across the stack and own projects through deployment and operation.
- Experience collaborating with cross-functional partners and subject-matter experts.
Responsibilities
- Design, build, and maintain the model-evaluation platform.
- Deliver end-to-end capabilities across evaluation libraries, distributed backend systems, data pipelines, APIs, and user-facing applications.
- Build abstractions for evaluation tasks, environments, graders, datasets, and model outputs.
- Ensure reproducibility and trustworthiness through versioning, provenance, observability, failure recovery, and quality controls.
- Partner with researchers to identify bottlenecks and convert bespoke workflows into self-serve systems.
Benefits
- Health, dental, and vision benefits.
- Unlimited PTO.
- Paid parental leave.
- Relocation support as needed.