Senior Research Scientist - Agent Flywheel
Databricks is a data and AI platform that lets organizations build analytics, AI agents, and applications on a unified, governed lakehouse.
Funding history
Investors
About Databricks
Data engineers, analysts, and AI teams use Databricks to process large datasets, build reliable pipelines, and train models on a single governed platform. Users can run SQL analytics, serve ML predictions in real time, and deploy AI agents grounded in enterprise data. Its open lakehouse architecture provides consistent security and governance across analytical and operational workloads.
Skills
Candidate Availability
About the Role
You will build the context layer for recursive self-improvement and autofix loops for agents. You'll turn raw production traces into deterministic metrics, calibrated LLM judges and failure clusters, and close the loop so signals drive training and agent improvement.
Requirements
- Expertise debugging failures in complex stochastic multi-component systems
- Skill in designing metrics and evals
- Experimental rigor and causal thinking
- Solid AI research and software engineering skills
- Strong communication skills
- Nice to have: experience with agent systems and evaluation environments
- Nice to have: research on learning from feedback such as post-training, RLHF or reward modeling
Responsibilities
- Turn raw production session traces from Genie Code and Genie One into online signals
- Train and calibrate LLM judges against expert human ratings
- Convert multi-turn agent trajectories into datasets and reward signals for post-training
- Design benchmarks and RL environments from real customer workloads
- Advance evaluation research including online evals, automated raters and verifiable rewards
- Keep up with deep learning developments and create new techniques
Benefits
- Annual performance bonus eligibility
- Equity eligibility