AI Engineer Data and Evals
Glean is an enterprise AI platform that connects company knowledge and systems to provide permission-aware search, AI assistance, and agents.
Funding history
About Glean
Glean develops enterprise Work AI software. Its platform provides enterprise search, a conversational AI assistant, and tools to build, govern, and orchestrate AI agents using a company’s connected data and permissions.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will build evaluation datasets, benchmarks, platforms, LLM verifiers, and enterprise simulations. You will run reliable evaluations across development and production, analyze failures, and turn findings into improvements to agent workflows, tool use, model routing, and task-specific models.
Requirements
- Go
- Python
- backend engineering
- API
- data modeling
- testing
- distributed pipeline
- LLM evaluation
- RAG
- agent system
- dataset curation
- synthetic data generation
- CI/CD
- OpenTelemetry
- SQL
Responsibilities
- Build evaluation datasets and benchmarks from curated, production, synthetic, and failure data.
- Build and scale evaluation platforms, experiment pipelines, trace capture and replay, and dashboards.
- Build and calibrate agentic LLM verifiers against human labels.
- Run evaluations in CI, pre-release suites, and sampled production traffic.
- Build enterprise simulation environments for multi-step agent tasks.
- Diagnose failures and turn evaluation findings into quality improvements.
Benefits
- Medical coverage
- Vision coverage
- Dental coverage
- Generous time-off policy
- 401k plan
- Home office improvement stipend
- Annual education stipend
- Annual wellness stipend
- Regular events
- Daily healthy lunches
Hiring Process
AI-focused exercise or discussion.