AI Data Analyst
Glean is an enterprise AI platform that connects company knowledge and systems to provide permission-aware search, AI assistance, and agents.
Funding history
About Glean
Glean develops enterprise Work AI software. Its platform provides enterprise search, a conversational AI assistant, and tools to build, govern, and orchestrate AI agents using a company’s connected data and permissions.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will evaluate AI responses and workflows, label failure modes, and maintain benchmark datasets, golden sets, and regression slices. You will analyze quality patterns, improve labeling rubrics, calibrate human and LLM judgments, support release-readiness reviews, and report actionable quality findings.
Requirements
- 3–5 years of experience in data labeling, data analysis, QA, or a related field
- Experience evaluating AI-generated outputs or working with NLP, search, recommendation, or other ML systems
- Familiarity with structured qualitative analysis, basic SQL/data retrieval, and spreadsheet workflows
- Written and verbal communication skills
Responsibilities
- Evaluate and label priority AI workflows
- Triage quality reports and categorize failure modes
- Analyze recurring quality issues
- Improve labeling guidelines and rubrics
- Maintain labeled datasets, golden sets, and regression slices
- Monitor dataset coverage, drift, leakage, and difficulty distribution
- Calibrate human graders and validate LLM-as-a-judge outputs
- Support benchmark updates and release-readiness decisions
- Report quality shifts, coverage gaps, and recommendations
Hiring Process
Brief AI-focused exercise or discussion during the interview process.
