Data Annotation Specialist Generalist
Toronto-founded enterprise AI company building foundation models, agentic AI, enterprise search, and private deployment products.
About Cohere
Cohere provides enterprise-focused AI models and end-to-end products, including North for agentic workplace AI, Compass for enterprise search, and models for generation and retrieval. It supports private, VPC, and on-premises deployment options.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will evaluate and rank model outputs, write clear justifications, and probe models for failure modes. You will create datasets, develop and apply rubrics, annotate multimodal data, calibrate quality standards, and report model-performance trends.
Requirements
- AI data annotation
- LLM evaluation
- Content moderation
- Research
- Quality assurance
- Preference ranking
- English
- JSON
- CSV
- TSV
- Markdown
- XML
- YAML
- Time management
Responsibilities
- Evaluate and rank model outputs and justify judgments
- Stress-test models and document reproducible failure cases
- Create training and evaluation datasets
- Build and apply rubrics and taxonomies
- Annotate, audit, and correct text, image, and structured data
- Participate in calibration and inter-annotator agreement checks
- Adapt to evolving experimental task types
- Report quality and performance trends in model and agent behavior
Hiring Process
Initial screening → multiple-choice assessment → comprehensive expertise assessment → video screen with Operations → Independent Contractor Agreement.