ML Annotation QA Engineer

Skills

About the Role

You will own quality analysis for annotated computer-vision data, review daily annotation output, and determine root causes for anomalies. You will maintain decision rules and quality documentation, build performance trackers, query data with Python and SQL, improve annotation tools, and support validation for new annotation programs.

Requirements

• BS in Computer Science, Engineering, Electrical Engineering, or equivalent experience
• 2–5 years of experience in ML QA, annotation quality, data quality, or analytics at an AI/ML company
• Experience working with annotated ML datasets and assessing label quality
• Strong understanding of statistics
• Root cause analysis experience
• Python and SQL or equivalent query-language familiarity
• Experience writing quality guidelines, decision rules, and labeling taxonomies
• Experience finding, diagnosing, and reporting data anomalies
• Understanding of data privacy and confidentiality requirements for customer operational data
• Jira familiarity
• Excellent documentation, communication, and collaboration skills

Responsibilities

• Analyze annotated data and produce in-house quality verdicts and root causes
• Own and refine verdict taxonomies, decision rules, and quality guidelines
• Build and maintain annotation-data performance trackers
• Detect anomalies and investigate their root causes
• Distinguish annotation errors from model, system, and field degradation
• Report reproducible findings and confidence levels to engineering and ML
• Document systematic failure patterns
• Query and analyze annotation data with Python and SQL
• Update annotation SOPs and instructions based on quality findings
• Specify and validate annotation-tool improvements
• Establish quality analysis and reporting for new annotation programs
• Track work in Jira and contribute to pre-release validation