Senior Product Manager, Multilingual AI and Evals

Skills

About the Role

You will own quality assurance and evaluation infrastructure for multilingual AI agents. You will improve automated translation quality gates, build AI-driven localization testing, create datasets and annotation tools, establish evaluation metrics and dashboards, and turn human feedback into agent improvements across global products.

Requirements

• Bachelor's degree or higher
• 3+ years in product management, with hands-on AI agent experience potentially substituting for part of the requirement
• Experience building content platforms, testing platforms, or SaaS products
• Hands-on experience with AI agents, agent harnesses, and evaluations
• Ability to design evaluation harnesses and golden datasets
• Experience shipping products across multiple markets or languages
• Cross-functional leadership across engineering, AI teams, linguists, design, and external partners

Responsibilities

• Enhance the evaluation agent that determines whether translations are ready for publication
• Balance automation coverage and risk across content tiers
• Diagnose systematic quality-gate failures and drive fixes into the agent
• Build an AI-driven tool that detects localization defects
• Ship an MVP that scans mobile and web applications for localization issues
• Integrate localization tests into internal product testing infrastructure
• Build golden datasets, annotation tools, automated evaluations, metrics dashboards, and feedback loops
• Integrate standardized real-time annotation workflows for linguists

Benefits

• Education subsidy
• Team building programs
• Company events
• Wellness allowance
• Meal allowance
• Comprehensive healthcare schemes for employees and dependants