Senior Product Manager, Conversational AI Chatbot & Agent Quality
OKX is a leading global cryptocurrency exchange offering a wide range of trading services, including spot and derivatives trading, as well as Web3 solutions.
Investors
Projects
About OKX
OKX is a global cryptocurrency exchange known for its trading services and financial products. The platform offers a wide range of features, including spot and derivatives trading, staking services, and a user-friendly interface suitable for both beginners and experienced traders.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will build and operate conversational AI products in production, focusing on knowledge quality, agent evaluation, chatbot operations, observability, and human-in-the-loop workflows. You will define evaluation frameworks, improve dialogue flows and retrieval, analyze failures, manage quality metrics, and partner with engineering on runtime and debugging capabilities.
Requirements
- 3–6 years of product management experience
- At least 2 years as the primary owner of a production chatbot or AI agent product
- Quantified business results with baseline metrics and numerical outcomes
- Hands-on knowledge base, labeling, and conversation analysis experience
- Familiarity with at least one chatbot or agent platform such as Coze, Dify, or Dialogflow
- Fluency in Mandarin Chinese
- English proficiency
- Agent evaluation harness design is a nice-to-have
- Internal agent platform product design is a nice-to-have
- LLM-as-judge evaluation experience is a nice-to-have
- Agent observability tooling familiarity is a nice-to-have
- Regression testing experience for non-deterministic systems is a nice-to-have
- Human-in-the-loop workflow specification experience is a nice-to-have
- Customer service, operations, or financial services background is a nice-to-have
Responsibilities
- Own knowledge base structure, content quality, retrieval coverage, and freshness governance
- Translate business processes into agent flows with edge-case handling and escalation paths
- Own annotation specifications, annotator quality assurance, and training batch impact tracking
- Review logs, triage failures, and maintain knowledge and flow updates
- Own resolution rate, fallback rate, per-intent accuracy, and CSAT metrics
- Define agent evaluation frameworks, test cases, automated scoring criteria, and regression coverage
- Operate the quality feedback loop from harness results to prioritized fixes, re-evaluation, and production deployment
- Define agent runtime requirements including observability, tool-call monitoring, failure alerts, and debugging tools
- Design human-in-the-loop case routing, reviewer interfaces, and correction data capture
- Track agent-version performance and maintain evaluation dashboards
Benefits
- L&D programs and education subsidy
- Team building programs and company events
- Wellness and meal allowances
- Comprehensive healthcare schemes for employees and dependants