Senior QA Engineer AI
Delta Exchange is a cryptocurrency derivatives exchange offering futures, options, and leveraged trading.
Maintainer signals as of 9/25/2026
Funding history
Investors
About Delta Exchange
Delta Exchange is a derivatives trading platform that offers features like futures, options, and leveraged trading for virtual digital assets. It provides a referral program where users can earn a commission on the trading fees paid by their invited friends. The platform is operated by Excelium Technologies Private Limited, a FIU (Govt. of India) registered entity.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will create and maintain evaluation datasets, design scoring methods, and run evaluations for release candidates. You will operate release gates, identify AI failure modes, investigate production traces and tool calls, turn findings into regression coverage, and define and improve quality metrics for AI product surfaces.
Requirements
- At least 1 year owning quality for AI products in a lead or primary-owner capacity
- Hands-on experience building or operating an AI evaluation harness
- 4–6 years of QA or SDET experience
- Working proficiency in Python
- Ability to analyse traces and tool calls using observability tooling
- Strong API testing experience
- Ability to build tooling and automation
- Clear written communication
Responsibilities
- Build and maintain verified golden datasets for AI products
- Design deterministic and model-graded scoring for AI outputs
- Schedule and execute evaluations for release candidates
- Compare evaluation results with baselines and triage failures
- Operate the release gate for prompt and model changes
- Identify AI failure modes, including hallucination, incorrect tool selection, context loss, and PII exposure
- Investigate production traces to determine retrieval, generation, and tool failures
- Convert findings into engineering evidence and regression coverage
- Define, report, and improve AI quality metrics