AI Engineer Product
Mistral AIVisit Mistral AI website
Paris-based AI company building frontier models, AI applications, developer tools, and compute infrastructure for enterprise and public-sector deployments.
Paris, France
About Mistral AI
Mistral AI develops open-weight and commercial language models and provides a full-stack AI platform spanning agents, application development, custom-model training, APIs, and AI cloud infrastructure.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will design evaluations and metrics for AI-powered product features, develop prompts and orchestration, and run A/B tests. You will implement observability, operate model releases, improve core product behaviors, create evaluation templates, and diagnose regressions with Science.
Requirements
- 3 to 4 years of experience
- TypeScript or Python skills
- Production LLM experience
- Experience with evaluations and A/B testing
- Experience implementing product code
- Experience with logging, tracing, dashboards, and alerting
Responsibilities
- Design evaluations for product features
- Define and track quality, safety, latency, and cost metrics
- Design, test, and iterate on prompts and system prompts
- Run A/B tests and make rollout or rollback decisions from data
- Implement logging, tracing, dashboards, and alerts for LLM calls
- Operate model releases using canary and shadow traffic
- Improve memory, intent classification, routing, tool-call reliability, or retrieval quality
- Create evaluation templates and documentation
- Diagnose regressions and lead post-mortems