Member of Technical Staff Answer Quality and Evals
53 minutes agoSeniorSalary: 200K - 350KSan Francisco; Palo AltoHybridFull TimeData ScienceJobs by Perplexity
PerplexityVisit Perplexity website
Perplexity is an AI-powered answer engine that provides real-time, cited answers and research capabilities.
San Francisco, United States
Funding history
About Perplexity
Private AI company founded in 2022. Its products combine conversational search and research with cited web sources; it also offers developer-facing Search, Agent, Router, and Embeddings APIs.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will build shared evaluation infrastructure and a platform for replaying and analyzing agent traces. You will operate scalable data systems, diagnose quality problems, turn findings into improvements, and collaborate with technical and product partners.
Requirements
- 4+ years of software, data, or machine learning engineering experience
- Python
- SQL
- System design
- Data modeling
- Distributed systems
- Big-data systems
- Distributed compute
- Large-scale storage
- High-volume data pipelines
- Production system operation
Responsibilities
- Build shared evaluation infrastructure
- Develop a platform for replaying and analyzing agent traces
- Build and operate scalable systems for processing, storing, and monitoring interaction, trace, and evaluation data
- Turn answer-quality problems into evaluations, analyses, and product improvements
- Shape how answer quality is measured and improved
Benefits
- Health insurance
- Dental insurance
- Vision insurance
- Retirement benefits
- Fitness account
- Commuter account
- Dependent care account