Senior ML Data Processing Developer

Skills

About the Role

You will build, automate, and scale data pipelines that transform web-scale raw data into training datasets. You will develop filtering, scoring, privacy, provenance, and leakage-detection capabilities; monitor data quality; acquire suitable text corpora; and create tools for researchers to explore datasets.

Requirements

• Degree in computer science, software engineering, or a related field
• 5+ years of experience in data processing, machine learning engineering, or natural language processing
• Experience handling massive unstructured text datasets
• Experience with Spark, Ray, or Flink
• Experience with PII scrubbing, content-safety filtering, and evaluation-contamination prevention
• Python proficiency and production-grade data-processing code experience
• Experience with Airflow, Prefect, or Dagster

Responsibilities

• Build, automate, scale, and manage data pipelines for training datasets
• Develop deduplication, quality scoring, filtering, toxicity removal, PII scrubbing, metadata extraction, and data transformations
• Maintain dataset versioning and provenance tracking
• Ensure ingested data meets compliance, governance, and legal requirements
• Develop scoring and filtering toolchains, including evaluators, classifiers, and review workflows
• Monitor data quality, guardrails, and alerts for regressions
• Analyze corpus coverage and support targeted data acquisition and licensing
• Design leakage detection mechanisms to prevent evaluation contamination
• Build tools for researchers to explore and query datasets

Benefits

• Comprehensive health benefits, including a mental health and wellness management account
• 20 days of vacation per year upon start
• Employer contribution of 4% to retirement savings with no employee match required
• Additional compensation equal to 8% of salary for retirement savings or bonuses