Data Developer

Skills

About the Role

You will design and build data pipelines for retrieval-augmented generation systems, ingest structured and unstructured data, optimize datasets for AI workloads, and deploy vector databases, embedding services, and processing pipelines. You will also monitor retrieval quality, improve data quality and latency, and collaborate with ML engineers on inference optimization.

Requirements

• Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field
• 2-5 years building data systems and pipelines in production environments
• Experience with RAG architectures and vector databases
• Proficiency in Python and DAG-based orchestration platforms
• Experience with embedding models and semantic search systems
• Experience with Apache Spark, Ray, or Dask
• Understanding of LLM inference optimization and prompt engineering
• Familiarity with Docker, containerization, and orchestration platforms
• Knowledge of data modeling, ETL/ELT patterns, and data quality

Responsibilities

• Design and build data pipelines for RAG systems
• Build ingestion pipelines for structured and unstructured data into a centralized data lake
• Develop workflows to prepare datasets for fine-tuning and inference
• Build monitoring and evaluation frameworks for retrieval quality and system performance
• Optimize data formats and storage patterns for GPU-accelerated inference
• Implement caching and data versioning systems
• Deploy and manage vector databases, embedding services, and data processing pipelines
• Improve data quality, latency, and retrieval accuracy
• Stay current with data engineering and AI technologies
• Contribute ideas for tools, process improvements, and technology adoption