Data Engineer

Skills

About the Role

You will support and improve large-scale data pipelines and infrastructure. You will work with data collection, processing, transformation, validation, and delivery while focusing on scalability, reliability, and performance. Your work will include distributed systems, large datasets, web scraping infrastructure, database systems, and production data workloads.

Requirements

• Bachelor’s degree or equivalent work experience
• Advanced Python including asynchronous programming multiprocessing and production-grade code
• High-volume web scraping experience including proxies rate limiting and anti-bot evasion
• Experience with platform APIs and large media or metadata datasets
• Experience designing and operating distributed data pipelines using task queues
• Practical experience with columnar or analytical data warehouses
• Experience with complex analytical queries partitioning and cost-aware cloud querying
• Experience with Docker Kubernetes Helm charts and deployment management
• Linux server and bare-metal operations experience
• CI/CD experience for data workflows using GitHub Actions or ArgoCD
• Experience writing scalable APIs

Responsibilities

• Maintain optimize and troubleshoot database queries and related data systems
• Assist in creating maintaining and improving large-scale data pipelines
• Develop test and maintain web scraping and data collection tools
• Monitor and troubleshoot pipeline issues and data quality concerns
• Document queries pipelines scraping workflows technical decisions and resolutions
• Participate in research and development projects to improve data products and workflows

Benefits

• Fully remote work
• Equity package