Associate Data Engineer Data Management Data Office

Skills

About the Role

You will design, build, and maintain scalable data pipelines and modern data platforms. You will develop batch and real-time ingestion processes, reusable engineering frameworks, and automation while ensuring data quality, security, governance, lineage, and observability. You will troubleshoot production issues, improve performance and resiliency, support CI/CD practices, and contribute to technical standards and documentation.

Requirements

• Collaborative experience working across global, cross-functional teams and taking ownership of major data platform components.
• Strong programming skills in Python, Java, and Scala.
• Experience with large-scale distributed data analytics engines, cloud data platforms, Snowflake, and SQL.
• Experience integrating and transforming CSV, TSV, Microsoft Excel, and database API sources.
• Typically 3–6 years of relevant software engineering, data engineering, or related technical experience.
• 4+ years of Java, Python, or Scala programming experience, including UDFs, reusable modules, and automated testing with frameworks such as pytest.
• 4+ years of experience building and optimizing large-scale data pipelines, architectures, and datasets.
• Familiarity with DAG-based workflow orchestration frameworks, dbt, and distributed event streaming and messaging platforms.
• 4+ years of experience developing production workloads with distributed data analytics engines, including resource allocation, performance tuning, and job optimization.
• 4+ years of experience with SQL-based analytics layers, workload monitoring, bucketing, partitioning, tuning, and schema-based serialization formats.
• 4+ years of experience with Transact-SQL, relational databases, non-relational databases, and GraphQL.
• Strong experience implementing solutions on Snowflake.
• Experience with data-quality and validation frameworks, particularly Great Expectations.
• Strong understanding of Swagger and OpenAPI for RESTful API design, documentation, and testing.
• Experience deploying and supporting solutions across cloud and hybrid environments using AWS, Azure, OpenStack, OCI containers, event streaming platforms, and container orchestration platforms.
• Familiarity with CI/CD tools including Jenkins, GitLab CI, and Azure DevOps.
• Experience with data governance, metadata management, data lineage, business glossaries, access controls, auditing, and centralized governance.
• Hands-on experience with Databricks notebooks, workflows, and machine learning integrations.
• Exposure to machine learning, artificial intelligence, generative AI, or AI-ready data platform engineering patterns is beneficial.

Responsibilities

• Design, develop, and maintain scalable data pipelines for enterprise data products.
• Build and optimize batch and real-time data ingestion, transformation, and publishing processes.
• Develop reusable data engineering frameworks, components, and automation.
• Ensure data products meet standards for quality, security, governance, lineage, and observability.
• Separate and segregate data according to relevant policies.
• Improve platform performance, scalability, resiliency, and cost efficiency.
• Troubleshoot production issues, perform root cause analysis, and implement sustainable solutions.
• Contribute to engineering standards, code reviews, testing, CI/CD automation, and technical documentation.
• Automate manual ingestion processes and optimize data delivery against service-level agreements.
• Partner with infrastructure teams to improve scalability.
• Evaluate emerging technologies and recommend improvements to the data ecosystem.
• Deliver high-quality engineering solutions while deepening expertise in modern data technologies.

Benefits

• Retirement plan
• Tuition reimbursement
• Comprehensive healthcare
• Support for working parents
• Flexible Time Off
• Employee Assistance Program
• Mental Health Ambassadors
• Parental leave