Senior Data Engineer
IG operates an award-winning trading platform and provides forex trading services, support, and educational resources through IG Academy.
About IG
IG is a trading-services company offering an award-winning trading platform, including forex trading services in the US. Its website highlights instant support and educational resources through IG Academy, and warns users about the risks of leveraged CFD trading.
Skills
Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will design, build, and operate batch and real-time data pipelines on GCP. You will implement data-quality controls and observability, maintain ingestion connectors and integrations, contribute to engineering standards, support data-science pipelines, and help ensure governance, lineage, reliability, and change control.
Requirements
- Production data-pipeline experience on GCP
- BigQuery
- Cloud Composer or Airflow
- Google Cloud Storage
- Cloud Run
- SQL
- Python
- Apache Kafka or equivalent streaming technology
- Event-driven ingestion
- dbt
- Medallion architecture
- Data contract
- Lineage tracking
- Observability
- Pipeline SLO monitoring
- Financial-services data governance
- Access control
- Audit trail
- Change control
Responsibilities
- Design and build batch and real-time data pipelines on GCP
- Implement Kafka event streams, BigQuery transformations, and Airflow workflows
- Define and implement data contracts, quality checks, and observability
- Maintain in-house connectors and third-party ingestion integrations
- Contribute to code review, dbt modelling, CI/CD, and change-control standards
- Support feature-engineering and model-data pipelines
- Implement lineage, reproducibility, and freshness safeguards for ML workloads
Benefits
- Extra time off for volunteering and community work