Lead Data Engineer
Skills
About the Role
You will drive the design and delivery of a cloud-native healthcare data platform, leading the implementation of scalable data solutions built on AWS, Apache Iceberg, Lake Formation, Glue Catalog, Athena, dbt, and modern orchestration frameworks. You'll combine deep hands-on engineering with technical leadership, collaborating across platform, analytics, and business teams to build reliable, high-performing data systems.
Requirements
- 8–12 years of Data Engineering experience
- Experience leading enterprise-scale data initiatives
- AWS S3, Glue, Athena, Lake Formation
- IAM and security controls
- Apache Iceberg (preferred)
- Delta Lake / Hudi
- dbt
- Airflow or Step Functions
- Python
- SQL
- PySpark / Spark
Responsibilities
- Lead delivery of complex data engineering projects across multiple teams
- Drive technical execution from design through production deployment
- Establish scalable implementation patterns
- Design and implement batch and near-real-time pipelines
- Build reusable ingestion, transformation, validation, and publishing frameworks
- Support modernization of legacy workloads
- Lead Apache Iceberg implementation and optimization
- Define standards for schema evolution, partitioning, compaction, and metadata management
- Ensure efficient storage and query performance
- Implement data quality frameworks
- Drive observability and monitoring practices
- Improve operational excellence and reliability
- Review architecture and design proposals
- Conduct code reviews and engineering reviews
- Mentor engineers and establish best practices