Senior Data Engineer

Skills

About the Role

You'll work directly with our trading desks to improve trading performance, ensuring data availability, breadth, and reliability so better trading decisions can be made. You'll be responsible for data services, including ETL pipeline implementation, data warehouse architecture, data quality automation, and analytic visualization. You'll need minimal guidance on architecture best practices and will communicate clearly to stakeholders across the organization. You'll solve challenging problems arising from petabytes of high-fidelity data, incrementally adding value to the trading desk while designing long-term architectures that scale.

Requirements

• Experience designing and developing big data warehouses and ETL pipelines
• Proven knowledge of SQL and Python
• Demonstrated ability to navigate and integrate data across multiple data platforms, including RDBMS, NoSQL, and Time Series
• Experience with real time messaging systems, such as Kafka, Kinesis, and Pulsar, and with developing a stream processing framework, such as Flink or Spark, a plus
• Proficiency in crafting high-performance BigQuery queries, optimizing for efficiency and scalability to handle large datasets effectively
• Excellent verbal and written communication, analytical, and problem-solving skills

Responsibilities

• Work with traders to understand analytic needs to improve trading performance
• Design and maintain a modular data architecture to facilitate and scale future analysis
• Ensure high data quality for all analytical trading datasets
• Hold other team members to data standards, including modular design, testing, and documentation
• Hold other team members to visual standards, including modular design, testing, and documentation
• Ensure technical solutions are simple and scale well for future use cases