Big Data Engineer, Web3

Skills

About the Role

Design, build, and operate large-scale distributed data systems and core compute and storage infrastructure. Develop orchestration, APIs, developer tools, telemetry, and automated operational workflows, including LLM-driven capabilities for scheduling, cost optimization, SLA monitoring, anomaly detection, context retrieval, and incident response.

Requirements

• Have 5+ years of experience building large-scale data platforms using Hadoop, Spark, Flink, or equivalent technologies
• Have deep expertise in distributed storage and compute systems, including MaxCompute, Hologres, ClickHouse, and Hive
• Have strong software engineering skills in Java, Scala, or Python
• Have experience with API-first design
• Have hands-on experience with task scheduling systems such as Airflow, DolphinScheduler, or equivalent systems
• Understand multi-cloud architectures and cost governance
• Be familiar with LLM integration patterns, including tool calling, RAG pipelines, and context management
• Have experience with MCP or similar agent-tool frameworks
• Have a current right to work in Singapore without requiring visa sponsorship

Responsibilities

• Design and operate large-scale distributed data systems
• Own big data compute and storage infrastructure using MaxCompute, ODPS, Hologres, and Spark
• Build and maintain multi-site task orchestration that selects engines dynamically and enforces policy
• Improve reliability and performance across batch and real-time pipelines
• Develop MCP tool interfaces for AI agents to interact with platform APIs
• Build scheduling and cost-optimization agents for resource allocation and alert severity
• Instrument platform telemetry for AI-driven SLA monitoring and anomaly detection
• Design RAG and vector-search context retrieval pipelines for SQL code and configuration knowledge bases
• Own the internal data development platform, including IDE integrations, code review automation, and deployment tooling
• Build API-first tools for backfills and ingestion automation
• Define platform contracts with data warehouse and service teams
• Establish SLA benchmarks, cost metrics, and latency dashboards
• Build automated incident-response and root-cause-analysis pipelines
• Define and enforce infrastructure policies across multi-cloud environments
• Design and maintain the cross-platform MCP tool layer

Benefits

• Wellness allowances
• Meal allowances
• Comprehensive healthcare schemes for employees and dependants