Senior Software Engineer (Streaming Data Pipeline)
Skills
MicroservicesFlinkToken StandardCdcCheckpointingPartitioningApi DesignSloContainerizationBlockchain DataObservabilityPerformance TuningCi/CdStreamingClickhouseSqlEvent IngestionReplicationIngestion PipelineAnalyticsDistributed MessagingWatermarkingEvent TimeProcessing TimeToken StandardsBackpressureKafkaData ModelingSystems Programming
About the Role
You will design and own high-throughput ingestion pipelines that process on-chain events with strong delivery guarantees. You will build stateful streaming jobs, operate distributed messaging at scale, and transform raw node output into a canonical on-chain data model for APIs and analytics. You will embed observability and automated recovery so systems self-heal under load, deploy containerized microservices via modern CI/CD and infrastructure automation in managed cloud environments, and participate in design and code reviews to rigorously test and ship secure, scalable software.
Requirements
- Bachelor’s degree in computer science or related field or equivalent experience
- 5+ years in backend or data infrastructure engineering with multi-year experience running real-time streaming systems
- Deep experience with distributed streaming and pub-sub platforms and concepts including watermarking, checkpointing, and back-pressure
- Proficiency in a statically typed systems language
- Strong SQL and API design skills
- Proven pipeline design and CDC or data-sink experience
- Experience with observability, SLOs, and performance tuning for latency, scalability, availability and cost
- Familiarity with blockchain data lifecycles, token standards, event ingestion and analytical databases
Responsibilities
- Design and own a high-throughput ingestion pipeline for on-chain events with strong delivery guarantees
- Build stateful streaming jobs and operate distributed messaging at scale
- Transform raw node output into a canonical on-chain data model for APIs and analytics
- Integrate external data sources where needed
- Embed robust observability and automate recovery for self-healing systems
- Deploy containerized microservices using CI/CD and infrastructure automation
- Conduct design and code reviews and rigorously test and ship secure, scalable software
