ES
Lead Data Engineer
Epsilon Solutions LTD
🇨🇦 Canada
On-site
Staff / Principal
4 weeks ago
- Debezium
- GCP
- Pub/Sub
- Dataflow
- Cloud SQL
- PostgreSQL
- Incident Management
- Java
4 weeks ago
Location: Toronto- Hybrid
Contract
| • 8+ years of experience in supporting and managing data engineering pipelines in production environments • Strong experience in Change Data Capture (CDC) using tools like Debezium and handling file-based ingestion (CSV) • Hands-on experience with GCP technologies including Pub/Sub, Dataflow (Apache Beam), and Datastream • Experience working with Cloud SQL (PostgreSQL) environments • Experience in managing staging and curated data layers for downstream consumption • Strong understanding of production monitoring, alerting, and incident management processes • Experience in API-based integrations within data pipelines • Strong programming experience in Java for data processing and support activities Roles & Responsibilities • Provide end-to-end production support for data ingestion and processing pipelines • Monitor and manage CDC pipelines and batch ingestion processes for multiple source systems • Handle incidents, perform root cause analysis (RCA), and implement preventive fixes • Work with GCP Pub/Sub topics, subscriptions, DLQs, and alerting mechanisms for pipeline monitoring • Support data processing layers including staging and transformed datasets • Perform production deployments and coordinate testing across lower environments • Troubleshoot and resolve data, performance, and pipeline failures within SLAs • Serve as technical lead for AMS support, coordinating day-to-day operations • Own incident communication, escalation, and stakeholder updates • Maintain operational runbooks and provide guidance to L2/L3 support teams • Identify recurring issues and drive continuous improvements for system stability and reliability |
Lead Data Engineer · Epsilon Solutions LTD