ES
Principal Data Engineer
Epsilon Solutions LTD
Location not stated
Remote
Staff / Principal
4 months ago
- Java
- Microservices
- Apache Hudi
- Apache
- Trino
- Azure
- AI
- AWS
- GCP
- CI/CD
- dbt
- Great Expectations
- Databricks
4 months ago
Location Canada - Remote
Job Type Contract
Ideal Experience
- Lead and guide the design and implementation of scalable distributed systems based on Java microservices
- Engineer and optimize data pipelines using solutions like Apache Hudi, Apache Trino, Azure ADLS
- Collaborate cross-functionally with product, analytics, and AI teams to ensure data is a strategic asset
- Principal Software Data Engineer with at least 10 years of professional experience in software or data engineering, including a minimum of 4 years focused on data pipelines (batch and streaming)
- Proven experience driving technical direction and mentoring engineers while delivering complex, high-scale solutions as a hands-on contributor
- Strong understanding of event-driven architectures and distributed systems, with hands-on experience implementing resilient, low-latency pipelines
- Practical experience with cloud platforms (AWS, Azure, or GCP) and containerized deployments for data workloads
- Fluency in data quality practices and CI/CD integration, including schema management, automated testing, and validation frameworks (e.g., dbt, Great Expectations)
- Operational excellence in observability, with experience implementing metrics, logging, tracing, and alerting for data pipelines using modern tools
- Solid foundation in data governance and performance optimization, ensuring reliability and scalability across batch and streaming environments
- Proven experience with Lakehouse architectures and related technologies, including Apache Hudi, Azure ADLS Gen2, HDFS, and other big data technologies (Trino, Databricks, Spark)
- Strong collaboration and communication skills, with the ability to influence stakeholders and evangelize modern data practices within your team and organization.
Principal Data Engineer · Epsilon Solutions LTD