SR DATA ENGINEER

Epsilon Solutions LTD

Location not stated

Hybrid

Senior

3 months ago

Apache Spark
Hadoop
Hive
HBase
Java
Scala
Python
SQL
XML
JSON
Parquet
Apache Avro
Protocol Buffers
Spring Boot
Microservices
Apache Kafka
AI
ChatGPT
Databricks
Delta Lake
Unity Catalog
MLflow

3 months ago

Auto apply with Likeremote

Not scoredNo CV on file. Upload one and this job gets a score out of 100.Upload CV

Job Title:Lead Data Engineer
Location: Mississauga / Hybrid
Employment Type: Full-time

Role Overview
We are seeking an experiencedData Engineer with strong expertise in large-scale data processing and cloud-based application development. The role requires deep hands-on technical skills along with a strong understanding of end-to-end data solutions and the ability to collaborate across teams to deliver scalable and high-performance systems.
Experience Requirements

8+ years of experience in software development, preferably indistributed data systems or large-scale applications
Minimum 5+ years of hands-on experience in designing and developingBig Data solutions
At leastone end-to-end Big Data implementation experience

Key Responsibilities

Design, develop, and maintain scalableBig Data pipelines and data processing systems
Build distributed data processing applications using modern frameworks
Develop backend services and APIs for data platforms
Optimize performance and troubleshoot issues in large-scale data systems
Work closely with cross-functional teams to deliver end-to-end solutions
Ensure data quality, scalability, and reliability of data pipelines

Technical Skills
Processing & Compute

Strong hands-on experience withApache Spark (mandatory)
Experience withHadoop ecosystem including HDFS, Hive, Impala, HBase

Programming

Proficiency inJava / Scala / Python

Data & Query

Strong expertise inSQL (query building, debugging, optimization)

Data Formats

Experience working withXML, JSON, Parquet, Avro, Protobuf

Application Development

Experience inSpring Boot / Microservices architecture
Strong experience in backend development and API design

Messaging & Streaming

Hands-on experience withApache Kafka for real-time data pipelines

AI & Productivity Tools

Ability to leverage tools such asGitHub Copilot, ChatGPT, or similar to enhance development efficiency, code quality, and debugging

Nice-to-Have Skills

Experience withDatabricks Lakehouse Platform, including:
- Delta Lake
- Databricks Workflows
- Unity Catalog
- MLflow

Soft Skills

Strong analytical and troubleshooting capabilities
Ability to work under pressure in a fast-paced environment
Strong multitasking and prioritization skills
High learning agility and proactive mindset
Adaptable, flexible, and energetic approach to problem-solving

SR DATA ENGINEER · Epsilon Solutions LTD

Auto apply with Likeremote