Senior Data Engineer

Job Title: Big data Developer
Work Location : Mississauga , Canada

Job Summary

• Seeking a Senior Python Data Engineer with 3 to 5 years of experience specializing in Apache Spark and Python to design and implement scalable data solutions

Job Description 

• Develop and maintain largescale data processing systems using Apache Spark Write efficient reusable and testable Python code for data ingestion transformation and analysis Collaborate with crossfunctional teams to understand data requirements and deliver robust data pipelines Optimize data workflows for performance and reliability in a distributed computing environment Participate in code reviews and contribute to best practices for data engineering and Python development Ensure data quality and integrity through rigorous testing and validation processes

Roles and Responsibilities 

•  Design build and deploy scalable data pipelines leveraging Apache Spark and Python Analyze complex data sets to identify trends patterns and insights to support business decisions Monitor and troubleshoot data pipeline issues to ensure high availability and performance Mentor junior team members and provide technical guidance on Python and Spark best practices Collaborate with data scientists analysts and other stakeholders to deliver datadriven solutions Stay updated with emerging technologies and recommend improvements to existing data infrastructure