ES
Big Data Developer with Python
Epsilon Solutions LTD
🇨🇦 Canada
On-site
6 months ago
- Python
- Apache Spark
- Hadoop
- Scala
6 months ago
Work Location : Mississauga, ON (Onsite role), hybrid
Job Type: Fulltime
Job Description:
- Seeking a Specialist experience in Apache Spark and Python to design develop and optimize scalable data processing solutions within the Python Data domain
- Develop and maintain scalable and robust data pipelines using Apache Spark and Python Collaborate with data scientists and analysts to enhance and optimize data workflows Ensure data quality integrity and consistency across large and complex datasets Implement efficient data transformations and aggregations to support analytics and reporting Integrate data solutions seamlessly into existing systems by working closely with cross functional teams Monitor and troubleshoot data processing jobs to maintain high reliability and performance Stay updated with emerging trends technologies and best practices in big data and Python development
- Design develop and optimize high performance data processing applications leveraging Apache Spark Write clean reusable and efficient Python code to support various data engineering initiatives Analyse complex datasets to extract actionable insights and identify trends Collaborate with stakeholders to gather requirements and deliver tailored data solutions that meet business objectives Mentor junior team members and promote knowledge sharing within the team Participate actively in code reviews to ensure adherence to coding standards and best practices Manage deployment of data pipelines and continuously monitor their health and performance in production environments
Mandatory Skills :Apache Spark, Big Data Hadoop Ecosystem, Python, Scala, SparkSQL
Big Data Developer with Python · Epsilon Solutions LTD