Python/Spark Developer

Pay rate range: $ 70 .00 - $ 75 .00 Negotiable based upon years of experience

Work Mode: Hybrid

Location: Mississauga, ON

This role is for an existing vacancy.

Responsibilities:

• Design, develop, optimize, and maintain scalable data pipelines for processing and analyzing large-scale financial data.
• Lead projects for developing batch pipeline processes for credit risk analytics, including stress loss and expected credit loss.
• Ensure efficient storage and retrieval of all risk data in big data platform systems such as Hadoop, Hive, Impala, and Spark.
• Implement best practices for Spark performance tuning, including partitioning, caching, and memory management.
• Maintain code quality through high-efficiency testing, CI/CD pipelines, and version control.

Requirements:

• Minimum 7 years of software development experience with at least 5 years of hands-on Python and Apache Spark development in enterprise environments.
• Experience building backend applications and services using Python, not solely data engineering or analytics solutions.
• Proficient in OOP, OOD, exception handling, concurrency, and functional programming concepts.
• Minimum 5 years of experience with strong SQL skills using enterprise databases such as Oracle, PostgreSQL, SQL Server, or MySQL. Experience with NoSQL platforms is a plus.
• Minimum 3 years of experience leading/owning software development projects from beginning to end with multiple team members.
• Experience working in financial market and credit risk enterprise software systems, including scenario analysis and stress testing.
• Experience with GitHub, Tekton, Harness, and other CI/CD pipeline technologies.
• Ability to work in a fast-paced TechFin environment.
• Experience leveraging AI-assisted development tools to improve engineering productivity.