Data Engineering Manager

Location: Toronto, Ontario, Canada (Hybrid/Onsite as required)
Employment Type: Full-Time

About the Role

We are seeking a highly skilled Data Engineering Manager to lead the design, development, and optimization of our Databricks and AWS-based data ecosystem. In this role, you will oversee a team of data engineers, drive engineering best practices, and ensure the performance, reliability, and scalability of the organization's data platforms. This position is ideal for a hands-on technical leader with deep expertise in Databricks, Spark, and modern cloud data frameworks.

What You'll Do

• Lead and mentor a team of data engineers in architecting and maintaining the Databricks Lakehouse Platform, including Delta Lake versioning and large-scale data processing.

• Manage and optimize Databricks infrastructure, including cluster lifecycle management, cost control, and integration with AWS services (S3, Glue, Lambda, and more).

• Architect and implement scalable ETL/ELT pipelines using Spark (Python/Scala), including streaming workloads when necessary.

• Drive technical excellence through advanced Spark performance tuning, cluster optimization, and I/O efficiency improvements.

• Implement strong data security and governance using Unity Catalog, ensuring alignment with industry and internal compliance standards.

• Champion Agile development practices, conduct code reviews, and serve as a technical advisor across cross-functional teams.

• Build and maintain monitoring and alerting solutions for data pipeline reliability, including SLAs, KPIs, and operational health checks.

• Develop and manage end-to-end CI/CD workflows using Git, automated testing, and continuous deployment pipelines.

What You Bring

• Bachelor's degree in Computer Science, Engineering, or a related field.

• 5+ years of hands-on experience with Databricks and Apache Spark, including building production-grade data pipelines.

• Proven experience leading and mentoring data engineering teams in a fast-paced environment.

• Deep expertise with AWS services: S3, EC2, Glue, EMR, Lambda, Step Functions.

• Strong programming proficiency in Python (PySpark) or Scala, with advanced SQL and data modeling skills.

• Experience with Infrastructure-as-Code tools such as Terraform or AWS CloudFormation.

• Strong understanding of data warehousing concepts, dimensional modeling, and RDBMS platforms (e.g., Postgres, Redshift).

• Proficiency with Git and CI/CD tools, including automated testing and deployment workflows.

• Excellent communication and stakeholder engagement capabilities.

• Demonstrated experience leveraging AI tools or techniques throughout the engineering lifecycle.

• Knowledge of the financial industry is preferred.
  
  
  
  Jeanhel Acuba
  Senior Technical Recruiter
  PRI Technology
  P:973.528.1947
  Jeanhel.Acuba@pritechnology.com