Azure Databricks Architect

Title: Azure Databricks Architect

Work Location - Toronto, Canada

Terms: Contract

Job Description:

• Experience Minimum 12 years of hands-on experience architecting and delivering largescale data platforms using Azure and Databricks.

• Architect and implement Azure based data solutions using Databricks Data Factory Synapse Analytics and Data Lake services.

• Design build and optimize a Medallion architecture Bronze Silver Gold supporting both HR and enterprise data domains.

• Develop data pipelines APIs and orchestration frameworks that enable seamless data ingestion transformation and delivery.

• Establish and maintain data governance metadata and lineage frameworks using Databricks Unity CatLog and Azure Purview.

• Build and maintain ETL frameworks data APIs and Realtime workflows connecting diverse HR and enterprise systems.

• Leverage AI and ML capabilities within Databricks including ML flow feature stores and the Agent Framework Agent Bricks to build intelligent data agents for automation monitoring and analytics enrichment.

• Design and implement AI agent pipelines that can reason over HR data models automate transformations and enhance analytical outcomes.

• Apply best practices for performance optimization scalability and cost efficiency within the Azure ecosystem.

• Integrate CICD pipelines and infrastructure as code approaches Terraform Bicep or Azure DevOps.

• Serve as the technical authority for Azure and Databricks architecture within HR Technology

• Partner with HR Analytics and Infrastructure leaders to define and execute the data and AI modernization roadmap

• Establish standards patterns and reusable frameworks for data engineering AI agents and automation

• Mentor engineers and champion a culture of innovation experimentation and delivery excellence.

• Expertise in Azure Data Lake Data Factory and Synapse Analytics

• Databricks Delta Lake Unity CatLog Spark optimization

• Python SQL PySpark and API development REST GraphQL

• AIML frameworks within Databricks including Agent Bricks and ML flow

• Strong grounding in data governance security and compliance best practices

• Excellent communication and collaboration skills ability to translate complex architecture into business outcomes

Skills

Mandatory Skills : Azure Data Factory, Azure Synapse Analytics, AZURE DATA LAKE