Senior Specialist - Data Engineering

ob Title Data Architect Azure Databricks
Experience
11 Years
Location
India HybridRemote
Job Summary
We are seeking an experienced Data Architect with 11 years of experience in designing and implementing enterprisescale data platforms and analytics solutions on Microsoft Azure The ideal candidate should possess strong expertise in Azure Databricks Azure Data Factory ADF SQL Python and PySpark with proven experience in building scalable secure and highperformance data ecosystems supporting business intelligence advanced analytics and AIML initiatives
Mandatory Skills

Azure Databricks
Microsoft Azure Cloud
Azure Data Factory ADF
SQL
Python
PySpark

Key Responsibilities
Data Architecture Design

Design and implement scalable cloudbased data architectures on Azure
Define enterprise data models data integration patterns and data governance standards
Architect modern Data Lakehouse solutions using Databricks and Delta Lake
Create conceptual logical and physical data models

Data Engineering Integration

Design and develop batch and realtime data pipelines using ADF and Databricks
Build scalable ETLELT frameworks using Python and PySpark
Integrate data from multiple source systems including databases APIs SaaS platforms and files
Optimize Spark jobs Databricks clusters and SQL workloads for performance and cost efficiency

Data Governance Security

Implement data quality lineage metadata and governance frameworks
Establish security controls rolebased access and compliance standards
Define best practices for data lifecycle management and monitoring

Stakeholder Engagement

Work closely with business stakeholders product owners and development teams to translate business requirements into technical solutions
Provide technical leadership and architectural guidance to data engineering teams
Participate in architecture reviews solution design sessions and technology evaluations

Delivery Leadership

Lead endtoend implementation of enterprise data platforms
Mentor data engineers and enforce engineering best practices
Drive continuous improvement and innovation in cloud data solutions

Required Qualifications

Bachelors or Masters degree in Computer Science Information Technology Engineering or related field
11 years of experience in Data Engineering Data Warehousing and Data Architecture
5 years of handson experience in Azure Cloud and Databricks ecosystem
Strong experience designing Lakehouse architectures and enterprisescale data platforms
Expertise in SQL optimization and advanced query development
Strong programming background in Python and PySpark
Experience with CICD DevOps and Agile delivery methodologies

Technical Competencies

Azure Databricks
Azure Data Factory ADF
Azure Data Lake Storage ADLS Gen2
Delta Lake
Spark SQL
Python
PySpark
Azure SQL Database
Data Modeling
ETLELT Design
Data Governance Security
Performance Tuning
Azure DevOps Git

Preferred Skills

Azure Synapse Analytics
Microsoft Fabric
Data Mesh Data Products Architecture
Streaming technologies Kafka Event Hub
AIML Data Platform experience
Databricks Unity Catalog
Certification in Azure Data Engineering or Azure Solutions Architecture

Soft Skills

Strong communication and stakeholder management
Excellent analytical and problemsolving abilities
Leadership and mentoring capabilities
Ability to drive architecture decisions across distributed teams

Key Deliverables

Enterprise Data Architecture
Lakehouse Data Platform Design
Scalable Data Pipelines
Data Governance Frameworks
Performance Optimized Analytics Solutions
Cloud Migration Modernization Strategies