ES
Databricks Lead / Data Architect
Epsilon Solutions LTD
Location not stated
Staff / Principal
4 weeks ago
- Databricks
- Apache Spark
- AWS
- Azure
- GCP
- AI/ML
- ETL
- ELT
- PySpark
- SQL
- Delta Lake
- Unity Catalog
- CI/CD
- Devops
- MLflow
- Python
- Data Modeling
- Git
- Airflow
- Azure Data Factory
- AI
- Snowflake
- IaC
- Terraform
- Kubernetes
- Data Architecture
4 weeks ago
4+ yrs Databricks " 1 Hybrid - Mississauga
"Job Summary
We are seeking an experienced Databricks Architect to design, build, and optimize scalable cloud-based data platforms and Lakehouse solutions. The ideal candidate will have strong expertise in Databricks, Apache Spark, cloud platforms (AWS/Azure/GCP), data engineering, and modern analytics architecture.
The role involves leading end-to-end architecture, migration, modernization, governance, performance optimization, and implementation of enterprise data solutions supporting analytics, AI/ML, and business intelligence initiatives.
Key Responsibilities
• Design enterprise-scale Lakehouse architecture using Databricks
• Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, and Databricks Workflows
• Implement Delta Lake, Medallion Architecture (Bronze/Silver/Gold), and Unity Catalog
• Lead migration of legacy data warehouses and ETL platforms to Databricks
• Design batch and real-time streaming data pipelines
• Optimize Spark jobs for performance, scalability, and cost efficiency
• Implement CI/CD pipelines and DevOps automation
• Define data governance, security, lineage, and access control standards
• Collaborate with business stakeholders, data engineers, analysts, and leadership teams
• Provide technical leadership, code reviews, architecture reviews, and best practices
• Support AI/ML enablement using MLflow and Databricks capabilities
• Establish monitoring, observability, and operational support frameworks
Required Skills
Technical Skills
• Strong hands-on experience with:
o Databricks
o Apache Spark
o PySpark
o SQL
o Python
• Experience with Delta Lake and Unity Catalog
• Expertise in data modeling and Lakehouse architecture
• Experience with cloud platforms:
o AWS
o Azure
o GCP
• Knowledge of:
o Data Lakes
o Data Warehousing
o Streaming frameworks
o CI/CD pipelines
o Git/DevOps
• Familiarity with orchestration tools:
o Airflow
o Azure Data Factory
o Databricks Workflows
Preferred Skills
• MLflow / AI & ML integration
• Kafka / Structured Streaming
• Snowflake integration
• Infrastructure as Code (Terraform)
• Kubernetes exposure
• Data governance and security frameworks
Experience Requirements
• 10+ years in Data Engineering / Data Architecture
• 5+ years working with Databricks
• Experience leading enterprise cloud modernization projects
• Strong stakeholder management and client-facing experience
• Experience designing scalable distributed data systems
"
Thanks & Regards
Aman Sharma
Direct Number:408-459-7174
e-mail : aman.s@epsilonsolutions.ca
Mississauga, Canada
Databricks Lead / Data Architect · Epsilon Solutions LTD