DL
Azure Databricks + PySpark
Diverse Lynx India
🇮🇳 India
On-site
Mid level
2 months ago
- Azure Databricks
- PySpark
- Azure Cloud
- Databricks
- SQL
- ETL
- ELT
- Azure Data Factory
- Delta Lake
- Agile
- Azure
- Apache Spark
- Python
- Azure Synapse
- Azure DevOps
- CI/CD
- Git
- Docker
- Kubernetes
- Snowflake
- Terraform
- IaC
2 months ago
Cognizant is looking for experienced Azure Databricks + PySpark Developers to join its Data Engineering practice. The ideal candidate should have strong expertise in Azure cloud technologies, Databricks, PySpark, and SQL, with hands-on experience building scalable data pipelines and modern data lake solutions.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks and PySpark.
- Build and optimize data processing solutions for large-scale datasets.
- Develop data pipelines using Azure Data Factory and Azure Data Lake Storage (ADLS).
- Write optimized Spark SQL and SQL queries for high-performance data processing.
- Implement Delta Lake architecture and optimize Databricks workloads.
- Collaborate with business analysts, data scientists, and application teams to deliver data solutions.
- Monitor, troubleshoot, and optimize production data pipelines.
- Ensure data quality, security, governance, and compliance.
- Participate in code reviews and follow Agile development practices.
- Strong hands-on experience with Microsoft Azure.
- Expertise in Azure Databricks.
- Strong experience with PySpark and Apache Spark.
- Advanced SQL programming.
- Experience with Azure Data Factory (ADF).
- Experience with Azure Data Lake Storage (ADLS Gen2).
- Knowledge of Delta Lake.
- Strong Python programming skills.
- Experience designing ETL/ELT pipelines.
- Good understanding of data warehousing concepts.
- Azure Synapse Analytics.
- Azure Event Hubs or Azure Service Bus.
- Azure DevOps and CI/CD.
- Git version control.
- Docker and Kubernetes.
- Snowflake experience.
- Terraform or Infrastructure as Code.
- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
- 5.5–9 years of relevant experience in Data Engineering.
- Strong communication and stakeholder management skills.
Azure Databricks + PySpark · Diverse Lynx India