DL
Azure Data Engineer
Diverse Lynx India
Location not stated
Mid level
1 month ago
- Azure
- Python
- Azure Data Factory
- PySpark
- ETL
- ELT
- Azure Synapse
- Azure Databricks
- SQL
- Git
- CI/CD
- Agile
- Scrum
- Azure DevOps
1 month ago
Experience Required: 8-10
Location: ~BANGALORE~HYDERABAD~CHENNAI~
Skills: Digital: Microsoft Azure~Digital : Python~Azure Data Factory
Job Summary
We are looking for an experienced Azure Data Engineer with 8–10 years of experience in designing, developing, and maintaining scalable data solutions on Microsoft Azure. The ideal candidate should have strong expertise in Azure data services, PySpark, and data migration testing, with experience in building robust ETL/ELT pipelines and validating large-scale data migrations.
Key Responsibilities
- Design, develop, and optimize data pipelines using Microsoft Azure services.
- Develop scalable data processing solutions using PySpark.
- Perform data migration testing, including source-to-target validation, data reconciliation, and data quality checks.
- Collaborate with business stakeholders, data architects, and application teams to understand data requirements.
- Build and maintain ETL/ELT workflows for structured and unstructured data.
- Monitor and troubleshoot data pipelines to ensure high availability and performance.
- Implement data quality, governance, and security best practices.
- Participate in code reviews, testing, and deployment activities.
Required Skills
- Strong experience with Microsoft Azure data services (Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, Azure Databricks).
- Hands-on experience with PySpark for large-scale data processing.
- Expertise in data migration testing, data validation, reconciliation, and quality assurance.
- Strong SQL skills and experience with relational databases.
- Knowledge of ETL/ELT concepts and data warehousing.
- Familiarity with version control tools such as Git and CI/CD practices.
- Good analytical, problem-solving, and communication skills.
Preferred Skills
- Experience working in Agile/Scrum environments.
- Knowledge of Python and Azure DevOps.
- Understanding of data governance and cloud security best practices.
- Azure certifications are an added advantage.
Azure Data Engineer · Diverse Lynx India