DL
Azure data engineer
Diverse Lynx India
Location not stated
Senior
1 month ago
- Azure Data Factory
- Databricks
- Data Architecture
- Azure Databricks
- ETL
- ELT
- SQL
- PySpark
- Scala
- Informatica
- Oracle
- Azure SQL
- Cosmos DB
- Unix
- Linux
- Maestro
- Azure DevOps
- GitHub
- CI/CD
- Copilot
- Machine Learning
1 month ago
Skills: Azure Data Factory
Experience Required: 4-6
Seeking a Senior Data Engineer with strong expertise in Azure Data Lake / Lakehouse architectures and Databricks, experienced in batch and real-time data ingestion, large-scale data processing, and enterprise data platforms. The role requires solid fundamentals in data architecture, data quality, and governance, across both legacy and modern data ecosystems.
________________________________________
Key Responsibilities
• Design and build batch and streaming data pipelines using Azure, Databricks, and Kafka
• Develop ETL/ELT solutions using SQL, PySpark, or Scala
• Implement and support Data Lake / Lakehouse architectures
• Integrate data from legacy and modern platforms
• Apply data quality checks and support data governance basics (metadata, lineage, access controls)
• Optimize Spark jobs for performance, reliability, and cost
• Collaborate with cross-functional teams and mentor junior engineers
________________________________________
Required Skills & Experience
• 7+ years of experience in data engineering
• Strong experience with Azure Data Lake, Databricks, ADF
• Hands-on expertise in SQL and PySpark or Scala
• Proven experience with batch & real-time ingestion
• Experience with Kafka and big data processing
• Experience with:
o Informatica PowerCenter
o Informatica IDMC or equivalent cloud data integration tools
o SQL Server, Oracle, Azure SQL
o Cosmos DB
o Unix/Linux
o Maestro (or similar enterprise schedulers)
o Data tools such as Precisely or equivalent
________________________________________
Added Advantage
• Insurance domain knowledge (Policy, Claims, Billing, Underwriting, Actuarial, Regulatory data)
• Experience with Azure DevOps (AzDO) and GitHub-based CI/CD
• Practical usage of Copilot for data engineering and development productivity
• Exposure to Generative AI / GenAI use cases in data platforms and analytics
Azure data engineer · Diverse Lynx India