DL
Azure Databricks, PySpark, Azure Data Factory - Pune
Diverse Lynx India
Location not stated
1 month ago
- Azure Databricks
- PySpark
- Azure Data Factory
- Apache Spark
- Power BI
- Azure
- Data Visualization
- SQL
- Unity Catalog
- CI/CD
- Databricks
- Agile
1 month ago
| Azure Databricks, PySpark, Azure Data Factory Responsibilities • Design, develop, and maintain data pipelines using Azure Data Factory. • Manage and optimize data storage using Azure Data Lake Gen 2. • Build and process large-scale data solutions with Azure Databricks and Apache Spark. • Create interactive reports and dashboards in Power BI for business insights. • Collaborate with cross-functional IT and business teams to translate requirements into data solutions. • Ensure data quality, security, and compliance in line with organizational standards. Core Skills Required • Azure Data Factory: Experience in building and managing data pipelines. • Azure Data Lake Gen 2: Proficient in data storage and management within Azure's data lake environment. • Azure Databricks / Apache Spark: Hands-on skill with distributed data processing, transformations, and analytics. • Power BI: Expertise in data visualization and reporting. Nice-to-Have Skills • Basic SQL Performance Tuning: Ability to write and optimize SQL queries. • Data Governance & Unity Catalog: Understanding of data governance principles and experience with Unity Catalog for data management. • Certification: Microsoft DP-203 (Azure Data Engineer Associate) certification. • CI/CD Pipelines: Experience implementing CI/CD pipelines for Azure Data Factory or Databricks projects. Qualifications • Bachelor's degree in Computer Science, Information Technology, or related field (or equivalent experience). • 7+ years of professional experience with Azure data services. • Strong analytical and problem-solving skills. • Effective communication and team collaboration abilities. Preferred Traits • Continuous learner, staying updated with Azure and data engineering advancements. • Experience working in Agile environments. • Passion for optimizing data workflows and enabling data-driven decisions Experience Required: 8+ relevant exp mandatory, anything below this will be rejected Loc- Pune |
Azure Databricks, PySpark, Azure Data Factory - Pune · Diverse Lynx India