Specialist - Data Engineering

Job Title: Specialist - Data Engineering

Location: Pune

Experience: 5-8 years

We are seeking a highly skilled Specialist - Data Engineering to join our team in Pune. This role involves designing, developing, and maintaining robust data solutions within the Azure cloud ecosystem, ensuring efficient data processing and availability for analytical insights. The ideal candidate will possess deep expertise in Azure data services and a proven track record of delivering scalable data platforms.

Job Description:

1. Design, develop, and implement scalable data pipelines using Azure Data Factory for data ingestion and transformation.

2. Manage and optimize data storage solutions within Azure Data Lake, ensuring data integrity and accessibility.

3. Develop and maintain complex data processing workflows utilizing Databricks for advanced analytics and machine learning initiatives.

4. Implement and manage relational databases in Azure SQL, ensuring high performance and reliability.

5. Optimize data storage and retrieval mechanisms, including the effective use of Azure BLOB storage.

6. Design and build data warehousing solutions leveraging Azure Synapse Analytics for business intelligence and reporting.

7. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver appropriate solutions.

8. Monitor data pipeline performance, troubleshoot issues, and implement optimizations to ensure efficient data flow.

9. Ensure adherence to data governance, security, and compliance standards across all data engineering activities.

10. Provide technical guidance and mentorship to junior team members on best practices in Azure data engineering.

Skills and Qualifications:

Required Skills:

5-8 years of professional experience in data engineering, with a strong focus on Microsoft Azure cloud services.

Proficiency in Azure Data Factory for orchestrating and automating data pipelines.

Extensive experience with Azure Data Lake for scalable data storage and processing.

Strong command of Azure SQL for database management and query optimization.

Demonstrated expertise in Databricks for big data processing, analytics, and machine learning.

Desired Skills:

Experience with Azure BLOB storage for unstructured data management.

Familiarity with Azure Synapse Analytics for integrated data warehousing and analytics.

Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field.

Excellent problem-solving abilities and a proactive approach to technical challenges.

Strong communication and collaboration skills to work effectively within a team environment.