DL
Azure Data Factory
Diverse Lynx India
🇮🇳 India
On-site
2 months ago
- PySpark
- ETL
- Databricks
- Python
- Azure
- Agile
- Scrum
- Azure Databricks
- Azure Data Factory
- SQL
- Data Modeling
- XML
- JSON
- Oracle
- Git
- Jenkins
2 months ago
• Candidate should have good understanding in Spark, pyspark, ADF, ADB and understanding of broadcast join, ETL.
• Data Engineer Design and develop ETL processes based on functional and non-functional requirements in Databricks| python PySpark within Azure platform.
• Understand the full end to end development activities from design to go live for ETL development in Azure platform.
• Recommend and execute improvements.
• Execute and provide support during testing cycles and post-production deployment| engage in peer code reviews.
• Apply automation and innovation on new and on-going data platforms that aligned to business strategies.
• Document component design for developers and for broader communication.
• Understand and adopt an Agile (SCRUM like) software development mindset.
• Follow established processes standards| business technology architecture for development| release management and deployment process.
• Design| develop and implement reporting platforms and complex ETL frameworks that meet business requirements.
• Provide data analysis and requirements within enterprise platform.
• Develop| maintain knowledge of data available from upstream sources and data within various platforms.
• Essential Skills: Must-have skills Azure Databricks| Azure Data Factory Python Spark Pyspark Shell ScriptSynapse5 years of experience in ETL Data area with minimum of 3 years of development experience in above technologies (PySPark| ADB| ADF) is must SQL| Working experience with data modeling| relational modeling and dimensional modeling| Data Warehousing| ETL General knowledge about file formats (e.g. XML| CSV| JSON)| databases (e.g. MS SQL| Oracle) and different type of connectivity is also very useful.Data files movement via mailbox Source-code version in gpromotion tools| e.g. Git Jenkins Good communication skill| team player
Azure Data Factory · Diverse Lynx India