NC
Databricks ETL Engineer
NR Consulting - India
🇮🇳 India
On-site
4 weeks ago
- Databricks
- ETL
- Alteryx
- PySpark
- ELT
- Delta Lake
- Python
- Unity Catalog
4 weeks ago
Location: Hyderabad
Exp: 3-6 Yrs
Job Description:
Responsibilities
• Translate Alteryx visual ETL logic into clean, optimized, code-based Databricks pipelines, ensuring full parity with source behavior.
• Recreate dynamic Alteryx behaviors such as parameter-driven execution and conditional logic in Databricks using PySpark.
• Validate migrated pipelines against source outputs and confirm measurable performance and scalability improvements post-migration.
• Design and develop ETL/ELT pipelines using PySpark, Delta Lake, and Databricks Workflows to process large-scale datasets efficiently.
• Produce clear technical documentation covering data flows, architecture decisions, and operational procedures; assist in developing communication materials to support accurate usage and interpretation of JLL data by business teams.
• Monitor data pipelines and Databricks workloads to support stability, performance, and reliability.
• Develop a thorough understanding of how data flows from various source systems and source types to continuously fine-tune data integration solutions.
• Collaborate with senior engineers to understand business requirements and contribute to appropriate technical solutions.
• Work independently or as part of a team to deliver data engineering projects on time and to specification; follow team coding standards and best practices and actively contribute to continuous process improvement.
Technical Skills & Competencies
• Databricks ETL/ELT Pipeline Design, Build & Optimization
• Python — Scripting, Automation, Pipeline Development
• PySpark — Distributed Processing, Transformations, Optimization
• Unity Catalog — Data Governance, Access Control, Lineage
Databricks ETL Engineer · NR Consulting - India