Databricks~Digital : PySpark

Diverse Lynx India

Location not stated

2 months ago

Azure Databricks
CI/CD
Devops
PySpark
Databricks

2 months ago

Not scoredNo CV on file. Upload one and this job gets a score out of 100.Upload CV

Role Descriptions: Design| develop| and maintain scalable data pipelines using Azure Databricks and PySpark.Build and optimize ETLELT workflows for processing structured and unstructured data.Develop data ingestion frameworks from multiple sources| including databases| APIs| files| and cloud storage.Implement data transformation| cleansing| and validation processes.Optimize Spark jobs for performance| scalability| and cost efficiency.Design and maintain data lakes| data warehouses| and analytical datasets.Ensure data quality| integrity| security| and governance standards are followed.Collaborate with data architects| analysts| and business teams to understand data requirements.Monitor| troubleshoot| and resolve data pipeline issues.Support deployment and automation through CICD and DevOps practices.
Essential Skills: We are seeking a highly skilled Data Engineer with strong expertise in Azure Databricks and PySpark to design| develop| and maintain scalable data pipelines and modern data platforms. The ideal candidate will have experience working with large-scale datasets| implementing ETLELT processes| and building cloud-based data engineering solutions on Microsoft Azure.The role will involve collaborating with business stakeholders| data analysts| architects| and development teams to enable reliable| high-quality| and efficient data processing solutions that support analytics| reporting| and business intelligence initiatives.
Desirable Skills:
Keyword:
Skills: Digital : Databricks~Digital : PySpark
Experience Required: 6-8

Databricks~Digital : PySpark · Diverse Lynx India

Auto apply with Likeremote