Pyspark Developer

Diverse Lynx India

Location not stated

Senior

4 weeks ago

PySpark
Python
SQL
Unix
ELT
ETL
Linux

4 weeks ago

Not scoredNo CV on file. Upload one and this job gets a score out of 100.Upload CV

We are seeking a highly skilledSenior PySpark Developer with strong expertise inPython programming, PySpark, SQL, UNIX scripting, and ELT/ETL data processing. The ideal candidate should have hands-on experience building scalable data pipelines, processing large datasets, and developing batch and streaming solutions in distributed environments. The candidate should be capable of independently designing, developing, optimizing, and supporting data engineering solutions while effectively collaborating with stakeholders.
Must-Have Skills

StrongPython programming skills with expertise in coding, data processing, and performance optimization.
Extensive hands-on experience inPySpark development, including application design, job execution, performance tuning, and troubleshooting.
Hands-on experience inSQL coding
Experience inELT/ETL pipeline development and data integration frameworks.
Proficiency inUNIX/Linux Shell Scripting, including creating, modifying, and troubleshooting scripts.
Experience withbatch and real-time/streaming data processing using modern data engineering technologies.
Ability to handle large-scale data processing and optimize data workflows for performance and scalability.
Good understanding of distributed computing concepts and big data architecture.
Experience in requirement gathering, solution design, technical documentation, and implementation.
Strong analytical and problem-solving skills.
Excellent verbal and writtencommunication skills with the ability to work effectively with technical and business stakeholders.

Pyspark Developer · Diverse Lynx India

Auto apply with Likeremote