|
Senior Big Data Engineer (Scala & Spark)-
Candidates with under 10 years of experience and strong hands-on expertise in Scala, Spark, and Big Data technologies, capable of contributing immediately to project delivery.
Key Responsibilities
- Design, develop, and maintain scalable data processing applications usingScala andApache Spark.
- Build and optimize ETL/ELT pipelines for processing large volumes of structured and unstructured data.
- Analyze complex business requirements and translate them into technical solutions.
- Work with distributed computing frameworks and Big Data platforms to ensure high performance and reliability.
- Collaborate with cross-functional teams including Data Engineers, Architects, Product Owners, and Business Stakeholders.
- Troubleshoot and resolve data processing, performance, and scalability issues.
- Follow coding standards, CI/CD practices, and best practices for data engineering development.
- Participate in code reviews and contribute to continuous improvement initiatives.
Required Skills
- Strong hands-on experience inScala development.
- Expertise inApache Spark (Spark Core, Spark SQL, DataFrames, Performance Tuning).
- Experience working withBig Data ecosystems such as Hadoop, Hive, HDFS, Kafka, or related technologies.
- Strong SQL and data modeling skills.
- Experience building and supporting large-scale distributed data processing applications.
- Understanding of software engineering best practices, version control, and CI/CD processes.
- Strong analytical and problem-solving skills.
Preferred Skills
- Experience with cloud platforms such asAWS,
- Exposure to data orchestration tools such as Airflow.
- Knowledge of containerization and DevOps practices.
- Experience in telecommunications or large enterprise data environments
|
