Likeremote

Subscribe to the latest remote jobs:

  • Likeremote jobs on https://LinkedIn.com/
  • Likeremote jobs on https://telegram.org/
  • Likeremote jobs on Reddit.com
ES

Java Spark Scala Developer

Epsilon Solutions LTD
Location not stated
7 months ago
  • Scala
  • Java
  • Apache Spark
  • Python
  • PySpark
  • Microservices
  • SQL
  • PostgreSQL
  • MySQL
  • Oracle
  • Hadoop
  • Yarn
  • Apache Airflow
  • Linux
  • Unix
  • AWS
  • EC2
  • EMR
  • Agile
  • Scrum
  • Hive
  • Azure
  • GCP
  • Git
  • Maven
  • Gradle
  • CI/CD
7 months ago
Auto apply with Likeremote
Not scoredNo CV on file. Upload one and this job gets a score out of 100.Upload CV
Position: Spark Scala with Java
Location: Toronto ON(Hybrid)


Job Description:

Key Responsibilities:
  • Design and Development:
    • Design, develop, and implement scalable and robust data pipelines and applications usingApache Spark with Scala, Python (PySpark), or Java.
    • Write efficient and optimized code for data processing, transformation, and analysis.
    • Develop and maintain APIs and microservices using Java or Python for integrating with various systems.
  • Data Management & Analysis:
    • Work extensively with SQL for querying, manipulating, and optimizing relational databases (e.g., PostgreSQL, MySQL, Oracle, MS SQL Server).
    • Design and implement database schemas, stored procedures, and complex queries.
    • Perform data analysis and develop insights from large datasets using Spark and relevant libraries.
  • Big Data Ecosystem:
    • Utilize and manage components within the Hadoop ecosystem (HDFS, YARN) for distributed storage and processing.
    • Implement and manage data ingestion and orchestration workflows (e.g., Apache Airflow, Oozie).
  • System Administration & Scripting:
    • Work proficiently in Linux/Unix environments, including shell scripting for automation, system monitoring, and deployment tasks.
    • Manage and configure servers and cloud resources (e.g., AWS EC2, EMR).
  • Collaboration & Best Practices:
    • Collaborate with data scientists, analysts, and other engineers to understand requirements and deliver solutions.
    • Participate in code reviews, promote best practices, and ensure code quality and maintainability.
    • Adhere to Agile/Scrum methodologies for software development.
  • Performance Optimization & Troubleshooting:
    • Identify and resolve performance bottlenecks in Spark jobs and data pipelines.
    • Troubleshoot issues related to data processing, system stability, and application functionality.
Required Skills:
  • Programming Languages: Expert-level proficiency inScala, Python, and Java.
  • Databases: Strong SQL skills, experience with relational databases (e.g., PostgreSQL, MySQL, Oracle).
  • Operating Systems: Proficient with Linux/Unix command-line interface and shell scripting.
Nice to have Skills:
  • Big Data Technologies: Deep understanding and hands-on experience with Apache Spark (including Spark SQL, DataFrames, RDDs), and ideally familiarity with other big data tools like Hive, Pig, Kafka.
  • Cloud Platforms: Experience with cloud platforms like AWS, Azure, or GCP is often preferred, including services like S3, EMR, EC2, Lambda.
  • Tools & Methodologies: Experience with version control systems (Git), build tools (Maven, Gradle, SBT), CI/CD pipelines, and Agile development methodologies.

Java Spark Scala Developer · Epsilon Solutions LTD

Auto apply with Likeremote