Subscribe to the latest remote jobs:

Hadoop Data Engineer

🇺🇸 United States

Scala

Management

Java

Python

AWS

Azure

Finance

Hadoop

Hive

Design

Project Management

NoSQL

SQL

Analyst

Hadoop Data Engineer

from 🇺🇸 United States

Hadoop Data Engineer responsible for designing, developing, and maintaining large-scale data processing systems within a distributed Hadoop ecosystem. The role focuses on enabling data-driven decision-making across banking operations, risk management, compliance, and customer analytics.

Key Responsibilities

  • Design, develop, and maintain scalabledata pipelines using Hadoop ecosystem tools (HDFS, Hive, Spark, Sqoop, Kafka).
  • Build and optimizeETL/ELT processes to support data ingestion from multiple banking systems.
  • Develop and managebig data solutions for structured and unstructured data.
  • Collaborate with data analysts, data scientists, and business stakeholders to deliver data solutions.
  • Ensuredata quality, integrity, and governance aligned with banking and regulatory standards.
  • Perform performance tuning and optimization of Hadoop/Spark jobs.
  • Implementdata security controls to comply with financial regulations (e.g., PCI, SOX).
  • Support real-time and batch data processing frameworks.
  • Troubleshoot production issues and provide continuous support for data platforms.
  • Work with cloud platforms (e.g., AWS, Azure) for modern data solutions.

Required Skills & Qualifications

Technical Skills

  • Strong experience with:
  • Hadoop ecosystem (HDFS, MapReduce, Hive, HBase)
  • Apache Spark (Scala/Python)
  • SQL & NoSQL databases
  • ETL tools (Informatica, Talend, or similar)
  • Kafka or other streaming tools
  • Proficiency in programming:
  • Python / Java / Scala
  • Experience with:
  • Data warehousing concepts
  • Workflow orchestration tools (Airflow, Oozie)
  • Unix/Linux environments
  • Knowledge ofcloud data platforms (AWS EMR, Azure Data Lake) is a plus

Domain Knowledge

  • Understanding ofbanking and financial services data
  • Exposure torisk, compliance, or fraud analytics is preferred

Soft Skills

  • Strong problem-solving and analytical abilities
  • Excellent communication and collaboration skills
  • Ability to work in Agile/Scrum environments

Education & Experience

  • Bachelor’s or Master’s degree in:
  • Computer Science, Information Technology, or related field
  • Typically5–10 years of experience in data engineering or big data development


by @maxrusakovic