Data Engineer

Technical Stack & Requirements:

·        Core Competencies: Python, Apache Spark (PySpark), AWS Glue ETL, Data Lakeconcepts (Medallion Architecture), Databricks, Sagemaker.

·        Infrastructure: CloudFormation/Terraform, CI/CD with GitHub Actions.

·        Core Responsibilities: Orchestrate data extraction from diverse legacy and modernsources, engineer high-performance scalable pipelines, and provide expert-level production support to ensure system reliability.

Qualifications:

·        Bachelor’s degree in Computer Science or related field.

·        Strong hands-on experience with Apache Spark and Glue.

·        Proven ability to work with Python-based data pipelines.

·        Experience with Git andGitHub Actionsis mandatory.

Required

Qualifications

·        Bachelor's degree in Computer Science, Engineering, Information

Systems, or related discipline.

·        5-7+ years of enterprise Data Engineering experience.

·        Strong Python development experience.

·        Advanced SQL programming skills.

·        Extensive experience with Apache Spark and PySpark.

·        Strong experience with Databricks application.

·        Hands-on experience with AWS data services including:

o   S3

o   AWS Glue

o   Athena

o   Lambda

o   Step

Functions

o   EventBridge

·        Experience building enterprise ETL and ELT pipelines.

·        Experience in EMR-based Query Cluster workloads, that includes Jupyter notebooks, Park Application and ETL Pipelines. 

·        Strong understanding of distributed data processing.

·        Experience working with large-scale cloud data platforms.

·        Strong knowledge of dimensional data modeling.

·        Experience implementing CI/CD pipelines.

·        Familiarity with Agile/Scrum methodologies.

·        Excellent communication and stakeholder management skills.

Preferred

Qualifications

·        Financial Services or Asset Management industry experience.

·        Knowledge of Apache Iceberg, Delta Lake, or Hudi.

·        Experience with Airflow.

·        Knowledge of Kafka or Kinesis streaming.

·        Experience supporting AI/ML and advanced analytics platforms.

·        AWS Professional or Specialty Certifications.