Python Data Engineer

Job Summary

Seeking a skilled Python Data Engineer with expertise in ETL pipeline design AWS data services and data quality engineering to support scalable data solutions

Job Description

Strong proficiency in Python data engineering using pandas and numpy Design and implementation of ETLdata pipelines that are schemadriven and configurationdriven Data modeling for hierarchical and dimensional data structures Experience with AWS data services including S3 IAM DynamoDB and SSM Expertise in data validation and dataquality engineering Proficient in SQL and working with DatabricksJDBC feeds Knowledge of columnar and analytics file formats such as Parquet and CSV pragmatics Handling and processing of timeseries data Implementing testing and ensuring reproducibility for data code Familiarity with Git GitHub Actions and Terraform for version control and infrastructure as code Integration experience with SageMaker and Chronos endpoints at the interface level Ensuring secure data handling practices throughout the data lifecycle

Roles and Responsibilities

Develop maintain and optimize ETL pipelines and data workflows following schema and configurationdriven approaches

Build and maintain data models for hierarchical and dimensional datasets to support analytics and reporting

Manage and utilize AWS data services to store secure and process large datasets

Perform rigorous data validation and quality checks to ensure data integrity

Collaborate with crossfunctional teams to integrate SQL queries and DatabricksJDBC data feeds

Work with columnar data formats and optimize data storage and retrieval

Handle timeseries data processing and analysis tasks

Write unit and integration tests to guarantee reproducibility and reliability of data code

Use version control tools and automation workflows to maintain codebase and infrastructure

Support integration of machine learning endpoints such as SageMaker and Chronos

Implement and enforce secure data handling and compliance standards