Senior Specialist - Data Engineering

Pipeline Development Design and implement highly modular automated Extract Transform Load ETL data pipelines using Python and related frameworks

Data Transformation Build Python logic using packages like Pandas or PySpark to clean format deduplicate and encrypt raw data into optimized staging areas

Orchestration Automation Schedule and orchestrate complex workflows using tools like Apache Airflow Prefect or Dagster to minimize delays and prevent performance bottlenecks

Database Warehouse Engineering Develop and tune SQL queries design data schemas and manage data ingestion into Relational Databases RDBMS or modern cloud data warehouses like Snowflake

Monitoring Error Handling Implement thorough logging frameworks exception handling and realtime ing systems to maintain pipeline integrity and data quality

API System Integration Extract data from heterogeneous systems including flat files thirdparty REST APIs web scraping and streaming sources

Documentation Maintain clear code documentation architectural mapping and pipeline workflow outlines