RC
Apache Airflow - Data Engineer
RAPS Consulting - India
Location not stated
2 months ago
- Apache Airflow
- Airflow
- ETL
- ELT
- CI/CD
- Git
- AWS
- Azure
- GCP
- Disaster Recovery
- Python
- SQL
2 months ago
We are seeking a skilledApache Airflow Engineer to design, develop, and maintain scalable workflow orchestration pipelines while ensuring alignment withinstitutional compliance.This role combines strong data engineering expertise with a deep understanding of financial compliance domain
Key Responsibilities
Airflow & Data Engineering
- Design, develop, and manageApache Airflow DAGs for scheduling, monitoring, and orchestrating complex data pipelines.
- Build scalableETL/ELT workflows integrating data from multiple sources (databases, APIs, cloud platforms).
- Optimize Airflow performance, managetask dependencies, retries, SLA monitoring, and alerting.
- ImplementCI/CD pipelines for DAG deployment and version control (Git-based workflows).
- Troubleshoot production issues and ensure high availability of workflows.
- Integrate Airflow with cloud platforms (AWS, Azure, GCP) and services such as S3, Ensurebusiness continuity and disaster recovery planning for data workflows.
- Strong hands-on experience withApache Airflow (DAG development, scheduling, monitoring).
- Proficiency inPython (core requirement for Airflow).
- Experience withSQL and relational/non-relational databases.
- Familiarity withcloud platforms (AWS/Azure/GCP).
Compliance & Governance Skills (Must-Have)
Working knowledge ofFinancial crime and compliance management
Apache Airflow - Data Engineer · RAPS Consulting - India