Lead Apache Airflow Engineer

We are seeking aLead Apache Airflow Engineer to drive the development and maintenance of Airflow-based workflows, guide technical delivery, and mentor team members while collaborating with cross-functional and distributed teams.

Responsibilities

• Develop new and maintain existing Airflow Operators, Hooks and Sensors
• Integrate new Airflow versions with Cloud Composer
• Fix bugs in Airflow and implement improvements
• Implement tests for Airflow, including Unit Testing and Solution Testing
• Create internal documentation for the work artifacts
• Coordinate development activities within the team and support delivery planning
• Collaborate with cross-functional and distributed teams to support delivery and resolve technical issues
• Support junior team members through knowledge sharing, technical guidance, and code review activities
• Participate in technical discussions with client, prioritization, and coordination of development activities within the team

Requirements

• 7+ years of experience with strong Python programming skills
• Experience with Cloud APIs (REST/RPC)
• Knowledge of DevOps/SysAdmin practices, Linux, and Kubernetes, including building container images
• Understanding of Cloud platforms, especially Google Cloud Platform
• Expertise in Airflow or similar experience in building workflows based on Luigi, Oozie, Argo, Azkaban, or Apache NiFi
• Experience working with open-source communities
• Proficiency in English communication, both spoken and written

Nice to have

• Understanding of cloud architecture, including scalability and IaaS principles
• Google Cloud Certification (e.g., GCP Architect or Data Engineer)