Senior Data Integration Engineer

We are actively seeking an experienced Senior Data Integration Engineer to join our team. In this role, you will play a crucial part in designing, developing, testing, deploying, maintaining, and enhancing data integration pipelines for our cutting-edge projects. If you possess a wealth of experience in data integration and pipeline development and thrive in a dynamic, collaborative environment, we welcome you to be a key player in shaping the future of our data-driven initiatives.

Responsibilities

• Design, develop, test, deploy, maintain, and improve data integration pipelines
• Utilize Python and common libraries for efficient data integration and pipeline development
• Employ advanced SQL for analytical purposes and to optimize complex queries
• Collaborate with cross-functional teams to ensure the success of data integration projects
• Implement and maintain data integration and pipeline developments in an AWS environment
• Optimize and debug user-defined functions, views, and indexes for efficient data integration
• Utilize GitLab for source control and Jenkins for build and continuous integration
• Contribute to the enhancement and maintenance of data integration pipelines
• Provide insights and recommendations for improving data integration processes
• Collaborate with the wider engineering team to drive standards of excellence across all data integration projects

Requirements

• At least 3+ years of hands-on experience in data integration and pipeline development
• Strong expertise in Amazon Web Services (AWS) Cloud for data integration with Apache Spark, Glue, Kinesis, and S3
• Proficiency in Python for data integration, with real-life experience in Python development
• Solid understanding of advanced SQL for analytical purposes and complex query optimization
• Experience with Databricks in an AWS environment for effective data integration
• Knowledge of source control systems such as GitLab and continuous integration tools like Jenkins
• Analytical experience with databases, including writing complex queries, query optimization, and debugging
• Strong communication and collaboration skills, essential for effective teamwork
• Fluent English language skills at an Upper-Intermediate level

Nice to have

• Familiarity with other data integration tools and technologies