Data Engineer

About this opportunity :

Join ourTeam as aData Lead. You'll work within multi-functional product engineering teams to deliver top-quality data capabilities, demonstrating your engineering expertise and cloud opportunities

What you'll be doing:

• Implement scalable, resilient, low-latency data systems.
• Ensure systems meet performance and reliability standards.
• Collaborate with engineering leadership and stakeholders.
• Set technical strategy and best practices for data engineering.
• Deliver solutions for multiple entities within the group.
• Ensure alignment with business needs and cloud opportunities.
• Promote secure, high-quality data pipelines.
• Implement standards for data integrity and compliance.
• Identify and eliminate recurring issues through automation.
• Improve operational efficiency and reduce manual intervention.
• Provide guidance and technical leadership.

What you'll need:

• Extensive industry experience in designing, building and supporting distributed systems and large-scale data processing systems in production with a proven track record.
• Expertise inGCP Data Productsfor storage, processing, analytics, and machine learning, enabling enterprises to manage, analyse, and derive insights from data at scale e.g. BigQuery, Dataflow, Cloud Storage, Pub/Sub, and Cloud Composer.
• Good hands on experience to Build Batch Data and Streaming Data Pipelines on Google Cloud
• Proven experience and knowledge of automation andCI/CD – Jenkins, GitHub & Spinnaker.
• Best practicecoding/scripting experience developed in a commercial/industry setting (Python, SQL, Java, Scala or Go).
• Strong working experience with operational data stores, data warehouse, big data technologies and data lakes
• Experience in using distributed frameworks (Spark, Flink, Beam,Hadoop)
• Good knowledge of containers (Docker, Kubernetes etc) and experience with cloud platforms such as GCP, Azure or AWS.
• Strong experience working with (Real Time streaming), such asKafka technologies.
• Clear understanding of data structures, algorithms, software design, design patterns and core programming concepts.
• Good understating of cloud storage, networking, and resource provisioning

And any experience of these would be really useful:

• Certification in GCP "Cloud Architect”, "Cloud Developer”, "Professional Data Engineer”

Certification in Apache Kafka (CCDAK)