Senior Data Engineer

Business Unit:

Company Details:

Job Details:

Key Responsibilities

The following are the key responsibilities of the position. It is expected that most, if not all of these, are met by the candidate:

·         Design, build, and optimize data pipelines using Azure Data Factory and Databricks (pyspark/spark SQL).

·         Develop robust ETL/ELT processes to ingest, transform, and validate large volumes of operational, telemetry, incident, and transactional data.

·         Implement scalable workflows leveraging Azure services such as Data Lake Storage, Azure Synapse, Azure SQL Databases, Key Vault, Logic Apps, and Functions.

·         Build efficient advanced SQL queries and stored procedures across Azure Synapse and Azure SQL Server.

·         Collaborate with data analysts, engineers, performance assurance, and operations teams to enable reliable, accurate datasets for reporting, KPIs and predictive analytics.

·         Design and maintain data models, schemas, tables, and metadata following COE architecture patterns.

·         Implement strong data quality, validation, and monitoring frameworks to ensure accuracy and reliability across global programs.

Required Skills and Qualifications

·         Bachelor’s/master’s degree in Comp Science, Data Analytics, Engineering, Mathematics, or related field.

·         5+ years of experience in data engineering roles.

·         Strong expertise with Azure Data Factory (ADF) pipelines, Azure Databricks, mapping data flows, and orchestrations.

·         Advanced experience with Azure Databricks (pyspark, spark SQL, Delta Lake, notebooks, clusters).

·         High proficiency in Python for ETL, automation, and data transformation.

·         Strong SQL skills across Azure Synapse and Azure SQL including query tuning and complex transformations.

·         Solid understanding of data modelling, warehousing, star/snowflake schema, Medallion architecture and distributed data processing.

·         Experience handling large-scale datasets and complex data domains.

·         Familiarity with CI/CD practices, Git branching, and DevOps pipelines.

·         Experience with data governance, data cataloging, and managing structured/unstructured data in cloud environments.

Preferred Skills

·         Experience with Azure Data Factory (ADF) pipelines, Azure Databricks, Databricks Workflows, or Apache Airflow.

·         Exposure to real-time/streaming data technologies (Event Hub, Kafka, Spark Streaming).

·         Exposure to Machine learning algorithms, feature engineering, model evaluation, hyperparameter tuning, LLM fundamentals and scikit-learn/XGBoost. Knowledge with deep learning, PyTorch/TensorFlow, MLflow, model deployment, and MLOps practices.

·         Experience integrating APIs, flat files, and operational systems such as ServiceNow.

·         Experience to machine learning pipelines, feature stores, or model operationalization.

·         Familiarity with Power BI dataset design and optimization.

·         Understanding of multi-region cloud environments and enterprise architecture patterns.

Soft Skills

·         Strong analytical mindset with exceptional problem-solving abilities.

·         Ability to work independently and take ownership of deliverables.

·         Excellent communication skills-capable of simplifying complex concepts for non-technical stakeholders.

·         Comfortable working in a fast-paced, global, multi-time zone team.

·         High attention to detail, data quality, and accuracy.

·         Strong sense of accountability, adaptability, and continuous improvement.

·         Ability to engage effectively with engineering, operations, and business stakeholders.

·         Proactive, resourceful, and committed to delivering high-quality outcomes.

Role Impact & Success Measures

The Senior Data Engineer will design, build, and maintain enterprise data warehouse solutions for analytics and reporting, develop and optimize ETL/ELT pipelines to load and transform data from multiple sources. Implement dimensional modelling (star/snowflake schemas) for performance and scalability, and ensure data quality, governance, and performance tuning across the warehouse whilst strengthening solutions using Lakehouse, Data Warehouse, and Real-Time Analytics.

Success in the First 3–6 Months

·         Build a solid understanding of CTS datasets, KPI frameworks, and key operational systems.

·         Gain strong understanding of CTS datasets, pipelines, and Azure architecture.

·         Deliver reliable ADF and Databricks pipelines for ingestion and transformation.

·         Improve data quality and performance of existing workflows.

·         Support analysts and operations teams with accurate, well-structured datasets.

Success in 6–12 Months

·         Take ownership of end-to-end analytical workstreams with minimal supervision.

·         Own end-to-end pipeline development with minimal supervision.

·         Implement scalable, reusable engineering patterns and metadata-driven frameworks.

·         Improve efficiency through automation, orchestration, and optimization.

·         Integrate new data sources to support KPIs, device analytics, and predictive modelling.

Long-Term Success (12+ Months)

·         Contribute to CTS Analytics strategy, standards, and global operating model improvements.

·         Contribute to global COE standards for data engineering and cloud architecture.

·         Become SME for key pipelines, data domains, or Azure components.

·         Lead continuous improvement initiatives around quality, governance, and automation.

Support junior engineers and uplift engineering capability within the COE.

‎

Worker Type:

‎

We are committed to creating an inclusive workplace and welcome applications from people of all backgrounds. We do not discriminate based on any protected characteristic under applicable law.