Specialist - Data Engineering

Job Summary

Seeking a Specialist with 5 to 7 years of experience in GCP BigQuery and Databricks to design develop and optimize scalable data solutions within the GCP Databricks ecosystem

Job Description

Design build and maintain scalable data pipelines and ETLELT processes using GCP BigQuery and Databricks Develop and optimize Databricks notebooks and jobs leveraging Apache Spark and PySpark for efficient data processing Implement data ingestion transformation and storage solutions using GCP native tools Ensure data quality validation and reconciliation across pipelines Collaborate with crossfunctional teams to align data solutions with business requirements Monitor and troubleshoot pipeline performance and optimize for scalability and reliability Maintain compliance with data governance security and best practices within cloud environments Stay current with emerging technologies and best practices in cloud data engineering and Databricks implementations

Roles and Responsibilities

Lead the design and implementation of data workflows and pipelines on GCP and Databricks platforms Develop and optimize Spark jobs notebooks and workflows for processing large enterprise datasets Apply data engineering best practices to enhance performance scalability and reliability of data solutions Collaborate with data scientists analysts and architects to support analytics AI and machine learning workflows Participate in code and design reviews to ensure highquality deliverables Support production data pipelines including monitoring incident resolution and continuous improvement Mentor and guide junior team members on technology evaluations design and implementation Integrate Databricks pipelines with upstream and downstream systems and cloud services