Lead Data Engineer (GCP)

Position Title:Lead Data Engineer (GCP)
Location:Nashville, TN (Remote)
Duration:Long Term Contract
 
Job description:
The Lead Data Engineer will be responsible for leading technical projects, mentoring a team of engineers, and driving the technical roadmap. This role requires a deep understanding of data engineering principles, proficiency in various programming languages and distributed systems, and hands-on experience with GCP technologies. The candidate will work closely with cross-functional teams to build and optimize data pipelines, manage data infrastructure, and support advanced analytics initiatives. Strong leadership and communication skills are essential to ensure successful project delivery and team development.
 
Required Skills:

• Technical leadership experience (minimum 5 years as a lead)
• Proficiency in SQL, Python, Java/Scala, and distributed systems (Spark, Beam)
• GCP Expertise: Hands-on with BigQuery, Dataflow, Dataproc, Pub/Sub, Composer (Airflow), and Terraform
• Experience with CloudSQL, Cloud Storage, GCP Suite, Cloud Run, and custom templates
• Experience with Kafka and Data Plex
• Leadership experience in leading projects, mentoring teams, and driving technical roadmaps
• Familiarity with DevOps practices, CI/CD, Git, and infrastructure-as-code

• GCP Certifications (e.g., Professional Data Engineer)
• Experience with real-time processing, Vertex AI, or multi-cloud environments
• Understanding of DBT Labs & clinical/CMS billing data

• Lead engineers, review code, and establish data engineering standards, CI/CD, and monitoring
• Build and optimize ETL/ELT pipelines using Dataflow (Apache Beam), Dataproc (Spark), Pub/Sub, and Cloud Functions
• Architect and manage BigQuery, Cloud Storage, and Data Fusion for efficient data ingestion, transformation, and storage
• Optimize pipeline performance, query efficiency, and GCP cost management
• Collaborate with data architects, scientists, and analysts to deliver self-service data platforms
• Support advanced analytics/ML use cases
• Implement data validation, lineage, and metadata management