Likeremote

Subscribe to the latest remote jobs:

  • Likeremote jobs on https://LinkedIn.com/
  • Likeremote jobs on https://telegram.org/
  • Likeremote jobs on Reddit.com
C

Lead Databricks Data Engineer

CirrusLabs
Location not stated
Staff / Principal
7 months ago
  • Databricks
  • ETL
  • ELT
  • Python
  • PySpark
  • SQL
  • Delta Lake
  • Unity Catalog
  • AWS
  • Azure
  • GCP
  • CI/CD
  • Scala
  • Data Modeling
  • Apache Airflow
  • Jenkins
7 months ago
Auto apply with Likeremote
Not scoredNo CV on file. Upload one and this job gets a score out of 100.Upload CV
We areCirrusLabs. Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and make things happen. We are committed to excellence. We are a dependable partner organization that delivers on commitments. We strive to maintain integrity with our employees and customers. Every action we take is driven by value. The core of who we are is through our well-knit teams and employees. You are the core of a values driven organization.

You have an entrepreneurial spirit. You enjoy working as a part of well-knit teams. You value the team over the individual. You welcome diversity at work and within the greater community. You aren't afraid to take risks. You appreciate a growth path with your leadership team that journeys how you can grow inside and outside of the organization. You thrive upon continuing education programs that your company sponsors to strengthen your skills and for you to become a thought leader ahead of the industry curve.

You are excited about creating change because your skills can help the greater good of every customer, industry and community. We are hiring a talented<Job Title>to join our team. If you're excited to be part of
a winning team, CirrusLabs (http://www.cirruslabs.io) is a great place to grow your career.

Role:Lead Databricks Data Engineer

Location: Atlanta, GA (Onsite)

Certification preferred: Databricks Certified Professional


Job Description

Designs, builds, and manages scalable data platforms, leading teams to create robust, cloud-based data pipelines (ETL/ELT) using tools like Python, PySpark, SQL, Databricks (Lakehouse, Delta Lake, Unity Catalog), and cloud services (AWS/Azure/GCP). Key responsibilities include technical leadership, mentoring junior engineers, ensuring data quality/governance, optimizing performance (Spark optimizations, Medallion Architecture), stakeholder communication, and driving DataOps practices for data-driven insights.

Key Responsibilities

  • Technical Leadership: Own architecture, set standards, guide technical direction, and lead complex projects.
  • Pipeline Development: Design, build, and optimize scalable data pipelines (ingestion, transformation, storage) on Databricks.
  • Mentorship: Coach and develop data engineers, conduct code reviews, and promote best practices.
  • Architecture: Implement Data Lakehouse (Bronze/Silver/Gold) and Medallion Architecture patterns.
  • Performance Tuning: Identify and resolve bottlenecks, optimize Spark configurations, and manage cluster optimization.
  • Collaboration: Work with architects, product managers, and business stakeholders to translate requirements into solutions.
  • Data Governance: Ensure data quality, integrity, security, and compliance.
  • DataOps/CI/CD: Implement modern engineering practices for automated, reliable data flows.

Required Skills & Experience

  • Databricks Expertise: Deep knowledge of the platform (Lakehouse, Delta Lake, Unity Catalog, SQL).
  • Programming: Python, PySpark, Scala, SQL.
  • Cloud Platforms: AWS, Azure, or GCP.
  • Big Data Technologies: Spark, Data Warehousing, ETL/ELT.
  • Leadership: Proven experience leading technical initiatives and teams.
  • Data Modeling: Strong understanding of data modeling concepts.

Desirable Skills

  • Apache Airflow, Kafka, CI/CD tools (Jenkins), orchestration.
  • Experience in specific domains (e.g., Healthcare).
  • DataOps, MDM, data governance.

Lead Databricks Data Engineer · CirrusLabs

Auto apply with Likeremote