Engineering Manager

Engineering Manager – Data Engineering & ETL

Position Overview

We are seeking an experiencedEngineering Manager – Data Engineering & ETL to lead a team of software and data engineers responsible for designing, developing, and supporting high-volume data processing and ETL pipelines. This role will combinetechnical leadership, people management, architecture, and hands-on collaboration to build scalable and reliable data solutions.

The ideal candidate has strong experience withApache Spark, ETL/data processing pipelines, distributed systems, and cloud technologies. Experience withDatabricks is highly desirable and considered a significant plus.

Key Responsibilities

• Lead, mentor, and develop a team of software/data engineers, providing technical direction and career development.
• Design and oversee scalableETL and data processing pipelines supporting large volumes of structured and unstructured data.
• Provide technical leadership for solutions utilizingApache Spark, PySpark, SQL, and distributed data processing frameworks.
• Drive architecture and engineering best practices across data ingestion, transformation, processing, and delivery.
• Partner with Product, Data, Analytics, Architecture, and other engineering teams to define technical requirements and deliver high-quality solutions.
• Establish standards fordata quality, reliability, scalability, performance, monitoring, and operational support.
• Lead the design and implementation of cloud-based data solutions usingAWS, GCP, or Azure.
• LeverageDatabricks for data engineering, ETL, analytics, and large-scale processing where appropriate.
• Conduct architecture and code reviews and ensure adherence to engineering standards.
• Identify opportunities to improve existing data pipelines, reduce processing times, optimize costs, and increase system reliability.
• Establish and monitor KPIs around engineering productivity, pipeline performance, reliability, and data quality.
• Collaborate with DevOps and Cloud Engineering teams on CI/CD, infrastructure, security, and deployment automation.
• Participate in technical planning, roadmap development, project estimation, and resource planning.
• Help establish a culture of ownership, accountability, innovation, and continuous improvement.

Required Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field.
• 8+ years of software/data engineering experience, with experience managing or leading engineering teams.
• Strong experience designing and developingETL/data processing pipelines.
• Strong hands-on experience withApache Spark and/or PySpark.
• Strong SQL and experience working with large-scale data environments.
• Experience with distributed systems and high-volume data processing.
• Experience with at least one major cloud platform:AWS, GCP, or Microsoft Azure.
• Strong understanding of data engineering architecture, data pipelines, data integration, and data processing technologies.
• Experience with software development methodologies, CI/CD, version control, testing, and production deployments.
• Demonstrated ability to lead technical teams while remaining engaged in architecture and technical decision-making.
• Strong communication, problem-solving, and stakeholder management skills.

Preferred Qualifications

• Databricks experience is a major plus.
• Experience withDelta Lake, Databricks Workflows, Unity Catalog, and Databricks SQL.
• Experience with cloud-native data services such asAWS Glue, EMR, Redshift, S3, Azure Data Factory, Synapse, or Google BigQuery/Dataflow.
• Experience withKafka or other real-time streaming technologies.
• Experience with Airflow or other workflow orchestration platforms.
• Experience with data warehousing and modern data lake/lakehouse architectures.
• Experience with Python, Scala, or Java.
• Experience implementing data governance, security, lineage, and data quality frameworks.
• Experience working in Agile/Scrum environments.

Technical Environment

Data & ETL: Apache Spark, PySpark, SQL, ETL/ELT, Data Pipelines
Data Platforms: Databricks, Delta Lake, Data Lakes, Data Warehouses
Cloud: AWS, GCP, and/or Azure
Programming: Python, Scala, Java, SQL
Streaming: Kafka and related technologies
DevOps: CI/CD, Git, Docker, Kubernetes, Infrastructure as Code
Orchestration: Airflow, Databricks Workflows, or similar technologies

Leadership Profile

The successful candidate will be ahands-on engineering leader who can move comfortably between people management, architecture, technical strategy, and delivery. They should be able to translate business requirements into scalable technical solutions while building and mentoring a high-performing engineering organization.