Data Engineer with Databricks

Request ID: 107834-1
Title: Data Engineer with Databricks
Locations: Tampa, FL
Duration: 6 Months
Pay Range: $44 - $47/Hour on W2/C2C (All inclusive)

We are seeking an experiencedData Engineer with 4–6 years of experience and strong hands-on expertise inDatabricks and Apache Spark. The ideal candidate will design, develop, and optimize scalable data pipelines to support analytics and data-driven business initiatives.

The candidate will work closely with data scientists, analysts, business stakeholders, and cross-functional teams to ensure data is reliable, secure, high-quality, and readily available.

Key Responsibilities

• Design, develop, and maintain scalableETL/ELT data pipelines using Databricks and Apache Spark.
• Build and optimizebatch and real-time/streaming data pipelines for large volumes of structured and unstructured data.
• Develop data engineering solutions usingDatabricks, Spark, and related data technologies.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
• Monitor, troubleshoot, and optimize data processing workflows to ensure reliability and performance.
• OptimizeDatabricks clusters for performance, scalability, and cost efficiency.
• Implement data quality, governance, security, and compliance standards across data pipelines.
• Perform data validation and troubleshooting to identify and resolve pipeline issues.
• Maintain technical documentation related todata architecture, data pipelines, workflows, and processes.
• Support continuous improvement of data engineering processes and platform capabilities.
• Explore and integrateAI/Agentic AI capabilities into data engineering workflows where applicable.

Required Skills

• 4–6 years of experience in Data Engineering.
• Strong hands-on experience withDatabricks.
• Strong experience withApache Spark.
• Experience developingETL/ELT pipelines.
• Experience with bothbatch and streaming data processing.
• Experience working with large volumes of structured and unstructured data.
• Strong understanding of data engineering and data processing concepts.
• Experience with data pipeline monitoring, troubleshooting, and performance optimization.
• Knowledge ofdata governance, security, and compliance.
• Experience optimizing Databricks clusters for cost and performance.
• Strong problem-solving and analytical skills.
• Good communication and collaboration skills.

Preferred Skills

• Experience withAI Agents / Agentic AI.
• Experience integrating AI capabilities into data platforms or data workflows.
• Knowledge of cloud data platforms such asAWS, Azure, or GCP.
• Experience with streaming technologies such as Kafka.
• Knowledge of data lake/lakehouse architecture.