AS
Senior Databricks Data Engineer - Remote
Axelon Services Corporation
- 🇺🇸 United States
- Remote
- Senior
- 18 hours ago
- $54.28 – $58.46 / hour
- Databricks
- PySpark
- SQL
- AWS Cloud
- Apache Spark
- ETL
- ELT
- AWS S3
- Snowflake
- Airflow
- AWS Glue
- AML
- Apache Airflow
- Data Modeling
- JSON
- Parquet
- Apache Avro
- CI/CD
- Devops
- Delta Lake
- Unity Catalog
- Agile
- Scrum
18 hours ago
Job Description:
We are seeking an experienced Senior Databricks Data Engineer with strong expertise in designing, developing, and optimizing large-scale data engineering solutions on the Databricks platform. The ideal candidate will have extensive hands-on experience in building enterprise-grade data pipelines, processing high-volume datasets, and delivering scalable analytics solutions using PySpark, Spark SQL, and AWS cloud technologies.
Key Responsibilities:
- Design, develop, and maintain scalable data engineering solutions using Databricks, PySpark, Spark SQL, and Apache Spark. Build and optimize end-to-end ETL/ELT pipelines integrating Databricks with AWS S3, Snowflake, Airflow, and AWS Glue.
- Process and transform large-scale structured and semi-structured datasets to support enterprise reporting, analytics, and regulatory requirements.
- Develop reusable frameworks and components for: Data ingestion Data cleansing and transformation Data validation and reconciliation Data enrichment Business rule implementation Implement robust data quality checks, monitoring, and governance controls to ensure data accuracy and reliability.
- Perform performance tuning and optimization of Spark applications and Databricks workloads to improve scalability and processing efficiency.
- Collaborate with business stakeholders, architects, and analytics teams to understand data requirements and deliver scalable solutions.
- Troubleshoot and resolve production data pipeline issues while ensuring high availability and reliability.
- Support advanced analytics, customer segmentation, campaign analytics, AML/regulatory reporting, and data validation initiatives.
- Technical Skills 7+ years of hands-on experience with Databricks and modern data engineering platforms.
- Strong expertise in: PySpark Spark SQL Apache Spark Databricks Lakehouse Architecture.
- Experience building and managing ETL/ELT pipelines on cloud platforms.
- Strong experience with: AWS S3 Snowflake Apache Airflow AWS Glue Knowledge of data modeling, data warehousing, and big data processing techniques.
- Experience working with structured and semi-structured data formats such as JSON, Parquet, Avro, and CSV.
- Hands-on experience implementing data quality frameworks and monitoring solutions. Strong SQL and data analysis skills. Preferred Skills Experience with regulatory reporting, AML, compliance, or financial data domains. Exposure to CI/CD and DevOps practices for data engineering.
- Experience with Delta Lake, Databricks Workflows, and Unity Catalog.
- Knowledge of Agile/Scrum delivery methodologies.
Customer Segmentation Analytics Campaign Analytics AML and Regulatory Reporting Enterprise Data Platforms Production Data Pipeline Support Data Validation and Reconciliation Large-Scale Data Migration and Modernization Programs
Senior Databricks Data Engineer - Remote · Axelon Services Corporation