AL
Data Engineer
Artech LLC
🇺🇸 United States
On-site
1 week ago
$45 – $50 / hour
- Databricks
- Scala
- ETL
- ELT
- SQL
- PySpark
- Delta Lake
- Unity Catalog
- Azure Databricks
- Devops
- Azure
- AKS
- Apache
- Java
- Docker
- Kubernetes
1 week ago
Title: Data Engineer
Locations: Quincy, MA - 100% ONSITE
Duration: 6 months
Pay Range: $45 - $50/Hour on W2/C2C (All inclusive)
JOB DESCRIPTION:
- Databricks, Scala- Data Engineer
- Must have atleast 8+ years of extensive Data engineering experience in Databricks, ETL/ELT using data pipelines, SQL/Procesurr experience
- Must have 8 years of hands on development proficiency in implementing Databricks solutions using scala and spark with data frames and notebooks/sql
- Must have 8+ years of hands on development and performance tuning/ enhancement experience in scala/pyspark and sql
- Must have strong experience with Databricks, including developing and optimizing spark jobs, data transformations, and data processing workflows
Must have experience and a good understanding on the below topics
- Databricks delta lake storage
- Unity catalog
- Autoloader
- Lakehouse architecture
- Medallion architecture
- Databricks serverless options
- Dataset and data frame
- Databricks clusters configuration and sizing
- Databricks job orchestration
- Spark structured streaming
- Must have experience with complex and large volume data streaming and data transformations using Azure Databricks
- Must have extensive experience in datawarehouse / data lakehouse implementation
- Must have experience with monitoring, debugging, and resolving issues in kive Databricks jobs
- Must have strong hands on expertise in troubleshooting devops pipelines, azure , and AKS services
- The candidate shd be able to provide data ingestion, streaming and transformations solutions and do development woek
- Experience with apache big data implementation with programming proficiency like pyspark, R or Java prior to Databricks is advantageous
- Good to have experience with containerization technologies such as docker and kubernetes
Company Benefits & Culture
• Opportunity to work with a dynamic team in a fast-paced environment
• Exposure to cutting-edge technologies and methodologies
• Supportive and collaborative work culture
For immediate consideration please click APPLY to begin the screening process with Alex.
Data Engineer · Artech LLC