TC
DataBrick Engineer
TechDigital Corporation
๐บ๐ธ United States
On-site
9 months ago
- Databricks
- PySpark
- SQL
- Delta Lake
- GCP
- ETL
- ELT
9 months ago
8+ years of experience in data engineering, with at least 3 years on Databricks.
- Strong proficiency in PySpark, SQL, and Delta Lake.
- Hands-on experience with GCP Dataproc.
Responsibilities:
1.Administration
- Lead the installation and configuration of Databricks on GCP cloud platforms.
- Monitor platform health, performance, and cost optimization.
- Implement governance, logging, and auditing mechanisms.
2.Development / Enhancements
- Design and develop scalable ETL/ELT pipelines using PySpark, SQL, and Delta Lake.
- Collaborate with data engineers and analysts to enhance data workflows and models.
- Optimize existing notebooks and jobs for performance and reliability.
3.Operations, Support & Troubleshooting
- Provide L2/L3 support for Databricks-related issues and incidents.
- Troubleshoot cluster failures, job errors, and performance bottlenecks.
- Maintain technical documentation for platform setup, operations, and development standards
DataBrick Engineer ยท TechDigital Corporation