PL
Capgemini - Databricks Engineer
PeopleNTech LLC
🇺🇸 United States
On-site
2 months ago
- Databricks
- PySpark
- ETL
- Delta Lake
- Shopify Liquid
- Unity Catalog
- Devops
- CI/CD
2 months ago
Warren, NJ
6-12 months approx.
Bill rate : $65/hr all Incl.
Here is a concise, 5-line job description tailored for this role:
* Design Frameworks: Architect and build scalable, metadata-driven ingestion and transformation frameworks using Databricks and PySpark.
* Automate Pipelines: Eliminate hardcoded ETL by developing reusable, configuration-based notebooks that dynamically execute data workflows.
* Optimize Delta Lake: Manage and tune Delta Lake tables utilizing advanced Medallion architecture, Z-Ordering, and Liquid Clustering.
* Enforce Governance: Implement robust data governance, access controls, and end-to-end data lineage via Databricks Unity Catalog.
* Streamline DevOps: Deploy and orchestrate pipelines using Databricks Asset Bundles (DABs), CI/CD workflows, and automated cloud infrastructure.
Capgemini - Databricks Engineer · PeopleNTech LLC