PL
Databricks Architect
PeopleNTech LLC
🇺🇸 United States
On-site
3 months ago
- Databricks
- ETL
- ELT
- Oracle
- Snowflake
- IAM
- RBAC
- ABAC
- SCIM
- Unity Catalog
- Purview
- Collibra
- Alation
- CI/CD
- Data Modeling
- Vault
- SQL
- PySpark
- Delta Lake
- AML
- KYC
- GDPR
- Airflow
3 months ago
Role :Databricks Architect
Location :Iselin, NJ (Hybrid)
Bill Rate : $80/hr - $90/hr
Submittal Rate: $70 an hr--$80/Hr
Responsibilities:
- Define and maintain reference architectures (Lakehouse, CDC, streaming) and domain data models (conceptual, logical, physical).
- Create and enforce data standards: naming conventions, data types, modeling practices, semantic definitions (aligned to business glossaries).
- Establish metadata operating model: ownership, stewardship, processes for Catalog, Glossary, Data Dictionary, and Data Lineage.
- Integrate lineage capture across pipelines (ETL/ELT/streaming),BI layers, and ML workflows.
- Architect cross-platform data flows across Databricks, Oracle/SQL Server, Snowflake and metadata tools.
- Define IAM models: RBAC/ABAC, SSO/federation, SCIM provisioning; directory-driven entitlements and periodic access reviews.
- Define catalog strategy (e.g., Unity Catalog/Purview/Collibra/Alation) and integrate with CI/CD for automated registration and lineage.
- Design reusable pipeline frameworks with configuration-driven IO, logging, metrics, retry/error handling, and data quality checks.
Skills:
- Data Modeling: Dimensional (star/snowflake), 3NF, Data Vault, business glossary-to-model mapping, SCD types, time-series/event modeling.
- Metadata & Governance: Practical use and integration of Data Catalogs, Lineage,
- Oracle/SQL Server (data modeling, migration/CDC patterns).
- Snowflake (roles, warehouses, performance tuning, tasks/streams, dynamic tables).
- Databricks/Spark (SQL/PySpark, Structured Streaming, Delta Lake; Unity Catalog).
- Security & Compliance: IAM/RBAC, masking, tokenization, encryption; PCI/AML/KYC/GDPR/DPDP awareness.
- Integration & Orchestration: Databricks Workflows, Airflow/ADF, API integrations with catalog tools; schema registry
- Exceptional interpersonal and collaboration skills within a team environment
Databricks Architect · PeopleNTech LLC