ES
Data Governance Technical Consultant (Collibra)
Epsilon Solutions LTD
🇨🇦 Canada
On-site
4 months ago
- Collibra
- Azure Databricks
- Databricks
- Unity Catalog
- ETL
- Informatica
- JSON
- Azure
- XML
- Change Management
- RBAC
- REST API
- Power BI
- Devops
- CI/CD
- Agile
4 months ago
Location: Vancouver
Duration: Contract
- Azure Databricks,Collibra,Databricks Unity Catalog,ETL,IT security,Informatica,JSON,Microsoft Azure Databricks,SGML format,XML,data cleansing,data formats,data governance,data integration,data management,data pre-processing,data privacy,data processing,data provisioning,data science,data science solutions,data security,file formats,information security,information technology,markup language,mathematics,programming,software development,statistics
- Someone local to Vancouver is preferred. Calgary is second best
- Teck - Internal/Interno
- Data Governance Technical Consultant
- Overview
- This Statement of Work (SOW) outlines the engagement of a Data Governance (DG) Technical Consultant to support the implementation of Collibra, with an initial focus on a pilot to establish scalable data governance practices across a modern data ecosystem built on Microsoft Azure and Databricks. The Technical Consultant will be taking direction from the DG Solution Owner, DG Solution Architect, and Manager of Enterprise Data.
- Objectives
- Implement Collibra aligned with enterprise data governance strategy
- Deliver a pilot focused on critical datasets
- Define and implement metadata and data quality requirements
- Enable integration with Databricks (Unity Catalog) and ecosystem tools
- Provide foundational automation to support change management (dev to prod)
- Scope of Work
- Includes:
- Current design assessment
- Collibra Configuration
- Metadata Model
- Integration with Databricks
- Governance Workflows
- Implementation and change management with documentation
- Validation & Knowledge Transfer
- Technical Requirements
- The consultant must demonstrate hands-on experience with:
- Collibra Data Intelligence Cloud (or equivalent on-prem setup)
- Configuration of:
- Data Catalog
- Business Glossary
- Data Lineage
- Data Helpdesk / workflows
- Customization of:
- Asset models and relationships
- Domains, communities, and responsibilities
- Workflow design using Collibra workflow engine
- Role-based access control (RBAC) configuration
- Design and implement custom metadata models in Collibra
- Experience defining:
- Critical Data Elements (CDEs)
- Business and technical metadata schemas
- Data classification frameworks (e.g., sensitivity, PII)
- Metadata standards including:
- Naming conventions
- Required attributes and completeness rules
- Experience aligning business glossary ? technical assets
- Hands-on exposure to integrating Collibra with:
- Databricks Unity Catalog
- Catalogs, schemas, tables, columns
- Permissions and ownership mapping
- Metadata ingestion from:
- Delta tables
- Notebooks / pipelines (where applicable)
- Understanding of the following is a plus:
- Lakehouse architecture
- Medallion architecture (Bronze / Silver / Gold layers)
- Experience using:
- Collibra REST APIs
- Collibra Import API / Edge / lineage ingestion
- Ability to build or configure:
- Automated metadata ingestion pipelines
- Incremental metadata sync processes
- Integration with:
- Power BI Integration for (reports and models)
- Familiarity with:
- JSON / XML metadata structures
- API authentication and security
- Experience implementing end-to-end data lineage in Collibra Data Lineage, including:
- Source ? transformation ? consumption layers
- (Nice to have) integration with:
- Databricks lineage (Unity Catalog lineage where available)
- External lineage tools (if applicable)
- Ability to:
- Map lineage across pipelines
- Validate lineage accuracy with engineering teams
- Define and implement data quality rules for critical datasets
- Integrate Collibra with data quality tools (if applicable)
- Experience with:
- Data quality dimensions (accuracy, completeness, timeliness, etc.)
- Monitoring and surfacing DQ metrics in Collibra
- Ability to link:
- Data quality rules to data assets to business definitions
- Design and implement workflows for:
- Data stewardship assignment
- Glossary term lifecycle (create/approve/update)
- Issue management and resolution
- Experience with:
- BPMN-based workflow configuration in Collibra
- Notifications and task assignments
- Implement governance-aligned access control:
- Role-based access (RBAC)
- Domain-level permissions
- Align Collibra roles with:
- Databricks roles and permissions
- Understanding of:
- Data privacy and compliance considerations
- Experience working across environments:
- Dev / Test / Prod
- Version control and deployment practices for:
- Collibra configurations
- Workflows and metadata models
- Familiarity with CI/CD concepts (nice to have)
- Collaborate on Collibra implementation for:
- Scalability across domains and datasets
- Efficient metadata ingestion
- Design for:
- API usage
- Job scheduling and ingestion frequency
- Deliverables
- Fully configured Collibra pilot environment
- Metadata model and governance standards
- Databricks metadata ingestion pipelines
- Configured workflows and RBAC
- Data lineage and (if applicable) data quality integration
- Technical documentation (architecture, integrations, workflows)
- Knowledge transfer sessions
- Scalable roadmap for enterprise rollout
- Be a strong team contributor following an agile methodology
- Timeline (6-9 months)
- Required Skills & Experience (Enhanced)
- 3+ Collibra implementations (end-to-end preferred)
- Proven experience with:
- Metadata modeling
- API-based integrations
- Workflow automation
- Experience in data governance operating model implementation
- Strong communication across business & technical stakeholders
Data Governance Technical Consultant (Collibra) · Epsilon Solutions LTD