DS
Software Developer - ETL - Senior
Damco Solutions Inc
- 🇨🇦 Canada
- On-site
- Senior
- 8 months ago
- ETL
- ELT
- Unit Testing
- Delta Lake
- Databricks
- Snowflake
- Azure Data Factory
- PySpark
- SQL
- dbt
- Azure
- PostgreSQL
- Apache Spark
- Azure Databricks
- Parquet
- JSON
- Azure Synapse
- Azure SQL
- Azure Functions
- RBAC
- Python
- PowerShell
- CI/CD
- Azure DevOps
- GitHub Actions
- Unity Catalog
- Power BI
- DAX
- Data Modeling
- MLflow
- AI
- System Design
8 months ago
Location: Toronto, ON
Contract
Maximum Number of Submissions - 2 (two)
MUST HAVES:
Please refer to the Technical Skills in the Experience and Skill Set Requirements
Description
· Design technical solutions for data acquisition and storage into our centralized data repository.
· Develop ELT scripts, design data-driven logic and conduct unit testing.
· Conduct database modeling and design as to improve overall performance.
· Produce design artifacts and documentation which will allow future support of the implemented solutions.
· Investigate and resolve incidents and identify whether the problem is caused by the data loading code or is due to bad data received from the data provider.
· Execute service requests related to routine and ad-hoc data loads.
· Provide the data quality check and report on the data quality issue.
NOTE
Extension/Amendment Attestation: Extension(s) only allowed using unused days/funds left on contract. No additional funds will be added beyond the maximum contract value and any extension options included in the original SOW.
Assignment Type : This position is listed as Onsite; the resource is expected to work 7.25 hours per calendar day between the standard working hours of 8:00-5:00 PM (excluding lunch breaks) every Monday to Friday inclusive at the identified OPS office location.
Skills
Experience and Skill Set Requirements
| Technical Skills | 10+ years experience in: · Designing and developing scalable Medallion Data Lakehouse architectures. · Expertise in data ingestion, transformation, and curation using Delta Lake and Databricks. · Experience integrating structured and unstructured data sources into star/snowflake schemas. · Building, automating, and optimizing complex ETL/ELT pipelines using Azure Data Factory (ADF), Databricks (PySpark, SQL, Delta Live Tables), and dbt. · Implementing orchestrated workflows and job scheduling in Azure environments. · Strong knowledge of relational (SQL Server, Synapse, PostgreSQL) and dimensional modeling. · Advanced SQL query optimization, indexing, partitioning, and data replication strategies. · Experience with Apache Spark, Delta Lake, and distributed computing frameworks in Azure Databricks. · Working with Parquet, ORC, and JSON formats for optimized storage and retrieval. · Deep expertise in Azure Data Lake Storage (ADLS), Azure Synapse Analytics, Azure SQL, Event Hubs, and Azure Functions. · Strong understanding of cloud security, RBAC, and data governance. · Proficiency in Python (PySpark), SQL, and PowerShell for data engineering workflows. · Experience with CI/CD automation (Azure DevOps, GitHub Actions) for data pipelines. · Implementing data lineage, cataloging, metadata management, and data quality frameworks. · Experience with Unity Catalog for managing permissions in Databricks environments. · Expertise in Power BI (DAX, data modeling, performance tuning). · Experience in integrating Power BI with Azure Synapse and Databricks SQL Warehouses. · Familiarity with MLflow, AutoML, and embedding AI-driven insights into data pipelines. |
50 points |
| Core Skill and Experience |
|
30 points |
| General Skills | · Demonstrated strong leadership and people management skills. · Proven technical leadership skills with ability to identify areas for improvement, and recommend solutions. · Exceptional analytical, problem solving and decision-making skills. · Demonstrated strong interpersonal, verbal and written communication, and presentation skills. · Proven troubleshooting and critical thinking experience · Demonstrated ability to apply strong listening skills to facilitate issue resolution. · Effective consulting skills to engage with all stakeholders with proven track record for building strong working relationships. · Strong interpersonal, facilitation and negotiation skills with ability to build rapport with stakeholders and drive negotiations to a successful outcome. · Excellent customer service skills, including tact and diplomacy to ensure client needs are managed effectively. · A motivated, flexible, detail-oriented and creative team player with perseverance, excellent organization and multi-tasking abilities, and a proven track record for meeting strict deadlines. |
15 points |
| Public Sector/Healthcare Experience | · Knowledge of Public Sector Enterprise Architecture artifacts (or similar), processes and practices, and ability to produce technical documentation that comply with industry standard practices. · Knowledge of Project Management Institute (PMI) and Public Sector I&IT project management methodologies. · Knowledge and understanding of Ministry policy and IT project approval processes and requirements. · Experience with large complex IT Health-related projects. |
5 points |
Software Developer - ETL - Senior · Damco Solutions Inc