TI
Billing Data Ingestion Engineer
Tranzeal Inc.
- 🇨🇦 Canada
- On-site
- 8 months ago
- ETL
- ELT
- AWS
- GCP
- BigQuery
- Azure
- CI/CD
- SQL
- Data Modeling
- dbt
- Snowflake
- Redshift
- Databricks
- Python
- Great Expectations
8 months ago
This engineer builds the foundational data models that ingest and normalize raw billing data
from all cloud providers into the data platform.
· 4-6 years of data engineering experience
· This role requires deep expertise in ETL/ELT patterns, handling schema evolution, and
building robust pipelines at scale. The complexity of cloud billing data (billions of rows, late-
arriving data, provider schema changes) demands someone who's solved these problems
before.
Key Deliverables
1. Source System Connectors: Build ingestion pipelines for AWS CUR (S3), GCP Billing
Export (BigQuery), and Azure Cost Management Exports (Blob Storage)
2. Staging Data Models: Create staging tables that preserve raw billing data with
appropriate partitioning and clustering
3. Normalized Base Models: Design and implement a unified schema that normalizes billing
data across all three providers
4. Data Quality Framework: Implement data quality tests, freshness checks, and
reconciliation models
5. Schema Evolution Handling: Build processes to detect and adapt to provider schema
changes
Engineering Responsibilities
• Design incremental ingestion logic handling daily/monthly billing updates
• Implement late-arriving data handling (billing adjustments, credits)
• Build reconciliation models comparing processed totals to provider invoices
• Create data contracts and documentation for downstream consumers
• Optimize model performance for large-scale billing datasets (partitioning, clustering)
• Implement CI/CD for data model deployments and testing
Required Technical Skills
• Expert SQL and data modeling (dbt, Dataform, or similar)
• Experience with data warehouse platforms (Snowflake, BigQuery, Redshift, Databricks)
• Python for data pipeline scripting and automation
• Understanding of cloud billing export formats (CUR, BigQuery export, Azure exports)
• Data quality testing frameworks (dbt tests, Great Expectations)
Billing Data Ingestion Engineer · Tranzeal Inc.