TC
Lead Data Engineer
TechDigital Corporation
🇵🇱 Poland | 🇺🇸 United States
On-site
Staff / Principal
4 months ago
- ETL
- ELT
- Python
- Scala
- Java
- Snowflake
- Azure
- AWS
- GCP
- BigQuery
- Redshift
- Data Architecture
- Devops
- CI/CD
4 months ago
Mandatory Skills:
• Advanced expertise in ETL / ELT pipeline design
• Handling: o Batch data processing
o Near real time / streaming data • Experience with structured and semi structured data
• Strong knowledge of: o Incremental loading
o CDC (Change Data Capture) • Pipeline orchestration and dependency management
• Strong programming skills in Python or Scala or Java (nice to have)
• Performance optimization for large scale data processing
• Solid understanding of: o Dimensional modeling (Star / Snowflake)
o Normalized and denormalized models • Strong experience on: Azure, AWS, or GCP
• Hands on with Data Warehouses (Snowflake, Synapse, BigQuery, Redshift)
Data Architecture & Solution Design
• Design end to end data engineering architectures
• Define scalable solutions for: o Data lakes / lakehouse
o Data warehouses
o Streaming and real time systems • Ensure alignment with enterprise architecture, security, and compliance standards
• Review and approve technical designs
Data Pipeline Development & Management
• Lead development of ETL / ELT pipelines
• Handle: o Batch and real time ingestion
o Structured and semi structured data • Optimize pipelines for performance, reliability, and cost
• Manage schema evolution and data dependencies
Data Quality, Reliability & Operations
• Establish data quality standards and validation rules
• Implement monitoring, alerting, and observability
• Perform root cause analysis for data incidents
• Drive operational excellence and stability
DevOps / DataOps Enablement
• Build CI/CD pipelines for data workloads
• Automate testing, deployment, and rollback
• Improve reliability through automation
Lead Data Engineer · TechDigital Corporation