TC
Data Engineer
TechDigital Corporation
- 🇺🇸 United States
- On-site
- 9 months ago
- Python
- PySpark
- ETL
- ELT
- Azure Data Factory
- Azure Databricks
- SQL
- Data Modeling
- Azure
- Key Vault
9 months ago
Desired Skills Python, PySpark
Key Responsibilities:
- Design and implementETL/ELT pipelines usingAzure Data Factory.
- Develop and optimize data processing workflows inAzure Databricks usingPySpark.
- Integrate data from multiple sources into a unified data platform.
- Ensure data quality, security and compliance across all processes.
- Collaborate with data architects, analysts and business teams to deliver reliable data solutions.
- Monitor and troubleshoot data pipelines for performance and reliability.
- Implement best practices fordata governance,metadata management anddata lineage.
- 6–8 years of experience in data engineering and data platform development.
- Strong hands-on experience withAzure Data Factory,Azure Databricks andPySpark.
- Proficiency inSQL and working with relational databases.
- Experience withdata modeling,data warehousing andbig data processing.
- Familiarity withAzure ecosystem (Data Lake, Synapse, Key Vault, etc.).
- Strong problem-solving and debugging skills.
Data Engineer · TechDigital Corporation