PL
Azure Data Engineer
PeopleNTech LLC
🇺🇸 United States
On-site
7 months ago
- Azure
- SQL
- Python
- Agile
- Devops
- Azure Data Factory
- Databricks
- ETL
- ELT
- PySpark
- CI/CD
- Azure DevOps
- Maven
- Parquet
- Scrum
- Apache Spark
- Azure Databricks
- Azure Blob Storage
7 months ago
Fort Worth, TX
Final Round will be Onsite
Rate : $115k / Annum
Seeking a 8 + ExpAzure Data Engineer to design, build, and optimize scalable, secure data solutions on Microsoft Azure. The role focuses on building batch/streaming pipelines withSQL, Python, Spark, and Azure-native services, followingAgile andDevOps best practices.
Responsibilities
- Design, develop, and maintaindata pipelines (batch & streaming) usingAzure Data Factory,Databricks, andEventHub.
- BuildETL/ELT workflows for ingestion, transformation, and curation acrossBlob Storage andAzure Data Lake (ADLS).
- Develop scalable processing jobs withPySpark/Spark SQL for large datasets and performance tuning.
- ImplementDevOps practices: version control, CI/CD, environment promotion usingAzure DevOps andMaven/Build pipelines.
- Ensuredata quality, reliability, and observability (logging, alerting, monitoring) across pipelines.
- Optimize storage formats and partitioning (e.g.,Parquet/Delta, Z-ordering) for cost and performance.
- Collaborate with product, analytics, and platform teams inAgile/Scrum to deliver incremental value.
- Enforcesecurity and governance (access controls, secrets, key vaults, data lineage) across Azure services.
- Write efficientSQL for transformations, validations, and performance optimization.
- Document solutions (data flows, runbooks) and support production operations and incident resolution.
Mandatory Skills
- 5–7 years building data solutions usingAgile andDevOps practices
- 5–7 years hands-on withSQL,Python, andApache Spark (PySpark/Spark SQL)
- 5–7 years withCloud Technologies, with strong preference forMicrosoft Azure
- Deep expertise with Azure services:
- Azure EventHub (stream ingestion)
- Azure Data Factory (ADF) (orchestration, ETL/ELT)
- Azure Databricks (Spark, notebooks, Delta)
- Azure DevOps (repos, pipelines/CI-CD)
- Azure Blob Storage
- Azure Data Lake (ADLS Gen2)
Azure Data Engineer · PeopleNTech LLC