Azure Data Engineer – Databricks / PySpark / Synapse

Job Title: Azure Data Engineer – Databricks / PySpark / Synapse
Remote or Hybrid: Remote (Project Tenure 3.5 Months), if hybrid is favored by candidate, they will work at Irving, TX location.
Location: If Hybrid, preferred, TX-Genpass Irving
Hours: 9AM -5PM CST
Interview Process: Teams interview. 1 round of hands-on coding session.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark.
• Extract, transform, and load (ETL/ELT) data from multiple source systems into Azure-based data platforms.
• Develop and optimize large-scale data processing workflows using PySpark and Apache Spark.
• Create, manage, and optimize database objects, including tables, views, stored procedures, and complex SQL queries.
• Build and maintain data solutions using Azure Data Lake Storage (ADLS) and Azure Synapse Analytics.
• Implement data integration and transformation processes to support reporting, analytics, and business intelligence requirements.
• Monitor, troubleshoot, and improve the performance, reliability, and scalability of data pipelines.
• Ensure data quality through validation, cleansing, reconciliation, and auditing processes.
• Collaborate with business stakeholders, data analysts, and data scientists to understand requirements and deliver data solutions.
• Implement data security, governance, and compliance standards across Azure data platforms.
• Support production deployments, incident management, and ongoing operational support.
• Create and maintain technical documentation, data flow diagrams, and operational procedures.
• Participate in code reviews and follow best practices for data engineering and software development.

Required Skills

• Strong experience with Azure Databricks, PySpark, and SQL.
• Hands-on experience with Azure Data Lake Storage (ADLS) and Azure Synapse Analytics.
• Strong understanding of ETL/ELT processes and data warehousing concepts.
• Experience in performance tuning and optimization of Spark and SQL workloads.
• Knowledge of data modeling techniques, including dimensional and relational modeling.
• Familiarity with Git, Azure DevOps, CI/CD pipelines, and Agile methodologies.
• Excellent analytical, troubleshooting, and problem-solving skills.