Senior Azure Data Engineer

About USEREADY

USEReady helps enterprises apply AI and agentic intelligence to improve decisions, automate operations, and build smarter, more autonomous business systems.

For more than a decade, we have built the foundations that make this possible by modernizing BI environments, migrating legacy platforms, improving data quality, and enabling governed, cloud-first architectures. These foundations now support the next step: AI-driven insights, automated intelligence, and agent-powered decision support that reduce complexity and accelerate outcomes.

We work closely with technology leaders such as AWS, Elementum, Snowflake, Tableau, Databricks, and others to help organizations modernize analytics, strengthen governance, and deploy agentic automation with confidence. We founded in 2011 and Headquartered in New York City with 450+ experts across the United States, Canada, India, and Singapore, we serve industries including financial services, healthcare, manufacturing, government, education, and retail. Our deep expertise, player-coach delivery model, and focus on fast, measurable results make us a trusted partner for building an AI-ready enterprise.

About the Role

We are looking for an experiencedSenior Azure Data Engineer with strong hands-on expertise inDatabricks, Azure Data Factory (ADF), PySpark, Python, and SQL. The ideal candidate will be responsible for designing, developing, optimizing, and maintaining scalable data engineering solutions on Azure.

You will work with modern Azure data technologies to build high-performance data pipelines, implement data governance, optimize Databricks workloads, and deliver reliable data platforms that support analytics and business intelligence initiatives.

Key Responsibilities

• Design, develop, and maintainscalable and reliable data pipelines usingAzure Databricks and Azure Data Factory (ADF).

• Develop data processing solutions usingPython, PySpark, SQL, and Apache Spark.

• Build and maintainDelta Lake tables and scalable data models for analytics and BI workloads.

• Implement and manageDatabricks Unity Catalog for data governance, security, access control, and data discovery.

• OptimizeDatabricks jobs, Spark workloads, SQL queries, and data pipelines for performance, scalability, and cost efficiency.

• Apply Apache Spark internals and tuning techniques to improve distributed data processing performance.

• Develop complex and optimizedSQL queries, including efficient joins and processing of large datasets.

• Design and implement ETL/ELT workflows usingAzure Data Factory.

• Work with Azure data services includingAzure Data Lake Storage (ADLS), Azure Synapse Analytics, and Azure SQL Database.

• Implement data quality checks, validation, reconciliation, monitoring, and error-handling mechanisms.

• Implement data governance, security, access control, anddata lineage across data platforms.

• Troubleshoot pipeline failures and performance issues and ensure high availability and reliability of data solutions.

• Collaborate with business stakeholders, data architects, analysts, and engineering teams to understand requirements and deliver effective data solutions.

• Automate manual processes and continuously improve data engineering workflows and operational efficiency.

• Mentor junior data engineers and promote best practices acrossAzure and Databricks data engineering.

Required Skills & Expertise

• 6+ years of experience in Data Engineering, with strong experience in Azure-based data platforms.

• Strong hands-on experience withAzure Databricks.

• Expertise inDatabricks Notebooks using Python and SQL.

• Strong experience withDatabricks Unity Catalog, governance, security, and access control.

• Hands-on experience withDatabricks performance optimization and cost/compute optimization.

• Strong experience withAzure Data Factory (ADF) and ETL/ELT orchestration.

• AdvancedPython and PySpark programming skills.

• Strong understanding ofApache Spark architecture, internals, and performance tuning.

• StrongSQL skills, including complex queries, query optimization, joins, aggregations, and large-volume data processing.

• Strong understanding ofdata warehousing concepts and data modeling, including Star and Snowflake schemas.

• Experience withDelta Lake and modern lakehouse architecture.

• Good knowledge ofAzure Data Lake Storage (ADLS).

• Experience withAzure Synapse Analytics and Azure SQL Database.

• Understanding ofdata governance, data security, access management, and data lineage.

• Experience working with large-scale distributed data processing environments.

• Strong problem-solving, communication, and collaboration skills.

Preferred Qualifications

• Databricks certifications are a plus.

• Microsoft Azure Data Engineering certifications are a plus.

• Experience working with enterprise-scale Azure data platforms.

• Experience implementing data quality, governance, and security frameworks.

• Experience mentoring or technically guiding junior data engineers.

Education

• Bachelor’s or Master’s degree inComputer Science, Information Technology, Engineering, or a related field is preferred.