Senior Technical Lead – Azure Data Engineer

About USEREADY

USEReady helps enterprises apply AI and agentic intelligence to improve decisions, automate operations, and build smarter, more autonomous business systems.

For more than a decade, we have built the foundations that make this possible by modernizing BI environments, migrating legacy platforms, improving data quality, and enabling governed, cloud-first architectures. These foundations now support the next step: AI-driven insights, automated intelligence, and agent-powered decision support that reduce complexity and accelerate outcomes.

We work closely with technology leaders such as AWS, Elementum, Snowflake, Tableau, Databricks, and others to help organizations modernize analytics, strengthen governance, and deploy agentic automation with confidence. We founded in 2011 and Headquartered in New York City with 450+ experts across the United States, Canada, India, and Singapore, we serve industries including financial services, healthcare, manufacturing, government, education, and retail. Our deep expertise, player-coach delivery model, and focus on fast, measurable results make us a trusted partner for building an AI-ready enterprise.

Job Summary

We are looking for aSenior Technical Lead – Azure Data Engineer with8–10 years of experience and strong hands-on expertise inMicrosoft Azure, Databricks, ADF, Python, PySpark, and SQL.

The ideal candidate will have strong experience designing and implementing scalable data engineering solutions, architecting Databricks environments, optimizing data pipelines and workloads, and implementing data governance and security usingUnity Catalog.

The candidate should be comfortable working with large-scale distributed data platforms, mentoring engineering teams, and collaborating with business and technical stakeholders to deliver high-quality data solutions.

Key Responsibilities

• Architect and design scalableAzure Databricks environments and data engineering solutions.

• Design and implement robust, scalable, and optimized data pipelines usingDatabricks and Azure Data Factory (ADF).

• Develop Databricks notebooks usingPython, PySpark, and SQL.

• Implement and manageUnity Catalog for data governance, security, access control, and data discovery.

• Design data models and define schema strategies, data structures, and pipeline organization.

• Build and maintainDelta Lake tables and data models supporting analytics and BI workloads.

• Optimize Databricks jobs, Spark workloads, SQL queries, and data pipelines forperformance, scalability, and cost efficiency.

• Apply knowledge ofApache Spark internals to troubleshoot and tune distributed data processing workloads.

• Develop complex and optimizedSQL queries, including efficient joins and processing of large datasets.

• Implement data quality, monitoring, validation, and reconciliation mechanisms.

• Work with Azure data services includingAzure Data Lake Storage (ADLS), Azure Synapse Analytics, and Azure SQL Database.

• Implement data governance, security, access controls, and data lineage.

• Troubleshoot pipeline failures and performance issues and ensure reliability and uptime.

• Collaborate with stakeholders to understand business requirements and translate them into scalable data solutions.

• Identify opportunities to automate manual processes and improve overall data platform efficiency.

• Mentor junior and mid-level engineers and establish best practices across Azure and Databricks development.

Mandatory Skills

• 8–10 years of overall experience in Data Engineering, with strong focus onAzure.

• Strong hands-on experience withAzure Databricks.

• Excellent experience withPython and PySpark.

• Strong knowledge ofApache Spark architecture, internals, and performance tuning.

• Strong SQL skills, including complex queries, query optimization, and large-scale data processing.

• Strong experience withAzure Data Factory (ADF).

• Hands-on experience withDatabricks Notebooks using Python and SQL.

• Experience withUnity Catalog, data governance, security, and access control.

• Strong understanding ofDelta Lake.

• Experience withAzure Data Lake Storage (ADLS).

• Good understanding ofdata warehousing and data modeling, including Star and Snowflake schemas.

• Experience working with large-scale distributed data processing environments.

• Strong troubleshooting and performance optimization skills.

Good to Have

• Experience withAzure Synapse Analytics.

• Experience withAzure SQL Database.

• Databricks or Azure Data Engineering certifications.

• Experience implementing enterprise-level data governance and data lineage.

• Experience mentoring technical teams and driving engineering best practices.

• Strong stakeholder management and communication skills.