Senior Azure Data Engineer - Databricks

🚀Senior Azure Data Engineer – Mindera

At Mindera, we build technology products that create meaningful business impact. We work with modern cloud technologies and encourage engineering ownership, collaboration, continuous learning, and self-organization.

Our Data Engineering teams build scalable, reliable, and high-performance data platforms that enable analytics, reporting, machine learning, and data-driven decision-making.

🧠 What We’re Looking For

• • Design, develop, and maintain scalabledata pipelines and ETL/ELT workflows using Azure Databricks.
  • Build data processing solutions usingApache Spark and PySpark for large-scale datasets.
  • Develop and optimizebatch and real-time/streaming data pipelines.
  • Implement modernMedallion Architecture using Bronze, Silver, and Gold data layers.
  • Work extensively withDelta Lake for reliable, scalable data storage and processing.
  • Develop complex and optimizedSQL queries for data transformation and analytics.
  • Integrate data from multiple sources including databases, APIs, files, cloud storage, and streaming platforms.
  • Work with Azure services such asADLS Gen2, Azure Data Factory, Azure Key Vault, Azure DevOps, and Azure Monitor.
  • Optimize Spark workloads through effective partitioning, caching, joins, cluster configuration, and file-format optimization.
  • Implement data quality, validation, monitoring, logging, and observability across data pipelines.
  • Apply appropriatedata modelling and dimensional modelling practices for analytics workloads.
  • Implement CI/CD, source control, automated testing, and deployment practices for data engineering solutions.
  • Contribute to architecture and technology decisions for modern cloud data platforms.
  • Troubleshoot production data pipelines and proactively improve reliability, performance, and cost efficiency.
  • Collaborate with Data Scientists, Data Analysts, Software Engineers, Product Managers, and business stakeholders.
  • Mentor junior and mid-level engineers and contribute to engineering best practices.
  • Create technical documentation covering data pipelines, architecture, processes, and operational procedures.

Must-Have Skills

• • 7+ years of professional experience inData Engineering.
  • Strong hands-on experience withAzure Databricks.
  • Excellent knowledge ofApache Spark / PySpark.
  • Strong programming experience inPython.
  • AdvancedSQL skills.
  • Hands-on experience building scalableETL/ELT data pipelines.
  • Strong understanding ofDelta Lake and Medallion Architecture.
  • Experience withAzure Data Lake Storage Gen2 (ADLS).
  • Experience withAzure Data Factory or equivalent orchestration tools.
  • Strong understanding of data warehousing, data lakes, and data modelling.
  • Experience with batch processing and preferablystreaming data processing.
  • Hands-on experience withGit and CI/CD practices.
  • Experience working in Agile/Scrum or similar engineering environments.
  • Strong analytical, problem-solving, and troubleshooting skills.
  • Excellent communication and stakeholder-management skills.

Good-to-Have Skills

• • Experience withUnity Catalog and Databricks governance.
  • Experience withAzure Event Hubs, Kafka, or Spark Structured Streaming.
  • Experience withdbt or similar transformation frameworks.
  • Experience withTerraform / Infrastructure as Code.
  • Knowledge of Azure Synapse Analytics.
  • Experience with data quality and observability frameworks.
  • Understanding of security, access control, lineage, and governance.
  • Experience with Databricks Workflows / Jobs and automated deployment.
  • Exposure to machine learning or AI data pipelines.
  • Experience optimizing cloud infrastructure and data-processing costs.

What We Look For

• • Strong ownership and ability to work independently.
  • A problem-solving mindset with attention to engineering quality.
  • Ability to design solutions rather than simply implement requirements.
  • Clear communication with technical and non-technical stakeholders.
  • Willingness to mentor others and share knowledge.
  • Curiosity and enthusiasm for learning new technologies.
  • Ability to work effectively in a collaborative, self-organized engineering environment.

🌟 Nice to Have

• Experience working withfeature stores and ML pipelines
• Exposure tostreaming technologies (Kafka, Kinesis, etc.)
• Understanding ofdata governance and data quality frameworks
• Experience withLLM-based applications.

We offer

• Flexible working hours (self-managed)
• Annual bonus, subject to company performance
• Access to Udemy online training and opportunities to learn and grow within the role

At Mindera we use technology to build products we are proud of, with people we love.

Software Engineering Applications, including Web and Mobile, are at the core of what we do at Mindera.

We partner with our clients, to understand their products and deliver high-performance, resilient and scalable software systems that create an impact on their users and businesses across the world.

You get to work with a bunch of great people, and the whole team owns the project together.

Our culture reflects our lean and self-organisation attitude.

We encourage our colleagues to take risks, make decisions, work in a collaborative way and talk to everyone to enhance communication. We are proud of our work and we love to learn all and everything while navigating through an Agile, Lean and collaborative environment.

Our offices are located: Porto, Portugal | Aveiro, Portugal | Coimbra, Portugal | Leicester, UK | San Diego, USA | Chennai, India | Bengaluru, India