Senior Data Engineer

We are seeking an experiencedSenior Data Engineer to join our team.

The project entails the development and optimization of a new Data Platform to adjust our current data processing system. The Senior Data Engineer will be responsible for designing, implementing, and maintaining the platform to meet the business's data needs.

f you have a passion for Data Software Engineering to deliver impactful web applications, we would love to hear from you.

Responsibilities

• Develop data processing pipelines using Azure Data Factory and Azure Functions
• Implement data ingestion and transformation processes using Databricks and PySpark
• Write efficient code in Python and SQL to extract, transform, and load data
• Collaborate with stakeholders to gather requirements and ensure the platform meets business needs
• Monitor and troubleshoot the platform's performance and address any issues
• Implement data quality assurance measures and process improvements
• Optimize data storage and retrieval operations
• Conduct data analysis and generate insights to guide decision-making
• Train and mentor junior Data Engineers
• Participate in peer code reviews and collaborative team activities

Requirements

• 3+ years of hands-on experience in big-data field and expertise in Data Software Engineering
• Strong knowledge and experience with Azure Data Factory, Azure Functions, and Databricks
• Proficiency in PySpark, Python, and SQL
• Understanding of batch processing and ETL principles in Data Warehouses
• Experience with data completeness signals and orchestration
• Knowledge of schema migrations and datasets evolution
• Familiarity with approaches for historical reprocessing and data correction
• Good problem-solving and communication skills
• Proficient in English with a B2+ level, strong speaking English

Nice to have

• Familiarity with Microsoft Power BI
• Competency in Azure Blob Storage, Azure File Storage