Data Engineer

Experience & Competencies:

• Previously worked closely with data architect or data modellers, can independently work on data analysis, data modelling.
• Proficient in manipulating data and drawing insights from large data sets using SQL
• Highly proficient in data preparation and integration, able to design, develop and modify data models and troubleshoot performance issues.
• Good in doing downstream analysis and understanding end to end data flow when multiple systems are involved.
• Experienced with working on Agile delivery.
• Expected to have strong at least 3 to 4 years of hands-on experience in SQL.
• Familiarity with data governance, data quality and control frameworks would certainly be useful in this role.
• Able to creatively use data and insights to uncover new opportunities, identify root causes and underlying risks to recommend solutions to the business.

Technical Skills:

• Advanced data engineering skills with strong experience using Spark, SQL, Python and Airflow
• Strong knowledge of big data querying tools such as Presto or Trino.
• Experience in data architecture principles, including data access patterns and data modelling.
• Experience with data quality measurement and monitoring
• Experience with metadata, including control totals and check sums for load assurance.
• Comfortable taking ownership of issues and driving resolution.
• Familiar with cloud computing concepts (AWS) including any or all of EC2, S3, IAM, EKS and RDS and Linux
• Good understanding of Data Warehousing/ETL concepts
• Experience with CI/CD tools
• Experienced with DevOps approaches