Talend/data bricks & snowflake

Key Responsibilities

• Design, develop, and maintain scalable and extensible data pipelines for collecting, storing, transforming, and analyzing large datasets from multiple sources.

• Build and optimize ETL/ELT pipelines using Talend, Databricks, Snowflake, Azure Data Factory, and related technologies.

• Develop and manage data ingestion and replication frameworks using Fivetran and HVR for batch and near real-time data movement.

• Design and implement scalable data lakehouse architectures using Databricks, Delta Lake, and Snowflake.

• Develop PySpark and Spark SQL solutions to process and transform large-scale datasets efficiently.

• Implement data quality frameworks, automated testing, monitoring, and reconciliation processes to ensure data integrity and reliability.

• Build and maintain API-based integrations to support enterprise-wide data consumption.

• Optimize Snowflake performance through warehouse tuning, query optimization, clustering strategies, and cost management.

• Adhere to data governance standards to ensure data quality, consistency, integrity, usability, and security.

• Implement CI/CD pipelines and DevOps best practices for automated deployment and platform reliability.

• Perform root cause analysis and troubleshoot complex data-related issues across multiple systems.

• Collaborate with business stakeholders, product teams, architects, and engineering teams to deliver high-quality data products and solutions.

• Support modernization initiatives by migrating legacy ETL workloads from Talend, Informatica, and DataStage to cloud-native platforms.

Minimum Qualifications

• 5+ years of experience in Data Engineering, Data Warehousing, and Analytics solutions.

• 5+ years of experience designing and developing ETL/ELT solutions using Talend, Informatica, DataStage, or similar tools.

• 3+ years of hands-on experience with Databricks, PySpark, and Spark SQL.

• 3+ years of experience with Snowflake, including performance tuning, data modeling, and workload optimization.

• Experience with Fivetran and/or HVR for Change Data Capture (CDC), replication, and data ingestion.

• Strong programming and scripting skills in SQL, Python, Stored Procedures, and Unix Shell Scripting.

• Experience designing and developing cloud-based data pipelines using Azure Data Factory.

• Strong understanding of dimensional modeling, data warehousing concepts, and data lake architectures.

• Experience creating test plans, test strategies, integration testing, and data validation frameworks.

• Experience implementing CI/CD and DevOps practices using GitHub, Jenkins, JFrog, and related tools.

• Knowledge of data governance, data quality, metadata management, and security best practices.

• Experience working with structured, semi-structured (JSON/XML), and unstructured datasets.

• Experience working in Agile and Scrum environments.

Preferred Qualifications

• Bachelor's degree in Computer Science, Engineering, Mathematics, Information Systems, or a related discipline.

• Experience building enterprise-scale analytics and reporting platforms using Azure Cloud services.

• Hands-on experience with Delta Lake and Lakehouse Architecture.

• Experience supporting retail, merchandising, customer analytics, or supply chain data platforms.

• Experience migrating legacy ETL solutions to Databricks and Snowflake.

• Exposure to streaming technologies such as Kafka or Azure Event Hubs.

• Knowledge of Infrastructure as Code (Terraform) and DataOps practices.

• Databricks, Snowflake, Azure Data Engineer, or related cloud certifications.

• Exposure to Power BI or other BI and visualization platforms.

Primary Screening Skills (Must-Have)

• Databricks

• Snowflake

• PySpark

• Python

• SQL

• Azure Data Factory

• Fivetran/HVR

• CI/CD and GitHub

Secondary Skills

• Talend / Informatica / DataStage

• Data Modeling

• DevOps

• API Integration

• Data Governance

Nice-to-Have Skills

• Delta Lake

• Kafka / Azure Event Hubs

• Terraform

• Power BI

• DataOps Practices