TI
Data Engineer - Hybrid _ Bengaluru
Tranzeal Inc.
🇮🇳 India
Hybrid
1 month ago
- Machine Learning
- PySpark
- AWS
- ETL
- ELT
- AWS S3
- Redshift
- EMR
- Athena
- Step Functions
- Apache Airflow
- AWS Glue
- Kinesis
- Snowflake
- CI/CD
- IAM
1 month ago
Job Title: Data Engineer - Hybrid
Location: Eco Space Centre, Bellandur Bengaluru
Experience: 6+
Salary: 22 - 25 LPA
JD:
We're looking for a skilled Data Engineer to design, build, and maintain scalable data pipelines that power analytics, reporting, and machine learning initiatives. You'll work extensively withPySpark for large-scale data processing and theAWS ecosystem to build reliable, cloud-native data infrastructure.
Key Responsibilities
- Design, develop, and maintain ETL/ELT pipelines usingPySpark for batch and streaming data processing
- Build and manage data lake and data warehouse solutions onAWS (S3, Redshift, Glue, EMR, Athena, Lake Formation)
- Develop and orchestrate workflows usingAWS Step Functions,Apache Airflow, orAWS Glue Workflows
- Optimize Spark jobs for performance, cost, and scalability (partitioning, caching, cluster tuning)
- Ingest data from multiple sources (APIs, databases, flat files, streaming platforms like Kafka/Kinesis)
- Implement data quality checks, validation frameworks, and monitoring/alerting for pipeline health
- Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements
- Design and maintain data models (star/snowflake schemas) for analytics use cases
- Write clean, well-documented, testable code following engineering best practices (CI/CD, version control)
- Ensure data security, governance, and compliance (IAM policies, encryption, access controls)
- Troubleshoot and resolve production data pipeline issues
Data Engineer - Hybrid _ Bengaluru · Tranzeal Inc.