DL
Kubernetes_PAN INDIA
Diverse Lynx India
🇮🇳 India
On-site
12 months ago
- Kubernetes
- AWS
- Apache Spark
- ETL
- Hadoop
- Snowflake
- BigQuery
- Machine Learning
- AWS Glue
12 months ago
Data EngineerSkill - Data Eng -Spark/spark, ,AWS, Kubernetes, ETLJD - 1. Design and implement scalable data pipelines using technologies such as Apache Spark, Kafka, and AWS Glue2. Develop and maintain ETL processes to ingest, transform, and store structured and unstructured data3. Optimize data storage solutions, including data lakes, data warehouses, and relational databases4. Ensure data quality, integrity, and governance by implementing validation, monitoring, and logging mechanisms5. Work with big data technologies such as Hadoop, Snowflake, or BigQuery for large-scale data processing6. Collaborate with Data Scientists, Analysts, and Software Engineers to support machine learning and analytics initiatives7. Monitor and troubleshoot data pipeline failures, optimizing performance and reliability8. Implement security best practices for data storage, access control, and encryption9. Stay up to date with the latest data engineering trends, tools, and frameworks to enhance the data ecosystem Design and implement scalable data pipelines using technologies such as Apache Spark, Kafka, and AWS Glue
Kubernetes_PAN INDIA · Diverse Lynx India