DL
AWS Devops-Nvidia
Diverse Lynx India
๐ฎ๐ณ India
On-site
2 months ago
- Devops
- AWS
- Apache Airflow
- EMR
- Kubernetes
- CI/CD
- Terraform
- CloudFormation
- Airflow
- EC2
- IAM
- CloudWatch
- VPC
- EKS
- Secrets Management
- Incident Response
2 months ago
- Design, deploy, and maintain scalable DevOps infrastructure onAWS, supporting data engineering and analytics workloads across production environments.
- Manage and optimize ApacheAirflow workflows for orchestration, scheduling, monitoring, and failure recovery of batch data pipelines.
- Support AmazonEMR clusters for large-scale distributed data processing, including cluster provisioning, configuration, job monitoring, and cost optimization.
- Deploy and operatecontainerized applications on Kubernetes, managing workloads, services, configurations, secrets, and autoscaling.
- Build and maintain CI/CD pipelines to automate application deployment, infrastructure changes, and data platform releases.
- Implemented infrastructure automation using tools such as Terraform, CloudFormation, or AWS CLI to improve consistency and reduce manual operations
- Monitor system health, pipeline performance, cluster utilization, and application logs using AWS-native and third-party observability tools.
- Troubleshoot production issues acrossAirflow DAGs, EMR jobs, Kubernetes pods, and AWS services to ensure platform reliability.
- Improved reliability and uptime by implementing alerting, retry strategies, failure notifications, and operational runbooks.
- Collaborate with data engineers, application teams, and cloud infrastructure teams to support secure and scalable data platform operations.
- Manage AWS resources includingEC2, S3, IAM, CloudWatch, VPC, EMR, EKS, and related networking/security components.
- Implement access controls,IAM roles, secrets management, and security best practices across cloud and Kubernetes environments.
- Optimize cloud costs by tuning EMR cluster sizing, Kubernetes resource requests/limits, autoscaling policies, and AWS resource usage.
- Support environment deployments across development, staging, and production with standardized release and rollback procedures.
- Maintain documentation for deployment processes, incident response, infrastructure architecture, and operational procedures.
AWS Devops-Nvidia ยท Diverse Lynx India