Data Platform Engineer

Diverse Lynx India

Location not stated

1 month ago

Kubernetes
Devops
Azure
AKS
Node.js
NGINX
RBAC
Disaster Recovery
DNS
Istio
Docker
Azure DevOps
CI/CD
YAML
Git
IaC
Bicep
Terraform
Azure Monitor
Azure Functions
Apache NiFi
GitOps
ArgoCD
Grafana
Prometheus
OpenTelemetry
Microsoft Defender
Secrets Management
Vulnerability Management
Azure Data Factory
PowerShell
Bash
Python

1 month ago

Not scoredNo CV on file. Upload one and this job gets a score out of 100.Upload CV

Big Data Platform Engineer (Platform Infrastructure)
Total Experience
Required: 8+ years of overall IT experience
Relevant Experience

5+ years in Cloud Infrastructure Engineering
4+ years working with Kubernetes platforms
3+ years supporting enterprise-scale Data Platforms
3+ years in DevOps/SRE practices
2+ years working with Azure Data Services

Must Have Skills
1. Azure Kubernetes Service (AKS)
Strong Expertise

AKS cluster deployment and administration
Cluster upgrades and patch management
Node pool management
Ingress Controllers (NGINX/Application Gateway)
RBAC implementation
AKS security hardening
Autoscaling (HPA/VPA/Cluster Autoscaler)
Persistent storage management
Troubleshooting cluster issues
Network policies

Experience

Production support of AKS environments
Multi-node and multi-environment AKS setups
High availability and disaster recovery implementation

2. Kubernetes Networking
Strong Knowledge

Kubernetes networking concepts
Services, Ingress, Load Balancers
DNS resolution
Pod-to-Pod communication
Pod-to-Service communication
Network Policies
Service Mesh concepts (Istio preferred)
CNI Architecture

Azure Networking

Virtual Networks (VNet)
Subnets
Private Endpoints
Peering
NSG
UDR
Azure Firewall
VPN connectivity

3. Docker & Containerization
Must Have

Docker image creation and optimization
Container lifecycle management
Registry management
Security scanning
Multi-stage builds
Container troubleshooting

Registries

Azure Container Registry (ACR)
Image versioning and governance

4. Azure DevOps
CI/CD Expertise

Azure DevOps Pipelines
YAML pipelines
Build and Release pipelines
Git repositories
Branching strategies

Infrastructure Deployment

Infrastructure as Code
ARM Templates
Bicep
Terraform (preferred)

Automation

Deployment automation
Environment promotion
Release management

5. Monitoring & Observability
Azure Monitor

Monitoring strategy implementation
Alert configuration
Dashboard creation

Log Analytics

KQL query development
Performance analysis
Troubleshooting

Application Monitoring

Azure Application Insights
Distributed tracing
Metrics collection

AKS Monitoring

Cluster health monitoring
Pod performance monitoring
Resource optimization

6. Event Streaming Platform
Azure Event Hub

Event Hub architecture
Namespace management
Consumer groups
Scaling and throughput units
Monitoring and troubleshooting
Security and access control

7. Azure Functions
Must Have

Deployment and operations support
Function App management
Scaling configuration
Managed Identity integration
Monitoring and troubleshooting
CI/CD deployment

8. Apache NiFi
Platform Administration

NiFi cluster setup and maintenance
Flow deployment
Security configuration
Performance tuning
Monitoring
Troubleshooting

Preferred

NiFi Registry
Cluster scaling
HA implementation

8. Infrastructure as Code
Must Have

Terraform
Bicep
ARM Templates

Capabilities

Platform provisioning
Environment consistency
Version-controlled infrastructure

Good to Have Skills
Platform Engineering

GitOps (ArgoCD/Flux)
Platform Automation
Self-Service Platform Concepts
Internal Developer Platform

Observability

Grafana
Prometheus
OpenTelemetry

Security

Microsoft Defender for Cloud
Kubernetes Security Best Practices
Container Security
Secrets Management
Vulnerability Management

Data Platform

Azure Data Factory
Synapse Analytics

Cloud

Multi-region architecture
Disaster Recovery planning
Landing Zone implementation

Scripting

PowerShell
Bash
Python

Key Responsibilities

Manage and support AKS-based Big Data platform infrastructure.
Administer NiFi, Event Hub, and Azure Functions.
Implement and maintain CI/CD pipelines using Azure DevOps.
Monitor platform health using Azure Monitor and Log Analytics.
Troubleshoot platform, networking, and infrastructure issues.
Implement infrastructure automation through Terraform/Bicep.
Ensure platform security, governance, and compliance.
Collaborate with Data Engineering teams for platform enablement.
Perform upgrades, patching, capacity planning, and optimization.
Support production incidents and platform reliability initiatives.

Data Platform Engineer · Diverse Lynx India

Auto apply with Likeremote