EI
Senior Kafka Engineer
eTeam Inc.
🇺🇸 United States
On-site
Senior
4 weeks ago
- Apache Kafka
- AWS
- SSL
- ACLS
- Kafka Connect
- AWS Cloud
- EC2
- IAM
- VPC
- EBS
- CloudWatch
- CloudTrail
- AWS Glue
- EMR
- Airflow
- IaC
- Devops
- CI/CD
- Jenkins
- GitHub Actions
- GitLab CI
- Terraform
- CloudFormation
- Ansible
- Prometheus
- Grafana
- Splunk
- Docker
- Kubernetes
- Python
- Bash
- Incident Management
- Linux
- RHEL
- Git
- GCP
- AI
- AI/ML
4 weeks ago
Work Location: Saint Louis, MO 63131
Duration: 12 months
Kafka Administration
- Deploy, configure, and manage Apache Kafka clusters and AWS MSK environments.
- Monitor broker health, partitions, replication factors, and consumer lag.
- Perform capacity planning and cluster scaling activities.
- Manage Kafka security using SSL, SASL, ACLs, and encryption standards.
- Troubleshoot producer, consumer, and broker performance issues.
- Support Kafka Connect, Schema Registry, Cruise Control, and MirrorMaker implementations.
- AWS Cloud Administration
- Manage cloud infrastructure services including EC2, S3, IAM, VPC, EBS, CloudWatch, CloudTrail and AWS Glue.
- Support AWS Managed Streaming for Kafka (MSK), EMR, Lambda, and Airflow environments.
- Implement cloud security best practices and governance controls.
- Perform infrastructure provisioning and automation using Infrastructure as Code (IaC).
- Monitor cloud resource utilization and optimize operational costs.
- DevOps & Automation
- Design and maintain CI/CD pipelines using Jenkins, GitHub Actions, GitLab CI, or similar tools.
- Automate infrastructure deployment using Terraform, CloudFormation, and Ansible.
- Manage source control repositories and release processes.
- Implement monitoring and alerting solutions using Prometheus, Grafana, Splunk, ELK, or CloudWatch.
- Support containerization technologies such as Docker and Kubernetes.
- Develop automation scripts using Python, Shell, or Bash.
- Operations & Support
Provide Level 2 and Level 3 production support.
- Participate in on-call support rotations and incident management activities.
- Perform root cause analysis (RCA) and implement preventive measures.
- Create and maintain operational documentation and standard operating procedures.
- Ensure compliance with security, audit, and regulatory requirements.
Required Skills:
- Apache Kafka Administration
- KRAFT and MSK
- AWS Cloud Services
- Linux (RHEL/Rocky Linux)
- Shell Scripting and Python
- Jenkins, Git, Ansible, Terraform
- Docker and Kubernetes
- Monitoring Tools (Grafana, Prometheus, Splunk)
- Networking, Security, and High Availability Concepts
- Performance Tuning and Capacity Planning
Preferred Qualifications:
Bachelor’s degree in Computer Science, Information Technology, or related field.
Experience with Cloudera CDP, AWS MSK, Airflow, and Spark.
AWS, GCP, Kafka, or Kubernetes certifications.
Experience supporting large-scale production environments handling petabyte-scale data workloads.
Key Achievements Expected
- Maintain platform availability above 99.9%.
- Adopt AI-assisted engineering practices to improve operational efficiency, reduce manual effort, and accelerate troubleshooting and documentation.
- Automate repetitive operational tasks.
- Improve cluster performance and resource utilization.
- Ensure secure, scalable, and reliable data platform operations.
- Support enterprise data engineering, analytics, and AI/ML workloads efficiently.
- Minimum years of experience
- 8-10 years
Senior Kafka Engineer · eTeam Inc.