Kafka Engineer

• Manage cloud infrastructure services including EC2, S3, IAM, VPC, EBS, CloudWatch, CloudTrail and AWS Glue.
• Support AWS Managed Streaming for Kafka (MSK), EMR, Lambda, and Airflow environments.
• Implement cloud security best practices and governance controls.
• Perform infrastructure provisioning and automation using Infrastructure as Code (IaC).
• Monitor cloud resource utilization and optimize operational costs.

• Design and maintain CI/CD pipelines using Jenkins, GitHub Actions, GitLab CI, or similar tools.
• Automate infrastructure deployment using Terraform, CloudFormation, and Ansible.
• Manage source control repositories and release processes.
• Implement monitoring and alerting solutions using Prometheus, Grafana, Splunk, ELK, or CloudWatch.
• Support containerization technologies such as Docker and Kubernetes.
• Develop automation scripts using Python, Shell, or Bash.

• Provide Level 2 and Level 3 production support.
• Participate in on-call support rotations and incident management activities.
• Perform root cause analysis (RCA) and implement preventive measures.
• Create and maintain operational documentation and standard operating procedures.
• Ensure compliance with security, audit, and regulatory requirements.

Required Skills

• Apache Kafka Administration
• AWS Cloud Services
• Linux (RHEL/Rocky Linux)
• Shell Scripting and Python
• Jenkins, Git, Ansible, Terraform
• Docker and Kubernetes
• Monitoring Tools (Grafana, Prometheus, Splunk)
• Networking, Security, and High Availability Concepts
• Performance Tuning and Capacity Planning

Preferred Qualifications

• Bachelor s degree in Computer Science, Information Technology, or related field.
• Experience with Cloudera CDP, AWS MSK, Airflow, and Spark.
• AWS, Google Cloud Platform, Kafka, or Kubernetes certifications.
• Experience supporting large-scale production environments handling petabyte-scale data workloads.

Key Achievements Expected

• Maintain platform availability above 99.9%.
• Adopt AI-assisted engineering practices to improve operational efficiency, reduce manual effort, and accelerate troubleshooting and documentation.
• Automate repetitive operational tasks.
• Improve cluster performance and resource utilization.
• Ensure secure, scalable, and reliable data platform operations.
• Support enterprise data engineering, analytics, and AI/ML workloads efficiently.

For applications and inquiries,