Kafka Integration Engineer
Senior Kafka & Integration Engineer IT-Integration & APIRole Type: Temporary / Contract (Support due to increased workload)
Role Overview:
We are looking for an experienced Kafka & Integration Engineer to join our IT-Integration & API team. In this role, you will design, deploy, and maintain highly available Apache Kafka and Confluent streaming platforms on Microsoft Azure, powering our event-driven enterprise architecture.
Key Responsibilities: Platform Engineering & Architecture:
- Design, implement, and manage highly available and scalable Apache Kafka / Confluent platforms hosted on Microsoft Azure.
- Manage core cluster components including brokers, topics, partitions, replication, security, and platform upgrades.
- Application & Data Integration:
- Develop and maintain robust Kafka producers, consumers, Kafka Connect integrations, Kafka Streams applications, and event-driven architectures.
- Build stream processing applications using Java, Python, or similar modern software engineering stacks.
- Disaster Recovery & High Availability:
- Implement and manage Confluent Cluster Linking, cross-cluster replication, and end-to-end Disaster Recovery (DR) solutions.
- Define and execute failover/failback strategies to satisfy enterprise business continuity requirements.
- Monitoring & Performance Optimization:
- Monitor, troubleshoot, and optimize Kafka production and development environments for peak performance, reliability, and scalability.
- Manage and minimize consumer lag, cluster throughput bottlenecks, and latency issues across environments.
- Automation, Infrastructure & DevOps:
- Automate Kafka platform operations and infrastructure management using Terraform, Ansible, Kubernetes, Docker, and Azure DevOps CI/CD pipelines.
- Establish and maintain platform standards, security controls, operational monitoring, technical documentation, and production support workflows.
- Required Technical Qualifications & SkillsEvent Streaming & Messaging: Apache Kafka, Confluent Platform / Confluent Cloud, Kafka Connect, Kafka Streams, Schema Registry, Confluent Cluster Linking.
- Cloud Platform: Microsoft Azure.
- Programming Languages: Java, Python, or equivalent object-oriented / functional languages.
- DevOps & Infrastructure as Code (IaC): Terraform, Ansible, Docker, Kubernetes (AKS), Azure DevOps CI/CD.Observability & Operations: Consumer lag monitoring, cluster tuning, disaster recovery, security controls, and enterprise support frameworks.