Kafka Administrator (MS Azure)

Kafka Administrator (MS Azure) - Job Description100% RemoteExtensive System admin and Cloud engineering experienceMust have hands on experience on• Cluster Management: Provisioning, configuring, and scaling Kafka clusters.• Azure Integration: Bridging Kafka with native services- eg. configure Azure Event Hubs (Kafka Surface) or manage data ingestion into Azure Data Lake Storage• Performance Tuning: Optimizing broker properties, partition strategies, and JVM settings to meet latency and SLAs requirements.• Security and compliance of kafka clusters/ services• Automating the deployment of streaming infrastructure using Terraform to ensure environment parity across Dev, UAT, and Production.Knowledge Areas• Distributed Systems: Deep understanding of partition leadership, replication factors, ISR (In-Sync Replicas), and \"exactly-once\" processing semantics.• Storage Optimization: Experience managing Azure Managed Disks (Premium SSDs) for Kafka brokers to balance IOPS and cost.• Disaster Recovery: Setting up multi-region clusters using Cluster Linking or MirrorMaker 2 to ensure high availability across Azure Regions.• Troubleshooting: Ability to diagnose complex issues like \"rebalance storms,\" consumer lag spikes, and disk pressure in a distributed environment.