Job Description

Required Skills:

• Hands-on experience provisioning and managing cloud-native streaming services—Amazon MSK, Azure Event Hubs, and AWS Kinesis Data Streams (KDS)—using Terraform modules and providers.

• Strong understanding of Kafka ecosystem components (brokers, zookeepers/quorum controllers, topics, partitions, ACLs) and ability to map them to managed cloud services like MSK, Event Hub namespaces, and KDS streams.

• Expertise designing scalable, secure event-streaming architectures across cloud platforms, including network integration (VPC/subnet design, Private Link, firewall/Sec Grp rules) using Infrastructure as Code.

• Skilled in implementing Terraform-based automation for Kafka cluster lifecycle management—cluster creation, versioning, encryption settings, monitoring configuration, and access control policies.

• Experience managing IAM, RBAC, and resource policies for streaming platforms (MSK IAM auth, Event Hub RBAC roles, Kinesis resource policies) using reusable Terraform modules.

• Strong knowledge of cloud-native observability stacks—such as CloudWatch, Azure Monitor, and MSK’s built-in monitoring—and incorporating them into Terraform deployments for end-to-end logging/metrics.

• Ability to operationalize streaming infrastructure by automating topic creation, schema registry integration, and consumer/producer configuration using Terraform, Ansible, or Python-based tooling.

• Demonstrated experience building CI/CD pipelines to continuously deploy changes to MSK, Event Hubs, and KDS, incorporating Terraform plan/apply automation and policy-as-code guardrails.

• Proficient in optimizing MSK, Event Hub, and KDS configurations for performance, cost, and fault tolerance—including scaling decisions, partitioning strategies, and retention tuning