kafka

Scaling a Kafka consumer group with Kubernetes operator and HPA

Kubernetes (k8s) has introduced a number of ways to extend its APIs and data structure for customising and managing complex workloads. In this post, I attempt to leverage k8s’s Operator pattern and Custom Resource Definition (CRD) to manage the life cycle of a Kafka consumer group running atop a k8s cluster.