upgrading EKS cluster hosting Kafka about eks-cluster-upgrade

[quixoticmonk]

To clarify, are you asking for a wait state between node upgrades based on user input ?

[bryantbiggs]

I don't know if this should be handled here - I think this is better handled through the normal Kubernetes highly available constructs such as startup probes, readiness probes, PDBs, etc.

How are you deploying Kafka - are you using the Strimzi operator?

[hjoshi84]

I don't know if this should be handled here - I think this is better handled through the normal Kubernetes highly available constructs such as startup probes, readiness probes, PDBs, etc.

How are you deploying Kafka - are you using the Strimzi operator?

I am using confluent kafka as operator

[hjoshi84]

Yes. before upgrading the next node script should wait until Kafka pods sync the partitions. otherwise, there will be a loss of data. I believe this will be a good feature addition.

[bryantbiggs]

and how would one check if a kafka pod has finished syncing? isn't this what statefulsets in combination with pod disruption budgets are intended to handle?