Today i saw something strange in a customer scenarios
kubectl get deployment my-dep showed running pods.
but if i LOOKED UP those pods, there were NO PODS running..
The issue ?
Kube controller manager was DOWN !!! That means that the state of the deployment isnt updated wrt the state of the overall system, i.e. the deployment never gets the message that the pods are down.
The root cause? Well, in this cluster, the CNI was down and Gatekeeper wasnt able to service requests, and this caused the Controller Manager to not be able to get a lock.
No comments:
Post a Comment