NetworkAvailability is meaningless : do we need to define Network Readiness in the Kubernetes Era of 2023 ????

The K8s E2Es have a concept of network readiness, you do to, so does your CNI, so does your cloud controlelr



So i filed https://github.com/kubernetes/kubernetes/issues/120486




Omg?

So if your new to this.


The Kubernetes landscape has been in constant evolution since its inception, with new features and innovations continuously reshaping our understanding of container orchestration and management. This evolution, in turn, has shaped our understanding and expectations of various concepts that once seemed straightforward. One such concept is "Network Readiness."

A short walk down memory lane: Close to a decade ago, while Stephen Watt and I were getting nginx to hum harmoniously on flannel0, right before that fateful ApacheCon, we were introduced to the concept of "NetworkUnavailable." This was a clear, binary state that helped developers understand the state of their deployments. The link here takes you to that story. However, fast forward to 2023, and this term may seem almost quaint given the complexities introduced by the rapid advancements in Kubernetes.

The current Kubernetes environment, rich with Windows nodes, multi-networking capabilities, pods with multiple NICs, sharded vClusters, and the intricate layering of netpols/service meshes, demands a more nuanced understanding. The term "Network Readiness" in this context feels almost reductionist. Let's delve into a few scenarios to illustrate:


Policy Controllers: What happens if my CNI network policy controller goes offline? Does that imply that the network is "unready"? i.e. antrea policy controller might be down but its not needed to start pods just to apply network policies....



Firewalls: If firewalls, which have not yet been removed, are interrupting nodePort communication, how do we define readiness?



Specific CNI Configurations: Suppose a particular CNI is configured only for host-network pods and secondary NIC pods. In this state, is our network "ready"?



Multus and Multiple CNIs: Modern Kubernetes configurations can have more than one CNI in a cluster, with tools like Multus allowing them to coexist. If different CNIs have varied readiness statuses, how do we holistically define the network's readiness?



Host-network Pods: Imagine a node designed exclusively for host-network pods. While it might be "ready" in the strictest sense, it's only prepared to handle specific, secure, high-performance host-network workloads.

Given these scenarios, it’s clear that the old binary understanding of network readiness is no longer sufficient. Today's Kubernetes ecosystem, vibrant and multifaceted, demands a more intricate evaluation of readiness.

In conclusion, as Kubernetes continues to evolve, so must our terminologies and our understanding. As developers and architects, it's pivotal that we revisit and redefine these concepts to ensure they are in sync with the current realities of our system. Perhaps it's time for a new terminology altogether or a set of terminologies to capture the varied shades of "readiness" in the Kubernetes world of 2023.