Let's get into it…
Kubernetes is the de facto standard for orchestrating containerized applications in almost any organization. Understanding the architecture of Kubernetes can be daunting for newcomers. In this rapid guide, we will try to explain the Kubernetes architecture with easy-to-understand explanations.
What will be covered?
10000 Fit Overview
Control Plane Components
Kube-apiserver
Kube-scheduler
ETCD
Kube-controller-manager
Cloud-controller-manager
Node Components
Kubelet
Container Run Time
Kube-proxy
1000 Fit Overview
Kubernetes was designed to manage and orchestrate containers across a cluster of machines, ensuring HA, scalability, and resilience. Kubernetes is divided into two main components: the control plan and worker nodes.
The control plane is the brain of the cluster, responsible for all of the functionality, such as scheduling pods, storing data on the cluster, exposing API, and more. The components in the control plane include the kube-apiserver, which serves as the frontend of the cluster; etcd, a highly available key-value store for cluster data; the kube-scheduler, which is responsible for assigning workloads to nodes; kube-controller-manager, which manages other various control loops; and cloud-controller-manager that interacts with the relevant cloud provider.
Worker nodes are responsible for running application workloads. Each node in the cluster has three components: the kubelet, responsible for various tasks on the node; the kube-proxy, responsible for implementing network rules in the node; and the container run time, which is responsible for pulling images, running the containers, and managing their lifecycle.
Control Plane Components - Kube API Server
The kube-apiserver is our entrypoint to the cluster. It’s used for client interaction and internal communication. The kube-apiserver has many responsibilities, including autorizing and authenticating API requests, executing read and write operations to the ETCD cluster, and actively getting reports from the cluster's kubelets.
As clients, we can interact with the kube-apiserver via CLI utilizing kubectl, via HTTP rest API requests, and through a different IDE.
The kube-apiserver is designed to scale horizontally (more pods), ensuring HA and the ability to work under heavy load. The kube-apiserver plays a critical role, serving as a central hub for the cluster and providing a consistent status of the cluster to the components and its users.
Control Plane Components - Kube Scheduler
The kube-scheduler is responsible for assigning pods to nodes in the cluster. The scheduler's goal is to optimize resource utilization across the cluster and ensure applications will run smoothly on their scheduled node.
The kubernetes schedule process is composed of two main phases: filtering and scoring. For example, for a pod that requires 2 CPUs and 4 Gib of RAM, the scheduler will filter out any node that doesn't have enough capacity to host the pod. After filtering, the nodes are scored based on different factors such as resource utilization, node affinity, and more. The node with the highest score will be picked for scheduling.
The kubernetes scheduler can be tuned, customized, and even changed, but most likely, it’s current tunning will be a great fit for any common use case.
As users, we can affect the pods scheduling by implementing the following:
Resources - limits and requests on CPU and memory requirements for the containers
NodeName - Assigning the pods to a specific node in the cluster
NodeSelector - Assigning pods to nodes that contain specific labels
Affinity - More expressive selection logic on the nodes
If we want to filter out some of the nodes, we can utilize taint and toleration. By tainting the nodes, only pods with the right toleration spec can be scheduled on them. By default, any control plane node comes with built-in taint to lock out applicable workloads from it.
Control Plane Components - ETCD
Etcd is a highly available key-value database responsible for storing cluster state and configuration data. Any information that we can access through the API server is stored in the database. It’s common practice, especially among cloud providers, to encrypt the Etcd database in rest to store its data safely in non-plain text.
In case we manage the control plane ourselves, we can access the etcd cluster utlizing the etcdctl command.
The data stored in the etcd cluster is in a hierarchical structure, similar to filesystems. For example:
/registry/pods - is a path used to store information on pods
/registry/deployments - is a path used to store information on deployments
Control Plan Components - Kube Controller Manager
The kube controller manager is responsible for core control loops that regulate the state of the cluster. Some of the control loops are the Deployment controller, Node Controller, Replicaset controller, Service controller, and more.
Each control loop monitors the relevant resources he responsible for. As an example Replicaset controller ensures the correct number of pods running for each replicaset.
Control Plan Components - Cloud Controller Manager
The cloud controller manager enables easy integration with the cloud providers and allows seamless integration with it. For example, in EKS kubernetes service type LoadBalancer will automatically create a classic loadbalancer in AWS.
Each provider has its own controller; some of them provide more functionality and some provide less. We will probably find ourselves adding controllers such as the AWS Loadbalancer controller and the aws-ebs-csi controller to introduce more functionality as needed for our requirements.
Node Components - Kubelet
Kubelet runs as a service on each node in the cluster. It acts as a kubernetes agent, responsible for creating containers (based on the provided pod spec), verifying that they are running and healthy. He is also responsible for sending back to the kube api reports on the node and containers health status.
The kubelet is not responsible for containers that were not created by kubernetes.
Node Components - Kube Proxy
Kube proxy runs on each node in the cluster and is responsible for implementing network rules based on current kubernetes services. Essentially, It ensures that the network communication to and from pods is properly routed and accessible, both internally within the cluster and externally.
For example, we know that implementing service type nodePort will expose a port in each node in the cluster and route the traffic to the underlying pods. Kube-proxy would be responsible for opening the port and setting up the routing so that even nodes that do not host the pods can route the traffic to another node in the cluster that has the relevant pods.
Node Components - Container Run Time
The container run time is responsible for pulling container images, creating, starting, and managing the container lifecycle. As of kubernetes 1.24, the default container run time is contained, which is a lighter version of our beloved docker.
In my CKS learning path I come to know other container runtime such as runC and GVisor (by Google) that can provide higher security and isolation between running containers and may be worthwhile for some further investigation in the future.
To Conclude,
This guide has an overview of the Kubernetes components and the working of a Kubernetes cluster. From the control plane to the nodes, we explored how each element plays a critical role in orchestrating and managing containerized applications. Understanding these components can benefit anyone who looks to leverage Kubernetes in their organization.
