Understanding Kubernetes

• Also known as k8s
• Orchestrates containers - deployment and management.
• Cloud-agnostic
• A minimum of 3 nodes are required: If one node becomes unavailable, the other 2 nodes can still operate and handle the workload without interruption
• Control plane runs on the master node, controls the cluster. All other are app specific workloads.

Control plane

It’s the brains of the cluster. Usually runs on the master node, but to guarantee high availability it may so be deployed in multiple machines.

Kubernetes API server

The primary entry point for interacting with the cluster's resources. It acts as the gatekeeper, authenticating clients and facilitating communication with pods, nodes, and services.

Kubernetes scheduler

Assigns unscheduled Pods to Nodes with the necessary available resources so that a Kubelet can run them^1, ensuring efficient distribution of workloads

func schedule_pod(pod_spec):
  feasible_nodes = filter(nodes, pod_spec)

  if feasible_nodes is empty:
    return not_schedulable
  else
    suitable_nodes = score_nodes(feasible_nodes, pod_spec)
    top_choice = select_highest_rank(suitable_nodes)
    schedule(top_choice)

To determine the most suitable place for a Pod, the scheduler first filters then scores the nodes.

Filtering feasible nodes: Collects a list of potential Nodes and discards candidates using multiple criteria. For example, not having enough available resources to meet a Pod's specific resource requests would discard a Node. ^1

Ranking the remaining nodes using multiple criteria (active scoring rules). Then select the top choice. If more than one node have the same score as the top choice, one will be picked at random.

Controller Manager

A loop is a never ending loop that that regulates the state a the system

A deamon(link to Wikipedia) that runs (embeds) control loops. In this case, the control loops are the Kubernetes controllers and third party controllers.

The controller manager interacts with the API server to monitor the state of the cluster and instructs changes towards the desired state.

For example, the replication controller, which manages the number of replicas running of a service may detect that a given workload is must to have at least 3 replicas. If less than 3 are running it would instruct the creation of a new Pod via the API server. Then the scheduler will assign the Pod to a Node for a Kubelet to run it.

Controllers have:

• CRD Custom resource definition. Among other things, defines the inputs the controller can take.
• Reconciler logic that — reconciles— the desired state of resources with their actual state by comparing, updating, or creating resources as needed.

Other examples of controllers are:

• Ingress
• Namespace

(TODO: Add more examples, external projects would be useful.)

Etcd

Distributed, reliable key-value store for the most critical data of a distributed system.^2

Basically, it stores the cluster configuration.

Worker Nodes

Node

Where pods run.

In Cloud platforms, a node represents a computing resource. For example, in AWS a single Node is a single EC2 instance. The instance could mount disks and provide access to a network (e.g. VPC), thus the Node would have storage and networks.

• A node may be a virtual or physical machine

Ref https://kubernetes.io/docs/concepts/architecture/nodes/

Kubelet

Responsible for managing the Node, the Pods and their containers.

As node agent

The kubelet is the primary node agent that runs on each node in a Kubernetes cluster. It is responsible for registering the actual Node against the Control Plane (calls the API Server to creates it own Node resource with the node's details, like memory, cpu, etc) and periodically send updates.

Pod management

Takes Pod specifications (PodSpecs), usually from the API Server and ensures that the containers are running and healthy by continuously monitoring the state of the pods and containers using different mechanisms/probes: liveness, readiness and startup probes.

Container management

It talks to the Container Runtime Interface (CRI), such as Docker, to control the lifecycle of containers. It instructs the runtime to start, stop, and manage containers to achieve the state described in the pod specifications.

Kube Proxy

Facilitates networking, by maintaining network rules on nodes. These rules enable the communication to the Pods from inside or outside the cluster, providing load balancing and routing functionalities.

If there is an OS packet filtering layer like IPVS or iptables and it's available, it relies on the kernel to efficiently handle large amounts of traffic. Otherwise, forwards traffic to itself.

Pods

The smallest unit of computing in k8s. It's a group of one or more containers that share resources (network, storage, ...)

There are 2 flavours: one container per Pod, which is the most common. And multi container, where multiple containers are deployed together because are tightly coupled and need to share resources. For example: A database migrations container (1) that needs to run before an API REST container (2), along with a Sidecar container like envoy for purposes (eg. Istio or other service mesh), and a sidecar container for telemetry and log collection (eg. prometheus, DataDog).

Fun fact: At Meta, they initially called "Tupeperware" their cluster management system that orchestrates containers. Later renamed Twine.

Communication

WIP

Scaling

WIP

Resources and References:

• https://kubernetes.io/docs/reference/glossary