In Kubernetes Engine, a container cluster consists of at least one cluster master and multiple worker machines called nodes. These master and node machines run the Kubernetes cluster orchestration system.
A container cluster is the foundation of Kubernetes Engine: the Kubernetes objects that represent your containerized applications all run on top of a cluster.
The cluster master runs the Kubernetes control plane processes, including the Kubernetes API server, scheduler, and core resource controllers. The master's lifecycle is managed by Kubernetes Engine when you create or delete a cluster. This includes upgrades to the Kubernetes version running on the cluster master, which Kubernetes Engine performs automatically, or manually at your request if you prefer to upgrade earlier than the automatic schedule.
Cluster master and the Kubernetes API
The master is the unified endpoint for your cluster. All interactions with the
cluster are done via Kubernetes API calls, and the master runs the Kubernetes
API Server process to handle those requests. You can make Kubernetes API calls
directly via HTTP/gRPC, or indirectly, by running commands from the
Kubernetes command-line client (
kubectl) or interacting with the UI in the
The cluster master's API server process is the hub for all communication for the cluster. All internal cluster processes (such as the cluster nodes, system and components, application controllers) all act as clients of the API server; the API server is the single "source of truth" for the entire cluster.
Master and node interaction
The cluster master is responsible for deciding what runs on all of the cluster's nodes. This can include scheduling workloads, like containerized applications, and managing the workloads' lifecycle, scaling, and upgrades. The master also manages network and storage resources for those workloads.
The master and nodes also communicate using Kubernetes APIs.
A container cluster typically has one or more nodes, which are the worker machines that run your containerized applications and other workloads. The individual machines are Compute Engine VM instances that Kubernetes Engine creates on your behalf when you create a cluster.
Each node is managed from the master, which receives updates on each node's self-reported status. You can exercise some manual control over node lifecycle, or you can have Kubernetes Engine perform automatic repairs and automatic upgrades on your cluster's nodes.
A node runs the services necessary to support the Docker containers that
make up your cluster's workloads. These include the Docker runtime and the
Kubernetes node agent (
kubelet) which communicates with the master and is
responsible for starting and running Docker containers scheduled on that node.
In Kubernetes Engine, there are also a number of special containers that run as per-node agents to provide functionality such as log collection and intra-cluster network connectivity.
Node machine type
Node OS images
Each node runs a specialized OS image for running your containers. You can specify which OS image your clusters and node pools use.
Minimum CPU platform
When you create a container cluster or node pool, you can specify a baseline minimum CPU platform for its nodes. Choosing a specific CPU platform can be advantageous for advanced or compute-intensive workloads. For more information, refer to Minimum CPU Platform.
Node allocatable resources
Some of a node's resources are required to run the Kubernetes Engine and Kubernetes node components necessary to make that node function as part of your cluster. As such, you may notice a disparity between your node's total resources (as specified in the machine type documentation) and the node's allocatable resources in Kubernetes Engine.
You can make a request for resources for your Pods or limit their resource usage. To learn how to request or limit resource usage for Pods, refer to Managing Compute Resources for Containers.
To inspect the node allocatable resources available in a cluster, run the following command:
kubectl describe node [NODE_NAME] | grep Allocatable -B 4 -A 3
The returned output contains
Allocatable fields with
measurements for ephemeral storage, memory, and CPU.
Allocatable memory and CPU resources
Allocatable resources are calculated in the following way:
Allocatable = Capacity - Reserved - Eviction Threshold
For memory resources, Kubernetes Engine reserves the following:
- 25% of the first 4GB of memory
- 20% of the next 4GB of memory (up to 8GB)
- 10% of the next 8GB of memory (up to 16GB)
- 6% of the next 112GB of memory (up to 128GB)
- 2% of any memory above 128GB
Kubernetes Engine reserves an additional 100 Mi memory on each node for kubelet eviction.
For CPU resources, Kubernetes Engine reserves the following:
- 6% of the first core
- 1% of the next core (up to 2 cores)
- 0.5% of the next 2 cores (up to 4 cores)
- 0.25% of any cores above 4 cores
The following table shows the amount of allocatable memory and CPU resources that are available for scheduling your cluster's workloads for each standard node machine type:
|Machine type||Memory capacity (GB)||Allocatable memory (GB)||CPU capacity (cores)||Allocatable CPU (cores)|
Allocatable local ephemeral storage resources
Beginning in Kubernetes Engine version 1.10, you can manage your local ephemeral storage resources as you do your CPU and memory resources. System reservations for local storage are made primarily for disk space used by container images.
If your node does not consume all reserved storage, Pods are still able to use the space. This does not prevent disk space from being used in any scenario.
Allocatable local ephemeral storage resources are calculated using the following formula, with an eviction threshold of 10% of storage capacity:
Allocatable = Capacity - Reserved - Eviction Threshold
|Disk Capacity (GB)||Reserved (GB)||Allocatable (GB)|