This page provides an overview of node taints on Google Kubernetes Engine (GKE). When you schedule workloads to be deployed on your cluster, node taints help you control which nodes they are allowed to run on.
Overview
When you submit a workload to run in a cluster, the scheduler determines where to place the Pods associated with the workload. The scheduler is free to place a Pod on any node that satisfies the Pod's CPU, memory, and custom resource requirements.
If your cluster runs a variety of workloads, you might want to exercise some control over which workloads can run on a particular pool of nodes.
A node taint lets you mark a node so that the scheduler avoids or prevents using it for certain Pods. A complementary feature, tolerations, lets you designate Pods that can be used on "tainted" nodes.
Node taints are key-value pairs associated with an effect. Here are the available effects:
NoSchedule
: Pods that do not tolerate this taint are not scheduled on the node; existing Pods are not evicted from the node.PreferNoSchedule
: Kubernetes avoids scheduling Pods that do not tolerate this taint onto the node.NoExecute
: Pod is evicted from the node if it is already running on the node, and is not scheduled onto the node if it is not yet running on the node.
Note that some system Pods (for example, kube-proxy and fluentd) tolerate all
NoExecute
and NoSchedule
taints, and will not be evicted.
Before you begin
Before you start, make sure you have performed the following tasks:
- Ensure that you have enabled the Google Kubernetes Engine API. Enable Google Kubernetes Engine API
- Ensure that you have installed the Cloud SDK.
Set up default gcloud
settings using one of the following methods:
- Using
gcloud init
, if you want to be walked through setting defaults. - Using
gcloud config
, to individually set your project ID, zone, and region.
Using gcloud init
If you receive the error One of [--zone, --region] must be supplied: Please specify
location
, complete this section.
-
Run
gcloud init
and follow the directions:gcloud init
If you are using SSH on a remote server, use the
--console-only
flag to prevent the command from launching a browser:gcloud init --console-only
-
Follow the instructions to authorize
gcloud
to use your Google Cloud account. - Create a new configuration or select an existing one.
- Choose a Google Cloud project.
- Choose a default Compute Engine zone.
Using gcloud config
- Set your default project ID:
gcloud config set project PROJECT_ID
- If you are working with zonal clusters, set your default compute zone:
gcloud config set compute/zone COMPUTE_ZONE
- If you are working with regional clusters, set your default compute region:
gcloud config set compute/region COMPUTE_REGION
- Update
gcloud
to the latest version:gcloud components update
Creating a cluster with node taints
When you create a cluster, you can assign node taints to the cluster. This assigns the taints to all nodes created with the cluster.
gcloud
To create a cluster with node taints, run the following command:
gcloud container clusters create cluster-name \
--node-taints key=value:effect
Replace the following:
- cluster-name is the name of the cluster to be created.
- effect is
PreferNoSchedule
,NoSchedule
, orNoExecute
. - key and value form a key-value pair associated with the effect.
For example, the following command applies a taint that has key dedicated
,
value experimental
, and effect PreferNoSchedule
:
gcloud container clusters create example-cluster \
--node-taints dedicated=experimental:PreferNoSchedule
Console
To create a cluster with node taints:
Visit the Google Kubernetes Engine menu in Cloud Console.
Click add_box Create.
Configure your cluster as desired.
From the navigation pane, under the node pool you want to modify, click Metadata.
In the Node taints section, click the Add taint button.
In the Effect drop-down list, select the desired effect.
Fill Key and Value with the desired key-value pair.
Add additional node taints, if desired.
Click Create.
API
When you use the API to create a cluster, include the nodeTaints
field
under nodeConfig
. Here's an example:
POST https://container.googleapis.com/v1/projects/project-id/zones/compute-zone/clusters
{
'cluster': {
'name': 'example-cluster',
'nodeConfig': {
'nodeTaints': [
{
'key': 'special',
'Value': 'gpu',
'effect': 'PreferNoSchedule'
}
]
...
}
...
}
}
Creating a node pool with node taints
When you apply a taint to a node, only Pods that tolerate the taint are allowed to run on the node. In a GKE cluster, you can apply a taint to a node pool, which applies the taint to all nodes in the pool.
To create a node pool with node taints, you can use the gcloud
command-line
tool, Cloud Console, or the GKE API.
gcloud
To create a node pool with node taints, run the following command:
gcloud container node-pools create pool-name \
--cluster cluster-name \
--node-taints key=value:effect
Replace the following:
- pool-name is the name of the node pool to be created.
- cluster-name is the name of the cluster in which the node pool is created.
- effect is
PreferNoSchedule
,NoSchedule
, orNoExecute
. - key and value form a key-value pair associated with the effect.
For example, the following command applies a taint that has key dedicated
,
value experimental
, and effect NoSchedule
:
gcloud container node-pools create example-pool --cluster example-cluster \
--node-taints dedicated=experimental:NoSchedule
This command applies a taint that has key special
, value gpu
, and
effect NoExecute
:
gcloud container node-pools create example-pool-2 --cluster example-cluster \
--node-taints special=gpu:NoExecute
Console
To create a node pool with node taints, perform the following steps:
Visit the Google Kubernetes Engine menu in Cloud Console.
Click the cluster's Edit button, which looks like a pencil.
From Node pools click Add node pool.
In the Node taints section, click Add taint.
From Effect, select the desired effect.
Fill Key and Value with the desired key-value pair.
Add more node taints if desired.
Click Save to exit the node pool modification overlay.
Click Create.
API
When you use the API to create a node pool, include the nodeTaints
field
under nodeConfig
. Here's an example:
POST https://container.googleapis.com/v1/projects/project-id/zones/compute-zone/clusters/cluster-id/nodePools
{
'nodePool': {
'name': 'example-pool',
'nodeConfig': {
'nodeTaints': [
{
'key': 'dedicated',
'Value': 'experimental',
'effect': 'NoSchedule'
}
]
...
}
...
}
}
Configuring Pods to tolerate a taint
You can configure Pods to tolerate a taint by including the tolerations
field
in the Pods' specification. Here's a portion of a
Pod specification.
This Pod can be scheduled on a node that has the taint dedicated=experimental:NoSchedule
:
tolerations:
- key: dedicated
operator: Equal
value: experimental
effect: NoSchedule
Adding a taint to an existing node
You can add taints to an existing node using the
kubectl taint
command:
kubectl taint nodes node-name key=value:effect
For example:
kubectl taint nodes node1 key=value:NoSchedule
You can also add taints to nodes with a specific label:
kubectl taint node -l myLabel=X dedicated=foo:PreferNoSchedule
For more information, refer to Taints and tolerations in the Kubernetes documentation.
Inspecting a node's taints
To see a node's taints, you can use the kubectl
command-line tool.
Get a list of all nodes in your cluster by running the following command:
kubectl get nodes
Inspect a node by running the following command:
kubectl describe node node-name
In the returned node description, look for the
Taints
field:Taints: key=value:effect
Removing a taint from a node
You can use kubectl taint
to remove taints. You can remove taints by key,
key-value, or key-effect.
For example, the following command removes from node foo
all the taints with
key dedicated
:
kubectl taint nodes foo dedicated-
Advantages of using the GKE node taints feature
Using the GKE node taints feature with clusters and node
pools instead of setting taints manually by using kubectl
has several
advantages:
- Taints are preserved when a node is restarted or replaced.
- Taints are created automatically when a node is added to a pool or cluster.
- Taints are created automatically during cluster autoscaling.
What's next
- Learn more about node pools.
- Read the Kubernetes documentation for taints and tolerations.
- Read the
kubectl taint
documentation.