Using Preemptible VMs with Kubernetes Engine

This page provides an overview of preemptible VMs support in Kubernetes Engine.

Overview

Preemptible VMs are Google Compute Engine VM instances that last a maximum of 24 hours and provide no availability guarantees. Preemptible VMs are priced lower than standard Compute Engine VMs and offer the same machine types and options.

You can use preemptible VMs in your Kubernetes Engine clusters or node pools to run batch or fault-tolerant jobs that are less sensitive to the ephemeral, non-guaranteed nature of preemptible VMs.

To learn more about preemptible VMs, refer to Preemptible VMs in the Compute Engine documentation.

Configure gcloud to use the v1beta1 API

To use this feature with gcloud, you must enable the v1beta1 API surface for gcloud, which allows you to run gcloud beta container clusters commands.

To configure the gcloud command-line tool to use the v1beta1 API, run one of the following commands in your shell or terminal window:

export CLOUDSDK_CONTAINER_USE_V1_API_CLIENT=false
or:
gcloud config set container/use_v1_api false

How preemptible VMs work

When Kubernetes Engine clusters or node pools create Compute Engine VMs, the VMs behave like a managed instance group. Preemptible VMs in Kubernetes Engine are subject to the same limitations as preemptible instances in a managed instance group.

Additionally, these preemptible VMs are given a Kubernetes label, cloud.google.com/gke-preemptible=true. Kubernetes labels can be used in the nodeSelector field for scheduling Pods to specific nodes.

Here is an example selector for filtering preemptible VMs:

apiVersion: v1
kind: Pod
spec:
  nodeSelector:
    cloud.google.com/gke-preemptible: "true"

Creating a cluster or node pool with preemptible VMs

You can use GCP Console or gcloud beta container clusters create to create a cluster or node pool with preemptible VMs.

Console

  1. Visit the Kubernetes Engine menu in GCP Console.

    Visit the Kubernetes Engine menu

  2. Click Create cluster.

  3. Configure your cluster as desired. Then, click More.
  4. From the Preemptible nodes drop-down menu, select Enabled.
  5. Click Create.

gcloud

You can create a cluster or node pool with preemptible VMs by specifying the --preemptible flag.

To create a cluster with preemptible VMs, run the following command:

gcloud beta container clusters create [CLUSTER_NAME] --preemptible \
[--zone [COMPUTE_ZONE] --project [PROJECT_ID]]

where [COMPUTE_ZONE] is the cluster's compute zone.

To create a node pool with preemptible VMs:

gcloud beta container node-pools create [POOL_NAME] --preemptible \
--cluster [CLUSTER_NAME] [--zone [COMPUTE_ZONE] --project [PROJECT_ID]]

Using node taints to avoid scheduling to preemptible VM nodes

Depending on your use case, you might want to avoid having critical Pods scheduled on a preemptible VM node. In such cases, you can use a node taint and toleration to avoid scheduling Pods to nodes with preemptible VMs.

To add a node taint for a node with preemptible VMs, run the following command:

kubectl taint nodes [NODE_NAME] cloud.google.com/gke-preemptible="true":NoSchedule

Now, only Pods that tolerate the node taint are scheduled to the node.

To add the relevant toleration to your Pods, add the following to your Pod's specification or your object's Pod template specification:

tolerations:
- key: cloud.google.com/gke-preemptible
  operator: Equal
  value: "true"
  effect: NoSchedule

Best practices

Because preemptible VMs have no availability guarantees, you should design your system under the assumption that any or all of your Compute Engine instances might be preempted and become unavailable. There are no guarantees as to when new instances become available.

If you want to ensure that your jobs or workloads are processed even if no preemptible VMs are available, you can create both non-preemptible and preemptible node pools in your cluster.

Known Issues

What's next

Send feedback about...

Kubernetes Engine