Change your container runtime

This page explains how to specify which container runtime your cluster uses to run Kubernetes Pods. You specify the container runtime by setting the value of the containerRuntime field in the cluster configuration file.

You can change the value of the containerRuntime field only when upgrading or installing a cluster. This field is optional and defaults to containerd if you leave it undefined.

Kubernetes 1.24 ends support of Docker Engine

The dockershim component in Kubernetes enables cluster nodes to use the Docker Engine container runtime. However, Kubernetes 1.24 removed the dockershim component. Since Google Distributed Cloud version 1.13 will run on Kubernetes 1.24, version 1.13 and higher clusters can no longer use Docker Engine.

When upgrading or creating clusters, note the following container runtime rules:

  • You must use containerd for version 1.13 clusters and higher. You can't upgrade a 1.12 cluster that uses Docker to version 1.13.
  • You can't create version 1.12 clusters that use the Docker Engine container runtime.
  • You can upgrade a 1.11 cluster that uses Docker to a 1.12 cluster that uses Docker. However, we strongly recommend that you switch to containerd before upgrading.

The Docker installation that you use in development to create images is unrelated to the Docker Engine container runtime inside your Kubernetes cluster. You can still use Docker to create images and build application containers. Those containers will still work inside your cluster.

Docker must be installed on your admin workstation. The bmctl command requires Docker for operations, such as cluster creation. This use of Docker is also unaffected by the dockershim deprecation.

For detailed instructions on how to specify the container runtime used in your cluster, see the following sections.

Before you begin

  • Identify what container runtime you want your clusters to use. You can only change the value of the container runtime field when upgrading or installing a cluster.

  • Ensure your deployment supports containerd version 1.4.6 or later. Google Distributed Cloud installs this version over any previously installed version of containerd.

  • Ensure Google Distributed Cloud can install the following files which are installed for both runtimes:

    • Binary files

      • /usr/bin/containerd
      • /usr/bin/containerd-shim
      • /usr/bin/containerd-shim-runc-v1
      • /usr/bin/containerd-shim-runc-v2
      • /usr/bin/crictl
      • /usr/bin/ctr
      • /usr/local/sbin/runc

    • Configuration files

      • /etc/crictl.yaml
      • /etc/systemd/system/containerd.service
      • /etc/containerd/config.toml
      • /etc/containerd/certs.d/
      • /etc/systemd/system/containerd.service.d/09-proxy.conf This file is only installed if you configure an HTTP proxy.

  • Ensure Google Distributed Cloud can install the following certificates on your nodes depending on your chosen runtime:

    • For containerd:
      • /etc/docker/certs.d/
      • /etc/containerd/certs.d/
    • For Docker: /etc/docker/certs.d/

Configure the container runtime for a new cluster

To set your container runtime for a new cluster, set the containerRuntime field in the nodeConfig section under spec in your cluster configuration file. Google Distributed Cloud supports the following values:

  • containerd
  • docker

If the field is empty or not set, Google Distributed Cloud uses containerd by default. To use Docker, the entry in your configuration should look similar to the following example:

apiVersion: baremetal.cluster.gke.io/v1
kind: Cluster
metadata:
  name: cluster1
  namespace: cluster-cluster1
spec:
  nodeConfig:
    containerRuntime: docker

Configure the container runtime when upgrading a cluster

You can configure Google Distributed Cloud to use a different container runtime when you upgrade. You can only change the value of the container runtime field when upgrading or installing a cluster.

Requirements

  • Upgrade admin clusters before user clusters
  • Set the value of the containerRuntime field in your configuration file before upgrading your clusters.

Change the runtime to Docker

To set your container runtime during an upgrade, set the containerRuntime field in the nodeConfig section under spec in your cluster configuration file. Google Distributed Cloud supports the following values:

  • containerd
  • docker

To use Docker, change your configuration file to look like the following example:

apiVersion: baremetal.cluster.gke.io/v1
kind: Cluster
metadata:
  name: cluster1
  namespace: cluster-cluster1
spec:
  nodeConfig:
    containerRuntime: docker

To set the container runtime of the cluster nodes to docker as part of their upgrade, run the following command to upgrade your clusters:

bmctl upgrade cluster

Update your clusters with Docker set as the runtime

If you set the container runtime to docker, the following command updates the image registry certificates in /etc/containerd/certs.d/

bmctl update cluster

Reset your clusters that use containerd as the runtime

If you use the default container runtime, the following command doesn't delete configurations, binary files, or directories:

bmctl reset cluster

During reset the containerd service continues to run, but the containers that Kubernetes scheduled are stopped and removed from the containerd runtime.

Delete the following binary files and directories to complete the reset:

  • /etc/crictl.yaml
  • /etc/systemd/system/containerd.service
  • /etc/systemd/system/containerd.service.d/
  • /etc/containerd/config.toml
  • /etc/containerd/certs.d/

If you are using Docker as your runtime, no cleanup is performed during reset.

Check the status of the runtime

To check the status of the runtime as part of the machine health check, run the following command:

systemctl status RUNTIME

Replace RUNTIME with your chosen runtime: containerd or docker.