Containers on Compute Engine

Software containers are a convenient way to run your apps in multiple isolated user-space instances. You can run containers on Linux or Windows Server public VM images, or on a Container-Optimized OS image. Containers let your apps run with fewer dependencies on the host virtual machine (VM) and run independently from other containerized apps that you deploy to the same VM instance. These characteristics make containerized apps more portable, easier to deploy, and easier to maintain at scale.

This document describes some of the more common container technologies that you can use to run containers on Compute Engine instances. You can use these technologies on most of the public VM images that Compute Engine provides.

Run containers on Compute Engine when you need complete control over your container environment and your container orchestration tools.

Alternatively, you can use Google Kubernetes Engine (GKE) to simplify cluster management and container orchestration tasks so that you don't need to manage the underlying VM instances. GKE provides a managed environment for deploying, managing, and scaling your containerized applications using Google infrastructure.

The GKE environment consists of multiple machines (specifically, Compute Engine instances) grouped together to form a cluster. When you run a GKE cluster, you gain the benefit of advanced cluster management features such as load-balancing, node pools, node auto-repair, automatic scaling and upgrades, and logging and monitoring that Google Cloud provides.

Learn how to create a GKE cluster with node pools running Microsoft Windows Server.

Try it for yourself

If you're new to Google Cloud, create an account to evaluate how Compute Engine performs in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

Try Compute Engine free

Container technologies that run on Compute Engine

In general, Compute Engine instances can run almost any container technology or tool. You can run several different types of containers on modern Linux operating systems. You can also run Docker on Windows Server 2016 or later. The following list includes several common tools that you can use to run and manage containerized apps:

  • Docker and Podman are two popular container technologies that let you run containerized apps.
  • Kubernetes is a container orchestration platform that you can use to manage and scale your running containers across multiple instances or within a hybrid-cloud environment.
  • Containers on Compute Engine provide an easy way to deploy containers to Compute Engine VM instances or managed instance groups.
  • You can convert your existing systems into LXD images and run them within Compute Engine VM instances for a lift-and-shift migration solution. LXD runs on Ubuntu images.

Additionally, you can use Artifact Registry to manage container image versions. Artifact Registry serves as a central location to store and manage your container images before you deploy those images to Kubernetes on Compute Engine or to Google Kubernetes Engine clusters.

Container-optimized VM images

Compute Engine provides several public VM images that you can use to create instances and run your container workloads. Some of these public VM images have a minimalistic container-optimized operating system that includes newer versions of Docker, Podman, or Kubernetes preinstalled. The following public image families are designed specifically to run containers:

  • Container-Optimized OS from Google
    • Includes: Docker, Kubernetes
    • Image project: cos-cloud
    • Image family: cos-stable
  • Fedora CoreOS
    • Includes: Podman, Docker
    • Image project: fedora-coreos-cloud
    • Image family: fedora-coreos-stable
  • Ubuntu
    • Includes: LXD
    • Image project: ubuntu-os-cloud
    • Image family: ubuntu-2004-lts

If you need to run specific container tools and technologies on images that do not include them by default, install those technologies manually.

Installing container technologies on your instances

To launch a single container on an instance, you can specify a container image when you create an instance. Compute Engine automatically supplies an up-to-date Container-Optimized OS image with Docker installed and launches your container when the VM starts up. For more information, see Deploying containers on VMs .

Alternatively, you can run your container workloads on Compute Engine using whatever container technologies and orchestration tools that you need. You can create an instance from a public VM image and then install the container technologies that you want. For example:

In some situations, you might require specific versions of these technologies to ensure that they operate together correctly. For example, Kubernetes usually runs best with specific versions of Docker. Typically, you can install the latest versions of these technologies for the best result.

Installing Docker on Windows Server images

Windows Server 2016 and later versions include container support. If you plan to run Docker containers on a Windows Server instance, you can either install Docker on a Windows Server base image or use marketplace images from Mirantis.

If you want to install Docker CE on the Windows Server base image, then follow the steps outlined below.

Start by creating a Windows Server instance using a Windows Server 2019 or later public image. For the best container support, we recommend that you use the most recent LTSC version of Windows Server. For more information about LTSC, see Windows Server servicing channels.

Install Docker

Install Docker on Windows. For more information, see Prep Windows for containers. Restart the instance after the installation is complete.

Additional setup steps

At this point you can use Docker to run containers in the instance. For example, the following command downloads the Windows nanoserver container image and runs a command prompt inside a nanoserver container:

PS C:\> docker run -it mcr.microsoft.com/windows/nanoserver:1809 cmd.exe

Docker's default network MTU is 1500 bytes. If you have configured your VPC network MTU to 1500 bytes as well, you can ignore the rest of this section. However, if you are using the default VPC MTU of 1460 bytes in your network, you have to do some additional configuration on each instance.

Set the MTU for all network interfaces (both Ethernet and vEthernet) to 1460 by running the following commands in a PowerShell terminal on each instance:

PS C:\> Get-NetAdapter | Where-Object Name -like "*Ethernet*" | ForEach-Object {
  & netsh interface ipv4 set subinterface $_.InterfaceIndex mtu=1460 store=persistent
}

PS C:\> netsh interface ipv4 show subinterfaces
   MTU  MediaSenseState   Bytes In  Bytes Out  Interface
------  ---------------  ---------  ---------  -------------
4294967295            1          0          0  Loopback Pseudo-Interface 1
  1460                1     306804     668688  Ethernet
  1460                1          0       1282  vEthernet (nat)

Even after changing the instance's MTU, connectivity from containers to the internet might be unstable because, by default, the container's network interface also uses an MTU of 1500. For commands to set the MTU correctly for every container, see the container MTU section.

You might need to periodically re-execute these MTU commands as you configure Docker networking. For full details, see the known issues section.

Running Windows containers

There are many resources available for getting started with Windows containers:

  • Microsoft provides extensive Windows containers documentation.

  • Docker Hub can be used as a repository for storing and pulling Windows containers.

Known issues with Windows containers

Deprecation of Windows Server 2019 for Containers images

Google offered a Windows Server image family with the Mirantis Container Runtime (previously Docker EE) pre-installed:

  • Windows
    • Includes: Docker
    • Image project: windows-cloud
    • Image family: windows-2019-core-for-containers

In early 2023, Microsoft ceased distributing and supporting the Mirantis Container Runtime for Windows Server. While Google was able to continue publishing this image family until October 30, 2023, the image family is now deprecated, and all images within the family have been marked obsolete.

Virtual machines based on these images created prior to October 30, 2023 will continue to run without interruption. However, instance creation for new instances based on these images will fail after that date.

Custom images derived from these images prior to October 30, 2023 will continue to function. However, custom images will not automatically receive any Patch Tuesday Windows updates, nor will they receive updated versions of the Mirantis Container Runtime.

If you want to continue using images with the Mirantis Container Runtime pre-installed, Mirantis Inc offers images on Google Cloud Marketplace. The cost of these VM images includes support and licensing for Mirantis Container Runtime directly from Mirantis. Customers interested in using the standalone Mirantis Container Runtime can also download it directly from the Mirantis website.

If you want to migrate to an alternative container runtime like Docker CE, please follow the instructions above.

Containers are incompatible across Windows versions

Containers built on earlier versions of Windows don't work in Compute Engine instances running more recent versions of Windows. Docker pulls the Windows Server 2019 version of a container by default. This means that running the following command in an instance running Windows Server version 1709 or newer results in an error:

PS C:\> docker run -it mcr.microsoft.com/windows/nanoserver cmd.exe
docker: Error response from daemon: container
9a1eb8bbcba4e91792be65f3c40b5a1aee062f02fbc60a78444b47d043438069 encountered an
error during CreateContainer: failure in a Windows system call: The operating
system of the container does not match the operating system of the host.
(0xc0370101)

Microsoft's Windows container version compatibility page contains more information. To work around Windows container version incompatibilities, specify the tag corresponding to your Windows version when pulling and running containers. For example, in a Windows Server, version 20H2 instance, use the following command to run a command prompt in the version 20H2 nanoserver container instead of the default 2019 LTSC (1809) container:

PS C:\> docker run -it mcr.microsoft.com/windows/nanoserver:1809 cmd.exe

MTU incompatibilities affect instance and container connectivity

When you create a container network on a Windows instance using the docker network create or New-VMSwitch commands, the MTU of the instance's network interface is typically forced to 1500. The default network interface inside of a new Docker container also typically uses an MTU of 1500. If your VPC network has an MTU of 1460, you might experience the following issues:

  • The RDP session can stop and you might be unable to reconnect. This is known to happen when creating a transparent container network.

  • DNS resolution inside the container might fail.

  • DNS resolution is successful, but establishing an HTTP connection from the container to the internet might fail.

The recommended workaround for these limitations requires two steps: setting the MTU for the instance's network interfaces to 1460 and setting the MTU for the container network interfaces to 1460. Alternatively, you can set the MTU for the VPC to 1500, but this requires stopping or migrating all of your VMs.

1. Setting the MTU for the Windows instance's network interfaces

Run the following command in a PowerShell terminal on the Windows instance to set the MTU for all network interfaces (both Ethernet and vEthernet):

PS C:\> Get-NetAdapter | Where-Object Name -like "*Ethernet*" | ForEach-Object {
  & netsh interface ipv4 set subinterface $_.InterfaceIndex mtu=1460 store=persistent
}

Check that the instance's Ethernet and vEthernet interface MTUs are set to 1460 using this command:

PS C:\> netsh interface ipv4 show subinterfaces
   MTU  MediaSenseState   Bytes In  Bytes Out  Interface
------  ---------------  ---------  ---------  -------------
4294967295            1          0          0  Loopback Pseudo-Interface 1
  1460                1  628295912    2613170  Ethernet
  1460                1      37793     223909  vEthernet (nat)

If you are unable to run these commands because you can no longer connect to an instance by using RDP, you can connect to the instance through the serial console, start a cmd prompt and run the netsh commands there to repair the MTU. To avoid having to do this, we recommend executing any docker network ... or New-VMSwitch commands as part of a script that also executes the MTU repair command.

2. Setting the MTU for the Windows container network interfaces

The MTU for a Windows container must be set while the container is running, either from inside the container or from the instance hosting the container. If PowerShell is available in your container, you can run this command interactively or from a script in the container to correctly set the MTU:

PS C:\> Get-NetAdapter | Where-Object Name -like "vEthernet*" | ForEach-Object {
  & netsh interface ipv4 set subinterface $_.InterfaceIndex mtu=1460 store=persistent
}

Or, you can run this command on the Windows instance to set the MTU for all running containers:

PS C:\> Get-NetIPInterface -IncludeAllCompartments |
  Where-Object InterfaceAlias -like "vEthernet*" |
  Set-NetIPInterface -IncludeAllCompartments -NlMtuBytes 1460

Hyper-V containers fail to start

Hyper-V containers are not supported on Compute Engine at this time.

What's next