Deploying Containers on VMs and Managed Instance Groups

This page describes how to deploy Docker images on Google Compute Engine virtual machine instances and managed instance groups.

To deploy and launch your container on a Compute Engine VM or a managed instance group, you provide a Docker image name and configure how your container should run when creating a VM or an instance template. Compute Engine will take care of the rest including supplying an up-to-date Container-Optimized OS image with Docker installed and launching your container when the VM starts up. For more information on the advantages of deploying containers on VMs, read Choosing to deploy containers on VMs and instance groups below.

For questions or to report issues and request new capabilities, please contact the Containers on Compute Engine team.

Before you begin

Choosing to deploy containers on VMs and instance groups

By deploying containers on Compute Engine, you can simplify application deployment while controlling your VM infrastructure.

  • Manage VMs that are running containers in the same way you would treat any other VM when configuring and managing your Compute Engine infrastructure.
  • Create scalable services using managed instance groups running containers, which offer features like autoscaling, autohealing, rolling updates, multi-zone deployments and load balancing.
  • Use familiar processes and tools such as the gcloud command-line tool or the Compute Engine API to manage your VMs with containers.

Alternatively, you might consider deploying to Kubernetes Engine to:

  • Run a large number of microservices
  • Have faster container startup time
  • Take advantage of Kubernetes automated orchestration, including auto upgrades, node auto repair, and autoscaling

Running each microservice on a separate VM on Compute Engine could make the operating system overhead a significant part of your cost. Kubernetes Engine allows you to deploy multiple containers and groups of containers for each VM instance, which can lead more efficient host VM utilization for microservices with a smaller footprint.

How deploying containers on Compute Engine works

The common methods of deploying software onto a Compute Engine VM instance include:

  • Deploying software on VM boot using a startup script or cloud-init.
  • Creating a custom boot disk image with software pre-installed.

Both of the above methods combine the tasks of configuring the application and setting up the host operating system environment. As the developer, you must carefully track and resolve any runtime dependencies. For example, if two applications running on a VM use different versions of the same library, you must install both versions and point to them through system variables.

A VM instance with applications deployed directly to the host operating system

You can also deploy software in a container directly onto a VM instance or to a managed instance group. Each container carries both application software and the required libraries and is isolated from the host OS applications and libraries. Containers can be easily moved between deployment environments without dealing with conflicting library versions in a container and its host OS.

A VM instance with applications deployed in a container

The following is the process for deploying a container on Compute Engine:

  1. You bundle your application and required libraries into a Docker image and publish the image to Container Registry (or publish publicly on Docker Hub or other registry).
  2. You specify a Docker image name and the docker run configuration when creating a VM instance or an instance template for a managed instance group.

Compute Engine executes the following tasks after you make the request to create a VM instance or instance template:

  1. Compute Engine creates a VM instance or an instance template using a Google-provided Container-Optimized OS image. This image includes Docker runtime and additional software, responsible for starting your container.
  2. Compute Engine stores your container settings in instance metadata under the gce-container-declaration metadata key.
  3. The Container-Optimized OS image pulls the container image from the repository and starts the container when the VM starts, using the docker run command configuration stored in the instance’s metadata.
Steps to create a VM instance or a managed instance group running a container

Limitations

  • You can only deploy one container for each VM instance. Consider Kubernetes Engine if you need to deploy multiple containers per VM instance. Contact the team if your use case requires you to deploy multiple containers on a Compute Engine instance.
  • You can only use Container-Optimized OS images with this deployment method.
  • You can only use this feature through the Google Cloud Platform Console or the gcloud command-line tool.

Preparing a container for deployment

Choose one of the methods below to make your container image accessible to Compute Engine:

  • Upload your Docker image to Google Container Registry.
  • Use any publicly available container images from Docker Hub or other registries.

Deploying a container on a new VM instance

You can deploy a container on a new VM instance using the Google Cloud Platform Console or the gcloud command line tool.

Console

The following example deploys a container from a Google-provided Nginx (gcr.io/cloud-marketplace/google/nginx1:1.12) Docker image to a VM instance. To use another Docker image, specify your desired image instead of gcr.io/cloud-marketplace/google/nginx1:1.12 in the examples below.

  1. Go to the VM instances page.

    Go to the VM instances page

  2. Click the Create instance button to create a new instance.
  3. Under the Container section, check Deploy container image.
  4. Specify a container image name under Container image and configure options to run the container if desired. For example, you can specify gcr.io/cloud-marketplace/google/nginx1:1.12 for the container image.
  5. Click Create.

gcloud

Use the gcloud beta compute instances create-with-container command:

 gcloud beta compute instances create-with-container [INSTANCE_NAME] \
     --container-image [DOCKER_IMAGE]

For example, the following command creates a new VM instance named nginx-vm which will launch and run Docker image gcr.io/cloud-marketplace/google/nginx1:1.12.

 gcloud beta compute instances create-with-container nginx-vm \
     --container-image gcr.io/cloud-marketplace/google/nginx1:1.12

Learn more about gcloud beta compute instances create-with-container command.

You must always specify a full Docker image name when using a public image from Docker Hub. For example, specify the following image name to deploy an Apache container image:

docker.io/httpd:2.4

Updating a container on a VM instance

You can update a Docker image and configuration options to run the container on a VM instance using Google Cloud Platform Console or gcloud command line tool.

When you update a VM running a container, Compute Engine performs two steps:

  • Updates container declaration on the instance. Compute Engine stores the updated container declaration in instance metadata under the gce-container-declaration metadata key.
  • Stops and restarts the instance to actuate the updated configuration, if the instance is running. If the instance is stopped, updates the container declaration and keeps the instance stopped. The VM instance will download the new image and launch the container on VM start.

Console

  1. Go to the VM instances page.

    Go to the VM instances page

  2. Click on the name of an instance that you need to update.
  3. Click Edit on the instance details page.
  4. Specify new container image and update the options to run the container as needed.
  5. Click Save and restart to save your changes. Compute Engine will save changes and restart the instance automatically to execute the update. After the VM restarts, it will download the new image and start the container with the update configuration.

gcloud

Update container declaration using the gcloud beta compute instances update-container command. For example:

gcloud beta compute instances update-container nginx-vm \
    --container-image gcr.io/cloud-marketplace/google/nginx1:1.13

This command sets the container image to gcr.io/cloud-marketplace/google/nginx1:1.13 and restarts the instance to actuate the changes. You can also update any of the properties described in Configuring Options to Run Your Container by adding corresponding flags.

Once the instance restarts, it will download the new container image and start the container with the new configuration.

Deploying a container on a managed instance group

You can deploy a container to a new managed instance group using Google Cloud Platform Console or the gcloud command line tool by following these steps:

  1. Create an instance template, based on a Docker image.
  2. Create a managed instance group from the new instance template.

Console

The following example creates an instance template that deploys a container from a Google-provided Nginx (gcr.io/cloud-marketplace/google/nginx1:1.12) Docker image to a managed instance group. To use other Docker images, specify your desired image instead of gcr.io/cloud-marketplace/google/nginx1:1.12 in the example below.

  1. Go to the Instance templates page.

    Go to the Instance Templates page

  2. Click the Create instance template button to create a new instance template.
  3. Under the Container section, check Deploy container image.
  4. Specify the Docker image name under Container image and configure options to run the container if desired. For example, you can specify gcr.io/cloud-marketplace/google/nginx1:1.12 for the container image.
  5. Click Create.

Next, create a managed instance group using the new instance template.

gcloud

Create an instance template for running Docker images using the gcloud beta compute instance-templates create-with-container command:

 gcloud beta compute instance-templates create-with-container [TEMPLATE_NAME] \
 --container-image [DOCKER_IMAGE]

You can also configure options to run your container if desired.

For example, the following command creates a new instance template with name nginx-template which includes information about the Docker image. A VM instance created from this template will launch and run the Docker image gcr.io/cloud-marketplace/google/nginx1:1.12 when the VM starts.

 gcloud beta compute instance-templates create-with-container nginx-template \
     --container-image gcr.io/cloud-marketplace/google/nginx1:1.12

Next, create a managed instance group using the new instance template.

Now that you have an instance template, you can create a managed instance group using the instance template. For example, to create a managed instance group using the gcloud tool with the nginx-template that you just created, run the following command:

gcloud compute instance-groups managed create example-group \
    --base-instance-name nginx-vm \
    --size 3 \
    --template nginx-template

Updating a managed instance group running a container

You can update a managed instance group to deploy a new version of a Docker image or a new version of the Container-Optimized OS image.

Updating a managed instance group to a new version of a container image

You can deploy a new version of a Docker image to a managed instance group using the Managed Instance Group Updater, in three steps:

  1. Prepare a new Docker image for deployment.
  2. Create an instance template, based on the new Docker image, the same way you would create a container-based template.
  3. Update a managed instance group with the new instance template using the Managed Instance Group Updater.

Updating a managed instance group to a new version of Container-Optimized OS image

Google updates Container-Optimized OS images regularly, and you might want to apply those updates to your containerized managed instance groups without changing your Docker image. You can update a managed instance group to a new version of Container-Optimized OS image using Google Cloud Platform Console or the gcloud command line tool in two steps:

  1. Create an instance template, based on the current version of your Docker image, the same way you would create a container-based template for a new managed instance group. The latest supported version of a Container-Optimized OS image will be used by default.
  2. Update a managed instance group with the new instance template using Managed Instance Group Updater.

Connecting to a container using SSH

You can connect to a container on a VM using SSH. In this case, use the gcloud beta compute ssh command instead of the standard gcloud compute ssh command. Using the gcloud tool, run the gcloud beta compute ssh command with the --container flag:

gcloud beta compute ssh [INSTANCE_NAME] \
   --container [CONTAINER_NAME]

where:

  • [INSTANCE_NAME] is the name of the VM instance.
  • [CONTAINER_NAME] is either the name of the standalone VM instance or the name of the instance template, if the instance belongs to a managed instance group.

Learn more about gcloud beta compute ssh command and its arguments.

Viewing container startup agent logs

Container startup agent logs are available in the serial console, in Stackdriver, and through the journald system service, included in the OS image.

Viewing serial console logs

Console

  1. Go to the VM instances page.

    Go to the VM instances page

  2. Select the VM instance for which you want to view startup agent logs.
  3. Under Logs, click on Serial port 1 (console) to view serial console logs.

gcloud

Use the get-serial-port-output command to view logs on the instance's serial port.

gcloud compute instances get-serial-port-output [INSTANCE_NAME]

where [INSTANCE_NAME] is the name of the VM instance.

For example, use the following command to view the serial port output of a VM instance named nginx-vm:

gcloud compute instances get-serial-port-output nginx-vm

Viewing journald logs

  1. Connect to your instance with a container using SSH.
  2. Execute the sudo journalctl command to see the VM startup and container startup logs. Use the following command to filter for container startup agent logs (konlet):

    sudo journalctl -u konlet*
    

Viewing Stackdriver logs

Console

  1. Go to the VM instances page.

    Go to the VM instances page

  2. Select the VM instance for which you want to view startup agent logs.
  3. Under Logs, click on Stackdriver Logging to view Stackdriver logs.

  4. Expand the dropdown menu and select Global resource type.

  5. Expand the All logs dropdown menu and select gcplogs-docker-driver.
  6. In the search field, type jsonPayload.instance.name:[INSTANCE_NAME] to filter logs to a specific VM. For example:

gcloud

Use the logging read command to view Stackdriver logs from the container startup agent.

gcloud logging read “resource.type=global AND jsonPayload.instance.name=[INSTANCE_NAME] AND logName=projects/[PROJECT_ID]/logs/gcplogs-docker-driver” \
    --limit 10

where:

  • [INSTANCE_NAME] is the name of the instance you want to get logs for.
  • [PROJECT_ID] is the project ID for this request.

For instance, use the following command to view the last 10 container-related Stackdriver logs of a VM instance named nginx-vm:

gcloud logging read “resource.type=global AND jsonPayload.instance.name=nginx-vm AND logName=projects/myproject/logs/gcplogs-docker-driver” \
    --limit 10

Viewing container logs

Logs from your container are available in Stackdriver. To view logs from your container in Stackdriver:

Console

  1. Go to the VM instances page.

    Go to the VM instances page

  2. Select the VM instance for which you want to view startup agent logs.
  3. Under Logs, click on Stackdriver Logging to view Stackdriver logs.

  4. Expand the dropdown menu and select Global resource type.

  5. Expand the All logs dropdown menu and select gcplogs-docker-driver.
  6. In the search field, type jsonPayload.container.name:[CONTAINER_NAME] to filter logs to a specific container. Your container is given the same name as the name of the VM instance it runs on. For example, if you created an NGINX container on an instance named nginx-vm, the container deployed to the instance would also be named nginx-vm. You would then specify jsonPayload.container.name:nginx-vm to view logs for the container:

gcloud

Use the logging read command to view Stackdriver logs from your container:

gcloud logging read “resource.type=global AND jsonPayload.container.name=/[CONTAINER_NAME] AND logName=projects/[PROJECT_ID]/logs/gcplogs-docker-driver” \
    --limit 10

where:

  • [CONTAINER_NAME] is the name of the container you want to get logs for.
  • [PROJECT_ID] is the proeject ID for this request.

For instance, use the following command to view the last 10 container-related Stackdriver logs of a VM instance named nginx-vm:

gcloud logging read “resource.type=global AND jsonPayload.container.name=/nginx-vm AND logName=projects/myproject/logs/gcplogs-docker-driver” \
    --limit 10

Note that your container has the same name as the name of the VM instance that it runs on. In this example, the name of the instance is nginx-vm. When specifying a container name in the jsonPayload.container.name filter, you must precede the container name with a /. For example, for a container name nginx-vm, specify /nginx-vm.

Specifying container-optimized images or image families

Containerized VM instances or instance templates are created using the latest supported container-optimized image by default. The image belongs to the cos-cloud project.

You can override this default with another image that is version 62 or higher, from the cos-stable, cos-beta, or cos-dev image families or with one of these image families (all in cos-cloud project).

These image families include container-optimized images from three different release channels:

  • The dev channel has all the latest changes and is updated frequently.
  • The beta channel is a qualified image in Beta and is updated less frequently.
  • The stable channel offers a longer tested and supported release with only rare and critical updates (if needed).

For example, using the gcloud tool, provide the --image flag to override the default container-optimized image or --image-family flag to pick the latest image from the specified family at VM creation time.

The following example creates a containerized VM instance using the latest image from cos-dev image family:

 gcloud beta compute instances create-with-container nginx-vm \
   --image-family cos-dev \
   --image-project cos-cloud \
   --container-image gcr.io/cloud-marketplace/google/nginx1:1.12

Configuring firewall rules

Containerized VMs launch containers with the network set to host mode. A container shares the host network stack, and all interfaces from the host are available to the container.

Google Cloud Platform firewall rules block all incoming connections to a VM instance and allow all outgoing connections from a VM instance by default.

Create firewall rules to allow incoming connections to your instance and therefore to the container.

Configuring options to run a container

You can configure the following options to run your container:

  • Specify a container restart policy.
  • Override container ENTRYPOINT (default command to be executed on container start).
  • Pass arguments to container ENTRYPOINT command.
  • Run a container in a privileged mode.
  • Mount a host directory or tmpfs as a data volume inside the container.
  • Set environment variables.
  • Allocate a buffer for STDIN in the container runtime.
  • Allocate a pseudo-TTY.

Learn more about configuring options to run your container.

Feedback and Questions

We welcome your feedback and questions! Please contact the Containers on Compute Engine team to ask questions, report issues, and request new capabilities.

What's next

Send feedback about...

Compute Engine Documentation