Deploying containers on VMs and MIGs

You can configure a VM instance or an instance template to deploy and launch a Docker container. Compute Engine supplies an up-to-date Container-Optimized OS (COS) image with Docker installed and launches your container when your VM starts up. For more information about the advantages of deploying containers on VMs, read Choosing to deploy containers on VMs and instance groups later in this topic.

Before you begin

• If you want to use the command-line examples in this guide:
  1. Install or update to the latest version of the gcloud command-line tool.
  2. Set a default region and zone.
• If you aren't familiar with Containers, read What are Containers and their benefits.
• If you aren't familiar with Docker, read the Docker documentation.
• Read about Container-Optimized OS.
• Read about Managed Instance Groups.

Choosing to deploy containers on VMs and instance groups

By deploying containers on Compute Engine, you can simplify app deployment while controlling your VM infrastructure.

• Manage VMs that are running containers in the same way you would treat any other VM when configuring and managing your Compute Engine infrastructure.
• Create scalable services using managed instance groups (MIGs) running containers, which offer features like autoscaling, autohealing, rolling updates, multi-zone deployments and load balancing.
• Use familiar processes and tools such as the gcloud command-line tool or the Compute Engine API to manage your VMs with containers.

Alternatively, you might consider deploying to Google Kubernetes Engine to:

• Run a large number of microservices
• Have faster container startup time
• Take advantage of Kubernetes automated orchestration, including auto upgrades, node auto repair, and autoscaling

Running each microservice on a separate virtual machine (VM) on Compute Engine could make the operating system overhead a significant part of your cost. Google Kubernetes Engine lets you deploy multiple containers and groups of containers for each VM instance, which can allocate host VM resources more efficiently to microservices with a smaller footprint.

How deploying containers on Compute Engine works

The common methods of deploying software onto a Compute Engine VM instance include:

• Deploying software on VM boot using a startup script or cloud-init.
• Creating a custom boot disk image with software pre-installed.

Both of the above methods combine the tasks of configuring the app and setting up the host operating system environment. As the developer, you must carefully track and resolve any runtime dependencies. For example, if two apps running on a VM use different versions of the same library, you must install both versions and point to them through system variables.

A VM instance with apps deployed directly to the host operating system

You can also deploy software in a container directly onto a VM instance or to a managed instance group. Each container carries both app software and the required libraries and is isolated from the host OS apps and libraries. Containers can be easily moved between deployment environments without dealing with conflicting library versions in a container and its host OS.

A VM instance with apps deployed in a container

The following process describes how you deploy a container on Compute Engine:

1. You bundle your app and required libraries into a Docker image and publish the image to Container Registry (or publish publicly on Docker Hub or other registry).
2. You specify a Docker image name and the docker run configuration when creating a VM instance or an instance template for a managed instance group.

Compute Engine executes the following tasks after you make the request to create a VM instance or instance template:

1. Compute Engine creates a VM instance or an instance template by using a Google-provided Container-Optimized OS image. This image includes a Docker runtime and additional software that is responsible for starting your container.
2. Compute Engine stores your container settings in instance metadata under the gce-container-declaration metadata key.
3. The Container-Optimized OS image pulls the container image from the repository and starts the container when the VM starts, using the docker run command configuration stored in the instance's metadata.

/>

Steps to create a VM instance or a managed instance group running a container

Limitations

• You can only deploy one container for each VM instance. Consider Google Kubernetes Engine if you need to deploy multiple containers per VM instance.
• You can only deploy containers from a public repository or from a private repository at Container Registry. Other private repositories are currently not supported.
• You can't map a VM instance's ports to the container's ports (Docker's -p option).
• You can only use Container-Optimized OS images with this deployment method.
• You can only use this feature through the Google Cloud Console or the gcloud command-line tool, not the API.

Preparing a container for deployment

Choose one of the methods below to make your container image accessible to Compute Engine:

• Upload your Docker image to Container Registry.
• Use any publicly available container images from Docker Hub or other registries.

Note: Consider running your container image builds on Cloud Build, with none of the overhead of managing your own build servers.

Deploying a container on a new VM instance

You can deploy a container on a new VM instance by using the Google Cloud Console or the gcloud command-line tool.

Console

The following example deploys a container from a Google-provided Nginx Docker image, https://gcr.io/cloud-marketplace/google/nginx1:latest, to a VM instance. To use another Docker image, specify your desired image instead in the examples below.

1. Go to the VM instances page.

   Go to the VM instances page

2. Click the Create instance button to create a new instance.
3. Under the Container section, check Deploy container image.
4. Specify a container image name under Container image and configure options to run the container if desired. For example, you can specify gcr.io/cloud-marketplace/google/nginx1:latest for the container image.
5. Click Create.

gcloud

Use the gcloud compute instances create-with-container command:

 gcloud compute instances create-with-container [INSTANCE_NAME] \
     --container-image [DOCKER_IMAGE]

For example, the following command creates a new VM instance named nginx-vm, which launches and runs the Docker image gcr.io/cloud-marketplace/google/nginx1:latest.

 gcloud compute instances create-with-container nginx-vm \
     --container-image gcr.io/cloud-marketplace/google/nginx1:latest

Learn more about gcloud compute instances create-with-container command.

You must always specify a full Docker image name when using a public image from Docker Hub. For example, specify the following image name to deploy an Apache container image:

docker.io/httpd:2.4

Updating a container on a VM instance

You can update a Docker image and configuration options to run the container on a VM instance using Google Cloud Console or the gcloud command-line tool.

When you update a VM running a container, Compute Engine performs two steps:

• Updates container declaration on the instance. Compute Engine stores the updated container declaration in instance metadata under the gce-container-declaration metadata key.
• Stops and restarts the instance to actuate the updated configuration, if the instance is running. If the instance is stopped, updates the container declaration and keeps the instance stopped. The VM instance downloads the new image and launches the container on VM start.

Note: If you specify a Docker image labeled with latest, the VM instance downloads the latest image and launches a container from the new image each time the VM starts. Do not use the latest label in production if you depend on a specific image version.

Console

1. Go to the VM instances page.

   Go to the VM instances page

2. Click the name of an instance that you need to update.
3. Click Edit on the instance details page.
4. Specify the new container image and update the options to run the container as needed.
5. Click Save and restart to save your changes. Compute Engine saves the changes and restarts the instance automatically to make the update. After the VM restarts, it downloads the new image and starts the container with the updated configuration.

gcloud

Update the container declaration by using the gcloud compute instances update-container command. For example:

gcloud compute instances update-container nginx-vm \
    --container-image gcr.io/cloud-marketplace/google/nginx1:latest

This command sets the container image to gcr.io/cloud-marketplace/google/nginx1:latest and restarts the instance to actuate the changes. You can also update any of the properties described in Configuring options to run your container by adding corresponding flags.

After the instance restarts, it downloads the new container image and starts the container with the new configuration.

Deploying a container on a managed instance group

You can deploy a container to a new managed instance group using Google Cloud Console or the gcloud command-line tool by following these steps:

1. Create an instance template that is based on a Docker image.

   Note: To maintain identical instances in your group, include a specific Docker image version in your instance template, such as nginx1:15. See Deterministic instance templates for more information.

2. Create a managed instance group from the new instance template.

Console

The following example creates an instance template that deploys a container from a Google-provided Nginx (gcr.io/cloud-marketplace/google/nginx1:15) Docker image to a managed instance group. To use other Docker images, specify your desired image instead of gcr.io/cloud-marketplace/google/nginx1:15 in the example below.

1. Go to the Instance templates page.

   Go to the Instance Templates page

2. Click the Create instance template button to create a new instance template.
3. Under the Container section, check Deploy container image.
4. Specify the Docker image name under Container image and configure options to run the container if desired. For example, you can specify gcr.io/cloud-marketplace/google/nginx1:15 for the container image.
5. Click Create.

Next, create a managed instance group using the new instance template.

gcloud

Create an instance template for running Docker images using the gcloud compute instance-templates create-with-container command:

 gcloud compute instance-templates create-with-container [TEMPLATE_NAME] \
 --container-image [DOCKER_IMAGE]

You can also configure options to run your container if desired.

For example, the following command creates a new instance template with name nginx-template, which includes information about the Docker image. A VM instance created from this template launches and runs the Docker image gcr.io/cloud-marketplace/google/nginx1:15 when the VM starts.

 gcloud compute instance-templates create-with-container nginx-template \
     --container-image gcr.io/cloud-marketplace/google/nginx1:15

Next, create a managed instance group using the new instance template.

Now that you have an instance template, you can create a managed instance group that uses the instance template. For example, to create a managed instance group using the gcloud tool with the nginx-template that you just created, run the following command:

gcloud compute instance-groups managed create example-group \
    --base-instance-name nginx-vm \
    --size 3 \
    --template nginx-template

Updating a managed instance group running a container

Beta

This feature is in a pre-release state and might change or have limited support. For more information, see the product launch stages.

You can update a managed instance group to deploy a new version of a Docker image or a new version of the Container-Optimized OS image.

Updating a managed instance group to a new version of a container image

You can deploy a new version of a Docker image to a managed instance group by using the Managed Instance Group Updater, in three steps:

1. Prepare a new Docker image for deployment.
2. Create an instance template based on the new Docker image in the same way you create a container-based template.
3. Update a managed instance group to the new instance template by using the Managed Instance Group Updater.

Updating a managed instance group to a new version of Container-Optimized OS image

Google updates Container-Optimized OS images regularly, and you might want to apply those updates to your containerized managed instance groups without changing your Docker image. You can update a managed instance group to a new version of a Container-Optimized OS image by using Google Cloud Console or the gcloud command-line tool in two steps:

1. Create an instance template based on the current version of your Docker image, the same way you create a container-based template for a new managed instance group. The latest supported version of a Container-Optimized OS image is used by default.
2. Update a managed instance group with the new instance template by using Managed Instance Group Updater.

Connecting to a container using SSH

You can connect to a container on a VM using SSH. Use the gcloud tool to run gcloud compute ssh with the --container flag:

gcloud compute ssh [INSTANCE_NAME] --container [CONTAINER_NAME]

where:

• [INSTANCE_NAME] is the name of the VM instance.
• [CONTAINER_NAME] is the name of the container.

Learn more about the gcloud compute ssh command and its arguments.

Viewing logs

You can view three types of logs related to containers.

1. Startup agent logs, also known as konlet logs. The startup agent parses the container's configuration and runs tasks to start the container on a Compute Engine VM instance.

2. Docker event logs report container events, including container start and stop events. These logs are available if you use COS 69 or higher.

3. Logs from your container include the STDOUT from apps that run in your container.

Viewing startup agent logs

Startup agent logs are available in the serial console, through the journald system service included in the OS image, and through Cloud Logging.

Viewing startup agent logs in the serial console

Console

1. Go to the VM instances page.

   Go to the VM instances page

2. Select the VM instance for which you want to view startup agent logs.

3. Under Logs, click Serial port 1 (console) to view serial console logs.

   

gcloud

Use the get-serial-port-output command to view logs on the instance's serial port.

gcloud compute instances get-serial-port-output [INSTANCE_NAME]

where [INSTANCE_NAME] is the name of the VM instance.

For example, use the following command to view the serial port output of a VM instance named nginx-vm:

gcloud compute instances get-serial-port-output nginx-vm

Viewing startup agent logs in journald

1. Connect to your instance with a container using SSH.

2. Execute the sudo journalctl command to see the VM startup and container startup logs. Use the following command to filter for container startup agent logs (konlet).

   sudo journalctl -u konlet*
   

Viewing startup agent logs in Logging

Console

1. In the Google Cloud Console, go to the VM instances page.

   Go to the VM instances page

2. Select the VM instance for which you want to view startup agent logs.

3. Under Logs, click Stackdriver Logging to view Cloud Logging logs.

   

4. Enter a search filter to retrieve startup agent logs.

   If you use COS 68 or lower, use the following filter.

   resource.type="global"
   logName="projects/[PROJECT_ID]/logs/gcplogs-docker-driver"
   jsonPayload.container.imageName:"gcr.io/gce-containers/konlet"
   jsonPayload.instance.name="[INSTANCE_NAME]"
   

   If you use COS 69 or higher, use the following filter.

   resource.type="gce_instance"
   logName="projects/[PROJECT_ID]/logs/cos_system"
   jsonPayload.SYSLOG_IDENTIFIER="konlet-startup"
   jsonPayload._HOSTNAME="[INSTANCE_NAME]"
   

   Where:

   • [PROJECT_ID] is the project ID that contains the instance.
   • [INSTANCE_NAME] is the name of the instance you want to get logs for.

gcloud

Use the gcloud logging read command with an appropriate filter to view container startup agent logs.

If you use COS 68 or lower, use the following command and filter.

gcloud logging read "resource.type=global AND \
    logName=projects/[PROJECT_ID]/logs/gcplogs-docker-driver AND \
    jsonPayload.instance.name=[INSTANCE_NAME]"

If you use COS 69 or higher, use the following command and filter.

gcloud logging read "resource.type=gce_instance AND \
    logName=projects/[PROJECT_ID]/logs/cos_system AND \
    jsonPayload.SYSLOG_IDENTIFIER=konlet-startup AND \
    jsonPayload._HOSTNAME=[INSTANCE_NAME]"

Where:

• [PROJECT_ID] is the project ID that contains the instance.
• [INSTANCE_NAME] is the name of the instance you want to get logs for.

For example, use the following command to view the last 10 startup agent logs in Logging for a VM instance named nginx-vm that's running COS 70 and that exists in my-project.

gcloud logging read "resource.type=gce_instance AND \
    logName=projects/my-project/logs/cos_system AND \
    jsonPayload.SYSLOG_IDENTIFIER=konlet-startup AND \
    jsonPayload._HOSTNAME=nginx-vm" \
    --limit 10

Viewing Docker event logs

With COS 69 or higher, you can view Docker event logs in journald and in Cloud Logging.

Viewing Docker event logs in journald

1. Connect to your instance with a container using SSH.

2. Execute the sudo journalctl command with the following filter to see Docker event logs.

   sudo journalctl -u docker-events-collector
   

Viewing Docker event logs in Logging

Console

1. Go to the VM instances page.

   Go to the VM instances page

2. Select the VM instance for which you want to view startup agent logs.

3. Under Logs, click Stackdriver Logging to view Cloud Logging logs.

   

4. Enter the following search filter to retrieve Docker event logs.

   resource.type="gce_instance"
   logName="projects/[PROJECT_ID]/logs/cos_system"
   jsonPayload._HOSTNAME="[INSTANCE_NAME]"
   jsonPayload.SYSLOG_IDENTIFIER="docker"
   

   Where:

   • [PROJECT_ID] is the project ID that contains the instance.
   • [INSTANCE_NAME] is the name of the instance you want to get logs for.

gcloud

Use the gcloud logging read command with an appropriate filter to view Docker event logs.

gcloud logging read "resource.type=gce_instance AND \
      logName=projects/[PROJECT_ID]/logs/cos_system AND \
      jsonPayload._HOSTNAME=[INSTANCE_NAME] AND \
      jsonPayload.SYSLOG_IDENTIFIER=docker"

Where:

• [PROJECT_ID] is the project ID that contains the instance.
• [INSTANCE_NAME] is the name of the instance you want to get logs for.

For example, use the following command to view the last 10 Docker event logs in Logging for a VM instance named nginx-vm that's running COS 70 and that exists in my-project.

gcloud logging read "resource.type=gce_instance AND \
    logName=projects/my-project/logs/cos_system AND \
    jsonPayload._HOSTNAME=nginx-vm AND \
    jsonPayload.SYSLOG_IDENTIFIER=docker" \
    --limit 10

Viewing container logs

Console

1. Go to the VM instances page.

   Go to the VM instances page

2. Select the VM instance for which you want to view startup agent logs.

3. Under Logs, click Stackdriver Logging to view Cloud Logging logs.

   

4. The Cloud Logging page loads with a default search filter. If you are using COS 69 or higher, examine the filter and copy the value for resource.labels.instance_id. You will use it later.

5. Update the search filter to retrieve container logs.

   If you use COS 68 or lower, use the following filter.

   logName="projects/[PROJECT_ID]/logs/gcplogs-docker-driver"
   jsonPayload.container.name="/[INSTANCE_NAME]"
   

   Where:

   • [PROJECT_ID] is the project ID that contains the instance.
   • [INSTANCE_NAME] is the name of the instance you want to get logs for.

   If you use COS 69 or higher, use the following filter.

   resource.type="gce_instance"
   logName="projects/[PROJECT_ID]/logs/cos_containers"
   resource.labels.instance_id="[INSTANCE_ID]"
   

   Where:

   • [PROJECT_ID] is the project ID that contains the instance.
   • [INSTANCE_ID] is the ID of the instance that you want to get logs for.

gcloud

Use the gcloud logging read command with an appropriate filter to view container logs.

If you use COS 68 or lower, use the following command and filter.

gcloud logging read "logName=projects/[PROJECT_ID]/logs/gcplogs-docker-driver AND \
    jsonPayload.container.name=/[INSTANCE_NAME]"

If you use COS 69 or higher:

1. Determine the ID for the instance that you want to get logs for.

   gcloud compute instances describe [INSTANCE_NAME] \
       --zone [ZONE] \
       --format="value(id)"
   

   Where:

   • [ZONE] is the zone where the instance is located.
   • [INSTANCE_NAME] is the name of the instance you want to get logs for.

2. Use the following command and filter to view the instance's container logs.

   gcloud logging read "resource.type=gce_instance AND \
       logName=projects/[PROJECT_ID]/logs/cos_containers AND \
       resource.labels.instance_id=[INSTANCE_ID]"
   

   Where:

   • [PROJECT_ID] is the project ID that contains the instance.
   • [INSTANCE_ID] is the ID of the instance.

   For example, use the following command to view the last 10 container logs in Cloud Logging for a VM instance that is running COS 70, that exists in my-project, and that has an instance ID of 555123456789012345.

   gcloud logging read "resource.type=gce_instance AND \
       logName=projects/my-project/logs/cos_containers AND \
       resource.labels.instance_id=555123456789012345" \
       --limit 10
   

Specifying container-optimized images or image families

Containerized VM instances or instance templates are created to use the latest supported container-optimized image by default. The image belongs to the cos-cloud project.

You can override this default with another image that is version 62 or higher, from the cos-stable, cos-beta, or cos-dev image families, or with one of these image families (all in cos-cloud project).

These image families include container-optimized images from three different release channels:

• The dev channel has all the latest changes and is updated frequently.
• The beta channel is a qualified image that is in beta and is updated less frequently.
• The stable channel offers a longer tested and supported release with only rare and critical updates (if needed).

Note: Read about container-optimized image release channels and update schedule and the container-optimized image support policy.

For example, using the gcloud tool, provide the --image flag to override the default container-optimized image or the --image-family flag to pick the latest image from the specified family at VM creation time.

The following example creates a containerized VM instance that uses the latest image from the cos-dev image family:

 gcloud compute instances create-with-container nginx-vm \
   --image-family cos-dev \
   --image-project cos-cloud \
   --container-image gcr.io/cloud-marketplace/google/nginx1:1.15

Configuring firewall rules

Containerized VMs launch containers whose network is set to host mode. A container shares the host network stack, and all interfaces from the host are available to the container.

Note: A VM instance starts a container with the --network="host" flag of the docker run command. Learn about container network settings and host mode.

By default, Google Cloud firewall rules block all incoming connections to a VM instance and allow all outgoing connections from a VM instance.

Create firewall rules to allow incoming connections to your instance and therefore to the container.

Note: This method of VM creation overrides the default host firewall configuration of container-optimized images and opens all ports for incoming TCP connections. The ports are still not accessible from outside the VM because of the default firewall settings of the Virtual Private Cloud (VPC) network.

Configuring options to run a container

You can configure the following options to run your container:

• Specify a container restart policy.
• Override container ENTRYPOINT (default command to be executed on container start).
• Pass arguments to container ENTRYPOINT command.
• Run a container in a privileged mode.
• Mount a host directory or tmpfs as a data volume inside the container.
• Set environment variables.
• Allocate a buffer for STDIN in the container runtime.
• Allocate a pseudo-TTY.

Learn more about configuring options to run your container.

What's next

• Learn about configuring options to run your container.
• Learn more about managed instance groups.
• Learn about Container-Optimized OS.