Tim Swast | Developer Programs Engineer | Google

Contributed by Google employees.

This tutorial shows you how to use the Google Cloud Logging driver to upload logs from your Docker containers to Cloud Logging.

• Run a Docker container configured with the Google Cloud Logging driver.
• View logs in the Cloud Console.

1. Create or select a Google Cloud project from the Cloud Console projects page.
2. Enable billing for your project.

This tutorial uses billable components of Google Cloud, including

• Compute Engine
• Cloud Logging

Use the pricing calculator to estimate the costs for your usage.

Create a new Compute Engine instance using the Container-Optimized OS stable image. Container-Optimized OS comes with Docker pre-installed and supports automatic system updates.

1. Open the Cloud Console.
2. Create a new Compute Engine instance.
3. Select the desired Zone, such as us-central1-f.
4. Select the desired Machine series, such as N1.
5. Select the desired Machine type, such as f1-micro.
6. Change the Boot disk to Container-Optimized OS stable.
7. Click Create to create the Compute Engine instance.

1. After the instance is created, click the SSH button to open a terminal connected to the machine.

2. To use the Google Cloud logging driver for Docker, specify the --log-driver=gcplogs command-line argument to the docker run command.

   Run the following command to start an NGINX container which writes logs to Cloud Logging:

   docker run -d \
       --name mysite \
       --log-driver=gcplogs \
       -p 80:80 \
       nginx
   

3. Make a request to your container, which will generate logs and push them to Cloud Logging:

   curl 127.0.0.1:80
   

When using Docker Compose, specify a logging driver for each service in the docker-compose.yml configuration file. For example:

version: '3'
services:
  web:
    logging:
      driver: gcplogs
    ...
  database:
    logging:
      driver: gcplogs
    ...

Now that you are uploading logs to Cloud Logging, you can view them in the Cloud Console Logs Viewer.

1. Open the Cloud Console Logs Viewer.

2. In the GCE VM instance menu, choose your instance, and then select the gcplogs-docker-driver label to limit to just the logs from your Docker containers.

3. Enter a search filter to narrow the logs further. Enter jsonPayload.container.name:nginx-proxy to limit to logs just from containers with the name nginx-proxy.

Instead of configuring the driver for each container, you can configure Docker to use the Google Cloud Logging driver by default.

To set the Google Cloud Logging driver as the default Docker logging driver, specify the --log-driver=gcplogs option in the dockerd command.

dockerd --log-driver=gcplogs

Container-Optimized OS starts Docker using systemd. To configure Docker to use the Google Cloud Logging driver when it is started by systemd, do the following:

1. Create /etc/docker/daemon.json:

   echo '{"log-driver":"gcplogs"}' | sudo tee /etc/docker/daemon.json
   

2. Restart the docker service:

   sudo systemctl restart docker
   

3. Test it out by running a container without specifying an explicit --log-driver:

   docker run -d --name mysite2 -p 80:80 nginx
   

   After making a few requests to this container, you should see the logs in the Logs Viewer as you did before.

On Container-Optimized OS, files in /etc/ are writable, but data does not persist across reboots. Instead, use cloud-init to configure Container-Optimized OS instances.

To configure cloud-init, update the instance metadata by writing a configuration to the user-data key.

You can write the configuration to the instance metadata from the command line or from the Cloud Console. Both methods are described in the following sections.

1. Go to the VM instances page.

2. Edit the instance.

3. Add a Custom metadata item with the key user-data and the value:

   #cloud-config
   
   write_files:
     - path: /etc/docker/daemon.json
       content: '{"log-driver":"gcplogs"}'
   
   runcmd:
     - systemctl restart docker
   

4. Save the changes to the instance.

5. Reboot the instance.

6. Verify that the /etc/docker/daemon.json file is present:

   sudo ls /etc/docker
   

If you have already written the metadata using the Cloud Console, you can skip this section.

From Cloud Shell or a development machine where you have installed and initialized the Cloud SDK, use the gcloud compute instances add-metadata command to add the user-data key to your instance.

1. Create a file instance-config.txt with the following contents:

   #cloud-config
   
   write_files:
     - path: /etc/docker/daemon.json
       content: '{"log-driver":"gcplogs"}'
   
   runcmd:
     - systemctl restart docker
   

2. Add the user-data key to your instance:

   gcloud compute instances add-metadata INSTANCE_NAME \
       --metadata-from-file user-data=./instance-config.txt
   

   Replace INSTANCE_NAME with the name of your instance.

3. Reboot the instance.

4. Verify that the /etc/docker/daemon.json file is present:

   sudo ls /etc/docker
   

• Install the Cloud Logging agent to stream system logs to Cloud Logging.
• Try out some other DevOps tutorials.