Edit on GitHub
Report issue
Page history

Using the Google Cloud logging driver for Docker

Author(s): @tswast ,   Published: 2017-04-24

Tim Swast | Developer Programs Engineer | Google

Contributed by Google employees.

This tutorial shows you how to use the Google Cloud logging driver to upload logs from your Docker containers to Cloud Logging.

Objectives

  • Run a Docker container configured with the Google Cloud logging driver.
  • View logs in the Cloud Console.

Before you begin

  1. Create or select a Google Cloud project from the Cloud Console projects page.
  2. Enable billing for your project.

Costs

This tutorial uses billable components of Google Cloud, including

Use the Pricing Calculator to estimate the costs for your usage.

Setting up the virtual machine

Create a new Compute Engine instance using the Container-Optimized OS stable image. Container-Optimized OS comes with Docker pre-installed and supports automatic system updates.

  1. Open the Cloud Console.
  2. Create a new Compute Engine instance.
  3. Select the desired Zone, such as "us-central1-f".
  4. Select the desired Machine series, such as "N1".
  5. Select the desired Machine type, such as "f1-micro".
  6. Change the Boot disk to "Container-Optimized OS stable".
  7. Click the Create button to create the Compute Engine instance.

Configuring the Google Cloud logging driver for a single container

  1. After the instance is created, click the SSH button to open a terminal connected to the machine.
  2. To use the Google Cloud logging driver for Docker, specify the --log-driver=gcplogs command-line argument to the docker run command.

    Run the following command to start an NGINX container which writes logs to Cloud Logging.

    docker run -d \
        --name mysite \
        --log-driver=gcplogs \
        -p 80:80 \
        nginx
    
  3. Make a request to your container, which will generate logs and push them to Cloud Logging.

    curl 127.0.0.1:80
    

Configuring the Google Cloud logging driver with Docker Compose

When using Docker Compose, specify a logging driver for each service in the docker-compose.yml configuration file. For example:

version: '3'
services:
  web:
    logging:
      driver: gcplogs
    ...
  database:
    logging:
      driver: gcplogs
    ...

Viewing your logs

Now that you are uploading logs to Cloud Logging, you can view them in the Cloud Console Logs Viewer.

  1. Open the Cloud Console Logs Viewer via that link or by opening Logging from the left menu.

    logging menu

  2. Select the Global logs.

    global logs

  3. Select the gcplogs-docker-driver label to limit to just the logs from your Docker containers.

    gcplogs docker driver logs

  4. Enter a search filter to narrow the logs further. Enter jsonPayload.container.name:nginx-proxy to limit to logs just from containers with the name nginx-proxy.

    logging container name filter

Setting the default logging driver

Instead of configuring the driver for each container, you can configure Docker to use the Google Cloud logging driver by default.

To set the Google Cloud logging driver as the default Docker logging driver, specify the --log-driver=gcplogs option in the dockerd command.

dockerd --log-driver=gcplogs

Container-Optimized OS starts Docker using systemd. To configure Docker to use the Google Cloud logging driver when it is started by systemd:

  1. Create /etc/docker/daemon.json.

    echo '{"log-driver":"gcplogs"}' | sudo tee /etc/docker/daemon.json
    
  2. Restart the docker service.

    sudo systemctl restart docker
    
  3. Test it out by running a container without specifying an explicit --log-driver.

    docker run -d --name mysite2 -p 80:80 nginx
    

    After making a few requests to this container, you should see the logs in the Logs Viewer as you did before.

Persisting configuration across reboots

On Container-Optimized OS, files in /etc/ are writable, but data does not persist across reboots. Instead, use cloud-init to configure Container-Optimized OS instances.

To configure cloud-init, update the instance metadata by writing a configuration to the user-data key.

You can write the configuration to the instance metadata from the command line or from the Cloud Console. Both methods are described in the following sections.

Writing metadata from the Cloud Console

  1. Go to the VM instances page.
  2. Edit the instance.
  3. Add a Custom metadata item with the key user-data and the value

    #cloud-config
    
    write_files:
      - path: /etc/docker/daemon.json
        content: '{"log-driver":"gcplogs"}'
    
    runcmd:
      - systemctl restart docker
    
  4. Save the changes to the instance.

  5. Reboot the instance.

  6. Verify that the /etc/docker/daemon.json file is present.

    sudo ls /etc/docker
    

Writing metadata from the command line

If you have already written the metadata using the Cloud Console, you can skip this section.

From Cloud Shell or a development machine where you have installed and initialized the Cloud SDK, use the gcloud compute intances add-metadata command to add the user-data key to your instance.

  1. Create a file instance-config.txt with the contents:

    #cloud-config
    
    write_files:
      - path: /etc/docker/daemon.json
        content: '{"log-driver":"gcplogs"}'
    
    runcmd:
      - systemctl restart docker
    
  2. Add the user-data key to your instance.

    gcloud compute instances add-metadata INSTANCE_NAME \
        --metadata-from-file user-data=./instance-config.txt
    

    Replace INSTANCE_NAME with the name of your instance.

  3. Reboot the instance.

  4. Verify that the /etc/docker/daemon.json file is present.

    sudo ls /etc/docker
    

Next steps

Submit a tutorial

Share step-by-step guides

Submit a tutorial

Request a tutorial

Ask for community help

Submit a request

View tutorials

Search Google Cloud tutorials

View tutorials

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies. Java is a registered trademark of Oracle and/or its affiliates.