Edit on GitHub
Report issue
Page history

Author(s): @{[ username ]}  Published: {[ TutorialCtrl.tutorial.date_published | date:'mediumDate' ]}

title: Using the Google Cloud Logging Driver for Docker description: Learn to use the gcplogs logging driver to save your Docker logs to Stackdriver Logging. author: tswast tags: Docker, Logging date_published: 2017-04-24


This tutorial shows you how to use the Google Cloud logging driver to upload logs from your Docker containers to Stackdriver Logging.

Objectives

  • Run a Docker container configured with the Google Cloud logging driver.
  • View logs in the Google Cloud Platform console.

Before you begin

  1. Create or select a Cloud Platform project from the Google Cloud Platform console's projects page.
  2. Enable billing for your project.

Costs

This tutorial uses billable components of Cloud Platform including

Use the Pricing Calculator to estimate the costs for your usage.

Setting up the virtual machine

Create a new Compute Engine instance using the Container-Optimized OS stable image. Container-Optimized OS comes with Docker pre-installed and supports automatic system updates.

  1. Open the Google Cloud Platform console.
  2. Create a new Compute Engine instance.
  3. Select the desired Zone, such as "us-central1-f".
  4. Select the desired Machine type, such as "micro" (f1-micro).
  5. Change the Boot disk to "Container-Optimized OS stable".
  6. Click the Create button to create the Compute Engine instance.

Configuring the Google Cloud logging driver for a single container

  1. After the instance is created, click the SSH button to open a terminal connected to the machine.
  2. To use the Google Cloud logging driver for Docker, specify the --log-driver=gcplogs command-line argument to the docker run command.

    Run the following command to start an NGINX container which writes logs to Stackdriver Logging.

    docker run -d --name mysite --log-driver=gcplogs -p 80:80 nginx
    

    You can specify additional options with the --log-opt command-line argument. This command tells Docker to also log the command that the container was started with.

    docker run -d \
        --name mysite \
        --log-driver=gcplogs \
        --log-opt gcp-log-cmd=true \
        -p 80:80 \
        nginx
    
  3. Make a request to your container so that it logs something.

    curl 127.0.0.1:80
    

Configuring the Google Cloud logging driver with Docker Compose

When using Docker Compose, specify a logging driver for each service in the docker-compose.yml configuration file. For example:

version: '2'
services:
  web:
    logging:
      driver: gcplogs
    ...
  database:
    logging:
      driver: gcplogs
    ...

Viewing your logs

Now that you are uploading logs to Stackdriver Logging, you can view them in the Google Cloud Platform Console Logs Viewer.

  1. Open the Google Cloud Platform Console Logs Viewer via that link or by opening Logging from the left menu.

    logging menu

  2. Select the Global logs.

    global logs

  3. Select the gcplogs-docker-driver label to limit to just the logs from your Docker containers.

    gcplogs docker driver logs

  4. Enter a search filter to narrow the logs further. Enter jsonPayload.container.name:nginx-proxy to limit to logs just from containers with the name nginx-proxy.

    logging container name filter

Setting the default logging driver

Instead of configuring the driver for each container, you can configure Docker to use the Google Cloud logging driver by default.

To set the Google Cloud logging driver as the default Docker logging driver, specify the --log-driver=gcplogs option in the dockerd command.

dockerd --log-driver=gcplogs

Container-Optimized OS starts Docker using systemd. To configure Docker to use the Google Cloud logging driver when it is started by systemd:

  1. Create /etc/docker/daemon.json.

    echo '{"log-driver":"gcplogs"}' | sudo tee /etc/docker/daemon.json
    
  2. Restart the docker service.

    sudo systemctl restart docker
    
  3. Test it out by running a container without specifying an explicit --log-driver.

    docker run -d --name mysite -p 80:80 nginx
    

    After making a few requests to this container, you should see the logs in the Logs Viewer as you did before.

Persisting configuration across reboots

On Container-Optimized OS, files in /etc/ are writable, but data does not persist across reboots. Instead, use cloud-init to configure Container-Optimized OS instances.

To configure cloud-init, update the instance metadata by writing a configuration to the user-data key.

You can write the configuration to the instance metadata from the command line or from the Cloud Platform Console. Both methods are described in the following sections.

Writing metadata from the Cloud Platform Console

  1. Go to the VM instances page.
  2. Edit the instance.
  3. Add a Custom metadata item with the key user-data and the value

    #cloud-config
    
    write_files:
      - path: /etc/docker/daemon.json
        content: '{"log-driver":"gcplogs"}'
    
    runcmd:
      - systemctl restart docker
    
  4. Save the changes to the instance.

  5. Reboot the instance.
  6. Verify that the /etc/docker/daemon.json file is present.

    sudo ls /etc/docker
    

Writing metadata from the command-line

If you have already written the metadata using the Cloud Platform Console, you can skip this section.

From Google Cloud Shell or a development machine where you have installed and initialized the Google Cloud SDK, use the gcloud compute intances add-metadata command to add the user-data key to your instance.

  1. Create a file instance-config.txt with the contents:

    #cloud-config
    
    write_files:
      - path: /etc/docker/daemon.json
        content: '{"log-driver":"gcplogs"}'
    
    runcmd:
      - systemctl restart docker
    
  2. Add the user-data key to your instance.

    gcloud compute instances add-metadata INSTANCE_NAME \
        --metadata-from-file user-data=./instance-config.txt
    

    Replace INSTANCE_NAME with the name of your instance.

  3. Reboot the instance.

  4. Verify that the /etc/docker/daemon.json file is present.

    sudo ls /etc/docker
    

Next steps

See more by @{[ username ]} and more tagged {[ tag ]}{[ $last ? '' : ', ' ]}

Submit a Tutorial

Share step-by-step guides

SUBMIT A TUTORIAL

Request a Tutorial

Ask for community help

SUBMIT A REQUEST

GCP Tutorials

Tutorials published by GCP

VIEW TUTORIALS

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies. Java is a registered trademark of Oracle and/or its affiliates.