title | description | author | tags | date_published |
---|---|---|---|---|
Using the Google Cloud Logging driver for Docker |
Learn to use the Google Cloud Logging driver to save your Docker logs to Cloud Logging. |
tswast |
Docker, Logging, devops |
2017-04-24 |
Tim Swast | Developer Programs Engineer | Google
Contributed by Google employees.
This tutorial shows you how to use the Google Cloud Logging driver to upload logs from your Docker containers to Cloud Logging.
- Run a Docker container configured with the Google Cloud Logging driver.
- View logs in the Cloud Console.
- Create or select a Google Cloud project from the Cloud Console projects page.
- Enable billing for your project.
This tutorial uses billable components of Google Cloud, including
Use the pricing calculator to estimate the costs for your usage.
Create a new Compute Engine instance using the Container-Optimized OS stable image. Container-Optimized OS comes with Docker pre-installed and supports automatic system updates.
- Open the Cloud Console.
- Create a new Compute Engine instance.
- Select the desired Zone, such as us-central1-f.
- Select the desired Machine series, such as N1.
- Select the desired Machine type, such as f1-micro.
- Change the Boot disk to Container-Optimized OS stable.
- Click Create to create the Compute Engine instance.
-
After the instance is created, click the SSH button to open a terminal connected to the machine.
-
To use the Google Cloud logging driver for Docker, specify the
--log-driver=gcplogs
command-line argument to thedocker run
command.Run the following command to start an NGINX container which writes logs to Cloud Logging:
docker run -d \ --name mysite \ --log-driver=gcplogs \ -p 80:80 \ nginx
-
Make a request to your container, which will generate logs and push them to Cloud Logging:
curl 127.0.0.1:80
When using Docker Compose, specify a logging driver for each
service in the docker-compose.yml
configuration file. For example:
version: '3'
services:
web:
logging:
driver: gcplogs
...
database:
logging:
driver: gcplogs
...
Now that you are uploading logs to Cloud Logging, you can view them in the Cloud Console Logs Viewer.
-
Open the Cloud Console Logs Viewer.
-
In the GCE VM instance menu, choose your instance, and then select the
gcplogs-docker-driver
label to limit to just the logs from your Docker containers. -
Enter a search filter to narrow the logs further. Enter
jsonPayload.container.name:nginx-proxy
to limit to logs just from containers with the namenginx-proxy
.
Instead of configuring the driver for each container, you can configure Docker to use the Google Cloud Logging driver by default.
To set the Google Cloud Logging driver as the
default Docker logging driver,
specify the --log-driver=gcplogs
option in the dockerd
command.
dockerd --log-driver=gcplogs
Container-Optimized OS starts Docker using systemd. To configure Docker to use the Google Cloud Logging driver when it is started by systemd, do the following:
-
Create
/etc/docker/daemon.json
:echo '{"log-driver":"gcplogs"}' | sudo tee /etc/docker/daemon.json
-
Restart the docker service:
sudo systemctl restart docker
-
Test it out by running a container without specifying an explicit
--log-driver
:docker run -d --name mysite2 -p 80:80 nginx
After making a few requests to this container, you should see the logs in the Logs Viewer as you did before.
On Container-Optimized OS, files in /etc/
are
writable, but data does not persist across reboots.
Instead,
use cloud-init to configure Container-Optimized OS instances.
To configure cloud-init,
update the instance metadata
by writing a configuration to the user-data
key.
You can write the configuration to the instance metadata from the command line or from the Cloud Console. Both methods are described in the following sections.
-
Go to the VM instances page.
-
Edit the instance.
-
Add a Custom metadata item with the key
user-data
and the value:#cloud-config write_files: - path: /etc/docker/daemon.json content: '{"log-driver":"gcplogs"}' runcmd: - systemctl restart docker
-
Save the changes to the instance.
-
Reboot the instance.
-
Verify that the
/etc/docker/daemon.json
file is present:sudo ls /etc/docker
If you have already written the metadata using the Cloud Console, you can skip this section.
From Cloud Shell or a development machine
where you have installed and initialized the Cloud SDK,
use the gcloud compute instances add-metadata
command to add the user-data
key to your instance.
-
Create a file
instance-config.txt
with the following contents:#cloud-config write_files: - path: /etc/docker/daemon.json content: '{"log-driver":"gcplogs"}' runcmd: - systemctl restart docker
-
Add the
user-data
key to your instance:gcloud compute instances add-metadata INSTANCE_NAME \ --metadata-from-file user-data=./instance-config.txt
Replace
INSTANCE_NAME
with the name of your instance. -
Reboot the instance.
-
Verify that the
/etc/docker/daemon.json
file is present:sudo ls /etc/docker