Setting Up Cloud Logging for Python

You can write logs to Cloud Logging from Python applications by using the standard Python logging handler, or by using the Cloud Logging API client library for Python directly. When you use the standard Python logging handler, you must attach a Cloud Logging handler to the Python root handler. This document illustrates that approach.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. Enable the Cloud Logging API.

   Enable the API

5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

6. Make sure that billing is enabled for your Google Cloud project.

7. Enable the Cloud Logging API.

   Enable the API

8. Prepare your environment for Python development.

   Go to the Python setup guide

Install the library

To install the Cloud Logging library for Python, see Install the client library for Python. This library lets you attach a Cloud Logging handler to the standard Python root handler. You can also use this library to send API requests to Cloud Logging.

For more information on installation, see the documentation for the Cloud Logging library for Python. You can also report issues using the issue tracker.

Write logs with the standard Python logging handler

To send all log entries that are written with the standard Python root handler to Cloud Logging, do the following:

1. Attach the Cloud Logging handler to the Python root logger by calling the setup_logging method:

   samples/snippets/handler.py

   View on GitHub

   # Imports the Cloud Logging client library
   import google.cloud.logging
   
   # Instantiates a client
   client = google.cloud.logging.Client()
   
   # Retrieves a Cloud Logging handler based on the environment
   # you're running in and integrates the handler with the
   # Python logging module. By default this captures all logs
   # at INFO level and higher
   client.setup_logging()

   For information about how to configure the logging handler, see Integration with logging Standard Library.

2. Write log data by using the Python root logger:

   samples/snippets/handler.py

   View on GitHub

   # Imports Python standard library logging
   import logging
   
   # The data to log
   text = "Hello, world!"
   
   # Emits the data using the standard logging module
   logging.warning(text)

   By default, any log whose severity level is at least INFO that is written by your application is sent to Cloud Logging.

   If messages are logged to Logging from App Engine or Google Kubernetes Engine, then the handler sends them to those environments' respective resource types; otherwise, logs are listed under the python log in the Global resource type.

Write logs with the Cloud Logging client library

For information on using the Cloud Logging client library for Python directly, see Cloud Logging Client Libraries.

Run on Google Cloud

For an application to write logs by using the Cloud Logging library for Python, the service account for the underlying resource must have the Logs Writer (roles/logging.logWriter) IAM role. Most Google Cloud environments automatically configure the default service account to have this role.

App Engine

Cloud Logging is automatically enabled for App Engine, and your app's default service account has the IAM permissions by default to write log entries.

For more information, see Writing and viewing logs.

Google Kubernetes Engine (GKE)

GKE automatically grants the default service account the Logs Writer (roles/logging.logWriter) IAM role. If you use Workload Identity with this default service account to let workloads access specific Google Cloud APIs, then no additional configuration is required. However, if you use Workload Identity with a custom IAM service account, then ensure that the custom service account has the role of Logs Writer (roles/logging.logWriter).

If needed, you can also use the following command to add the logging.write access scope when creating the cluster:

gcloud container clusters create example-cluster-name \
    --scopes https://www.googleapis.com/auth/logging.write

Compute Engine

Note: To use the Cloud Logging library for Python on a Compute Engine VM instance, you do not need to install the Cloud Logging agent.

When using Compute Engine VM instances, add the cloud-platform access scope to each instance. When creating a new instance through the Google Cloud console, you can do this in the Identity and API access section of the Create Instance panel. Use the Compute Engine default service account or another service account of your choice, and select Allow full access to all Cloud APIs in the Identity and API access section. Whichever service account you select, ensure that it has been granted the Logs Writer role in the IAM & Admin section of the Google Cloud console.

Run locally and elsewhere

To use the Cloud Logging library for Python outside of Google Cloud, including running the library on your own workstation, on your data center's computers, or on the VM instances of another cloud provider, you must supply your Google Cloud project ID and appropriate service account credentials directly to the Cloud Logging library for Python.

For existing service accounts, do the following:

1. Grant the service account the IAM the Logs Writer (roles/logging.logWriter) IAM role. For more information on IAM roles, see Access control.

2. Set up Application Default Credentials.

If you don't have a service account, then create one. For information about this process, see Create service accounts.

For general information about the methods that you can use to authenticate, see Terminology: service accounts.

View the logs

In the Google Cloud console, select Logging, and then select Logs Explorer, or click the following button:

Go to the Logs Explorer

In the Logs Explorer, you must specify one or more resources, but the resource selection might not be obvious. Here are some tips to help you get started:

• If you are deploying your application to App Engine or using the App Engine-specific libraries, set your resource to GAE Application.

• If you are deploying your application on Compute Engine, set the resource to GCE VM Instance.

• If you are deploying your application on Google Kubernetes Engine, your cluster's logging configuration determines the resource type of the log entries. For a detailed discussion on the Legacy Google Cloud's operations suite and the Google Cloud's operations suite Kubernetes Monitoring solutions, and how those options affect the resource type, see Migrating to Google Cloud's operations suite Kubernetes Monitoring.

• If your application is using the Cloud Logging API directly, the resource is dependent on the API and your configuration. For example, in your application, you can specify a resource or use a default resource.

• If you don't see any logs in the Logs Explorer, to see all log entries, switch to the advanced query mode and use an empty query.

  1. To switch to the advanced query mode, click menu (▾) at the top of the Logs Explorer and then select Convert to advanced filter.
  2. Clear the content that appears in the filter box.
  3. Click Submit Filter.

  You can examine the individual entries to identify your resources.

For additional information, see Using the Logs Explorer.