Setting Up Stackdriver Logging for Python

Beta

This product or feature is in a pre-release state and might change or have limited support. For more information, see Product launch stages.

You can write logs to Logging from Python applications by using the Python logging handler included with the Logging client library, or by using Stackdriver Logging API Cloud client library for Python directly.

Before you begin

1. Sign in to your Google Account.

   If you don't already have one, sign up for a new account.

2. Select or create a GCP project.

   Note: If you don't plan to keep the resources you create in this procedure, create a new project instead of selecting an existing project. After you finish following these steps, you can delete the project, removing all resources associated with the project.

   Go to the Project selector page

3. Make sure that billing is enabled for your Google Cloud Platform project.

   Learn how to enable billing

4. Enable the Stackdriver Logging API.

   Enable the API

5. Prepare your environment for Python development.

   Go to the Python setup guide

Installing the library

To install the Stackdriver Logging library for Python, see Installing the client library.

Once installed, this library includes logging handlers to connect Python's standard logging module to Stackdriver, as well as an API client library to access Stackdriver Logging manually.

Connecting the library to Python logging

To send all log entries to Stackdriver by attaching the Stackdriver Logging handler to the Python root logger, use the setup_logging helper method:

logging/cloud-client/handler.py

View on GitHub

# Imports the Google Cloud client library
import google.cloud.logging

# Instantiates a client
client = google.cloud.logging.Client()

# Connects the logger to the root logging handler; by default this captures
# all logs at INFO level and higher
client.setup_logging()

Using the Python root logger

Once the handler is attached, any logs at, by default, INFO level or higher which are emitted in your application will be sent to Logging:

logging/cloud-client/handler.py

View on GitHub

# Imports Python standard library logging
import logging

# The data to log
text = 'Hello, world!'

# Emits the data using the standard logging module
logging.warn(text)

If messages are logged to Logging from App Engine or Google Kubernetes Engine, the handler will send them to those environments' respective resource types; otherwise, logs will by default appear under the python log in the Global resource type.

Configuring the logging handler

To attach the Stackdriver logging handler to only select Python loggers, or to otherwise configure the logging handler, go to the API library documentation.

For more information on installation, see the documentation for the Stackdriver Logging library for Python. You can also report issues using the issue tracker.

Using the Cloud Client Library directly

For information on using the Stackdriver Logging Cloud client library for Python directly, see Stackdriver Logging Client Libraries.

Running on Google Cloud Platform

Using Stackdriver Logging library for Python requires the Cloud IAM Logs Writer role on GCP. Most GCP environments provide this role by default.

App Engine

App Engine grants the Logs Writer role by default.

The Stackdriver Logging library for Python can be used without needing to explicitly provide credentials.

Stackdriver Logging is automatically enabled for App Engine applications. No additional setup is required.

Note: Logs written to stdout and stderr are automatically sent to Stackdriver Logging for you, without needing to use Stackdriver Logging library for Python.

Google Kubernetes Engine

On Google Kubernetes Engine, you must add the logging.write access scope when creating the cluster:

gcloud container clusters create example-cluster-name --scopes https://www.googleapis.com/auth/logging.write

Compute Engine

Note: To use the Stackdriver Logging library for Python on a Compute Engine VM instance, you do not need to install the Stackdriver Logging agent.

When using Compute Engine VM instances, add the cloud-platform access scope to each instance. When creating a new instance through the Google Cloud Platform Console, you can do this in the Identity and API access section of the Create Instance panel. Use the Compute Engine default service account or another service account of your choice, and select Allow full access to all Cloud APIs in the Identity and API access section. Whichever service account you select, ensure that it has been granted the Logs Writer role in the IAM & admin section of the GCP Console.

Running locally and elsewhere

To use the Stackdriver Logging library for Python outside of Google Cloud Platform, including running the library on your own workstation, on your data center's computers, or on the VM instances of another cloud provider, you must supply your GCP project ID and appropriate service account credentials directly to the Stackdriver Logging library for Python.

You can create and obtain service account credentials manually. When specifying the Role field, use the Logs Writer role. For more information on Cloud Identity and Access Management roles, go to Access control guide.

Viewing the logs

After deployment, you can view the logs in the GCP Console Logs Viewer.

Go to the Logs Viewer

In the Logs Viewer, you must specify one or more resources, but the resource selection might not be obvious. Here are some tips to help you get started:

• If you are deploying your application to App Engine or using the App Engine-specific libraries, set your resource to GAE Application.

• If you are deploying your application on Compute Engine, set the resource to GCE VM Instance.

• If you are deploying your application on Google Kubernetes Engine, your cluster's logging configuration determines the resource type of the log entries. For a detailed discussion on the Legacy Stackdriver and the Stackdriver Kubernetes Monitoring solutions, and how those options affect the resource type, see Migrating to Stackdriver Kubernetes Monitoring.

• If your application is using the Stackdriver Logging API directly, the resource is dependent on the API and your configuration. For example, in your application, you can specify a resource or use a default resource.

• If you don't see any logs in the Logs Viewer, to see all log entries, switch to the advanced filter mode and use an empty filter.

  1. To switch to the advanced filter mode, click menu (▾) at the top of the Logs Viewer and then select Convert to advanced filter.
  2. Clear the content that appears in the filter box.
  3. Click Submit Filter.

  You can examine the individual entries to identify your resources.

For additional information, see Viewing Logs and Advanced logs filters.