Integration with logging Standard Library

We recommend that you use google-cloud-logging to integrate with the Python logging standard library. This way, you can write logs using Python standards, and still have your logs appear in Google Cloud Logging.

Automatic Configuration

To integrate google-cloud-logging with the standard logging module, call setup_logging() on a Client instance.

# Imports the Cloud Logging client library
import google.cloud.logging

# Instantiates a client
client = google.cloud.logging.Client()

# Retrieves a Cloud Logging handler based on the environment
# you're running in and integrates the handler with the
# Python logging module. By default this captures all logs
# at INFO level and higher
client.setup_logging()

This setup_logging() function chooses the best configurations for the environment your code is running on. For more information, see the Google Cloud Logging documentation.

Manual Handler Configuration

Automatic Configuration automatically determines the appropriate handler for the environment. To specify the handler yourself, construct an instance manually and pass it in as an argument to setup_logging():

from google.cloud.logging.handlers import CloudLoggingHandler
from google.cloud.logging_v2.handlers import setup_logging

handler = CloudLoggingHandler(client)
setup_logging(handler)

There are two supported handler classes to choose from:

  • CloudLoggingHandler:

      * Sends logs directly to Cloud Logging over the network ([gRPC or HTTP](grpc-vs-http.md))


  * Logs are transmitted according to a Transport class


  * This is the default handler on most environments, including local development

  • StructuredLogHandler:

      * Outputs logs as [structured JSON](https://cloud.google.com/logging/docs/structured-logging#special-payload-fields)

    to standard out, to be read and parsed by a GCP logging agent

      * This is the default handler on Kubernetes Engine, Cloud Functions and Cloud Run

Handler classes can also be specified via dictConfig:

import logging.config

import google.cloud.logging

client = google.cloud.logging.Client()

LOGGING = {
    "version": 1,
    "handlers": {
        "cloud_logging_handler": {
            "class": "google.cloud.logging.handlers.CloudLoggingHandler",
            "client": client,
        },
        "structured_log_handler": {
            "class": "google.cloud.logging.handlers.StructuredLogHandler"
        },
    },
    "root": {"handlers": [], "level": "WARNING"},
    "loggers": {
        "cloud_logger": {"handlers": ["cloud_logging_handler"], "level": "INFO"},
        "structured_logger": {
            "handlers": ["structured_log_handler"],
            "level": "INFO",
        },
    },
}

logging.config.dictConfig(LOGGING)

Note that since CloudLoggingHandler requires an already initialized Client, you must initialize a client and include it in the dictConfig entry for a CloudLoggingHandler.

Standard Library

After you setup the Google Cloud Logging library with the Python logging standard library, you can send logs with the standard logging library as you normally would:

# Imports Python standard library logging
import logging

# The data to log
text = "Hello, world!"

# Emits the data using the standard logging module
logging.warning(text)

For more information on using the Python logging standard library, see the logging documentation

Logging JSON Payloads

Although the Python logging standard library expects all logs to be strings, Google Cloud Logging allows JSON payload data.

To write JSON logs using the standard library integration, do one of the following:

  1. Use the json_fields extra argument:
import logging

data_dict = {"hello": "world"}
logging.info("message field", extra={"json_fields": data_dict})
  1. Log a JSON-parsable string:
import logging
import json

data_dict = {"hello": "world"}
logging.info(json.dumps(data_dict))

Automatic Metadata Detection

The Google Cloud Logging library attempts to detect and attach additional LogEntry fields . The following fields are currently supported:

  • labels

  • trace

  • span_id

  • trace_sampled

  • http_request

  • source_location

  • resource

  • json_fields

NOTE: More information about trace, span_id, and trace_sampled can be found here.http_request requires a supported Python web framework.## Manual Metadata Using the extra Argument

The Python logging standard library accepts an “extra” argument when writing logs. You can use this argument to populate LogRecord objects with user-defined key-value pairs. Google Cloud Logging uses the extra field as a way to pass in additional metadata to populate LogEntry fields.

my_labels = {"foo": "bar"}
my_http = {"requestUrl": "localhost"}
my_trace = "01234"

logging.info(
    "hello", extra={"labels": my_labels, "http_request": my_http, "trace": my_trace}
)

All of the LogEntry fields that can be autodetected can also be set manually through the extra argument. Fields sent explicitly through the extra argument override any automatically detected fields.

CloudLoggingHandler Transports

Transport classes define how the CloudLoggingHandler transports logs over the network to Google Cloud. There are two Transport implementations (defined as subclasses of transports.base.Transport):

You can set a Transport class by passing it as an argument when initializing CloudLoggingHandler manually.

You can use both transport options over gRPC or HTTP.

NOTE: StructuredLogHandler prints logs as formatted JSON to standard output, and does not use a Transport class.