Shape the future of software operations and make your voice heard by taking the 2021 State of DevOps survey.

Python and OpenTelemetry

This page is designed for application developers who want to collect Cloud Trace data for Python applications by using OpenTelemetry. OpenTelemetry is a set of instrumentation libraries for collecting trace and metric data; these libraries work with multiple backends. To collect traces with OpenTelemetry and Python, you do the following, as described on this page:

  • Install the OpenTelemetry packages.
  • Configure your application to export spans to Cloud Trace.
  • Configure your platform.

For release information, see the following:

For the latest details about OpenTelemetry for Python, along with additional documentation and examples, see OpenTelemetry.

Before you begin

  • You must use Python 3.6 or later.
  • Verify that the Cloud Trace API is enabled for your Google Cloud project:

    1. Click the following button or, in the Google Cloud Console, select APIs & Services, and then select Cloud Trace API:

      Go to Trace API

    2. On the Cloud Trace API page, if a button labeled Enable is displayed, then click it. If this button isn't displayed, then the Cloud Trace API is enabled for the selected project.

Install the OpenTelemetry packages

To install the required OpenTelemetry packages, do the following:

  1. (Optional) Upgrade to the latest version of pip:

    pip install --upgrade pip
    
  2. Install the following OpenTelemetry packages using pip:

    pip install opentelemetry-api \
      opentelemetry-sdk \
      opentelemetry-exporter-gcp-trace
    

Import trace packages

Update your application to import the following packages and classes:

  • trace
  • CloudTraceSpanExporter
  • TracerProvider
  • An export span processor:

    • To send spans with a background process, use the BatchSpanProcessor processor. We recommend that you use this processor unless you are using Cloud Run. Cloud Run doesn't support background processes.

    • To send spans with a foreground process, use the SimpleSpanProcessor processor. If you are using Cloud Run, then you must use this processor. This processor might slow your application.

  • (Optional) If you want to link spans, then import the Link class.

The following example illustrates these import statements:


from opentelemetry import trace
from opentelemetry.exporter.cloud_trace import CloudTraceSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.trace import Link

Configure the export of spans to Cloud Trace

To send spans to Cloud Trace, modify your application to use the CloudTraceSpanExporter exporter. The following example illustrates the required steps:


tracer_provider = TracerProvider()
cloud_trace_exporter = CloudTraceSpanExporter()
tracer_provider.add_span_processor(
    # BatchSpanProcessor buffers spans and sends them in batches in a
    # background thread. The default parameters are sensible, but can be
    # tweaked to optimize your performance
    BatchSpanProcessor(cloud_trace_exporter)
)
trace.set_tracer_provider(tracer_provider)

tracer = trace.get_tracer(__name__)

Add attributes to a span

To add an attribute to a span, call the span's set_attribute method. For example, the following code adds multiple attributes to the span named foo_with_attribute:


with tracer.start_span("foo_with_attribute") as current_span:
    do_work()

    # Add attributes to the spans of various types
    current_span.set_attribute("string_attribute", "str")
    current_span.set_attribute("bool_attribute", False)
    current_span.set_attribute("int_attribute", 3)
    current_span.set_attribute("float_attribute", 3.14)

Add events to a span

To add an event to a span, call the span's add_event method. For example, the following code adds an event to the span named foo_with_event:


# Adding events to spans
with tracer.start_as_current_span("foo_with_event") as current_span:
    do_work()
    current_span.add_event(name="event_name")

To link two spans together, import the Link class and then use the links field in the start_as_current_span method. When you link two spans, you can include attributes in the links field.

The following code illustrates two different ways you can link a span to the span named link_target:


# Adding links to spans
with tracer.start_as_current_span("link_target") as link_target:
    # Using start_as_current_span() instead of start_span() will make spans
    # created within this scope children of foo_with_attribute

    # Creates a span "span_with_link" and a link from
    # "span_with_link" -> "link_target"
    with tracer.start_as_current_span(
        "span_with_link", links=[Link(link_target.context)]
    ):
        do_work()

    # Creates a span "span_with_link" and a link from
    # "span_with_link" -> "link_target". This link also has the attribute
    # {"link_attr": "string"}
    with tracer.start_as_current_span(
        "span_with_link_and_link_attributes",
        links=[Link(link_target.context, attributes={"link_attr": "string"})],
    ):
        do_work()

Sample Flask application

OpenTelemetry's Flask instrumentation is designed to simplify capturing trace content related to HTTP requests. This means that you don't need to add specific instrumentation to your routes for these requests:

  • Flask uses the configured propagator to extract the span context from the incoming HTTP requests.
  • Flask automatically creates spans with attributes that describe the request and the response.

For an end-to-end example using Flask and OpenTelemetry, see flask_e2e. The Git README includes information on how to install, configure, and run the example.

This section highlights Flask-specific configuration steps that are included in the example's server.py file. The client file, client.py, uses the Requests instrumentation to enable tracing of HTTP requests made by the requests library.

Import and configuration

To use OpenTelemetry's Flask instrumentation, you must import the FlaskInstrumentor.

When you want to ensure that spans created by different Google Cloud products are associated with the same trace, you must configure the propagator with the Cloud Trace propagator. This propagator specifies the use of the X-Cloud-Trace-Context header. If you don't configure a propagator, then OpenTelemetry uses the default propagator. In this case, spans created by different Google Cloud products, such as Cloud Run and App Engine, are in separate traces.

The following sample illustrates the required import and configuration statements, and the configuration of the Cloud Trace propagator:


import time

from flask import Flask
from opentelemetry import trace
from opentelemetry.exporter.cloud_trace import CloudTraceSpanExporter
from opentelemetry.instrumentation.flask import FlaskInstrumentor
from opentelemetry.propagate import set_global_textmap
from opentelemetry.propagators.cloud_trace_propagator import (
    CloudTraceFormatPropagator,
)
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

set_global_textmap(CloudTraceFormatPropagator())

You don't need to add any Flask-specific statements when you configure the CloudTraceSpanExporter exporter; the configuration shown in Configure export of spans to Cloud Trace is sufficient.

Initialize Flask

Configure the FlaskInstrumentor to instrument your application. The following sample illustrates how to perform this step:


app = Flask(__name__)
FlaskInstrumentor().instrument_app(app)


@app.route("/")
def hello_world():
    # You can still use the OpenTelemetry API as usual to create custom spans
    # within your trace
    with tracer.start_as_current_span("do_work"):
        time.sleep(0.1)

    return "Hello, World!"

Configure your platform

You can use Cloud Trace on Google Cloud and other platforms.

Running on Google Cloud

When your application is running on Google Cloud, you don't need to provide authentication credentials in the form of a service account to the client library. However, you do need to ensure that your Google Cloud platform has the Cloud Trace API access scope enabled.

For the following configurations, the default access-scope settings enable the Cloud Trace API:

  • App Engine flexible environment
  • App Engine standard environment

  • Google Kubernetes Engine (GKE)

  • Compute Engine

If you use custom access scopes, then you must ensure that Cloud Trace API access scope is enabled:

  • For information about how to configure the access scopes for your environment by using the Google Cloud Console, see Configuring your Google Cloud project.

  • For gcloud users, specify access scopes using the --scopes flag and include the trace.append Cloud Trace API access scope. For example, to create a GKE cluster with only the Cloud Trace API enabled, do the following:

    gcloud container clusters create example-cluster-name --scopes=https://www.googleapis.com/auth/trace.append

Running locally and elsewhere

If your application is running outside of Google Cloud, then you must provide authentication credentials in the form of a service account to the client library. The service account must contain the Cloud Trace agent role. For instructions, see Creating a service account.

Google Cloud client libraries use Application Default Credentials (ADC) to find your application's credentials. You provide these credentials by setting the GOOGLE_APPLICATION_CREDENTIALS environment variable:

Linux/macOS

    export GOOGLE_APPLICATION_CREDENTIALS=path-to-your-service-accounts-private-key

Windows

    set GOOGLE_APPLICATION_CREDENTIALS=path-to-your-service-accounts-private-key

PowerShell:

    $env:GOOGLE_APPLICATION_CREDENTIALS="path-to-your-service-accounts-private-key"

Viewing the traces

After deployment, you can view the traces in the Cloud Console Trace Viewer.

Go to the Trace Viewer page

Troubleshooting

For information on troubleshooting issues with Cloud Trace, go to the Troubleshooting page.

Resources