This document provides a brief overview on how to instrument your application for Cloud Trace. For detailed instructions on setting up Cloud Trace, see the language-specific setup pages.

Cloud Trace provides distributed tracing data for your applications. After instrumenting your application, you can inspect latency data for a single request and view the aggregate latency for an entire application in the Cloud Trace console.

Cloud Trace recommends using OpenTelemetry. OpenTelemetry is an open-source product from the merger between OpenCensus and OpenTracing.

When to instrument your application

For your application to submit traces to Cloud Trace, it must be instrumented. You can instrument your code by using the Google client libraries. However, it's recommended that you use OpenTelemetry or OpenCensus to instrument your application. These are open source tracing packages. OpenTelemetry is actively in development and is the preferred package.

Instrumenting tracing for applications

There are three ways to implement tracing for your applications:

  • Use OpenTelemetry and the associated Cloud Trace client library. This is the recommended way to instrument your applications.

  • Use OpenCensus if an OpenTelemetry client library is not available for your language.

  • Use the Cloud Trace API and write custom methods to send tracing data to Cloud Trace.

The following table lists the recommended instrumentation for each programming language:

Language Recommended instrumentation
C# .NET Cloud Trace API
Go OpenTelemetry
Java OpenTelemetry
Node.js OpenTelemetry
PHP OpenCensus
Python OpenTelemetry
Ruby Cloud Trace API

When to create spans

The Cloud Trace client libraries typically maintain a global trace context that holds information about the current span, including its trace ID and whether the trace is sampled. These libraries usually create spans on RPC boundaries. However, you might need to create spans if the default creation algorithm isn't sufficient for your needs.

The current active span can be accessed by the global trace context, which is sometimes wrapped in a Tracer object. You can add information relevant to your application by using custom annotations and tags to existing spans, or you can create new child spans with their own annotations and tags to trace the application's behavior with finer granularity. Because the context is global, multi-threaded applications that update the context must use appropriate isolation.

When to provide authentication credentials

You don't need to provide authentication credentials to your application or specify your Google Cloud project ID in your application when you are running on Google Cloud. For some languages, you do need to specify your Google Cloud project ID even if you are running on Google Cloud.

If you are running outside of Google Cloud, you need to provide authentication credentials to your application. You also need to specify your Google Cloud project ID in your application.

For details, go to the language-specific setup pages.

How to force a request to be traced

Cloud Trace doesn't sample every request. For example, if you use Java and OpenCensus, then only 1 request out of every 10,000 is traced. If you are using App Engine, requests are sampled at a rate of 0.1 requests per second for each App Engine instance. If you use the Cloud Trace API, then you can configure custom rates. Some packages, such as the Java OpenCensus package, support configuring the sampling rate.

If you configure a microservice with a sampling rate, then that rate applies only to requests that start at that microservice. If you don't configure a sampling rate for a microservice, then the sampling rate of the parent context determines the sampling rate for the microservice.

To force a specific request to be traced, add an X-Cloud-Trace-Context header to the request. The header specification is:

"X-Cloud-Trace-Context: TRACE_ID/SPAN_ID;o=TRACE_TRUE"


  • TRACE_ID is a 32-character hexadecimal value representing a 128-bit number. It should be unique between your requests, unless you intentionally want to bundle the requests together. You can use UUIDs.

  • SPAN_ID is the decimal representation of the (unsigned) span ID. It should be randomly generated and unique in your trace. For subsequent requests, set SPAN_ID to the span ID of the parent request. See the description of TraceSpan (REST, RPC) for more information about nested traces.

  • TRACE_TRUE must be 1 to trace this request. Specify 0 to not trace the request.

For example, to force a trace with curl:

curl "http://www.example.com" --header "X-Cloud-Trace-Context:

How to create Cloud Trace exemplars

OpenCensus creates Cloud Trace exemplars when it writes time-series data for gRPC services. To view these exemplars, you create a chart and use the chart's toolbar to enable their display. For more information, see Using the chart toolbar.

If you are interested in writing your own custom metrics, see the following resources:

If you are writing custom metrics, you can create exemplars. The following Go code snippet illustrates creating a single Point in a time series. The first element is the time interval, labeled by Interval, and the second is the value. The value is an instance of a TypedValue object and must resolve to a distributionValue.

import (

	googlepb "github.com/golang/protobuf/ptypes/timestamp"
	distributionpb "google.golang.org/genproto/googleapis/api/distribution"
	monitoringpb "google.golang.org/genproto/googleapis/monitoring/v3"

func createDataPointWithExemplar() *monitoringpb.Point {
	end := time.Now().Unix()
	dataPoint := &monitoringpb.Point{
		Interval: &monitoringpb.TimeInterval{
			StartTime: &googlepb.Timestamp{Seconds: end - 60},
			EndTime:   &googlepb.Timestamp{Seconds: end},
		Value: &monitoringpb.TypedValue{Value: &monitoringpb.TypedValue_DistributionValue{
			DistributionValue: &distributionpb.Distribution{
				Count: 14,
				BucketOptions: &distributionpb.Distribution_BucketOptions{Options: &distributionpb.Distribution_BucketOptions_LinearBuckets{
					LinearBuckets: &distributionpb.Distribution_BucketOptions_Linear{NumFiniteBuckets: 2, Width: 3, Offset: 0},
				BucketCounts: []int64{5, 6, 3},
				Exemplars: []*distributionpb.Distribution_Exemplar{
					{Value: 1, Timestamp: &googlepb.Timestamp{Seconds: end - 30}},
					{Value: 4, Timestamp: &googlepb.Timestamp{Seconds: end - 30}},
	return dataPoint

Configuring your Google Cloud project

To use Cloud Trace, your Google Cloud project must have the Cloud Trace API enabled. This setting lets your Google Cloud project receive trace data from authenticated sources.

By default, Google Cloud projects have the Cloud Trace API enabled and you don't need to take any action. If you have modified the access scopes of your Google Cloud project and want to verify your settings, do the following:

  1. From the Google Cloud Console, go to APIs and Services:

    Go to APIs and Services

  2. Click Enable APIs and Services

  3. In the search bar, enter Trace API.

  4. If API enabled is displayed, this API is already enabled and there is nothing for you to do. Otherwise, click Enable.

What's next

For detailed configuration information, samples, and links to GitHub and other open source repositories, go to the setup page for your language: