Go and OpenTelemetry

This page is designed for application developers who want to collect Cloud Trace data for Go applications by using OpenTelemetry. OpenTelemetry is a vendor-neutral instrumentation framework that you can use to collect trace and metric data. For information about instrumenting your code, see Instrumentation and observability.

  • Install the OpenTelemetry packages.
  • Configure your application to export spans to Cloud Trace.
  • Configure your platform.

For release information, see the following:

For OpenTelemetry reference content, see the following:

For the latest details about OpenTelemetry for Go, along with additional documentation and examples, see OpenTelemetry.

Before you begin

  1. In the navigation panel of the Google Cloud console, select APIs & Services, click Enable APIs and Services, and then enable the Cloud Trace API:

    Go to Cloud Trace API settings

  2. If API enabled is displayed, then the API is already enabled. If not, click the Enable button.

Installing, initializing, and using the client

See the following instructions to instrument your Go applications on Compute Engine and Google Kubernetes Engine. For a general example of using OpenTelemetry, see the OpenTelemetry GitHub repository for Go.

Compute Engine

Install the OpenTelemetry package:

go get go.opentelemetry.io/otel
go get github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace

Import OpenTelemetry and Cloud Trace export packages:

import (
	"context"
	"errors"
	"log"
	"os"

	texporter "github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace"
	"go.opentelemetry.io/contrib/detectors/gcp"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/sdk/resource"
	sdktrace "go.opentelemetry.io/otel/sdk/trace"
	semconv "go.opentelemetry.io/otel/semconv/v1.24.0"
)

Create the exporter and the trace provider:

func main() {
	// Create exporter.
	ctx := context.Background()
	projectID := os.Getenv("GOOGLE_CLOUD_PROJECT")
	exporter, err := texporter.New(texporter.WithProjectID(projectID))
	if err != nil {
		log.Fatalf("texporter.New: %v", err)
	}

	// Identify your application using resource detection
	res, err := resource.New(ctx,
		// Use the GCP resource detector to detect information about the GCP platform
		resource.WithDetectors(gcp.NewDetector()),
		// Keep the default detectors
		resource.WithTelemetrySDK(),
		// Add your own custom attributes to identify your application
		resource.WithAttributes(
			semconv.ServiceNameKey.String("my-application"),
		),
	)
	if errors.Is(err, resource.ErrPartialResource) || errors.Is(err, resource.ErrSchemaURLConflict) {
		log.Println(err)
	} else if err != nil {
		log.Fatalf("resource.New: %v", err)
	}

	// Create trace provider with the exporter.
	//
	// By default it uses AlwaysSample() which samples all traces.
	// In a production environment or high QPS setup please use
	// probabilistic sampling.
	// Example:
	//   tp := sdktrace.NewTracerProvider(sdktrace.WithSampler(sdktrace.TraceIDRatioBased(0.0001)), ...)
	tp := sdktrace.NewTracerProvider(
		sdktrace.WithBatcher(exporter),
		sdktrace.WithResource(res),
	)
	defer tp.Shutdown(ctx) // flushes any pending spans, and closes connections.
	otel.SetTracerProvider(tp)

	// Create custom span.
	tracer := otel.GetTracerProvider().Tracer("example.com/trace")
	err = func(ctx context.Context) error {
		ctx, span := tracer.Start(ctx, "foo")
		defer span.End()

		// Do some work.

		return nil
	}(ctx)
}

When you create the exporter, you provide information about your Google Cloud project identifier. In this example, the identifier is stored in the environment variable GOOGLE_CLOUD_PROJECT.

The example application calls the function WithBatcher to configure the trace provider to send spans to Cloud Monitoring by using a background process. The example is also configured to call the exporter's Shutdown function on application exit. When Shutdown executes, it sends all pending spans to Cloud Monitoring. The configuration shown in the sample is the recommended implementation for all operating environments, including for Cloud Run where containers can be shutdown at any time.

When you create the Tracer instance, you provide it with a name. In the example, the name is example.com/trace. We recommend that you name these instances after the component being traced as this strategy lets you have multiple instances.

When the example is executed, a single trace named foo is created.

GKE

Add the following to your Dockerfile:

RUN go get go.opentelemetry.io/otel
RUN go get github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace

Import OpenTelemetry and Cloud Trace export packages:

import (
	"context"
	"errors"
	"log"
	"os"

	texporter "github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace"
	"go.opentelemetry.io/contrib/detectors/gcp"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/sdk/resource"
	sdktrace "go.opentelemetry.io/otel/sdk/trace"
	semconv "go.opentelemetry.io/otel/semconv/v1.24.0"
)

Create the exporter and the trace provider:

func main() {
	// Create exporter.
	ctx := context.Background()
	projectID := os.Getenv("GOOGLE_CLOUD_PROJECT")
	exporter, err := texporter.New(texporter.WithProjectID(projectID))
	if err != nil {
		log.Fatalf("texporter.New: %v", err)
	}

	// Identify your application using resource detection
	res, err := resource.New(ctx,
		// Use the GCP resource detector to detect information about the GCP platform
		resource.WithDetectors(gcp.NewDetector()),
		// Keep the default detectors
		resource.WithTelemetrySDK(),
		// Add your own custom attributes to identify your application
		resource.WithAttributes(
			semconv.ServiceNameKey.String("my-application"),
		),
	)
	if errors.Is(err, resource.ErrPartialResource) || errors.Is(err, resource.ErrSchemaURLConflict) {
		log.Println(err)
	} else if err != nil {
		log.Fatalf("resource.New: %v", err)
	}

	// Create trace provider with the exporter.
	//
	// By default it uses AlwaysSample() which samples all traces.
	// In a production environment or high QPS setup please use
	// probabilistic sampling.
	// Example:
	//   tp := sdktrace.NewTracerProvider(sdktrace.WithSampler(sdktrace.TraceIDRatioBased(0.0001)), ...)
	tp := sdktrace.NewTracerProvider(
		sdktrace.WithBatcher(exporter),
		sdktrace.WithResource(res),
	)
	defer tp.Shutdown(ctx) // flushes any pending spans, and closes connections.
	otel.SetTracerProvider(tp)

	// Create custom span.
	tracer := otel.GetTracerProvider().Tracer("example.com/trace")
	err = func(ctx context.Context) error {
		ctx, span := tracer.Start(ctx, "foo")
		defer span.End()

		// Do some work.

		return nil
	}(ctx)
}

When you create the exporter, you provide information about your Google Cloud project identifier. In this example, the identifier is stored in the environment variable GOOGLE_CLOUD_PROJECT.

The example application calls the function WithBatcher to configure the trace provider to send spans to Cloud Monitoring by using a background process. The example is also configured to call the exporter's Shutdown function on application exit. When Shutdown executes, it sends all pending spans to Cloud Monitoring. The configuration shown in the sample is the recommended implementation for all operating environments, including for Cloud Run where containers can be shutdown at any time.

When you create the Tracer instance, you provide it with a name. In the example, the name is example.com/trace. We recommend that you name these instances after the component being traced as this strategy lets you have multiple instances.

When the example is executed, a single trace named foo is created.

How to create a custom span

You can add additional information to the system-created trace by creating a custom span.

To create a custom span with the name foo, add the following to the source code:

// Create custom span.
tracer := otel.GetTracerProvider().Tracer("example.com/trace")
err = func(ctx context.Context) error {
	ctx, span := tracer.Start(ctx, "foo")
	defer span.End()

	// Do some work.

	return nil
}(ctx)

Here, example.com/trace refers to the name of the tracer instance.

Configure your platform

You can use Cloud Trace on Google Cloud and other platforms.

Running on Google Cloud

When your application is running on Google Cloud, you don't need to provide authentication credentials in the form of a service account to the client library. However, you do need to ensure that your Google Cloud platform has the Cloud Trace API access scope enabled.

For a list of supported Google Cloud environments, see Environment support.

For the following configurations, the default access-scope settings enable the Cloud Trace API:

  • App Engine flexible environment
  • App Engine standard environment

  • Google Kubernetes Engine (GKE)

  • Compute Engine

  • Cloud Run

If you use custom access scopes, then you must ensure that Cloud Trace API access scope is enabled:

  • For information about how to configure the access scopes for your environment by using the Google Cloud console, see Configuring your Google Cloud project.

  • For gcloud users, specify access scopes using the --scopes flag and include the trace.append Cloud Trace API access scope. For example, to create a GKE cluster with only the Cloud Trace API enabled, do the following:

    gcloud container clusters create example-cluster-name --scopes=https://www.googleapis.com/auth/trace.append

Running locally and elsewhere

If your application is running outside of Google Cloud, then you must provide authentication credentials in the form of a service account to the client library. The service account must contain the Cloud Trace agent role. For instructions, see Creating a service account.

Google Cloud client libraries use Application Default Credentials (ADC) to find your application's credentials.

You can provide these credentials in one of three ways:

  • Run gcloud auth application-default login

  • Place the service account in a default path for your operating system. The following lists the default paths for Windows and Linux:

    • Windows: %APPDATA%/gcloud/application_default_credentials.json

    • Linux: $HOME/.config/gcloud/application_default_credentials.json

  • Set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path to your service account:

Linux/macOS

    export GOOGLE_APPLICATION_CREDENTIALS=path-to-your-service-accounts-private-key

Windows

    set GOOGLE_APPLICATION_CREDENTIALS=path-to-your-service-accounts-private-key

PowerShell:

    $env:GOOGLE_APPLICATION_CREDENTIALS="path-to-your-service-accounts-private-key"

View traces

In the navigation panel of the Google Cloud console, select Trace, and then select Trace explorer:

Go to Trace explorer

Troubleshooting

For information on troubleshooting issues with Cloud Trace, go to the Troubleshooting page.