Instrumentation samples overview

This document describes the structure of the instrumentation samples provided for the Go, Java, Node.js, and Python languages. These samples provide guidance about how to instrument an application.

You might be interested in other samples that illustrate different configurations:

Correlate metrics and traces by using exemplars describes how to configure a Go application to generate exemplars. An exemplar is an example data point attached to a metric data point. You can use exemplars to correlate your trace and metric data.
Use the Ops Agent and OpenTelemetry Protocol (OTLP) describes how you can configure the Ops Agent and an OTLP receiver to collect metrics and traces from an application.

How the samples work

The samples for Go, Java, Node.js, and Python use the OpenTelemetry protocol to collect trace and metric data. The samples configure a logging framework to write structured logs and the OpenTelemetry collector is configured to read from the application's stdout stream. For framework recommendations, see Choose an instrumentation approach.

The applications are built and deployed by using Docker. You don't have to use Docker when you instrument an application with OpenTelemetry.

You can run the samples in the Cloud Shell, on Google Cloud resources, or on a local development environment.

Deep dive

The samples use the OpenTelemetry Collector as a sidecar to receive and enrich the application's telemetry, which is then sent to your Google Cloud project by using a Google Cloud exporter. The exporter converts the telemetry into a format compatible with the Cloud Trace API, Cloud Monitoring API, or Cloud Logging API. Next, they send the transformed data to your Google Cloud project by issuing an API command.

The samples show how to do the following:

Configure OpenTelemetry to collect metrics and traces by using the OpenTelemetry collector.

If you review the samples, you'll notice that the complexity of this step is language dependent. For example, for Go, this step configures the main function to call a function that configures the collection of metrics and traces. For Go, the HTTP server and client are also updated.
Configure a logging framework to write structured logs.

We recommend that your applications write structured logs, which results in the log payload being formatted as a JSON object. For these logs, you can construct queries that search specific JSON paths and you can index specific fields in the log payload.

Some services, like Google Kubernetes Engine, have built-in agents that scrape structured logs and send those logs to your Google Cloud project. Other services, like Compute Engine, require that you install an agent, which scrapes and sends your logs. If you want to learn about agents you install, see Ops Agent overview.

You don't need to install any agents to use these samples.
Configure Docker files. All samples contain the following yaml files:
- docker-compose.yaml: Configures the services for the application, the OpenTelemetry collector, and a load generator. For example, the service for the OpenTelemetry collector, otelcol, specifies an image, a volume, and environment variables. The endpoint for the OpenTelemetry collector is set by the OTEL_EXPORTER_OTLP_ENDPOINT environment variable, which is specified in the app service.
- otel-collector-config.yaml: Configures the receivers, exporters, processors, and pipelines.
  
  The telemetry service defines pipelines for trace, metric, and log data. Each pipeline entry specifies a receiver, a processor, and an exporter. The same receiver, otlp, is used for metrics and traces.
  
  The exporters section describes how collected data is exported to a Google Cloud project. For all telemetry, a Google Cloud exporter is utilized. The exporter converts the telemetry into a format compatible with the Cloud Trace API, Cloud Monitoring API, or Cloud Logging API. Next, it sends the transformed data to your Google Cloud project by issuing an API command.
- docker-compose.creds.yaml: This file optionally mounts a Google Cloud credentials file in the otelcol container. This file is needed when a sample is run on a local machine where the Application Default Credentials (ADC) are available only as a file.

Required permissions

If you run the samples in the Cloud Shell, on Google Cloud resources, or on a local development environment, then the permissions listed in this section are sufficient. For production applications, typically a service account provides the credentials to write log, metric, and trace data.

To get the permissions that you need to for the sample applications to write log, metric, and trace data, ask your administrator to grant you the following IAM roles on your project:
- Logs Writer (roles/logging.logWriter)
- Monitoring Metric Writer (roles/monitoring.metricWriter)
- Cloud Trace Agent (roles/cloudtrace.agent)

To get the permissions that you need to view your log, metric, and trace data, ask your administrator to grant you the following IAM roles on your project:
- Logs Viewer (roles/logging.viewer)
- Monitoring Viewer (roles/monitoring.viewer)
- Cloud Trace User (roles/cloudtrace.user)
For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Required APIs

The following provides information about the APIs required to send telemetry data to a Google Cloud project:

Google Cloud console

Enable the Cloud Logging, Cloud Monitoring, and Cloud Trace APIs.

Enable the APIs

Google Cloud CLI

Enable the Cloud Logging, Cloud Monitoring, and Cloud Trace APIs:

gcloud services enable logging.googleapis.com monitoring.googleapis.com cloudtrace.googleapis.com