Trace labels

Trace provides a mechanism for you to annotate your spans with meaningful data. If you are using the Cloud Trace API V1, you create these annotations by using the labels field in the Cloud Trace API TraceSpan object. If you are using the Cloud Trace API V2, you can annotate with Attributes. For more information about annotation options, see Annotating spans.

Most labels are displayed in the Labels table when the Trace Waterfall View is displayed:

Trace List page with labels and a stack trace displayed.

There are two other locations where labels are displayed. The Details pane lists the most common HTTP labels while the Call trace table lists the content of the stacktrace label.

The labels field is a map of key-value pairs. They keys and values are stored as strings. For string length restrictions, see the API reference for labels.

There is a limit of 32 labels per span.

Predefined keys

The following table lists the supported predefined keys, includes a description, and in some cases, includes an example:

Label keyDescription
Example
/agent Trace agent identifier.
"node@google-cloud/trace-agent v3.0.0"
/component Component identifier.
"grpc"
/error/message An error message.
"Rendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE details = OS Error."
/error/name Display name for the error.
/http/client_city The city of the client.
"NYC"
/http/client_country The country of the client.
"US"
/http/client_protocol HTTP protocol identifier.
"1.0"
/http/client_region The region of the client.
"us-east4"
/http/host The value of the Host Header.
"default.example.com"
/http/method Set to the HTTP request method.
Don't include this key for non-HTTP requests.
"GET"
/http/path Request-URL path.
"/cart/checkout"
/http/redirected_url The URL prior to redirection.
In this case, use the key HTTL URL label to store the final URL.
/http/request/size Number of bytes in the request.
/http/response/size Number of bytes in the response.
/http/route The matched route.
"/cart/checkout/:item_id"
/http/status_code HTTP response status code.
"200"
/http/url Complete HTTP request URL.
"http://example.com"
/http/user_agent Information about the HTTP User-Agent that originated the request.
"python-requests/2.19.1"
/stacktrace JSON-formatted stack trace. Stack traces aren't indexed for search.
This label is displayed in the Call Stack table.
    "stackTrace": {
      "stackFrames": {
        "frame": [
          {
            "functionName": {
              "value": "serverMethodTrace [as func]"
            },
            "fileName": {
              "value":
              "/usr/src/app/node_modules/@google-cloud/trace-agent/build/src/plugins/plugin-grpc.js"
            },
            "lineNumber": "249",
            "columnNumber": "28"
          },
          {
            "functionName": {
              "value": "anonymous function"
            },
            "fileName": {
              "value": "/usr/src/app/node_modules/grpc/src/server.js"
            },
            "lineNumber": "592",
            "columnNumber": "13"
          }
        ]
      }
    },

This label and its value are displayed in the Details table that is included in the Trace details view.

Example

The following is a partial JSON representation of a Trace object that includes multiple labels:

{
  "projectId": "a-sample-project",
  "traceId": "00000000000000004db6dd68e7d37f57",
  "spans": [
    {
      "spanId": "12913864118554233534",
      "kind": "RPC_SERVER",
      "name": "http://xx.xxx.xxx.xxx/",
      "startTime": "2019-04-02T19:37:34.149058Z",
      "endTime": "2019-04-02T19:37:34.151136Z",
      "parentSpanId": "5599906629317525335",
      "labels": {
        "/component": "default",
        "/http/host": "xx.xxx.xxx.xxx",
        "/http/status_code": "200",
        "/http/url": "http://xx.xxx.xxx.xxx/",
        "zipkin.io/http.route": "/**",
        "/http/method": "GET",
        "zipkin.io/endpoint.ipv4": "10.16.1.6",
        "zipkin.io/http.path": "/",
        "zipkin.io/mvc.controller.class": "ResourceHttpRequestHandler"
      }
    }
  ]
}

The above trace was taken from a system using a Zipkin collector. In this case, the labels with zipkin.io keys were added by that collector.

Prefix g.co

In some cases, labels displayed in the Trace waterfall view contain a g.co prefix. The g.co prefix indicates that this label was generated by a Google service. If you are running your service on App Engine or other Google Cloud infrastructure, you might see labels such as the following:

Label key Sample label value
g.co/agent opencensus-go[0.15.0]
g.co/gce/instanceid 12345678900000000

Custom labels

You can create custom labels. If you create a custom label, we recommend that you use the following formats:

  • /category/product/key for agents of well-known products. For example, /db/mongodb/read_size.
  • short_host/path/key for domain-specific keys. For example, g.co/agent.

If you create hundreds or thousands of custom labels, then that might incur performance impacts due to the increased label cardinality.

As illustrated in the Example section, if you use a Zipkin collector or a library such as OpenCensus, it might add labels to your trace spans.

Using OpenCensus

If you are using OpenCensus libraries, you must use the OpenCensus attribute name. To find a table that cross-lists the OpenCensus attribute name with the Cloud Trace predefined label name, see Attributes.