Viewing trace details

Stackdriver Trace shows the details for a trace in the Trace details pane of the Trace list window. This view shows a summary of details about the request, a graphical timeline that shows the root span for the request and subspans for any RPC calls, and a detailed view of latency data collected for the spans.

Viewing trace details

  1. In the Google Cloud Platform Console, go to the Trace list page:

    Go to Trace list

  2. There are two ways to display detailed information about a trace:

    • Click its URI listed in table.
    • Click its dot in the graph.

    After you choose a trace to examine, two new panes are displayed on the Trace list window: a Timeline pane and a span-information pane. Additionally, in the Timeline pane, the root span is highlighted:

    Stackdriver Trace details pane.

Timeline

The Timeline pane displays a graphical representation of latency data for the request. It shows a root span that represents the time it took for the application to process the request end-to-end, as well as spans that represent the time it took to complete RPC calls performed when handling the request.

Each row in the Timeline corresponds to a span in the trace. Displayed on the span are two attributes:

  • The name of the RPC call in the format service_name.call_name. For example, datastore_v3.RunQuery.

    For App Engine standard environment, the internal name reported here might not match the name from a language-specific service API.

  • The time it took to complete the round-trip RPC call.

Span information

The pane next to the Timeline displays details about the highlighted span. By default, the highlighted span corresponds to the root span. However, you can view the span information for any span displayed in the Timeline by selecting that span.

The first two rows in the span information are the relative start time and the name of the RPC call:

Property Description
Relative start time The first entry is the relative start time for the span.

For root spans, this value is always @0 ms.

For subspans, this is the time when the subspan started relative to the request start. That is, this value displays how long it takes from the start of the overall request, for this RPC call to begin.
Name The next entry is the name of the RPC call. Names are formatted as service_name.call_name. For example, datastore_v3.RunQuery.

For App Engine standard environment, the internal name reported here might not match the name from a language-specific service API

When a root span is highlighted, the next entry is the Summary table. There is a row in the table for the root span, and one row for each RPC called by a subspan:

Column Description
Name Name of the RPC call in format service_name.call_name. Displayed in the span label.
RPCs The number of times the procedure was called.
Total Duration The total time spent performing the RPCs.

The final entry in the pane is the Detail table. This table lists metadata about the span highlighted in the Timeline. The following properties are always displayed:

Property Description
Timestamp The time at which the application received the incoming request.
Log A link to a log entry.
Report A link to the most recent analysis report that includes this trace.
Labels A table of labels and values in the request. Labels are application specific. A few common labels, and their meaning, are listed in the following table:
http/host Hostname where the application is running.
http/response/size Number of bytes in the HTTP response body.
http/url Relative URL for the request.
gae/request_log_id Internal App Engine request ID.

Additional properties might be listed when your request type is HTTP or when your application is running on App Engine. The following table displays a partial list of some commonly observed properties:

Property Description
Traced time (HTTP only) The aggregate time it took for all RPC calls to be completed.
Untraced time (HTTP only) The time during which no RPC calls were made. That is, this measures the time spent locally in the application. Note that a longer than typical untraced time at the beginning of a request can often be caused by a new instance being created to handle the load.
HTTP method (HTTP only) The HTTP method for the request.
Service (App Engine only) The App Engine service that handled the request. For more information, go to App Engine.
Version (App Engine only) The version of the application that handled the request..

Insights

Insights on the performance of the request are shown below the timeline, when insights are available.

For more information, go to Insights.

Annotating spans

You can annotate a trace by using Stackdriver Trace API v1 or Stackdriver Trace API v2. This section describes the different options available to you.

Annotating with labels

You can add annotations to spans by creating a labels object and attaching it to the TraceSpan object when you use Stackdriver Trace API v1 patchTraces.

These annotations are displayed as labels when you view a traces' details. For more detail, go to Viewing trace details on this page.

Annotating with attributes

You can annotate spans by creating an attributes object and attaching it to the Span object when you use Stackdriver Trace API v2 batchWrite.

These annotations are displayed as labels when you view a traces' details. For more detail, go to Viewing trace details on this page.

Annotating with time events

You can add annotations and message event to spans by creating a TimeEvents object and attaching it to the Span object when you use Stackdriver Trace API v2 batchWrite.

The TimeEvents object is an array of TimeEvent objects, each one contains a message event and an annotation.

To view the TimeEvents for a trace, go to the traces' Timeline view and click Show events. The individual TimeEvent entries are displayed in the trace Timeline. When you select a TimeEvent, its details are displayed in information pane next to the Timeline pane.

Log entries

Trace associates logs with traces by using annotations in the log entries. App Engine standard environment automatically associates its log entries with traces, but not all services do.

When you use the Trace client libraries, you can associate a trace with a log entry by setting the trace field in the LogEntry object. The trace field must be set to a string with the following format:

   projects/[PROJECT_ID]/traces/[TRACE_ID]

where [PROJECT_ID] is your GCP project ID and [TRACE_ID] is the trace identifier

You can associate a span with a log entry by setting the span_id field in the LogEntry object. In this case, set the span_id field to the 16-character hexadecimal encoding of the span's ID. For example, a span with an ID of 74 would be represented as 000000000000004a. When you are using trace sampling, it's possible that a log entry is created when the trace itself isn't captured.

Viewing logs

You can view the log entry for a trace alongside the Timeline pane or in the Logs Viewer. When you use the Logs Viewer, it's automatically restricted to the timestamp range of the trace. If there are no log entries to display, the Logs Viewer displays the message No entries found matching current filter.

From the Trace details pane, to view the log entry for the trace, do one of the following:

  • To display the trace log entries alongside the Timeline, click Show logs in the Timeline pane. When Show logs isn't displayed, no log entries are available.

  • To view the log entry in the Logs Viewer, click View next to the label Log in the Details section. Note that when you have a Cloud Load Balancing trace, click View next to the label VM Log.

For more information about viewing log entries in Stackdriver Logging, see Viewing logs.

Log viewing permissions

To view any log entries, you must have the logging.logEntries.list permission in your project. This permission is provided by the Logs Viewer and Project Viewer Cloud Identity and Access Management (Cloud IAM) roles.

To view VM instance logs, you must have the compute.instances.get permission in your project. This permission is provided by the Compute Engine Network Viewer and Project Viewer Cloud IAM roles.

What's next

Was this page helpful? Let us know how we did:

Send feedback about...

Stackdriver Trace
Need help? Visit our support page.