The Stackdriver Profiler interface displays a flame graph and a provides a set of controls that let you analyze the data collected by the profiling agent. This page describes how to interpret the flame graph and describes each of the controls. Subsequent pages describe specific concepts in more detail:
- Filtering profiles describes how to use filters to change the flame graph to show specific data of interest.
- Focusing the graph describes how to change the flame graph to focus on a specific function.
- Comparing profiles describes how to compare profiles for different versions of your application or by metric consumption.
Before you start
Before you can use the Profiler to analyze data, you must have some profiling data collected to analyze. For information on using the Profiler agent to collect profiling data, see:
- Profiling Go applications
- Profiling Java applications
- Profiling Node.js applications
- Profiling Python applications
- Profiling applications running outside Google Cloud Platform
Opening the Profiler interface
To use the Profiler interface:
In the Google Cloud Platform Console, select your project:
In the navigation panel, scroll to Stackdriver and then click Profiler:
Stackdriver Profiler displays profiling data by using flame graphs. For a conceptual introduction, go to Flame graphs.
The flame graph is composed of frames. Each frame represents a function in the service:
The width of a frame corresponds to its consumption of the metric being analyzed. The top frame, for example, represents the entire service and consumes 100% of the metric being analyzed. Therefore, this frame is shown as full width.
By default, the frame color corresponds, where possible, to the function's package. If package information is unavailable, as with Node.js, the names of the source files are used to color the function blocks. In a call stack, a change in block color means a transition from one package to another. You can change change the way the frames are colored. For more information, see Color mode.
The flame graph shows one or more call stacks with the values averaged over a set of profiles:
- Each individual profile represents data collected one time per minute from a single instance of the configured service in a single Compute Engine zone. The collection period for a profile varies with the profile type. See Profile collection for more information.
- A maximum of 250 profiles are averaged. If there are more than 250 profiles available, 250 of them are selected randomly as a sample set.
- The top frame, or "root", in a flame graph represents the entire service. Under the "root" frame, is another frame or set of frames making up the second row in the graph. Each of these color-coded frames is a top-level call made by the service. Under each of those colored function frames is another set of function frames, each of which is responsible for some part of the resource of the frame above it. The hierarchy of function frames in the graph represents the call sequence, and the width of a frame represents that function or method's contribution to the resource consumption.
For example, the profiler graph of the consumed heap for the hotapp-service
shows that the service calls different top-level frames, one
of them is the Go runtime's
main function. The width of these
top-level frames show the majority of the heap is consumed by the call stack
involving Go runtime's
What happens if I hover on frame?
When you hold the pointer over a frame in the flame graph, a frame tooltip opens and displays additional information about the frame:
- Function name
- Source file location
- Metric consumption information
In the metric consumption information, the prefix total indicates that the metric-value and percentage are for a function and its children. In contrast, the prefix self indicates the metric-value and percentage are for the function with the consumption of its children excluded.
The dialog also includes buttons for three filtering actions:
In this example, the tooltip shows that the
is located in the
/usr/local/go/src/runtime/proc.go file. The
metric consumption information shows that
runtime.main and its children
consume 55.41 MiB of
memory, or about 92% of the total memory consumed by the program:
What happens if I click on frame?
When you click a frame, the flame graph is redrawn with that frame shown full width and the colors higher in the call stack are muted. Clicking on a frame doesn't change any settings, it only changes how the graph is displayed. To revert to the original view, click the root (top) frame.
In the following example, the pointer
is on a tiny frame and it is difficult to view the call stack. The tooltip
(*huffmanBitWriter).write as the frame function:
After you click the frame, the graph is redrawn, making the call stack more visible:
You use the Profiler interface control to determine which profiles are analyzed and how that information is displayed in the profiler graph.
Range of time
You use the Timespan menu, Now button, and End time menu to control the range of time for which profiling data is displayed. When you change any of the time fields, Profiler identifies all of the available profiles that meet the current time range and randomly selects 250 for analysis. This action might change your graph because different profiles are included in the analysis. The time fields aren't automatically updated during a profiling session.
By default, the time fields have the following settings:
- Timespan is set to 7 days.
- Now button is shaded by a blue background.
- End time contains the time when Profiler was started and cannot be modified.
With these settings, Profiler analyzes profiles captured in the previous 7 days.
To set the timespan, click the Timespan down arrow, and then select an option from the list. Your choices range from 10 minutes to 30 days, the limit of the retention period for profile data.
To update the end time to the current time, click Now. The background of this button toggles between blue and white. In either case, a single click updates the end time field to the current time.
To set the end time, do the following:
If the End time text isn't changeable, as shown in the following image, click Now:
In the End time field, enter a date and time, or use the calendar option to select a date:
Use the Service menu to select the service whose data you want to analyze. The service name is one of the values you (or the runtime environment) specify when you run an application with profiling enabled. For more information on service names, see the profiling guides for Go, Java, Node.js, or Python.
Use the Profile type menu to select the type of profiling data to analyze:
Each profile type captures a different kind of information:
- CPU time: information about CPU usage.
- Heap: information about the memory allocated in the program's heap when the profile was collected.
- Allocated Heap: information about the total memory that was allocated in the program's heap, including memory that is freed and no longer in use.
- Contention: information about mutex usage.
- Threads: information about thread usage.
- Wall time: information about total time to run.
To restrict the analysis to instances of the service running in a specific Compute Engine zone, click the Zone down arrow, and then select the zone from the list.
The default setting for this field is All zones.
To restrict the analysis to a specific version of the named service, click the Version down arrow, and then select the version of interest. The service version is an optional value you (or the runtime environment) specify when an application is run with profiling enabled. For more information on service versions, see the profiling guides for Go, Java, Node.js, or Python.
The default setting for this field is All versions.
To analyze only the profiles captured when the metric consumption was within a selected percentage of peak consumption, click the Weight down arrow, and make a selection from the menu. An example Weight menu is as follows:
All (3.07 s - 3.25 s), 20146 profiles Top 50% (3.16 s - 3.25 s), 12974 profiles Top 25% (3.21 s - 3.25 s), 1019 profiles Top 10% (3.23 s - 3.25 s), 7 profiles Top 5% (3.24 s - 3.25 s), 2 profiles Top 1% (3.25 s - 3.25 s), 2 profiles
Most rows in the Weight menu start with
Top followed by a percentage.
indicates that only profiles that were collected during the top 5% of
metric consumption are available for analysis. The two values in the
parentheses list the corresponding range of metric consumption. The last value
is the number of profiles collected over this range.
Top 5% row, 2 profiles were collected.
The first row is the default setting for the Weight field. The word
All indicates that all collected profiles, or equivalently 100% of the
collected profiles, are available for analysis.
The following screenshot shows a weight-filtered graph:
The default setting for this field is all profiles.
To visually compare two profiles of the same type, taken from the same service with a project, use the Compare To feature; for more information, see Comparing profiles.
The default setting for this field is None.
To focus the flame graph on a specific function, click List Focusing the graph., and then select the function from the Select focus function table. When you focus the frame graph on a function, the graph displays all code paths that flow into, and out of, that function. For examples, go to
Focusing the flame graph on a function is optional.
You choose which data in the selected set of profiles to display with filters.
Each filter is specified by a predefined
FILTER-OPTION that is paired
with a user-defined
[FILTER-OPTION] : [VALUE]
The filters being used are displayed in the filter bar as filter chips.
Each filter chip displays the filter option, the filter value, and
a Close close button.
In this example, there is one filter chip that displays
Metric : Bytes:
Profiler automatically creates a filter chip with a
Metric and a
VALUE based on your selected profile type.
You can change
VALUE for some profile types. You cannot remove this filter
To add a filter, use one of the following approaches:
- Click Filters , select an option from the list, and then enter the value.
- Click the gray text Add profile data filter in the filter bar, and then enter the filter option and value.
- For the Focus, Show from frame, and Show stacks filter options, you can also place your pointer on the frame, and then select the option from the frame tooltip.
To remove a filter, click Close close on the filter chip.
When preparing the data to display, Profiler searches for
matches between a frame and a filter.
When a match occurs Profiler uses the
determine what action to execute. A frame matches the filter when the frame's
function name or the function's source file name contains
A case-sensitive comparison is performed.
For example, if the filter is
Hide frames : oo, then
frames with functions named
busyloop match are hidden
from the flame graph.
For more information on the filtering options and examples, see the Filtering profiles page.
Stackdriver Profiler keeps your profile data for 30 days, but you can download the data for longer-term storage or for other reasons. When you click download, Profiler writes a compressed file to your local file system. The file name is:
SERVICE_NAMEcontains your Service selection
PROFILE_TYPEcontains your Profile type selection
TO_DATEcontains your time-range specifications
ZONEcontains your Zone selection
VERSIONcontains your Version selection
The data is written in the serialized protocol
.pb) format. The open-source
tool can read this format, so you can analyze this data, not just store it.
pprof documentation for more information.