Build and save queries by using the Logging query language

This document describes how to retrieve and analyze logs when you use the Logs Explorer by writing queries in the query-editor field, and by making selections from the filter menus by using options included with log entries. The queries you build are written in the Logging query language.

You can also save your queries in the Logs Explorer page, or by using the Logging API method savedQueries.create.

Before you begin

To get the permissions that you need to read log data to build queries, use private saved queries, or to list and get shared queries, ask your administrator to grant you the Logs Viewer (roles/logging.viewer) IAM role on your project.

This predefined role contains the permissions required to read log data to build queries, use private saved queries, or to list and get shared queries. To see the exact permissions that are required, expand the Required permissions section:
Required permissions

The following permissions are required to read log data to build queries, use private saved queries, or to list and get shared queries:
- Use private saved queries: logging.queries.usePrivate
- List and get shared queries:
  - logging.queries.listShared
  - logging.queries.getShared

To get the permissions that you need to create, update, and delete shared queries, ask your administrator to grant you the Logging Admin (roles/logging.admin) IAM role on your project.

This predefined role contains the permissions required to create, update, and delete shared queries. To see the exact permissions that are required, expand the Required permissions section:
Required permissions

The following permissions are required to create, update, and delete shared queries:
- logging.queries.share
- logging.queries.updateShared
- logging.queries.deleteShared

For more information about the necessary IAM permissions, see Permissions for the Google Cloud console.

Build queries

To build queries by using the Google Cloud console, do the following:

In the Google Cloud console, go to the Logs Explorer page:
Go to Logs Explorer

If you use the search bar to find this page, then select the result whose subheading is Logging.
Select the Google Cloud project or other Google Cloud resource for which you want to view logs.
Use the Query pane to build your query.

The Query pane provides multiple ways to build and run query expressions:
- Search for text across all log fields.
- Select options from filter menus.
- Write or modify queries by using the query editor.
- View and run saved, recent, and suggested queries from the Query library.

Search for text across log fields

To search for text across all log fields and find all matching log entries, enter your search terms in the search field of the Query pane.

You can search for words and phrases, and your search terms can include Boolean operators and regular expressions:

To perform a case-sensitive search, you must use a regular expression.
To perform a case-insensitive search along token boundaries, enter the search terms without backticks or double quotes.

For example, to search for log entries that contain the word hello and the word world, enter hello world. This command, which is converted to SEARCH("hello world"), matches log entries that contain the tokens hello and world, in any order. Because the search is case-insensitive, the search also matches a log entry that contains the tokens Hello and World. The search doesn't match the token worlds.
To perform a case-insensitive search for a phrase along token boundaries, wrap the phrase in backticks.

For example, to search for the phrase hello world, enter `hello world`. This command, which is converted to SEARCH("`hello world`"), matches log entries that contain the token hello world. The search doesn't match the token hello worlds.
To perform a case-insensitive search for a substring, wrap the text in double quotes. For example, "hello world" matches Hello World and Hello world. The same query also matches hello worlds, because the search isn't performed along token boundaries.

Note: Substring searches are slower than token-based searches. Therefore, we recommend that you use token-based searches when possible.

To see your search terms within the query expression, enable Show query.

After you enter your search terms, click Run query or press the Enter key. The results of the query are displayed in the Query results pane.

Boolean operators

Your search-field entries are converted into Boolean expressions that specify a subset of all the log entries in your selected Google Cloud resource.

The search field supports the usage of the Boolean operators AND, OR, and NOT. When using Boolean operators in your search expressions, note the following:

You can't use parentheses to nest rules. Any parentheses in the search expression are parsed as search terms.
You must capitalize Boolean operators. Lowercase and, or, and not are parsed as search terms, not as operators.

If you don't include any operators, all search terms and phrases are joined by AND. You can omit the AND operator between search terms.

The AND and OR operators are short-circuit operators. You can combineAND and OR rules in the same expression. For example, when the two operators are mixed, the expression a AND b OR c AND d turns into the following Logging query language expression:

"a"
"b" OR "c"
"d"

The NOT operator has the highest precedence, followed by OR and AND in that order.

The NOT operator performs a negation of the subsequent term. For example, NOT error returns log entries that don't contain error. You can also replace the NOT operator with the - (minus) operator. For example, the following two queries are the same:

"response" AND "successful" AND NOT "error"

"response successful" -"error"

This logic also works with a phrase, if the - (minus) operator is outside the quotation marks. For example, the following two queries are the same:

-"response successful"

NOT "response successful"

Construct queries with filter menus

You can use the filter menus in the Query pane to add resource, log name, log severity, and correlation parameters to the query-editor field. These options correspond to the LogEntry fields for all logs in Logging.

The options in the Resource and Log name menus are derived from the log entries that are stored by Cloud Logging.

Resource: Lets you specify the resource.type and associated resource.labels. You can select a single resource type using this filter menu, and zero or more resource labels to apply to your query. The resource parameters are joined by the logical operator AND.
Log name: Lets you specify the logName. You can select multiple log names to apply to your query. When selecting multiple log names, the logical operator OR is used.
Severity: Lets you specify the severity. You can select multiple severity levels at the same time to add to apply to your query. When selecting multiple severity levels, the logical operator OR is used.
Correlate by: Lets you group and view log entries in a "parent-child" format. For more information, see Correlate log entries.

To use any of the filter menus, do the following:

Expand the Menu on any of the filter menus in the Query pane.
Refine the filter parameters.
Click Apply. You see the parameters in the query-editor field.

To see your search terms within the query expression, enable Show query.
After you review the query, click Run query. The results of the query are displayed in the Query results pane.

For certain Compute Engine resource types, such as gce_instance and gce_network, you see the resource name with the resource ID as subtext. For example, for the gce_instance resource type, you see the VM name alongside the VM ID. The resource names help you identify the correct resource ID, on which you can build queries.

View logs by time range

There are two ways to display logs that were written in a specific time range:

Use the time-range selector.
Include a timestamp expression in the query-editor field.

Use the time-range selector

The default time range is one hour, but you can select from preset time options, specify a custom start and end time, or center the time range around a specific timestamp by using the time-range selector. For example, if you want to view the data for the past week, then select Last 1 week from the time-range selector.

You can also set your time zone preferences by using the time-range selector.

Include a timestamp expression in the query-editor field

To add a timestamp expression directly to the query-editor field, use the Logging query language.

If the query-editor field contains an expression with a timestamp, then the time-range selector is disabled, and the query uses the timestamp expression as its time-range restriction. If a query doesn't use a timestamp expression, then the query uses the time-range selector as its time-range restriction.

Write advanced queries using the Logging query language

You can use the Logging query language to build more advanced queries in the Logs Explorer query-editor field:

If you don't see the query-editor field in the Query pane, enable Show query.
Enter your query expressions directly into the query-editor field.

If you added any search terms in the search field or selected any parameters in the filter menus, then those also appear in the query-editor field and are evaluated as part of your query expression.
After you review your query, click Run query.

Logs that match your query are listed under the Query results pane. The Histogram and Log fields panes also adjust according to the query expression.

For examples of common queries you might want to use, see Sample queries using the Logs Explorer.

Use recent queries

When you run any query, the query is added to the Query library, which contains the last 10,000 unique queries over a 30-day period.

To view your recent queries, select the Query library button in the primary toolbar. For recent queries, you have the following options:

Stream: To run the query and stream the results, choose this option.
Run: To run the query, choose this option.
More options: Lets you view the query expression with the options to run the query or save it to your list of Saved queries. You can also select the query directly to get these options.

To save the query, do the following:
1. Click Save. The Save query dialog opens.
2. Complete the following fields:
  - Name (Required): Provide a name for your query. Names are limited to 64 characters.
  - Description (Optional): Provide a description to help identify the purpose of the query.
  - Include summary fields (Optional): Enable Include summary fields and enter the summary fields that you want to display.
  - Truncate summary fields (Optional): Enable Truncate summary fields and select the number of characters to truncate to and whether truncation occurs at the beginning or end of the fields.
3. Click Save query. The query is now available in your Saved queries list.

You can also sort and filter your recent queries; the filter matches on the text in your query expression.

Save and share queries

Saved queries let you store query expressions to help you explore your logs more consistently and efficiently. The Logs Explorer features a Query library, where you can access your saved queries. You can also save your queries by using the Logging API method savedQueries.create.

You can save your query so that it is private and visible only to you, or you can share it with other members of the Google Cloud project. Once you share a query, the query is no longer owned by you, and any member in the project with the necessary permissions can access the query.

Console

To save a query expression that you've built in the query-editor field, do the following:

Click Save in the Query pane. The Save query dialog opens, with your query expression in the query-editor field.
Complete the following fields:
- Name (Required): Provide a name for your query. Names are limited to 64 characters.
- Description (Optional): Provide a description to help identify the purpose of the query.
- Include custom summary fields (Optional): Enable Include summary fields and enter the summary fields that you want to display.
- Truncate summary fields (Optional): Enable Truncate summary fields and select the number of characters to truncate to and whether truncation occurs at the beginning or end of the fields.
- Share with project: Optionally, enable Share with project to share your query with other members of the Google Cloud project.
Click Save query. Your saved queries appear in a list under the Saved tab.

To run a saved query, click Run. To run the query and stream the results, click Stream.

To modify a saved query, select More options, and then select Edit. You can also select the query, make modifications, and then save the modified query.

To delete a saved query, select More options, and then select Delete.

You can also sort and filter your saved queries; the filter matches the text in your query expression.

API

To save a query by using the Logging API, use the savedQueries.create method. For more information about this method, its parameters, and the response data, see the reference page for savedQueries.create.

You can execute the savedQueries.create method by using the APIs Explorer widget on the method's reference page. For Logs Explorer queries, you must specify the loggingQuery field. The following example illustrates a sample request body, which contains an instance of SavedQuery:

{
  "parent": "projects/my-project/locations/global"
  "savedQueryId": "compute-query"
  {
    "displayName": "compute-admin-activity-query",
    "description": "Queries for Compute Engine Admin Activity logs.",

    "loggingQuery":
      {
        "filter": resource.type="gce_instance" AND log_id("cloudaudit.googleapis.com/activity"),
      },
    "visibility": "PRIVATE"
  }
}

To share the query with other members of the Google Cloud project, specify a value of SHARED in the visibility field.

View saved queries

You can view both private queries and queries that are shared with other members in the Google Cloud project by clicking the Query library button:

Console

In the Google Cloud console, go to the Logs Explorer page:
Go to Logs Explorer

If you use the search bar to find this page, then select the result whose subheading is Logging.
Select the Google Cloud project or other Google Cloud resource for which you want to view logs.
Click the Query library button, and click Saved.

You can sort the table by any header. The Visibility column indicates if the queries are shared or private:

Shared: Queries that are shared with other members of the Google Cloud project.
Private: Queries that you have saved and are only visible to you.

API

You can use the Logging API to view private and shared queries by using the savedQueries.list method.

For example, the following request body lists all shared Logs Explorer queries with a wildcard location ID:

{
  "parent": "name": projects/PROJECT_ID/locations/-
  "visibility": "SHARED"
  "filter": "explorer"
}

Use suggested queries

Logging generates suggested queries based on the context of your Google Cloud project, such as the Google Cloud products you're using. Suggested queries can help you identify issues and provide you with insights into the overall health of your systems. For example, detecting that you're using Google Kubernetes Engine, Logging might suggest a query that finds all the error logs for your containers.

To view and run suggested queries, click the Query library button and click Suggested. In the Suggested tab, you have the following options:

Stream: To run the query and stream the results, choose this option.
Run: To run the query, choose this option.
More options: Lets you view the details of the query expression with the options to run the query or save it. You can also select the query directly to get these options.

To review the details of a suggested query, do either of the following:
- Select the query's row.
- Click More and select View. The Query details dialog opens.
In the Query details dialog, you see the query and the options to Run, Stream or Save as:
- To save the query, do the following:
  1. Click Save query.
  2. Complete the fields in the Save query dialog.
  The edited query shows up in your Saved list, where you can choose to run the query later.
- To run the query now, click Run. The query runs and appears in the query-editor field.
- To run the query now and stream the results, click Stream.
- To close the dialog and return to the suggested queries list, click Close.

Note the following expected behaviors:

Successive page loads might not show the same queries in the same order.
You might see zero suggested queries.
Sometimes running a suggested query returns zero logs.

Select queries from the library

Logging provides a library of queries based on common use cases and Google Cloud products. These queries can help you efficiently find logs during time-critical troubleshooting sessions and explore your logs to better understand what Logging data is available.

To view and run the library's queries, do the following:

In the Google Cloud console, go to the Logs Explorer page:
Go to Logs Explorer

If you use the search bar to find this page, then select the result whose subheading is Logging.
Select the Google Cloud project or other Google Cloud resource for which you want to view logs.
Click the Query library button. You see categories of available queries and subsets of queries based on Google Cloud products. To narrow the selection of queries that you see, click any of the products.

You can also use the search field to search the available queries by category, description, or the contents of the query expression.
To review a query expression, do either of the following:

a. Click the query's row.

b. Click More and select View.
In the Query details dialog, you see the query and the options to Run, Stream or Save as:
- To save the query, do the following:
  1. Click Save query.
  2. Complete the fields in the Save query dialog.
  The edited query shows up in your Saved list, where you can choose to run the query later.
- To run the query now, click Run. The query runs and appears in the query-editor field.
- To run the query now and stream the results, click Stream.
- To close the dialog and return to the suggested queries list, click Close.

What's next

Sample queries

Build and save queries by using the Logging query language

Before you begin

Required permissions

Required permissions

Build queries

Search for text across log fields

Boolean operators

Construct queries with filter menus

View logs by time range

Use the time-range selector

Include a timestamp expression in the query-editor field

Write advanced queries using the Logging query language

Use recent queries

Save and share queries

Console

API

View saved queries

Console

API

Use suggested queries

Select queries from the library

What's next