Build queries by using the Logging query language

This document describes how to retrieve and analyze logs when you use the Logs Explorer. You retrieve logs by writing and executing queries. You can build queries by making selections from the filter menus, by using options included with log entries, and by using the query-editor field. The queries you build are written in the Logging query language.

Before you begin

• To build queries, you must have the permissions to read log data. These permissions are included in the Logging Viewer (roles/logging.viewer) role. For details on the necessary IAM permissions, see Permissions for the Google Cloud console.

• To share queries, your Identity and Access Management role must include the logging.queries.share permission. This permission is included in the Owner (roles/owner) and Logging Admin (roles/logging.admin) roles.

  For a list of permissions associated with each Logging role, see Logging roles.

Build queries

To build queries by using the Google Cloud console, do the following:

1. In the navigation panel of the Google Cloud console, select Logging, and then select Logs Explorer:

   Go to Logs Explorer

2. Select the Google Cloud project or other Google Cloud resource for which you want to view logs.

   To view the logs that you're sending from an Amazon Web Services (AWS) account to Logging, select the AWS connector project in the Google Cloud console resource picker and then use the Logs Explorer. The AWS connector project stores the Amazon Resource Name (ARN) for your AWS account and links your AWS account to Google Cloud services. For more information, see Collect metrics from AWS accounts.

3. Use the Query pane to build your query.

   

   The Query pane provides multiple ways to build and run query expressions:

   • Search for text across all log fields.
   • Select options from filter menus.
   • Write or modify queries by using the query editor.
   • View, edit, or run the queries in the Recent, Saved, Suggested and Library tabs.

Search for text across log fields

To search for text across all log fields and find all matching log entries, enter your search terms in the search field:

You can search for words and phrases, and your search terms can include Boolean operators and regular expressions:

• To perform a case-insensitive search along token boundaries, enter the search terms without backticks or double quotes.

  For example, to search for log entries that contain the word hello and the word world, enter hello world. This command, which is converted to SEARCH("hello world"), matches log entries that contain the tokens hello and world, in any order. Because the search is case-insensitive, the search also matches a log entry that contains the tokens Hello and World. The search doesn't match the token worlds.

• To perform a case-insensitive search for a phrase along token boundaries, wrap the phrase in backticks.

  For example, to search for the phrase hello world, enter `hello world`. This command, which is converted to SEARCH("`hello world`"), matches log entries that contain the token hello world. The search doesn't match the token hello worlds.

• To perform a case-insensitive search for a substring, wrap the text in double quotes. For example, "hello world" matches Hello World and Hello world. The same query also matches hello worlds, because the search isn't performed along token boundaries.

  Note: Substring searches are slower than token-based searches. Therefore, we recommend that you use token-based searches when possible.

To see your search terms within the query expression, enable Show query.

After you enter your search terms, click Run query or press the Enter key. The results of the query are displayed in the Query results pane.

Boolean operators

Your search-field entries are converted into Boolean expressions that specify a subset of all the log entries in your selected Google Cloud resource.

The search field supports the usage of the Boolean operators AND, OR, and NOT. When using Boolean operators in your search expressions, note the following:

• You can't use parentheses to nest rules. Any parentheses in the search expression are parsed as search terms.
• You must capitalize Boolean operators. Lowercase and, or, and not are parsed as search terms, not as operators.

If you don't include any operators, all search terms and phrases are joined by AND. You can omit the AND operator between search terms.

The AND and OR operators are short-circuit operators. You can combineAND and OR rules in the same expression. For example, when the two operators are mixed, the expression a AND b OR c AND d turns into the following Logging query language expression:

"a"
"b" OR "c"
"d"

The NOT operator has the highest precedence, followed by OR and AND in that order.

The NOT operator performs a negation of the subsequent term. For example, NOT error returns log entries that don't contain error. You can also replace the NOT operator with the - (minus) operator. For example, the following two queries are the same:

"response" AND "successful" AND NOT "error"

"response successful" -"error"

This logic also works with a phrase, if the - (minus) operator is outside the quotation marks. For example, the following two queries are the same:

-"response successful"

NOT "response successful"

Construct queries with filter menus

You can use the filter menus in the Query pane to add resource, log name, and log severity parameters to the query-editor field. These options correspond to the LogEntry fields for all logs in Logging.

The options in the Resource and Log name menus only display logs that are currently stored in Cloud Logging.

• Resource: Lets you specify the resource.type and associated resource.labels. You can select a single resource type using this filter menu, and zero or more resource labels to apply to your query. The resource parameters are joined by the logical operator AND.
• Log name: Lets you specify the logName. You can select multiple log names to apply to your query. When selecting multiple log names, the logical operator OR is used.
• Severity: Lets you specify the severity. You can select multiple severity levels at the same time to add to apply to your query. When selecting multiple severity levels, the logical operator OR is used.

To use any of the filter menus, do the following:

1. Expand the Menu on any of the filter menus in the Query pane.

2. Refine the filter parameters.

3. Click Apply. You see the parameters in the query-editor field.

   To see your search terms within the query expression, enable Show query.

4. After you review the query, click Run query. The results of the query are displayed in the Query results pane.

For certain Compute Engine resource types, such as gce_instance and gce_network, you see the resource name with the resource ID as subtext. For example, for the gce_instance resource type, you see the VM name alongside the VM ID. The resource names help you identify the correct resource ID, on which you can build queries.

View logs by time range

There are two ways to display logs that were written in a specific time range:

1. Use the time-range selector.
2. Include a timestamp expression in the query-editor field.

Use the time-range selector

The default time range is one hour, but you can select from preset time options, specify a custom start and end time, or center the time range around a specific timestamp by using the time-range selector. For example, if you want to view the data for the past week, then select Last 1 week from the time-range selector.

You can also set your time zone preferences by using the time-range selector.

Include a timestamp expression in the query-editor field

To add a timestamp expression directly to the query-editor field, use the Logging query language.

If the query-editor field contains an expression with a timestamp, then the time-range selector is disabled, and the query uses the timestamp expression as its time-range restriction. If a query doesn't use a timestamp expression, then the query uses the time-range selector as its time-range restriction.

Write advanced queries using the Logging query language

You can use the Logging query language to build more advanced queries in the Logs Explorer query-editor field:

1. If you don't see the query-editor field in the Query pane, enable Show query.

2. Enter your query expressions directly into the query-editor field.

   If you added any search terms in the search field or selected any parameters in the filter menus, then those also appear in the query-editor field and are evaluated as part of your query expression.

3. After you review your query, click Run query.

   Logs that match your query are listed under the Query results pane. The Histogram and Log fields panes also adjust according to the query expression.

For examples of common queries you might want to use, see Sample queries using the Logs Explorer.

Note: The length of a query can't exceed 20,000 characters.

Use recent queries

When you run any query, the query is added to your Recent queries list, which contains the last 10,000 unique queries over a 30-day period.

To view your recent queries, select the Recent tab in the Query pane. Within the Recent tab, you have the following options:

• Stream: To run the query and stream the results, choose this option.
• Run: To run the query, choose this option.

• More options: Lets you view the query expression with the options to run the query or save it to your list of Saved queries. You can also select the query directly to get these options.

  To save the query, do the following:

  1. Click Save as. The Save query dialog opens.

  2. Complete the following fields:

     • Name (Required): Provide a name for your query. Names are limited to 64 characters.
     • Description (Optional): Provide a description to help identify the purpose of the query.
     • Include summary fields (Optional): Enable Include summary fields and enter the summary fields that you want to display.
     • Truncate summary fields (Optional): Enable Truncate summary fields and select the number of characters to truncate to and whether truncation occurs at the beginning or end of the fields.

  3. Click Save query. The query is now available in your Saved queries list.

You can also sort and filter your recent queries; the filter matches on the text in your query expression.

Save queries

Note: If your data is managed through an Assured Workloads environment, then this feature might be impacted or restricted. For information, see Restrictions and limitations in Assured Workloads.

The Query pane features a Saved tab, where you can access your saved queries. Saved queries let you store query expressions to help you explore your logs more consistently and efficiently.

To save a query expression that you've built in the query-editor field, do the following:

1. Click Save in the Query pane. The Save query dialog opens, with your query expression in the query-editor field.

2. Complete the following fields:

   • Name (Required): Provide a name for your query. Names are limited to 64 characters.
   • Description (Optional): Provide a description to help identify the purpose of the query.
   • Include summary fields (Optional): Enable Include summary fields and enter the summary fields that you want to display.
   • Truncate summary fields (Optional): Enable Truncate summary fields and select the number of characters to truncate to and whether truncation occurs at the beginning or end of the fields.
   1. Click Save query. Your saved queries appear in a list under the Saved tab.

To run a saved query, click Run. To run the query and stream the results, click Stream.

You can also sort and filter your saved queries; the filter matches the text in your query expression.

Share queries

Shared queries let users of a Google Cloud project share their saved queries with each other. You can view shared queries within the Saved tab.

For the roles and permissions needed to view and edit shared queries, see Google Cloud console permissions. Note that users who have the IAM role roles/logging.admin or roles/editor can edit other users' shared queries.

Create a shared query

You can share queries that you've already saved, or you can share a new query.

To create and share a query, do the following:

1. Enter a query in the query-editor field.

2. Click Save.

3. Complete the fields in the Save query dialog.

4. Enable Share with project.

5. Click Save query.

Your query is now shared with other users of the Google Cloud project.

To share an already-saved query, do the following:

1. Select the Saved tab.

2. Select More options and then select Edit, or select the query directly.

3. In the Edit query dialog, enable Share with project, and then click Update query.

Your query is now shared with other users of the Google Cloud project.

View shared queries

To quickly view all shared queries, sort the Visibility column to show shared queries first:

1. Select the Saved tab.

2. Click All.

3. Sort the Visibility column.

The Visibility column indicates if and how the queries are shared:

• Shared by me: Queries that you have saved and shared with other users of the Google Cloud project.
• Shared: Queries that other users of the Google Cloud project have shared.
• Private: Queries that you have saved and didn't share with other users of the Google Cloud project.

View only your queries

To view saved queries that you created or shared, click Mine. You now see a list of queries that you've created and saved. In the Visibility column, you can see your unshared Private queries. Queries that you've shared are denoted by Shared by me.

Use suggested queries

Logging generates suggested queries based on the context of your Google Cloud project, such as the Google Cloud products you're using. Suggested queries can help you identify issues and provide you with insights into the overall health of your systems. For example, detecting that you're using Google Kubernetes Engine, Logging might suggest a query that finds all the error logs for your containers.

To view and run suggested queries, select the Suggested tab in the Query pane. The Suggested tab shows you a list of queries, each with descriptions and the following options:

• Stream: To run the query and stream the results, choose this option.
• Run: To run the query, choose this option.

• More options: Lets you view the details of the query expression with the options to run the query or save it. You can also select the query directly to get these options.

  To review the details of a suggested query, do either of the following:

  • Select the query's row.

  • Click More and select View. The Query details dialog opens.

  In the Query details dialog, you see the query and the options to Run, Stream or Save As:

  • To save the query, do the following:

    1. Click Save As.
    2. Complete the fields in the Save query dialog.

    The edited query shows up in your Saved list, where you can choose to run the query later.

  • To run the query now, click Run. The query runs and appears in the query-editor field.

  • To run the query now and stream the results, click Stream.

  • To close the dialog and return to the suggested queries list, click Close.

Note the following expected behaviors:

• Successive page loads might not show the same queries in the same order.
• You might see zero suggested queries.
• Sometimes running a suggested query returns zero logs.

Select queries from the library

Logging provides a library of queries based on common use cases and Google Cloud products. These queries can help you efficiently find logs during time-critical troubleshooting sessions and explore your logs to better understand what logging data is available.

To view and run the library's queries, do the following:

1. Select the Library tab in the Query pane.

2. In the All queries column, you see broad categories of available queries and subsets of queries based on Google Cloud products. To narrow the selection of queries that you see, click on any of the products.

   You can also use the search field to search the available queries by category, description, or the contents of the query expression.

3. To review a query expression, do either of the following:

   a. Click on the query's row.

   b. Click More and select View.

4. In the Query details dialog, you see the query and the options to Run, Stream or Save As:

   • To save the query, do the following:

     1. Click Save As.
     2. Complete the fields in the Save query dialog.

     The edited query shows up in your Saved list, where you can choose to run the query later.

   • To run the query now, click Run. The query runs and appears in the query-editor field.

   • To run the query now and stream the results, click Stream.

   • To close the dialog and return to the suggested queries list, click Close.

What's next

• Sample queries