Building log queries

This page describes how to build queries in the Google Cloud Console Logs Explorer to retrieve, refine, and analyze logs.

Before you begin

You don't need a Workspace to use Logging, unless you are sending logs from Amazon Web Services (AWS) to Logging.

The Logs Explorer displays logs for a single Google Cloud project. If you are using a Workspace, Logging doesn't combine the logs from the monitored projects. You must select a specific project to view its logs.

If you're using a Workspace and AWS, select the AWS connector project to see the AWS logs.

Getting started

To navigate to the Logs Explorer, do the following:

  1. Go to the Google Cloud navigation menu and select Logging > Logs Explorer:
    Go to the Logs Explorer
  2. Select a Google Cloud project.
  3. From the Upgrade menu, switch from Legacy Logs Viewer to Logs Explorer.

You're now in the Logs Explorer.

Building queries

The query-builder pane provides multiple ways to retrieve logs:

  1. Query builder drop-down menus.
  2. Queries using the query builder language.
  3. Recent, Saved, and Suggested queries tabs.

query-builder-pane

The following sections describe how to build and run queries to retrieve your logs.

Query builder drop-down menus

The drop-down menus let you add query parameters to the Query builder. You can use the drop-down menus to select resources, log names, log severity, and time ranges. These options correspond to the LogEntry fields for all Google Cloud's operations suite logs.

Drop down menus for query builder

  • Resource: Lets you specify resource.type. You can select a single resource at a time to add to the Query builder. Entries use the logical operator AND.
  • Log name: Lets you specify logName. You can select multiple log names at once to add to the Query builder. When selecting multiple entries, the logical operator OR is used.
  • Severity: Lets you specify severity. You can select multiple severity levels at once to add to the Query builder. When selecting multiple entries, the logical operator OR is used.

To use any of the search parameter menus, expand them and select a parameter, or multiple parameters, then click Add.

Once you've built your query, click Run Query to retrieve your desired log entries.

Queries with a time restriction

There are two ways to query logs based on time:

  1. Query using a timestamp expression
  2. Query using the time-range selector

    Logs Explorer is showing two ways to filter by time

If you have a query with a timestamp, the time-range selector is disabled, and the query uses the timestamp expression as its time-range restriction. If a query doesn't use a timestamp expression, then the query uses the time-range selector as its time-range restriction.

Queries using the query builder language

You can use the query builder language to build queries in the Cloud Logging query-builder pane, as well as in the Logging API, or the command-line interface.

For more details, see query builder language.

The following are some suggestions for optimizing your queries.

Optimize your queries

To find log entries more efficiently, do the following:

  • Search using indexed fields.
  • Minimize the number of log entries that must be searched.

Use indexed fields

To take advantage of the index, specify exact values for indexed fields using the equality operator. Don't use substring matches.

The following LogEntry fields are indexed:

The next sections explain how to use these indexed fields to minimize the number of log entries to be queried.

Find log entries quickly

Make your searches faster by reducing the number of logs, the number of log entries, or the time span of your searches. Even better, reduce all three.

Example: Use the right log name

Specify the log containing the log entries you're interested in. Be sure you know the real log name by inspecting one of your log entries. For example, the Logs Explorer shows that there is a log in the Compute Engine section named "activity_log". On closer inspection of the activity log entries, the log is actually named "compute.googleapis.com/activity_log".

The following comparison is incorrect. It doesn't match anything because it uses the wrong log name:

    logName = "projects/my-project-id/logs/activity_log"   # WRONG!

The following comparison is correct. It chooses log entries from the activity log. You must URL-encode the log name, as shown:

    logName = "projects/my-project-id/logs/compute.googleapis.com%2Factivity_log"

Example: Choose the right log entries

If you know that the log entries you want are coming from a particular VM instance, then specify it. Check for the right label names by inspecting one of the log entries that you want to search for. In the following example, instance_id is one of the indexed labels:

    logName = "projects/my-project-id/logs/compute.googleapis.com%2Factivity_log"
    resource.type = "gce_instance" AND
    resource.labels.instance_id = "6731710280662790612"

You must specify a value for the resource type. Otherwise, the comparison of instance_id doesn't use the index.

Example: Choose the right time period

You should specify a time period to search in. A quick way of determining useful timestamps in RFC 3339 format is to use the Gnu/Linux date command:

$ date --rfc-3339=s
2016-06-27 17:39:00-04:00
$ date --rfc-3339=s --date="3 hours ago"
2016-06-27 14:40:00-04:00
$ date --rfc-3339=s --date="5 hours ago"
2016-06-27 12:40:00-04:00

Use the values of these timestamps in the following queries. To create a timestamp acceptable to Logging, replace the space between the date and time with the letter T.

For example, to search within the last three hours:

    timestamp >= "2016-06-27T14:40:00-04:00"

As another example, to search between three and five hours ago:

    timestamp >= "2016-06-27T12:40:00-04:00" AND
    timestamp <= "2016-06-27T14:40:00-04:00"

For another example of using timestamps, go to Temporary field indexes on this page.

Minimize global and substring searches

Avoid the temptation to take shortcuts when typing logging queries.

Example: Don't use substrings in indexed fields

You are searching for a log entry from the Apache2 web server. You know that the Apache logs are named apache-access and apache-error. What do you do?

  • Don't use a substring match to save yourself some typing:

        logName:apache   # THIS CAUSES A SLOW SEARCH!
    
  • Do use exact-matches when searching indexed fields:

        logName = ("projects/my-project-id/logs/apache-access" OR
                   "projects/my-project-id/logs/apache-error")
    

An indexed field loses all of its quickness when you do a substring search.

Example: Don't use global searches

You're searching for a log entry with "Hello, Kitty" in the payload:

  • Don't use a global search. One reason is they are all substring searches:

        "Hello, Kitty"   # THIS CAUSES A SLOW SEARCH!
    
  • Do limit the search to a single field, even if you must keep the substring search:

        textPayload:"Hello, Kitty"
    
  • Do use an equality test if you can:

        textPayload = "Hello, Kitty"
    
  • Do reference individual fields in a payload, if your log entries have structured payloads:

        jsonPayload.my_favorite_cat = "Hello, Kitty"
    
  • Do use an indexed field to restrict the search:

        logName = "projects/my-project_id/logs/somelog" AND
        jsonPayload.my_favorite_cat = "Hello, Kitty"
    

Searching examples

The log entries shown are the ones that match the query in the search-query box. If the Jump to date menu contains a value, then the display scrolls to that point in time. Here are some query examples:

resource.type=gae_app

Finds the same log entries as the basic query interface would show if you selected GAE Application (All module IDs) from the resource log menu and All logs from the log name menu. For a list of resource types, go to Monitored resource list.

As you type, the Logs Explorer suggests completions for fields like resource.type.

resource.type=gae_app AND logName:request_log

Finds log entries for App Engine apps from log names containing request_log. Note several things:

  • The = operator is exact equality. The resource type must be exactly "gae_app" except for letter case.
  • The : operator means "has". The logName field must contain request_log, in any letter case. The actual log name is much longer. Using : might result in slower searches.
  • The two comparisons are joined by AND. You can also use OR, but AND is assumed if you leave out the operator.
resource.type = (gce_instance OR aws_ec2_instance) AND severity >= ERROR

Finds log entries with either of two resource types: Compute Engine VM instance or AWS EC2 VM instance. The log entries must have severity of at least ERROR, which is equivalent to selecting ERROR in the basic query interface's severity menu. You cannot view logs from multiple resource types in the basic query interface.

logName = "projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity"

Finds all the Admin Activity audit log entries in the project [PROJECT_ID]. Audit logs all use the same log name in a project, but have different resource types. The log ID, cloudaudit.googleapis.com/activity must be URL-encoded in the log name. Using equality in the comparison speeds up the search. For more information, review Understanding audit logs.

unicorn

Finds log entries containing unicorn in any field, in any letter case. A search term that isn't part of a field comparison is an "all fields" query.

unicorn phoenix

Finds log entries that contain unicorn in some field and phoenix in some field.

textPayload:(unicorn phoenix)

Finds log entries whose textPayload field contains both unicorn and phoenix in any order—the AND is implicit between the two words.

textPayload:"unicorn phoenix"

Finds log entries whose textPayload field contains the string "unicorn phoenix". This is the same as in the basic query interface.

NOT textPayload: "unicorn phoenix"

Finds log entries whose textPayload field does not contain the string "unicorn phoenix". This type of query reduces unwanted log entries.

timestamp >= "2016-11-29T23:00:00Z" timestamp <= "2016-11-29T23:30:00Z"

Finds log entries within a 30-minute period.

Saved queries

The query-builder pane features a Saved tab, where you can access your saved queries.

List of Saved queries

Saved queries let you store query expressions to help you explore your logs more consistently and efficiently.

To save a query that you have built in the query-builder pane, do the following:

  1. Click Save in the query-builder pane. The Save query dialog opens, with your query expression in the Query field.

  2. Add a Name for your query.

    Names are limited to 64 characters.

  3. Optional: To add summary fields to your query, toggle Include summary fields.

  4. Optional: Add a Description for your query.

    Descriptions are limited to 1,000 characters. Don't include any sensitive information.

  5. Click Save query in the dialog.

Your saved queries appear in a list under the Saved tab in the query-builder pane.

To run a saved query, click Run query.

Recent queries

The query-builder pane features a Recent tab. When you run a query, the query is added to your Recent queries list, which contains the last 10,000 unique queries over a 30-day period.

The recent tab shows recent queries.

To view your recent queries, select the Recent tab in the query-builder pane. Within the Recent tab, you have the following options:

  • Run. Runs the query.
  • More options . Lets you view the query parameters with the options to run the query or save it to your list of Saved queries. You can also select the query directly to get these options.

    To run the query, select Run.

    To save the query, select Save as, and complete the following fields:

    • Name (Required): Provide a name.
    • Description (Optional): Provide a description to help identify the purpose of the query.
    • Include summary fields (Optional): Enable Include summary fields and enter the fields you want displayed.
    • Truncate summary fields (Optional): Enable Truncate summary fields and select the number of characters to truncate to and whether truncation occurs at the beginning or end of the fields.

    Select Save query. The query is now available in your Saved queries list.

Note that you can also filter your recent queries; the filter matches on the text in your query's expression.

Suggested queries

Logging generates suggested queries based on its understanding of the Google Cloud products you're using. It proposes queries that can help you pinpoint issues and provide you with insights into the overall health of your systems. For instance, detecting that you're using Google Kubernetes Engine, Logging might suggest a query that finds all the error logs for your containers.

To view and run Suggested queries in the query-builder pane, do the following:

  1. Click on the Suggested tab.

    Suggested queries tab that show a set of queries

  2. Review the description of the Suggested query.

  3. To review the details of a Suggested query, do either of the following:

    a. Click on the row.

    b. Click More and select View.

  4. In the query-builder pane, you see the query and the options to Run or Save it.

    a. To save the query, click Save. The query shows up in your Saved list, where you can choose to run the query later.

    b. To run the query, click Run. The query shows up under Query preview in the query-builder pane.

  5. After you review the query, click Run query. The results of the suggested query are displayed under Query results.

Suggested queries are created dynamically based on the Google Cloud project context. Successive page loads might not show the same queries in the same order. This is expected behavior.

Troubleshooting

Unquoted text

You can omit quotation marks around text strings that don't contain whitespace or certain special characters. This is called unquoted text, and an example is the word ERROR in the preceding example. The string "v1.compute.instances.insert" is quoted because it contains periods. If you want to include a quotation mark within a string, precede the quotation mark with a backslash.

Syntax issues

If you encounter issues using the query builder language, check the following:

  • Your query obeys the syntax rules, with matched parentheses and quotation marks. Your query can't include comments.

  • Your log entry field names are correctly spelled.

  • Boolean operations are in upper-case letters (AND, OR, NOT).

  • Boolean expressions as global restrictions or as the right-hand side of comparisons should be parenthesized for clarity. For example, the two queries below look the same, but are not:

    insertId = "ABC-1" OR "ABC-2"  # ERROR!?
    insertId = ("ABC-1" OR "ABC-2")
    
  • Unquoted text must not contain any special characters. When in doubt, add double quotation marks. For example, the first comparison below is illegal because of the embedded substring operator (:). The comparison must be written with quotation marks:

    insertId = abc:def  # ILLEGAL!
    insertId = "abc:def"
    
  • gcloud logging requires the query to be in double quotes. To use double quotes for escaping special characters using the gcloud logging command, wrap the entire query with single quotes instead:

    gcloud logging read 'resource.type=gce_instance AND jsonPayload.message="Stopped Unattended Upgrades Shutdown."'
    gcloud logging read 'timestamp>="2020-06-17T21:00:00Z"'
    
  • If you are writing a query that includes a timestamp, you must select No limit from the time-range selector below the search-query box.