Before you begin
You don't need a Workspace to use Logging, unless you are sending logs from Amazon Web Services (AWS) to Logging.
The Logs Explorer displays logs for a single Google Cloud project. If you are using a Workspace, Logging doesn't combine the logs from the monitored projects. You must select a specific project to view its logs.
If you're using a Workspace and AWS, select the AWS connector project to see the AWS logs.
To navigate to the Logs Explorer, do the following:
Go to the Google Cloud navigation menu menu and
select Logging > Logs Explorer:
Go to the Logs Explorer
- Select a Google Cloud project.
- From the Upgrade menu, switch from Legacy Logs Viewer to Logs Explorer.
You're now in the Logs Explorer.
The query-builder pane provides multiple ways to retrieve logs:
- Query builder drop-down menus.
- Queries using the query builder language.
- Recent, Saved, and Suggested queries tabs.
The following sections describe how to build and run queries to retrieve your logs.
Query builder drop-down menus
The drop-down menus let you add query parameters to the Query builder. You can use the drop-down menus to select resources, log names, log severity, and time ranges. These options correspond to the LogEntry fields for all Google Cloud's operations suite logs.
- Resource: Lets you specify resource.type. You can select a
single resource at a time to add to the Query builder. Entries use
the logical operator
- Log name: Lets you specify logName. You can select multiple
log names at once to add to the Query builder. When selecting
multiple entries, the logical operator
- Severity: Lets you specify [severity][severity]. You can select multiple
severity levels at once to add to the Query builder. When selecting
multiple entries, the logical operator
To use any of the search parameter menus, expand them and select a parameter, or multiple parameters, then click Add.
Once you've built your query, click Run Query to retrieve your desired log entries.
Queries with a time restriction
There are two ways to query logs based on time:
- Query using a timestamp expression
Query using the time-range selector
If you have a query with a timestamp, the time-range selector is disabled, and the query uses the timestamp expression as its time-range restriction. If a query doesn't use a timestamp expression, then the query uses the time-range selector as its time-range restriction.
Queries using the query builder language
For more details, see query builder language.
The following are some suggestions for optimizing your queries.
Optimize your queries
To find log entries more efficiently, do the following:
- Search using indexed fields.
- Minimize the number of log entries that must be searched.
Use indexed fields
To take advantage of the index, specify exact values for indexed fields using the equality operator. Don't use substring matches.
The following LogEntry fields are indexed:
- indexed resource labels in resource.labels
The next sections explain how to use these indexed fields to minimize the number of log entries to be queried.
Find log entries quickly
Make your searches faster by reducing the number of logs, the number of log entries, or the time span of your searches. Even better, reduce all three.
Example: Use the right log name
Specify the log containing the log entries you're interested in. Be sure you know the real log name by inspecting one of your log entries. For example, the Logs Explorer shows that there is a log in the Compute Engine section named "activity_log". On closer inspection of the activity log entries, the log is actually named "compute.googleapis.com/activity_log".
The following comparison is incorrect. It doesn't match anything because it uses the wrong log name:
logName = "projects/my-project-id/logs/activity_log" # WRONG!
The following comparison is correct. It chooses log entries from the activity log. You must URL-encode the log name, as shown:
logName = "projects/my-project-id/logs/compute.googleapis.com%2Factivity_log"
Example: Choose the right log entries
If you know that the log entries you want are coming from a particular VM
instance, then specify it. Check for the right label names by inspecting one of
the log entries that you want to search for. In the following example,
instance_id is one of the indexed labels:
logName = "projects/my-project-id/logs/compute.googleapis.com%2Factivity_log" resource.type = "gce_instance" AND resource.labels.instance_id = "6731710280662790612"
You must specify a value for the resource type. Otherwise, the comparison of
instance_id doesn't use the index.
Example: Choose the right time period
You should specify a time period to search in. A quick way of determining useful
timestamps in RFC 3339 format is to use the Gnu/Linux
$ date --rfc-3339=s 2016-06-27 17:39:00-04:00 $ date --rfc-3339=s --date="3 hours ago" 2016-06-27 14:40:00-04:00 $ date --rfc-3339=s --date="5 hours ago" 2016-06-27 12:40:00-04:00
Use the values of these timestamps in the following queries. To create a
timestamp acceptable to Logging, replace the space between the
date and time with the letter
For example, to search within the last three hours:
timestamp >= "2016-06-27T14:40:00-04:00"
As another example, to search between three and five hours ago:
timestamp >= "2016-06-27T12:40:00-04:00" AND timestamp <= "2016-06-27T14:40:00-04:00"
Minimize global and substring searches
Avoid the temptation to take shortcuts when typing logging queries.
Example: Don't use substrings in indexed fields
You are searching for a log entry from the Apache2 web server. You know that
the Apache logs are named
apache-error. What do you do?
Don't use a substring match to save yourself some typing:
logName:apache # THIS CAUSES A SLOW SEARCH!
Do use exact-matches when searching indexed fields:
logName = ("projects/my-project-id/logs/apache-access" OR "projects/my-project-id/logs/apache-error")
An indexed field loses all of its quickness when you do a substring search.
Example: Don't use global searches
You're searching for a log entry with "Hello, Kitty" in the payload:
Don't use a global search. One reason is they are all substring searches:
"Hello, Kitty" # THIS CAUSES A SLOW SEARCH!
Do limit the search to a single field, even if you must keep the substring search:
Do use an equality test if you can:
textPayload = "Hello, Kitty"
Do reference individual fields in a payload, if your log entries have structured payloads:
jsonPayload.my_favorite_cat = "Hello, Kitty"
Do use an indexed field to restrict the search:
logName = "projects/my-project_id/logs/somelog" AND jsonPayload.my_favorite_cat = "Hello, Kitty"
The log entries shown are the ones that match the query in the search-query box. If the Jump to date menu contains a value, then the display scrolls to that point in time. Here are some query examples:
Finds the same log entries as the basic query interface would show if you selected GAE Application (All module IDs) from the resource log menu and All logs from the log name menu. For a list of resource types, go to Monitored resource list.
As you type, the Logs Explorer suggests completions for fields like
resource.type=gae_app AND logName:request_log
Finds log entries for App Engine apps from log names containing
request_log. Note several things:
=operator is exact equality. The resource type must be exactly
"gae_app"except for letter case.
:operator means "has". The
logNamefield must contain
request_log, in any letter case. The actual log name is much longer. Using
:might result in slower searches.
- The two comparisons are joined by
AND. You can also use
ANDis assumed if you leave out the operator.
resource.type = (gce_instance OR aws_ec2_instance) AND severity >= ERROR
Finds log entries with either of two resource types: Compute Engine VM instance or AWS EC2 VM instance. The log entries must have
severityof at least
ERROR, which is equivalent to selecting ERROR in the basic query interface's severity menu. You cannot view logs from multiple resource types in the basic query interface.
logName = "projects/[PROJECT_ID]/logs/cloudaudit.googleapis.com%2Factivity"
Finds all the Admin Activity audit log entries in the project
[PROJECT_ID]. Audit logs all use the same log name in a project, but have different resource types. The log ID,
cloudaudit.googleapis.com/activitymust be URL-encoded in the log name. Using equality in the comparison speeds up the search. For more information, review Understanding audit logs.
Finds log entries containing
unicornin any field, in any letter case. A search term that isn't part of a field comparison is an "all fields" query.
Finds log entries that contain
unicornin some field and
phoenixin some field.
Finds log entries whose
textPayloadfield contains both
phoenixin any order—the
ANDis implicit between the two words.
Finds log entries whose
textPayloadfield contains the string
"unicorn phoenix". This is the same as in the basic query interface.
NOT textPayload: "unicorn phoenix"
Finds log entries whose
textPayloadfield does not contain the string
"unicorn phoenix". This type of query reduces unwanted log entries.
timestamp >= "2016-11-29T23:00:00Z" timestamp <= "2016-11-29T23:30:00Z"
Finds log entries within a 30-minute period.
The query-builder pane features a Saved tab, where you can access your saved queries.
Saved queries let you store query expressions to help you explore your logs more consistently and efficiently.
To save a query that you have built in the query-builder pane, do the following:
Click Save in the query-builder pane. The Save query dialog opens, with your query expression in the Query field.
Add a Name for your query.
Names are limited to 64 characters.
Optional: To add summary fields to your query, toggle Include summary fields.
Optional: Add a Description for your query.
Descriptions are limited to 1,000 characters. Don't include any sensitive information.
Click Save query in the dialog.
Your saved queries appear in a list under the Saved tab in the query-builder pane.
To run a saved query, click Run query.
The query-builder pane features a Recent tab. When you run a query, the query is added to your Recent queries list, which contains the last 10,000 unique queries over a 30-day period.
To view your recent queries, select the Recent tab in the query-builder pane. Within the Recent tab, you have the following options:
- Run. Runs the query.
More options more_vert. Lets you view the query parameters with the options to run the query or save it to your list of Saved queries. You can also select the query directly to get these options.
To run the query, select Run.
To save the query, select Save as, and complete the following fields:
- Name (Required): Provide a name.
- Description (Optional): Provide a description to help identify the purpose of the query.
- Include summary fields (Optional): Enable Include summary fields and enter the fields you want displayed.
- Truncate summary fields (Optional): Enable Truncate summary fields and select the number of characters to truncate to and whether truncation occurs at the beginning or end of the fields.
Select Save query. The query is now available in your Saved queries list.
Note that you can also filter your recent queries; the filter matches on the text in your query's expression.
Logging generates suggested queries based on its understanding of the Google Cloud products you're using. It proposes queries that can help you pinpoint issues and provide you with insights into the overall health of your systems. For instance, detecting that you're using Google Kubernetes Engine, Logging might suggest a query that finds all the error logs for your containers.
To view and run Suggested queries in the query-builder pane, do the following:
Click on the Suggested tab.
Review the description of the Suggested query.
To review the details of a Suggested query, do either of the following:
a. Click on the row.
b. Click More more_vert and select View.
In the query-builder pane, you see the query and the options to Run or Save it.
a. To save the query, click Save. The query shows up in your Saved list, where you can choose to run the query later.
b. To run the query, click Run. The query shows up under Query preview in the query-builder pane.
After you review the query, click Run query. The results of the suggested query are displayed under Query results.
Suggested queries are created dynamically based on the Google Cloud project context. Successive page loads might not show the same queries in the same order. This is expected behavior.
You can omit quotation marks around text strings that don't
contain whitespace or certain special characters. This is called
unquoted text, and an example is the word
ERROR in the preceding
example. The string
"v1.compute.instances.insert" is quoted because it
contains periods. If you want to include a quotation mark within a string,
precede the quotation mark with a backslash.
If you encounter issues using the query builder language, check the following:
Your query obeys the syntax rules, with matched parentheses and quotation marks. Your query can't include comments.
Your log entry field names are correctly spelled.
Boolean operations are in upper-case letters (
Boolean expressions as global restrictions or as the right-hand side of comparisons should be parenthesized for clarity. For example, the two queries below look the same, but are not:
insertId = "ABC-1" OR "ABC-2" # ERROR!? insertId = ("ABC-1" OR "ABC-2")
Unquoted text must not contain any special characters. When in doubt, add double quotation marks. For example, the first comparison below is illegal because of the embedded substring operator (
:). The comparison must be written with quotation marks:
insertId = abc:def # ILLEGAL! insertId = "abc:def"
gcloud logging requires the query to be in double quotes. To use double quotes for escaping special characters using the
gcloud loggingcommand, wrap the entire query with single quotes instead:
gcloud logging read 'resource.type=gce_instance AND jsonPayload.message="Stopped Unattended Upgrades Shutdown."' gcloud logging read 'timestamp>="2020-06-17T21:00:00Z"'
If you are writing a query that includes a timestamp, you must select No limit from the time-range selector below the search-query box.