This document explains how you can find log entries that you routed to Cloud Logging buckets. Log buckets are Cloud Logging storage containers in your Google Cloud projects that hold your logs data. You can create log sinks to route all, or just a subset, of your logs to any bucket in Cloud Logging. This flexibility allows you to choose which Google Cloud project your logs are stored in and what other logs are stored with them.
Logs that you route to Cloud Logging buckets are available immediately.
For information about viewing logs routed to other destinations, see the following documents:
Before you begin
For instructions about how to create and manage log buckets, see the following documents:
For a conceptual discussion of sinks, see Overview of routing and storage models: Sinks.
For instructions about how to route your logs, see Route logs to supported destinations.
View logs
To troubleshoot and view individual log entries in a log bucket, do the following:
-
In the Google Cloud console, select Logging, and then select Logs Explorer, or click the following button:
- In the Action toolbar, select Refine scope.
- On the Refine scope dialog, select Scope by storage.
Select one or more log buckets and then click Apply.
For information about how to run queries, see Build queries in the Logs Explorer.
To perform analytics on log entries stored in a log bucket that is upgraded to use Log Analytics, do the following:
-
In the Google Cloud console, select Logging, and then select Log Analytics, or click the following button:
In the Log views list, find the view, and then select Query. The Query pane is populated with a default query, which includes the name of the table that is queried.
You can also enter a query in the Query pane, or edit a displayed query.
To query all logs in the log bucket, select the
_AllLogs
view.In the toolbar, click Run query.
The query is executed and the result of the query is shown in the Results tab.
You can use the toolbar options to format your query, clear the query, and open the BigQuery SQL reference documentation.
For information about how to run queries, see Build queries using SQL.
Log entries organization
Logging log entries are objects of type LogEntry
.
Log entries with the same log type, referred to as [LOG_ID]
in the
LogEntry
reference, usually have the same format. The following
table shows sample log entries:
syslog
The Compute Engine syslog
is a custom log type produced by the
logging agent,
google-fluentd
, which runs on
virtual machine instances:
{
logName: "projects/my-gcp-project-id/logs/syslog",
timestamp: "2015-01-13T19:17:01Z",
resource: {
type: "gce_instance",
labels: {
instance_id: "12345",
zone: "us-central1-a",
project_id: "my-gcp-project-id"
}
},
insertId: "abcde12345",
textPayload: "Jan 13 19:17:01 my-gce-instance /USR/SBIN/CRON[29980]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)"
}
request_log
The App Engine request_log
has log entries containing
protoPayload
fields which hold objects of type
RequestLog
:
{
logName: "projects/my-gcp-project-id/logs/appengine.googleapis.com%2Frequest_log",
timestamp: "2015-01-13T19:00:39.796169Z",
resource: {
type: "gae_app",
labels: {
module_id: "default",
zone: "us6",
project_id: "my-gcp-project-id",
version_id: "20150925t173233"
}
}
httpRequest: {
status: 200
}
insertId: "abcde12345",
operation: {
id: "abc123",
producer: "appengine.googleapis.com/request_id",
first: true,
last: true
}
protoPayload: {
@type: "type.googleapis.com/google.appengine.logging.v1.RequestLog"
versionId: "20150925t173233",
status: 200,
startTime: "2017-01-13T19:00:39.796169Z",
# ...
appId: "s~my-gcp-project-id",
appEngineRelease: "1.9.17",
}
}
activity
The activity
log is an Admin Activity audit log.
Its payload is a JSON representation of the
AuditLog
type:
{
logName: "projects/my-gcp-project-id/logs/cloudaudit.googleapis.com%2Factivity"
timestamp: "2017-04-22T13:41:32.245Z"
severity: "NOTICE"
resource: {
type: "gce_instance"
labels: {
instance_id: "2403273232180765234"
zone: "us-central1-b"
project_id: "my-gcp-project-id"
}
}
insertId: "54DC1882F4B49.A4996C2.6A02F4C1"
operation: {
id: "operation-1492868454262-54dc185e9a4f0-249fe233-f73d472a"
producer: "compute.googleapis.com"
last: true
}
protoPayload: {
@type: "type.googleapis.com/google.cloud.audit.AuditLog"
authenticationInfo: {
principalEmail: "649517127304@cloudservices.gserviceaccount.com"
}
requestMetadata: {…}
serviceName: "compute.googleapis.com"
methodName: "v1.compute.instances.delete"
resourceName: "projects/my-gcp-project-id/zones/us-central1-b/instances/abc123"
}
}
Troubleshooting
If logs seem to be missing from your sink's destination or you otherwise suspect that your sink isn't properly routing logs, then see Troubleshoot routing and sinks.
Pricing
Cloud Logging doesn't charge to route logs to a
supported destination; however, the destination might apply charges.
With the exception of the _Required
log bucket,
Cloud Logging charges to stream logs into log buckets and
for storage longer than the default retention period of the log bucket.
For more information, see Cloud Logging pricing.
Note also that if you send and then exclude your Virtual Private Cloud flow logs from Cloud Logging, VPC flow log generation charges apply in addition to the destination charges.