Stackdriver Logging is part of the Stackdriver suite of products in Google Cloud Platform (GCP). It includes storage for logs, a user interface called the Logs Viewer, and an API to manage logs programmatically. Stackdriver Logging lets you read and write log entries, search and filter your logs, export your logs, and create logs-based metrics.
Logs are associated primarily with GCP projects, although organizations can also have logs. The Logs Viewer shows only the logs from one project, but using the API you can read log entries from multiple projects.
A log entry records status or an event. The entry might be created by GCP services, AWS services, third-party applications, or your own applications. The "message" the log entry carries is called the payload, and it can be a simple string or structured data.
Your project receives log entries when you begin to use the services that routinely produce log entries, like Compute Engine or BigQuery. You also get log entries when you connect Stackdriver to AWS, when you install the Stackdriver Logging agent on your VM instances, and when you call the entries.write method in the API.
A log is a named collection of log entries within a GCP project or organization.
Each log entry includes the name of its log. A log name can be a simple
syslog, or a structured name including the log's author, like
compute.googleapis.com/activity. Logs exist only if they have log entries.
Log entries are held in Stackdriver Logging for a limited time known as the retention period. After that, the entries are deleted. If you want to keep your log entries longer, export them outside of Stackdriver Logging.
The retention periods for different types of logs are listed in the Stackdriver Logging Quota Policy.
Each log entry indicates where it came from by including the name of a monitored resource. Examples are individual Compute Engine VM instances, individual Amazon EC2 VM instances, database instances, and so on. For a complete listing of monitored resource types, see Monitored Resources and Services.
An advanced logs filter is an expression in the Stackdriver Logging filter language. It is used in the Logs Viewer and the Stackdriver Logging API to select log entries, such as those from a particular VM instance or those arriving in a particular time period with a particular severity level.
Exporting logs using sinks
Log entries received by Stackdriver Logging can be exported to Cloud Storage buckets, BigQuery datasets, and Cloud Pub/Sub topics. You export logs by configuring log sinks, which then continue to export log entries as they arrive in Stackdriver Logging. A sink includes a destination and a filter that selects the log entries to export.
Metrics are a feature of Stackdriver Monitoring. A logs-based metric is a metric whose value is the number of log entries that match a filter that you specify. Creation of logs-based metrics is restricted according to your service tier.
An audit log is a permanent log written by a GCP service to record administrative or user actions. Audit logs appear in projects alongside other logs. For more information, see Audit Logs.
Most logs can be read by anyone with Viewer access to the project. Reading private logs requires Owner access to the project. You can also share the ability to read logs in a project using Cloud Identity and Access Management. For more information, see Access Control.
To read private logs, primarily Data Access audit logs, you must have special permission or be an owner of the project containing the private logs.
Service tiers and Stackdriver accounts
A Stackdriver account collects a number of GCP projects so that you can monitor them together. Stackdriver accounts are required for using Stackdriver Monitoring, but not for using only Stackdriver Logging. Without a Stackdriver account, your project is considered to be in the Basic service tier, which is free but imposes certain restrictions, including a cap on the amount of logs your projects can receive each month for free. For more information, see Stackdriver Accounts and Stackdriver Pricing.