Available logs

This document provides basic information about the broad categories of logs that are available in Cloud Logging, as well as next steps for viewing and managing logs.

For an index of logging-related terms, see Basic concepts.

Categories of logs

Logging receives, indexes, and stores log entries from many sources, including Google Cloud, virtual machine (VM) instances running the Logging agent, and other cloud services providers.

The following sections broadly categorize the kinds of logs supported by Logging. The log categories are meant to help describe the logging information available to you; the categories aren't mutually exclusive.

Google Cloud platform logs

Google Cloud platform logs are service-specific logs that can help you debug and troubleshoot issues, as well as better understand the Google Cloud services you're using.

The Google Cloud platform logs visible to you in Cloud Logging vary, depending on which Google Cloud monitored resources you're using in your Cloud project, folder, or organization.

To learn more about the available Google Cloud platform logs, see Google Cloud platform logs.

Note that some Google Cloud platform logs are sent by an agent.

VPC Flow Logs record a sample of network flows sent from and received by VM instances. For details, see Using VPC Flow Logs.

User-written logs

User-written logs are written to Cloud Logging by the user in one of the common ways that users write their own logs: using the logging agent, the Cloud Logging API, or the Cloud Logging client libraries. User-written logs contain information related to their custom applications and services.

As a counterexample, App Engine writes logs to Cloud Logging on behalf of the user's software components; these component logs aren't considered user-written logs because, in that counterexample, App Engine is writing the logs to Cloud Logging on the user's behalf.

Agent logs

The Logging agent is a process that collects logs from user applications and writes them to the Cloud Logging API. For more information, see Default logging agent logs.

Component logs

Component logs are a hybrid between platform and user-written logs. These logs might serve a similar purpose to platform logs but they follow a different log entry structure.

Some logs are generated by software components that are provided by Google but that run on your VM or compute hardware, such as the fluentd logging agent or Knative logging on Google Kubernetes Engine. The software component generates the logs which, in some cases, provide a service to the users running the component.

For example, GKE provides software components that users can run on their own VM or in their own data center. Logs are generated from the user's GKE instances to a user's Cloud project, but GKE uses the logs or their metadata to provide user support.

Security logs

Cloud Logging provides two kinds of security-related logs, Cloud Audit Logs and Access Transparency logs; details are as follows.

Audit logs

Cloud Audit Logs provides the following audit logs for each Cloud project, folder, and organization:

  • Admin Activity audit logs
  • Data Access audit logs
  • System Event audit logs
  • Policy Denied audit logs

Cloud Audit Logs provide audit trails of administrative changes and data accesses of your Google Cloud resources.

For a list of Google Cloud services that write audit logs, see Google services with audit logs.

For more information about audit logging, see Cloud Audit Logs.

Access Transparency logs

Access Transparency provides you with logs of actions taken by Google staff when accessing your Google Cloud content. Access Transparency logs can help you track compliance with your organization's legal and regulatory requirements.

For a list of Google Cloud services that write Access Transparency logs, see Google services with Access Transparency logs.

For more information, including how to enable Access Transparency logs, see Access Transparency.

Multi-cloud and hybrid-cloud logs

Cloud Logging can ingest logs arriving from other cloud services providers, including Microsoft Azure and Amazon Web Services (AWS). These logs are called "multi-cloud" logs.

Cloud Logging also supports logs from your on-premises infrastructure and apps. If extending Cloud Logging to include your on-premises resources, the logs that are received by Cloud Logging are known as "hybrid-cloud" logs.

For details on how to ingest logs from your on-premises or other cloud sources, see Logging on-premises resources.

Understand and manage logs

The following sections provide basic information about using and managing logs in Cloud Logging.

Log entry structure

Cloud Logging represents logs using a single data type, LogEntry, which defines certain common data for all log entries as well as the payloads that Google Cloud services can include.

Each log entry is characterized by the following information:

  • A log name. This includes the identifier of the Cloud project, folder, or organization that contains the log entry and the identifier of the log type, LOG_ID.
  • The resource from which the log entry originated. This consists of a resource type from the Monitored resource list and corresponding label values that identify a specific instance.
  • A timestamp.
  • A payload, which can be represented as one of textPayload, jsonPayload, or (for some Google Cloud services) protoPayload.

By examining the LOG_ID string in each log entry's logName field, you can generally tell which aforementioned log category that the log entry falls into:

  • If LOG_ID contains cloudaudit.googleapis.com, it is a Cloud Audit Logs or Access Transparency log entry.
  • If LOG_ID is prefixed by serviceName.googleapis.com, it is a platform log entry.
  • If LOG_ID has no service prefix, it is most likely a user-written log entry.

For more information about log entry contents, see the Logging API reference for the LogEntry type.

View logs

There are several ways to view your log entries:

Route logs

You can control how your log entries are routed and stored. The Log Router checks each log entry against existing rules to determine which log entries to discard, which log entries to store in Logging, and which log entries to include in exports to other destinations.

For details, see Routing and storage overview.

Logs retention

Log entries are stored in log buckets for a specified length of time and are then deleted. For more information, see Routing and storage overview: retention.

Pricing

For information on logs pricing, see Logging pricing.