Cloud Logging

This page explains how to use Cloud Logging to collect and query logs from Google Kubernetes Engine (GKE) clusters.

Overview

When Logging is enabled in your cluster, your logs are stored in a dedicated, persistent datastore. Your Google Cloud project has several logs that are relevant to a GKE cluster. These include the Admin Activity log, the Data Access log, and the Events log.

A log is a collection of log entries, and each log entry applies to a certain type of resource. These are the resource types that are most relevant to GKE clusters:

Resource type	Display name
k8s_cluster	Kubernetes Cluster
gke_cluster	GKE Cluster Operations
gke_container	GKE Container
gke_nodepool	GKE Node Pool Operations

For detailed information about log entries that apply to the Kubernetes Cluster and GKE Cluster Operations resource types, see Audit logging.

While GKE itself stores logs, these logs are not stored permanently. For example, GKE container logs are removed when their host Pod is removed, when the disk on which they are stored runs out of space, or when they are replaced by newer logs. System logs are periodically removed to free up space for new logs. Cluster events are removed after one hour.

For container and system logs, GKE deploys a per-node logging agent that reads container logs, adds helpful metadata, and then stores them. The logging agent checks for container logs in the following sources:

Standard output and standard error logs from containerized processes
kubelet and container runtime logs
Logs for system components, such as VM startup scripts

For events, GKE uses a deployment in the kube-system namespace which automatically collects events and sends them to Logging.

Logging is compatible with JSON format. Logs are stored for up to 30 days.

Before you begin

To prepare for this task, perform the following steps:

Ensure that you have enabled the Google Kubernetes Engine API.

Enable Google Kubernetes Engine API

Ensure that you have installed the Cloud SDK.
Set your default project ID:
```
gcloud config set project [PROJECT_ID]
```
If you are working with zonal clusters, set your default compute zone:
```
gcloud config set compute/zone [COMPUTE_ZONE]
```
If you are working with regional clusters, set your default compute region:
```
gcloud config set compute/region [COMPUTE_REGION]
```
Update gcloud to the latest version:
```
gcloud components update
```

Note: You can override these default settings in gcloud commands using the --project, --zone, and --region operational flags.

Follow the Quickstart for Cloud Logging.

Enabling Logging

You can create a cluster with Logging enabled, or enable Logging in an existing cluster.

Creating a cluster with Logging

gcloud

When you create a cluster, the --enable-cloud-logging flag is automatically set, which enables Logging in the cluster.

To disable this default behavior, set the --no-enable-cloud-logging flag.

Console

In the Google Cloud console, go to the Kubernetes Engine > Kubernetes clusters page:

Go to Kubernetes clusters
Click Create cluster.
Configure the cluster as needed.
Click Advanced options. In the Additional features section, enable Enable Stackdriver Logging service.
Click Create.

Enabling Logging for an existing cluster

gcloud

To enable Logging for an existing cluster, run the following command, where [CLUSTER_NAME] is the name of the cluster.

gcloud container clusters update [CLUSTER_NAME] --logging-service logging.googleapis.com

console

In the Google Cloud console, go to the Kubernetes Engine > Kubernetes clusters page:

Go to Kubernetes clusters
Click Edit.
Set the Stackdriver Logging drop-down value to Enabled.
Click Save.

Viewing logs

You can view logs in the in the Google Cloud console's Logs Explorer.

To learn more about viewing logs, including how to construct basic and advanced queries, refer to Viewing logs in the Logging documentation.

Example queries

Query logs from Pods labelled nginx-deployment:

resource.type="container"
resource.labels.pod_id:"nginx-deployment-"

Query logs in the prod namespace with severity WARNING or above:
```
resource.type="container"
resource.labels.namespace_id="prod"
severity>WARNING
```
For more infromation about severity levels, see Log Severity.

Query logs from a specific container, my-container, in a specific Pod, my-pod, with a text payload:

resource.type="container"
resource.labels.container_name="my_container"
resource.labels.pod_id="my_pod"
textPayload:"abracadabra"

Limitations

Multi-line entries (entries with line feed characters) might not be processed correctly. To avoid this issue, wrap your logs in single-line JSON strings.

Disabling logging

gcloud

To disable Legacy Logging and Monitoring for an existing cluster, run the following command, where [CLUSTER_NAME] is the name of the cluster.

gcloud beta container clusters update [CLUSTER_NAME] --logging-service none

If you are running Cloud Operations for GKE in your cluster, you must disable both monitoring and logging by using gcloud beta to set the following flags in your cluster:

gcloud beta container clusters update [CLUSTER_NAME] --logging-service none --monitoring-service none

Console

In the Google Cloud console, go to the Kubernetes Engine > Kubernetes clusters page:

Go to Kubernetes clusters
Click Click Edit for the cluster you want to change.
Set the Stackdriver Logging drop-down value to Disabled.
Click Save.

What's next

To learn more Google Cloud's operations suite's costs, see Pricing.
For information about Cloud Logging, see the Cloud Logging documentation.
To learn about Google Cloud's operations suite's resources and services, see Monitored resources and services.