Installing monitoring and logging support

Google Kubernetes Engine (GKE) includes integration with Stackdriver Monitoring and Stackdriver Logging. Installing Stackdriver support lets you monitor your running GKE clusters, manage your system and debug logs, and analyze your system's performance using advanced profiling and tracing capabilities.

This page describes how to:

• Create a new cluster and configure Stackdriver Kubernetes Engine Monitoring or Legacy Stackdriver.
• Select whether to collect application logs or not.
• Disable Stackdriver Monitoring and Stackdriver Logging for a cluster.

Stackdriver Kubernetes Engine Monitoring provides a single option that enables improved support for both Stackdriver Monitoring and Stackdriver Logging in your clusters. With Legacy Stackdriver, you could enable Monitoring without Logging, but this option is not supported in Stackdriver Kubernetes Engine Monitoring.

Note: Security logs, including basic audit logs, are available for GKE and most other Google Cloud services without going through this procedure. See Cloud Audit Logs for more information.

Before you begin

• Determine which monitoring and logging integration option you want to use.

• Determine whether application logs should be collected.

• You must be an Owner of the project containing your cluster. Your project must be associated with a Workspace.

Controlling the collection of application logs

By default, GKE collects logs for workloads deployed to the cluster. This is a powerful tool for instrumenting your application. However, this automated collection may not be preferred. Beginning with version 1.15.7, you can configure Stackdriver Kubernetes Engine Monitoring to only capture system logs and not collect application logs.

When the system-only option is selected, these logs are collected:

• All pods running in namespaces kube-system, istio-system,knative-serving,gke-system, andconfig-management-system`.
• Key services that are not containerized including docker/containerd runtime, kubelet, kubelet-monitor, node-problem-detector, and kube-container-runtime-monitor.
• The node’s serial ports output, if the VM instance metadata serial-port-logging-enable is set to true.

Configuring Monitoring without Logging for a new cluster

When you create a new GKE cluster, Stackdriver Kubernetes Engine Monitoring is enabled by default. Alternatively, you can select Legacy Stackdriver integration or disable Stackdriver all together.

    Note: If Stackdriver Kubernetes Engine Monitoring and Legacy Stackdriver Logging is disabled,
    support is offered on best-effort basis and will require additional
    effort from your engineering team.

The cluster-creation instructions in this section cover only the options relevant to monitoring and logging. For complete instructions on creating a GKE cluster, see Creating a cluster.

CONSOLE

1. Go to the GKE Kubernetes clusters page for your project. The following button takes you there:

   Go to Kubernetes clusters

2. Click Create Cluster.

3. In the dialog, specify the following fields in addition to any other properties you want in your cluster. For more information, see Creating a Cluster.

   1. Master version: Based on the version selected, this will change the Stackdriver Monitoring and Stackdriver Logging options available.

   2. Click Additional features. Locate the Stackdriver section towards the bottom.

      • For Stackdriver Kubernetes Engine Monitoring support, select Enable Stackdriver Kubernetes Engine Monitoring. Then select System and workload logging and monitoring (recommended).

        

        or

        System logging and monitoring only (beta) (for container_name_short version 1.15.7 and greater only)

        

      • For Legacy Stackdriver support (GKE version 1.14 and earlier), clear Enable Stackdriver Kubernetes Engine Monitoring then select Legacy Stackdriver from the drop-down box. Legacy Stackdriver Monitoring and Legacy Stackdriver Logging can be enabled or disabled separately.

        

      • To disable Stackdriver integration, clear the Enable Stackdriver Kubernetes Engine Monitoring checkbox.

4. Click Create to create the cluster.

GCLOUD

Use gcloud to create your cluster:

1. Open a terminal window with Cloud SDK and gcloud installed. One way to do this is to use Cloud Shell:

   In the Cloud Console, go to Cloud Shell.

   Go to Cloud Shell

   At the bottom of the Cloud Console, a Cloud Shell session opens and displays a command-line prompt. Cloud Shell is a shell environment with the Cloud SDK already installed, including the gcloud command-line tool, and with values already set for your current project. It can take a few seconds for the session to initialize.

2. To select Stackdriver Kubernetes Engine Monitoring support for your new cluster, include the --enable-stackdriver-kubernetes option in the create command:

   gcloud beta container clusters create [CLUSTER_NAME] \
     --zone=[ZONE] \
     --project=[PROJECT_ID] \
     --cluster-version=[CLUSTER_VERSION] \
     --enable-stackdriver-kubernetes
   

   In the previous expression, [CLUSTER_NAME] is your clusters' name, [ZONE] is the compute zone for the cluster, and [PROJECT_ID] is the Google Cloud project ID, not the project name.

   [CLUSTER_VERSION] is the GKE version to install on the cluster. You can enter the specific patch release to install, for example 1.12.8-gke.10. Otherwise, you can enter the minor version to install, for example, 1.12.8, and your cluster is created with the latest patch version. For more information on GKE versioning, see Versioning and upgrades.

3. Alternatively, to select Legacy Stackdriver for your a GKE cluster version 1.14 and earlier, create your cluster and include the --enable-cloud-logging and/or --enable-cloud-monitoring options in the create command. For example, a simple cluster creation might look like the following: ``` gcloud container clusters create [CLUSTER_NAME]
   --zone=[ZONE]
   --project=[PROJECT_ID]
   --cluster-version=[CLUSTER_VERSION]

Configuring Stackdriver for an existing cluster

The following section details how to modify the monitoring and logging integration for an existing GKE cluster.

Note: Changing your monitoring and logging support, and changing the Kubernetes version of your cluster are distinct actions. Changing the Kubernetes version of your cluster doesn't change your configured monitoring and logging support.

Which monitoring and logging support does my cluster use?

To see which monitoring and logging support your cluster is using, complete the following steps:

1. In the Google Cloud Console, click your cluster's name:

   Go to Kubernetes clusters

2. In the Details panel for your cluster, see the status for Legacy Stackdriver Monitoring, Legacy Stackdriver Logging, and Stackdriver Kubernetes Engine Monitoring.

   

Modifying your cluster

To change the monitoring and logging configuration for an existing cluster, follow the instructions below:

CONSOLE

1. Go to the GKE Kubernetes clusters page for your project:

   Go to Kubernetes clusters

2. Upgrade your cluster to version 1.12.7 or higher. For instructions, see Upgrading clusters. Allow the upgrade to complete.

3. Click Edit for your cluster:

   

4. Select the appropriate integration option for your cluster.

   • To change to Stackdriver Kubernetes Engine Monitoring, in the Stackdriver Kubernetes Engine Monitoring drop-down box, select System and workload logging and monitoring (recommended).

     

   • To change to Stackdriver Kubernetes Engine Monitoring with system logs only, in the Stackdriver Kubernetes Engine Monitoring drop-down box, select System logging and monitoring only.

     

   • To change to Legacy Stackdriver support (for GKE version 1.14 and below, do the following:

     1. In the Stackdriver Kubernetes Engine Monitoring drop-down box, select Legacy Stackdriver.

     2. In the Legacy Stackdriver Logging drop-down box, select Enabled.

     3. In the Legacy Stackdriver Monitoring drop-down box, select Enabled.

        

   • To completely disable Stackdriver integration, select Disabled in the Stackdriver Kubernetes Engine Monitoring drop-down box.

5. Click Save.

GCLOUD

The following gcloud instructions cover upgrading your cluster's Stackdriver support using the gcloud container clusters update command. Notice that you use the update command, not the upgrade command, even though we sometimes refer to this process as "upgrading."

1. In the Cloud Console, go to Cloud Shell.

   Go to Cloud Shell

   At the bottom of the Cloud Console, a Cloud Shell session opens and displays a command-line prompt. Cloud Shell is a shell environment with the Cloud SDK already installed, including the gcloud command-line tool, and with values already set for your current project. It can take a few seconds for the session to initialize.

2. If you have not done so, set up default values for your cluster in Cloud SDK:

   gcloud config set project [PROJECT_ID]
   gcloud config set compute/zone [ZONE]
   

3. Update your cluster to the desired Kubernetes version and select the Stackdriver support you want in the cluster. Stackdriver Kubernetes Engine Monitoring is supported in all versions of container_name_short. Legacy Stackdriver is supported in GKE 1.14 and earlier.

4. The following update command that enables Stackdriver Kubernetes Engine Monitoring only shows the options needed for Stackdriver:

   gcloud beta container clusters update [CLUSTER_NAME] \
     --zone=[ZONE]  --region=[REGION]  \
     --enable-stackdriver-kubernetes
     

   The field [REGION] is the compute region of the cluster.

   Note: You must use Cloud SDK version 248.0.0 or higher.

5. Alternatively, the following update command that enables Legacy Stackdriver only shows the options needed for Stackdriver:

   gcloud beta container clusters update [CLUSTER_NAME] \
     --zone=[ZONE]  --region=[REGION]  \
     --logging-service logging.googleapis.com \
     --monitoring-service monitoring.googleapis.com
     

   Note: The system only logging option is not available in gcloud.

What's next

• If you are using Legacy Stackdriver, you should begin to become familiar with the changes in Stackdriver Kubernetes Engine Monitoring by reading Migrating to Stackdriver Kubernetes Engine Monitoring.

• You can deploy your application software to your cluster as usual. For more information, see Overview of deploying workloads.

• To use Prometheus with this release of Stackdriver Kubernetes Engine Monitoring, see Using Prometheus.