Installing monitoring and logging support

Google Kubernetes Engine (GKE) includes integration with Stackdriver Monitoring and Stackdriver Logging. This integration lets you monitor your running GKE clusters, manage your system and debug logs, and analyze your system's performance using advanced profiling and tracing capabilities.

This page describes how to do the following:

  • Create a new cluster and configure Stackdriver Kubernetes Engine Monitoring or Legacy Stackdriver.

  • Select whether to collect application logs or not.

  • Disable Stackdriver Monitoring and Stackdriver Logging for a cluster.

Stackdriver Kubernetes Engine Monitoring provides a single option that enables improved support for both Stackdriver Monitoring and Stackdriver Logging in your clusters. With Legacy Stackdriver, you could enable Monitoring without Logging, but this option is not supported in Stackdriver Kubernetes Engine Monitoring.

Before you begin

  • Determine which monitoring and logging integration option you want to use.

  • Determine whether application logs should be collected.

  • Ensure you are an Owner of the project containing your cluster. Your project must be associated with a Workspace.

Controlling the collection of application logs

By default, GKE collects logs for workloads deployed to the cluster. This is a powerful tool for instrumenting your application; however, this automated collection may not be preferred. Beginning with GKE version 1.15.7, you can configure Stackdriver Kubernetes Engine Monitoring to only capture system logs and not collect application logs.

When the system-only option is selected, the following logs are collected:

  • All pods running in namespaces kube-system, istio-system, knative-serving,gke-system, and config-management-system.

  • Key services that are not containerized including docker/containerd runtime, kubelet, kubelet-monitor, node-problem-detector, and kube-container-runtime-monitor.

  • The node’s serial ports output, if the VM instance metadata serial-port-logging-enable is set to true.

Configuring monitoring and logging support for a new cluster

When you create a new GKE cluster, Stackdriver Kubernetes Engine Monitoring is enabled by default. Alternatively, you can select Legacy Stackdriver integration for clusters running GKE version 1.14 or lower, or you can disable Stackdriver all together.

The cluster-creation instructions in this section only cover the options relevant to monitoring and logging. For complete instructions on creating a GKE cluster, see Creating a cluster.

CONSOLE

  1. Go to the GKE Kubernetes clusters page for your project. The following button takes you there:

    Go to Kubernetes clusters

  2. Click Create Cluster.

  3. In the dialog, specify the following fields in addition to any other properties you want in your cluster.

    1. Master version: Based on the version selected, this will change the Stackdriver Monitoring and Stackdriver Logging options available.

    2. Click Availability, networking, security, and additional features. Locate the Stackdriver section towards the bottom.

    3. Select the logging and monitoring support for your cluster:

      • For new clusters, Stackdriver Kubernetes Engine Monitoring is enabled by default.

        Select Stackdriver Kubernetes Engine Monitoring support.

        • If you select Master version 1.15.7 or greater, you get the additional option of selecting System and workload logging and monitoring (recommended) or System logging and monitoring only (beta).

          Select Stackdriver Kubernetes Engine Monitoring support.

      • For GKE version 1.14 and earlier, you can enable Legacy Stackdriver support by clearing Enable Stackdriver Kubernetes Engine Monitoring and then selecting Enable legacy Stackdriver Logging and Enable legacy Stackdriver Monitoring. Legacy Stackdriver Monitoring and Legacy Stackdriver Logging can be enabled or disabled separately.

        Select Legacy Stackdriver support.

      • To disable Stackdriver integration, clear the Enable Stackdriver Kubernetes Engine Monitoring checkbox.

  4. Click Create to create the cluster.

GCLOUD

Use gcloud to create your cluster:

  1. Open a terminal window with Cloud SDK and gcloud installed. One way to do this is to use Cloud Shell.

  2. In the Cloud Console, go to Cloud Shell.

    Go to Cloud Shell

    At the bottom of the Cloud Console, a Cloud Shell session opens and displays a command-line prompt. Cloud Shell is a shell environment with the Cloud SDK already installed, including the gcloud command-line tool, and with values already set for your current project. It can take a few seconds for the session to initialize.

  3. To select Stackdriver Kubernetes Engine Monitoring support for your new cluster, include the --enable-stackdriver-kubernetes option in the create command:

    gcloud beta container clusters create [CLUSTER_NAME] \
      --zone=[ZONE] \
      --project=[PROJECT_ID] \
      --cluster-version=[CLUSTER_VERSION] \
      --enable-stackdriver-kubernetes
    

    In the previous expression, [CLUSTER_NAME] is your clusters' name, [ZONE] is the compute zone for the cluster, and [PROJECT_ID] is the Google Cloud project ID, not the project name.

    [CLUSTER_VERSION] is the GKE version to install on the cluster. You can enter the specific patch release to install, for example 1.12.8-gke.10. Otherwise, you can enter the minor version to install, for example, 1.12.8, and your cluster is created with the latest patch version. For more information on GKE versioning, see Versioning and upgrades.

  4. Alternatively, to select Legacy Stackdriver for your a GKE cluster version 1.14 and earlier, create your cluster and include the --enable-cloud-logging and/or --enable-cloud-monitoring options in the create command. For example, a simple cluster creation might look like the following:

    gcloud container clusters create [CLUSTER_NAME] \
      --zone=[ZONE] \
      --project=[PROJECT_ID] \
      --cluster-version=[CLUSTER_VERSION]
    

Configuring monitoring and logging support for an existing cluster

The following section details how to modify the monitoring and logging integration for an existing GKE cluster.

Which monitoring and logging support does my cluster use?

To see which monitoring and logging support your cluster is using, complete the following steps:

  1. In the Google Cloud Console, click your cluster's name:

    Go to Kubernetes clusters

  2. In the Details panel for your cluster, see the status for Legacy Stackdriver Monitoring, Legacy Stackdriver Logging, and Stackdriver Kubernetes Engine Monitoring.

    Cluster details screen.

Modifying your cluster

To change the monitoring and logging configuration for an existing cluster, follow the instructions below:

CONSOLE

  1. Go to the GKE Kubernetes clusters page for your project:

    Go to Kubernetes clusters

  2. Upgrade your cluster to version 1.12.7 or higher. For instructions, see Upgrading clusters. Allow the upgrade to complete.

  3. Click Edit for your cluster:

    Edit a GKE cluster.

  4. Select the appropriate integration option for your cluster.

    • To change to Stackdriver Kubernetes Engine Monitoring, in the Stackdriver Kubernetes Engine Monitoring drop-down box, select System and workload logging and monitoring (recommended).

      Choose Stackdriver Kubernetes Engine Monitoring support.

    • To change to Stackdriver Kubernetes Engine Monitoring with system logs only, in the Stackdriver Kubernetes Engine Monitoring drop-down box, select System logging and monitoring only.

      Choose Stackdriver Kubernetes Engine Monitoring support.

    • To change to Legacy Stackdriver support (for GKE version 1.14 and below, do the following:

      1. In the Stackdriver Kubernetes Engine Monitoring drop-down box, select Legacy Stackdriver.

      2. In the Legacy Stackdriver Logging drop-down box, select Enabled.

      3. In the Legacy Stackdriver Monitoring drop-down box, select Enabled.

        Choose Legacy Stackdriver supports.

    • To completely disable Stackdriver integration, select Disabled in the Stackdriver Kubernetes Engine Monitoring drop-down box.

  5. Click Save.

GCLOUD

The following gcloud instructions cover upgrading your cluster's monitoring and logging support using the gcloud container clusters update command. Notice that you use the update command, not the upgrade command, even though we sometimes refer to this process as "upgrading."

  1. Open a terminal window with Cloud SDK and gcloud installed. One way to do this is to use Cloud Shell.

  2. In the Cloud Console, go to Cloud Shell.

    Go to Cloud Shell

    At the bottom of the Cloud Console, a Cloud Shell session opens and displays a command-line prompt. Cloud Shell is a shell environment with the Cloud SDK already installed, including the gcloud command-line tool, and with values already set for your current project. It can take a few seconds for the session to initialize.

  3. If you have not done so, set up default values for your cluster in Cloud SDK:

    gcloud config set project [PROJECT_ID]
    gcloud config set compute/zone [ZONE]
    
  4. Update your cluster to the desired Kubernetes version and select the monitoring and logging support you want in the cluster. Stackdriver Kubernetes Engine Monitoring is supported in all versions of container_name_short. Legacy Stackdriver is supported in GKE 1.14 and earlier.

  5. The following update command that enables Stackdriver Kubernetes Engine Monitoring only shows the options needed for Stackdriver:

    gcloud beta container clusters update [CLUSTER_NAME] \
      --zone=[ZONE]  --region=[REGION]  \
      --enable-stackdriver-kubernetes
      

    The field [REGION] is the compute region of the cluster.

  6. Alternatively, the following update command that enables Legacy Stackdriver only shows the options needed for Stackdriver:

    gcloud beta container clusters update [CLUSTER_NAME] \
      --zone=[ZONE]  --region=[REGION]  \
      --logging-service logging.googleapis.com \
      --monitoring-service monitoring.googleapis.com
      

What's next