Version 1.8. This version is no longer supported. For information about how to upgrade to version 1.9, see Upgrading Anthos on bare metal in the 1.9 documentation. For more information about supported and unsupported versions, see the Version history page in the latest documentation.
This page shows how to configure a cluster for Google Distributed Cloud so that
custom logs and metrics from user applications are sent to Cloud Logging and
Cloud Monitoring.
Enabling Logging and Monitoring for user applications
The configuration for Logging and Monitoring is
held in a Stackdriver object named stackdriver.
To enable the collection of custom metrics from an application, add the
prometheus.io/scrape: "true" annotation to the application's Service or
Deployment manifest.
Run an example application
In this section, you create an application that writes custom logs and exposes
a custom metric.
Save the following Service and Deployment manifests to a file named
my-app.yaml. Notice that the Service has the annotation
prometheus.io/scrape: "true":
Click Resource. Under ALL_RESOURCE_TYPES, select Kubernetes
Container.
Under CLUSTER_NAME, select the name of your user cluster.
Under NAMESPACE_NAME, select default.
Click Add and then click Run Query.
Under Query results, you can see log entries from the
monitoring-example Deployment. For example:
{"textPayload":"2020/11/14 01:24:24 Starting to listen on :9090\n","insertId":"1oa4vhg3qfxidt","resource":{"type":"k8s_container","labels":{"pod_name":"monitoring-example-7685d96496-xqfsf","cluster_name":...,"namespace_name":"default","project_id":...,"location":"us-west1","container_name":"prometheus-example-exporter"}},"timestamp":"2020-11-14T01:24:24.358600252Z","labels":{"k8s-pod/pod-template-hash":"7685d96496","k8s-pod/app":"monitoring-example"},"logName":"projects/.../logs/stdout","receiveTimestamp":"2020-11-14T01:24:39.562864735Z"}
gcloud
Run this command:
gcloudloggingread'resource.labels.project_id="PROJECT_ID" AND \ resource.type="k8s_container" AND resource.labels.namespace_name="default"'
Replace PROJECT_ID with the ID
of your project.
In the output, you can see log entries from the monitoring-example
Deployment. For example:
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-09-03 UTC."],[[["\u003cp\u003eThis guide explains how to configure a Google Distributed Cloud cluster to send custom logs and metrics from user applications to Cloud Logging and Cloud Monitoring.\u003c/p\u003e\n"],["\u003cp\u003eTo enable logging and monitoring for user applications, you must edit the \u003ccode\u003estackdriver\u003c/code\u003e object within the \u003ccode\u003ekube-system\u003c/code\u003e namespace and set \u003ccode\u003eenableStackdriverForApplications\u003c/code\u003e to \u003ccode\u003etrue\u003c/code\u003e.\u003c/p\u003e\n"],["\u003cp\u003eCustom metrics collection requires the \u003ccode\u003eprometheus.io/scrape: "true"\u003c/code\u003e annotation to be added to the application's Service or Deployment manifest.\u003c/p\u003e\n"],["\u003cp\u003eAn example application is provided, demonstrating how to write custom logs and expose a custom metric, which can then be viewed in the Google Cloud console's Logs explorer and Metrics explorer, respectively.\u003c/p\u003e\n"],["\u003cp\u003eYou can verify that logs from your applications are being sent and received in Cloud Logging by querying the \u003ccode\u003ek8s_container\u003c/code\u003e resource type in the Logs Explorer or the gcloud logging command, and you can verify metrics in the metrics explorer.\u003c/p\u003e\n"]]],[],null,["# Enable application logging and monitoring\n\n\u003cbr /\u003e\n\nThis page shows how to configure a cluster for Google Distributed Cloud so that\ncustom logs and metrics from user applications are sent to Cloud Logging and\nCloud Monitoring.\n\nEnabling Logging and Monitoring for user applications\n-----------------------------------------------------\n\nThe configuration for Logging and Monitoring is\nheld in a Stackdriver object named `stackdriver`.\n\n1. Open the `stackdriver` object for editing:\n\n kubectl --kubeconfig=\u003cvar label=\"kubeconfig file name\" translate=\"no\"\u003eCLUSTER_KUBECONFIG\u003c/var\u003e --namespace kube-system edit stackdriver stackdriver\n\n Replace \u003cvar scope=\"CLUSTER_KUBECONFIG\" translate=\"no\"\u003eCLUSTER_KUBECONFIG\u003c/var\u003e\n with the path of your user cluster kubeconfig file.\n2. Under `spec`, set `enableStackdriverForApplications` to `true`:\n\n apiVersion: addons.sigs.k8s.io/v1alpha1\n kind: Stackdriver\n metadata:\n name: stackdriver\n namespace: kube-system\n spec:\n projectID: ...\n clusterName: ...\n clusterLocation: ...\n proxyConfigSecretName: ...\n enableStackdriverForApplications: true\n enableVPC: ...\n optimizedMetrics: true\n\n3. Close the edited file.\n\nAnnotate workloads\n------------------\n\nTo enable the collection of custom metrics from an application, add the\n`prometheus.io/scrape: \"true\"` annotation to the application's Service or\nDeployment manifest.\n\nRun an example application\n--------------------------\n\nIn this section, you create an application that writes custom logs and exposes\na custom metric.\n\n1. Save the following Service and Deployment manifests to a file named\n `my-app.yaml`. Notice that the Service has the annotation\n `prometheus.io/scrape: \"true\"`:\n\n kind: Service\n apiVersion: v1\n metadata:\n name: \"monitoring-example\"\n namespace: \"default\"\n annotations:\n prometheus.io/scrape: \"true\"\n spec:\n selector:\n app: \"monitoring-example\"\n ports:\n - name: http\n port: 9090\n ---\n apiVersion: apps/v1\n kind: Deployment\n metadata:\n name: \"monitoring-example\"\n namespace: \"default\"\n labels:\n app: \"monitoring-example\"\n spec:\n replicas: 1\n selector:\n matchLabels:\n app: \"monitoring-example\"\n template:\n metadata:\n labels:\n app: \"monitoring-example\"\n spec:\n containers:\n - image: gcr.io/google-samples/prometheus-dummy-exporter:latest\n name: prometheus-example-exporter\n imagePullPolicy: Always\n command:\n - /bin/sh\n - -c\n - ./prometheus-dummy-exporter --metric-name=example_monitoring_up --metric-value=1 --port=9090\n resources:\n requests:\n cpu: 100m\n\n2. Create the Deployment and the Service:\n\n kubectl --kubeconfig \u003cvar scope=\"CLUSTER_KUBECONFIG\" translate=\"no\"\u003eCLUSTER_KUBECONFIG\u003c/var\u003e apply -f my-app.yaml\n\nView application logs\n---------------------\n\n### Console\n\n1. Go to the Logs explorer in the Google Cloud console.\n\n [Go to the Logs explorer](https://console.cloud.google.com/logs/query)\n2. Click **Resource** . Under **ALL_RESOURCE_TYPES** , select `Kubernetes\n Container`.\n\n3. Under **CLUSTER_NAME**, select the name of your user cluster.\n\n4. Under **NAMESPACE_NAME** , select `default`.\n\n5. Click **Add** and then click **Run Query**.\n\n6. Under **Query results** , you can see log entries from the\n `monitoring-example` Deployment. For example:\n\n {\n \"textPayload\": \"2020/11/14 01:24:24 Starting to listen on :9090\\n\",\n \"insertId\": \"1oa4vhg3qfxidt\",\n \"resource\": {\n \"type\": \"k8s_container\",\n \"labels\": {\n \"pod_name\": \"monitoring-example-7685d96496-xqfsf\",\n \"cluster_name\": ...,\n \"namespace_name\": \"default\",\n \"project_id\": ...,\n \"location\": \"us-west1\",\n \"container_name\": \"prometheus-example-exporter\"\n }\n },\n \"timestamp\": \"2020-11-14T01:24:24.358600252Z\",\n \"labels\": {\n \"k8s-pod/pod-template-hash\": \"7685d96496\",\n \"k8s-pod/app\": \"monitoring-example\"\n },\n \"logName\": \"projects/.../logs/stdout\",\n \"receiveTimestamp\": \"2020-11-14T01:24:39.562864735Z\"\n }\n\n### gcloud\n\n1. Run this command:\n\n gcloud logging read 'resource.labels.project_id=\"\u003cvar label=\"project ID\" translate=\"no\"\u003ePROJECT_ID\u003c/var\u003e\" AND \\\n resource.type=\"k8s_container\" AND resource.labels.namespace_name=\"default\"'\n\n Replace \u003cvar label=\"project ID\" translate=\"no\"\u003ePROJECT_ID\u003c/var\u003e with the ID\n of your project.\n2. In the output, you can see log entries from the `monitoring-example`\n Deployment. For example:\n\n insertId: 1oa4vhg3qfxidt\n labels:\n k8s-pod/app: monitoring-example\n k8s- pod/pod-template-hash: 7685d96496\n logName: projects/.../logs/stdout\n receiveTimestamp: '2020-11-14T01:24:39.562864735Z'\n resource:\n labels:\n cluster_name: ...\n container_name: prometheus-example-exporter\n location: us-west1\n namespace_name: default\n pod_name: monitoring-example-7685d96496-xqfsf\n project_id: ...\n type: k8s_container\n textPayload: |\n 2020/11/14 01:24:24 Starting to listen on :9090\n timestamp: '2020-11-14T01:24:24.358600252Z'\n\nView application metrics in the Google Cloud console\n----------------------------------------------------\n\nYour example application exposes a custom metric named `example_monitoring_up`.\nYou can view the values of that metric in the Google Cloud console.\n\n1. Go to the Metrics explorer in the Google Cloud console.\n\n [Go to the Metrics explorer](https://console.cloud.google.com/monitoring/metrics-explorer)\n2. For **Resource type** , select `Kubernetes Pod`.\n\n3. For **metric** , select `external/prometheus/example_monitoring_up`.\n\n4. In the chart, you can see that `example_monitoring_up` has a repeated value\n of 1."]]