Configure a observabilidade do GKE Dataplane V2

Esta página mostra como configurar clusters do Google Kubernetes Engine (GKE) com a observabilidade do GKE Dataplane V2, a partir das versões 1.28 ou posteriores do GKE. Para mais informações sobre as vantagens e os requisitos da observabilidade do GKE Dataplane V2, consulte o artigo Acerca da observabilidade do GKE Dataplane V2.

Antes de começar

Antes de começar, certifique-se de que realizou as seguintes tarefas:

  • Ative a API Google Kubernetes Engine.
    • Ative a API Google Kubernetes Engine
  • Se quiser usar a CLI gcloud para esta tarefa, instale-a e, em seguida, inicialize-a. Se instalou anteriormente a CLI gcloud, execute gcloud components update para obter a versão mais recente.

Configure as métricas do GKE Dataplane V2

Para recolher métricas, tem de configurar as métricas do GKE Dataplane V2. Pode configurar as métricas do GKE Dataplane V2 quando cria um cluster ou atualiza um cluster em execução com o GKE Dataplane V2. Pode ativar ou desativar as métricas do GKE Dataplane V2 através da CLI gcloud.

Recomendamos que ative as métricas do GKE Dataplane V2 e o serviço gerido da Google Cloud para o Prometheus no seu cluster do GKE. Depois de ambos estarem ativados, as métricas do GKE Dataplane V2 são enviadas para o serviço gerido da Google Cloud para o Prometheus.

Crie um cluster do Autopilot com as métricas do GKE Dataplane V2 ativadas

Quando cria novos clusters do GKE Autopilot, o GKE ativa as métricas do GKE Dataplane V2 por predefinição no cluster sem precisar de uma flag específica.

Para usar as métricas do GKE Dataplane V2 do cluster do GKE Autopilot com o serviço gerido do Google Cloud para Prometheus, configure o recurso ClusterPodMonitoring para extrair as métricas e enviá-las para o serviço gerido do Google Cloud para Prometheus.

  1. Crie um ClusterPodMonitoringmanifesto:

    # Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: monitoring.googleapis.com/v1
kind: ClusterPodMonitoring
metadata:
  name: advanced-datapath-observability-metrics
spec:
  selector:
    matchLabels:
      k8s-app: cilium
  endpoints:
  - port: flowmetrics
    interval: 60s
    metricRelabeling:
    # only keep denormalized pod flow metrics
    - sourceLabels: [__name__]
      regex: 'pod_flow_(ingress|egress)_flows_count'
      action: keep
    # extract pod name
    - sourceLabels: [__name__, destination]
      regex: 'pod_flow_ingress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
      replacement: '${2}'
      targetLabel: pod_name
      action: replace
    - sourceLabels: [__name__, source]
      regex: 'pod_flow_egress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
      replacement: '${2}'
      targetLabel: pod_name
      action: replace
    # extract workload name by removing 2 last "-XXX" parts
    - sourceLabels: [pod_name]
      regex: '([a-zA-Z0-9-\.]+)((-[a-zA-Z0-9\.]+){2})'
      replacement: '${1}'
      targetLabel: workload_name
      action: replace
    # extract workload name by removing one "-XXX" part when pod name has only 2 parts (eg. daemonset)
    - sourceLabels: [pod_name]
      regex: '([a-zA-Z0-9\.]+)((-[a-zA-Z0-9\.]+){1})'
      replacement: '${1}'
      targetLabel: workload_name
      action: replace
    # extract pod namespace
    - sourceLabels: [__name__, destination]
      regex: 'pod_flow_ingress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
      replacement: '${1}'
      targetLabel: namespace_name
      action: replace
    - sourceLabels: [__name__, source]
      regex: 'pod_flow_egress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
      replacement: '${1}'
      targetLabel: namespace_name
      action: replace
    # extract remote workload name
    - sourceLabels: [__name__, source]
      regex: 'pod_flow_ingress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
      replacement: '${2}'
      targetLabel: remote_workload
      action: replace
    - sourceLabels: [__name__, destination]
      regex: 'pod_flow_egress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
      replacement: '${2}'
      targetLabel: remote_workload
      action: replace
    # extract remote workload namespace
    - sourceLabels: [__name__, source]
      regex: 'pod_flow_ingress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
      replacement: '${1}'
      targetLabel: remote_namespace
      action: replace
    - sourceLabels: [__name__, destination]
      regex: 'pod_flow_egress_flows_count;([a-zA-Z0-9-\.]+)/([a-zA-Z0-9-\.]+)'
      replacement: '${1}'
      targetLabel: remote_namespace
      action: replace
    # default remote workload class to "pod"
    - replacement: 'pod'
      targetLabel: remote_class
      action: replace
    # extract remote workload class from reserved identity
    - sourceLabels: [__name__, source]
      regex: 'pod_flow_ingress_flows_count;reserved:([^/]*)'
      replacement: '${1}'
      targetLabel: remote_class
      action: replace
    - sourceLabels: [__name__, destination]
      regex: 'pod_flow_egress_flows_count;reserved:([^/]*)'
      replacement: '${1}'
      targetLabel: remote_class
      action: replace
  targetLabels:
    metadata: []

  2. Aplique o manifesto ClusterPodMonitoring:

    kubectl apply -f ClusterPodMonitoring.yaml

Crie um cluster padrão com as métricas do GKE Dataplane V2 ativadas

Para ativar as métricas do GKE Dataplane V2, crie um cluster com a flag --enable-dataplane-v2-metrics:

gcloud container clusters create CLUSTER_NAME \
    --enable-dataplane-v2 \
    --enable-ip-alias \
    --enable-managed-prometheus \
    --enable-dataplane-v2-metrics

Substitua o seguinte:

  • CLUSTER_NAME: o nome do cluster.

A flag --enable-managed-prometheus indica ao GKE que deve usar as métricas com o serviço gerido da Google Cloud para o Prometheus.

Ative as métricas do GKE Dataplane V2 num cluster existente

Para ativar as métricas do GKE Dataplane V2 num cluster existente, execute o seguinte comando:

gcloud container clusters update CLUSTER_NAME \
    --enable-dataplane-v2-metrics

Substitua CLUSTER_NAME pelo nome do seu cluster.

Desative as métricas do GKE Dataplane V2

Para desativar as métricas do GKE Dataplane V2:

gcloud container clusters update CLUSTER_NAME \
    --disable-dataplane-v2-metrics

Substitua CLUSTER_NAME pelo nome do seu cluster.

Configure as ferramentas de observabilidade do GKE Dataplane V2

Pode usar um ponto final privado para aceder às ferramentas de resolução de problemas de observabilidade do GKE Dataplane V2. Para ativar as ferramentas de observabilidade do GKE Dataplane V2, tem de ter um cluster configurado com o GKE Dataplane V2. Pode ativar as ferramentas de observabilidade do GKE Dataplane V2 num novo cluster ou num cluster existente.

Crie um cluster do Autopilot com a observabilidade ativada

Para criar um cluster do GKE Autopilot com a observabilidade do GKE Dataplane V2 ativada:

gcloud container clusters create-auto CLUSTER_NAME \
    --enable-dataplane-v2-flow-observability \
    --location COMPUTE_LOCATION

Substitua o seguinte: * CLUSTER_NAME: o nome do cluster. * COMPUTE_LOCATION: a localização do Compute Engine para o cluster.

Crie um cluster padrão com a observabilidade ativada

Para criar um cluster padrão do GKE com a observabilidade do GKE Dataplane V2 ativada:

gcloud container clusters create CLUSTER_NAME \
    --enable-dataplane-v2 \
    --enable-ip-alias \
    --enable-dataplane-v2-flow-observability \
    --location COMPUTE_LOCATION

Substitua o seguinte: * CLUSTER_NAME: o nome do cluster. * COMPUTE_LOCATION: a localização do Compute Engine para o cluster.

Ative as ferramentas de observabilidade do GKE Dataplane V2 num cluster existente

Para ativar a observabilidade do GKE Dataplane V2 num cluster existente, execute o seguinte comando:

gcloud container clusters update CLUSTER_NAME \
    --enable-dataplane-v2-flow-observability \
    --location COMPUTE_LOCATION

Substitua o seguinte:

Desative as ferramentas de observabilidade do GKE Dataplane V2

Para desativar as ferramentas de observabilidade do GKE Dataplane V2 num cluster existente, execute o seguinte comando:

gcloud container clusters update CLUSTER_NAME \
    --disable-dataplane-v2-flow-observability

Substitua CLUSTER_NAME pelo nome do seu cluster.

Como usar a CLI Hubble

Use a ferramenta CLI do Hubble no cluster depois de ativar a funcionalidade de observabilidade do plano de dados V2 do GKE.

  1. Defina o alias para o binário hubble-cli:

    alias hubble="kubectl exec -it -n gke-managed-dpv2-observability deployment/hubble-relay -c hubble-cli -- hubble"

  2. Para verificar o estado do Hubble, com a funcionalidade de observabilidade do GKE Dataplane V2 ativada, use a CLI do Hubble em todos os clusters do Autopilot:

    hubble status

  3. Para ver o tráfego atual, use a CLI do Hubble da seguinte forma:

    hubble observe

Como implementar a distribuição binária da IU do Hubble

Depois de ativar a observabilidade do plano de dados V2 do GKE, pode implementar a IU do Hubble de código aberto.

  1. Ative a observabilidade no seu cluster do GKE:

    1. Crie um cluster do GKE com a observabilidade ativada:

      gcloud container clusters create-auto hubble-rc-auto \
    --location COMPUTE_LOCATION \
    --cluster-version VERSION \
    --enable-dataplane-v2-flow-observability

      Substitua o seguinte:

    2. Em alternativa, ative a observabilidade num cluster existente:

      gcloud container clusters update CLUSTER_NAME \
    --location COMPUTE_LOCATION \
    --enable-dataplane-v2-flow-observability

      Substitua o seguinte:

  2. Configure o kubectl para se ligar ao cluster:

    gcloud container clusters get-credentials CLUSTER_NAME \
    --location COMPUTE_LOCATION

    Substituir

  3. Implemente a IU do Hubble:

    # Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: hubble-ui
  namespace: gke-managed-dpv2-observability
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: hubble-ui
  labels:
    app.kubernetes.io/part-of: cilium
rules:
  - apiGroups:
      - networking.k8s.io
    resources:
      - networkpolicies
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - componentstatuses
      - endpoints
      - namespaces
      - nodes
      - pods
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - apiextensions.k8s.io
    resources:
      - customresourcedefinitions
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - cilium.io
    resources:
      - "*"
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: hubble-ui
  labels:
    app.kubernetes.io/part-of: cilium
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: hubble-ui
subjects:
  - kind: ServiceAccount
    name: hubble-ui
    namespace: gke-managed-dpv2-observability
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: hubble-ui-nginx
  namespace: gke-managed-dpv2-observability
data:
  nginx.conf: |
    server {
        listen       8081;
        # uncomment for IPv6
        # listen       [::]:8081;
        server_name  localhost;
        root /app;
        index index.html;
        client_max_body_size 1G;
        location / {
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            # CORS
            add_header Access-Control-Allow-Methods "GET, POST, PUT, HEAD, DELETE, OPTIONS";
            add_header Access-Control-Allow-Origin *;
            add_header Access-Control-Max-Age 1728000;
            add_header Access-Control-Expose-Headers content-length,grpc-status,grpc-message;
            add_header Access-Control-Allow-Headers range,keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout;
            if ($request_method = OPTIONS) {
                return 204;
            }
            # /CORS
            location /api {
                proxy_http_version 1.1;
                proxy_pass_request_headers on;
                proxy_hide_header Access-Control-Allow-Origin;
                proxy_pass http://127.0.0.1:8090;
            }
            location / {
                # double `/index.html` is required here
                try_files $uri $uri/ /index.html /index.html;
            }
        }
    }
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: hubble-ui
  namespace: gke-managed-dpv2-observability
  labels:
    k8s-app: hubble-ui
    app.kubernetes.io/name: hubble-ui
    app.kubernetes.io/part-of: cilium
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: hubble-ui
  template:
    metadata:
      labels:
        k8s-app: hubble-ui
        app.kubernetes.io/name: hubble-ui
        app.kubernetes.io/part-of: cilium
    spec:
      securityContext:
        fsGroup: 1000
        seccompProfile:
          type: RuntimeDefault
      serviceAccount: hubble-ui
      serviceAccountName: hubble-ui
      containers:
        - name: frontend
          image: quay.io/cilium/hubble-ui:v0.11.0
          ports:
            - name: http
              containerPort: 8081
          volumeMounts:
            - name: hubble-ui-nginx-conf
              mountPath: /etc/nginx/conf.d/default.conf
              subPath: nginx.conf
            - name: tmp-dir
              mountPath: /tmp
          terminationMessagePolicy: FallbackToLogsOnError
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 1000
            runAsGroup: 1000
            capabilities:
              drop:
                - all
        - name: backend
          image: quay.io/cilium/hubble-ui-backend:v0.11.0
          env:
            - name: EVENTS_SERVER_PORT
              value: "8090"
            - name: FLOWS_API_ADDR
              value: "hubble-relay.gke-managed-dpv2-observability.svc:443"
            - name: TLS_TO_RELAY_ENABLED
              value: "true"
            - name: TLS_RELAY_SERVER_NAME
              value: relay.gke-managed-dpv2-observability.svc.cluster.local
            - name: TLS_RELAY_CA_CERT_FILES
              value: /var/lib/hubble-ui/certs/hubble-relay-ca.crt
            - name: TLS_RELAY_CLIENT_CERT_FILE
              value: /var/lib/hubble-ui/certs/client.crt
            - name: TLS_RELAY_CLIENT_KEY_FILE
              value: /var/lib/hubble-ui/certs/client.key
          ports:
            - name: grpc
              containerPort: 8090
          volumeMounts:
            - name: hubble-ui-client-certs
              mountPath: /var/lib/hubble-ui/certs
              readOnly: true
          terminationMessagePolicy: FallbackToLogsOnError
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsUser: 1000
            runAsGroup: 1000
            capabilities:
              drop:
                - all
      volumes:
        - configMap:
            defaultMode: 420
            name: hubble-ui-nginx
          name: hubble-ui-nginx-conf
        - emptyDir: {}
          name: tmp-dir
        - name: hubble-ui-client-certs
          projected:
            # note: the leading zero means this number is in octal representation: do not remove it
            defaultMode: 0400
            sources:
              - secret:
                  name: hubble-relay-client-certs
                  items:
                    - key: ca.crt
                      path: hubble-relay-ca.crt
                    - key: tls.crt
                      path: client.crt
                    - key: tls.key
                      path: client.key
---
kind: Service
apiVersion: v1
metadata:
  name: hubble-ui
  namespace: gke-managed-dpv2-observability
  labels:
    k8s-app: hubble-ui
    app.kubernetes.io/name: hubble-ui
    app.kubernetes.io/part-of: cilium
spec:
  type: ClusterIP
  selector:
    k8s-app: hubble-ui
  ports:
    - name: http
      port: 80
      targetPort: 8081

  4. Aplique o manifesto hubble-ui-128.yaml:

    kubectl apply -f hubble-ui-128.yaml

  5. Exponha o serviço com o encaminhamento de portas:

    kubectl -n gke-managed-dpv2-observability port-forward service/hubble-ui 16100:80 --address='0.0.0.0'

  6. Aceda à IU do Hubble no seu navegador de Internet:

    http://localhost:16100/

O que se segue?