Viewing deprecation insights and recommendations


This page shows you how to view insights and recommendations about Kubernetes deprecations for your Google Kubernetes Engine (GKE) clusters. To learn more about how to manage Kubernetes deprecations with GKE, see GKE deprecations.

What are deprecation insights and recommendations?

When GKE detects that a cluster is using a Kubernetes feature or API that is deprecated and will be removed in an upcoming minor version, the following happens:

Deprecation insights and recommendations are available from Recommender, a service that provides insights and recommendations for using resources on Google Cloud. To learn more about how Recommender works with GKE, see Optimize your usage of GKE with insights and recommendations.

For the deprecations topic with Recommender:

  • An insight explains that your cluster uses a feature or API that is deprecated and will be removed in an upcoming minor version.
  • A recommendation provides guidance on what to do to mitigate your cluster's exposure to the deprecation.

For example, an insight might explain that your cluster is using a beta API that is deprecated and will be removed in the next minor version. The recommendation explains that you should migrate your workloads from using the beta APIs to using v1 APIs. Each recommendation includes a reference to a migration guide for the deprecated feature or API. This migration guide is also available in the corresponding table for Kubernetes deprecations.

When GKE generates an insight and recommendation pair, it covers usage of any deprecated Kubernetes APIs or features in a particular insight subtype for one cluster in your project. For example, if any user agents call any of the APIs addressed by the DEPRECATION_K8S_1_22_V1BETA1_API subtype, one insight and recommendation appear to recommend that you migrate this cluster from using these APIs. If another cluster is also using these APIs, GKE generates another insight and recommendation.

Use of deprecated Kubernetes features and APIs detected by GKE

Deprecation Removed in GKE version Insight subtype
TLS certificates signed with SHA-1 algorithm 1.29 DEPRECATION_K8S_SHA_1_CERTIFICATE
Kubernetes 1.29 deprecated APIs 1.29 DEPRECATION_K8S_1_29_API
Kubernetes 1.27 deprecated APIs 1.27 DEPRECATION_K8S_1_27_API
Kubernetes 1.26 deprecated APIs 1.26 DEPRECATION_K8S_1_26_API
Kubernetes 1.25 deprecated APIs 1.25 DEPRECATION_K8S_1_25_API
PodSecurityPolicy 1.25 DEPRECATION_K8S_1_25_PODSECURITYPOLICY
Docker-based node images 1.24 DEPRECATION_K8S_1_24_DOCKERSHIM
X.509 Common Name field in webhook certificates 1.23 DEPRECATION_K8S_1_23_CERTIFICATE
Kubernetes 1.22 deprecated APIs,
Kubernetes Ingress Beta APIs removed in GKE 1.23
1.22, 1.23 DEPRECATION_K8S_1_22_V1BETA1_API

Remember that for upcoming deprecations, you must independently assess your cluster environment's exposure and mitigate any issues.

Before you begin

Before you start, make sure you have performed the following tasks:

  • Enable the Google Kubernetes Engine API.
  • Enable Google Kubernetes Engine API
  • If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running gcloud components update.

Required roles

Ensure that you have the necessary permissions for Kubernetes insights and recommendations, either with the following basic roles or predefined roles:

View insights and recommendations

You can view insights and recommendations with the Google Cloud CLI, in the Google Cloud console, or with the Recommender API.

Console

  1. Go to the Google Kubernetes Engine page in the Google Cloud console:

    Go to Google Kubernetes Engine

  2. Check the Notifications column for specific clusters to find insights.

  3. Click on the insight to reveal more information. In the revealed sidebar panel, you can see details about this insight, including any associated recommendation.

gcloud

An insight typically has a corresponding recommendation. Insights are fetched with gcloud recommender insights and recommendations are fetched with gcloud recommender recommendations.

  1. View the list of insights for the clusters of a specific zone (for zonal clusters) or a specific region (for regional clusters):

    gcloud recommender insights list \
        --insight-type=google.container.DiagnosisInsight \
        --location=LOCATION \
        --project=PROJECT_ID \
        --format=FORMAT \
        --filter="insightSubtype:SUBTYPE"
    

    Replace the following:

    • PROJECT_ID: the Google Cloud project ID for the project where the cluster(s) are located.
    • LOCATION: the exact region or zone of the cluster. For zonal clusters, you must provide the exact zone (e.g. us-central1-c). For regional clusters, you must provide the exact region (e.g. us-central1).
    • FORMAT: change the output format to YAML. This flag is optional.
    • SUBTYPE: an insight subtype, for example, DEPRECATION_K8S_1_23_CERTIFICATE. This limits the output to insights of a specified subtype. This flag is optional.

    Or, if you already have an insight ID, you can see details about the insight by running the following command:

    gcloud recommender insights describe INSIGHT \
        --insight-type=google.container.DiagnosisInsight \
        --location=LOCATION \
        --project=PROJECT_ID \
        --format=FORMAT
    

    Replace the INSIGHT with the value of insight ID, specified at the end of the URL from the name field for the insight.

    The output includes additional details about what behavior GKE has detected with your cluster.

  2. View the list of recommendations for the clusters of a specific zone (for zonal clusters) or a specific region (for regional clusters):

    gcloud recommender recommendations list \
        --recommender=google.container.DiagnosisRecommender \
        --location=LOCATION \
        --project=PROJECT_ID \
        --format=FORMAT \
        --filter="recommenderSubtype:SUBTYPE"
    

    Replace SUBTYPE with a recommender subtype, for example, DEPRECATION_K8S_1_23_CERTIFICATE. This limits the output to recommendations of a specified subtype. This flag is optional.

    Or, if you already have a recommendation ID, you can view the recommendation by running the following command:

    gcloud recommender recommendations describe RECOMMENDATION_ID \
        --recommender=google.container.DiagnosisRecommender \
        --location=LOCATION \
        --project=PROJECT_ID \
        --format=FORMAT
    

    Replace the RECOMMENDATION_ID with the saved value of associatedRecommendations.recommendation from the command output in Step 1.

    The output provides guidance on what action you might need to take to optimize your cluster's usage of GKE.

API

Each insight has a corresponding recommendation. Insights are fetched with REST Resource: projects.locations.insightTypes.insights and recommendations are fetched with REST Resource: projects.locations.recommenders.recommendations.

  1. View the list of insights for the clusters of a specific zone (for zonal clusters) or a specific region (for regional clusters):

    GET https://recommender.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/insightTypes/google.container.DiagnosisInsight/insights?filter=insightSubtype%20%3D%20SUBTYPE
    

    Replace the following:

    • PROJECT_ID: the Google Cloud project ID for the project where the cluster(s) are located.
    • LOCATION: the exact region or zone of the cluster. For zonal clusters, you must provide the exact zone (e.g. us-central1-c). For regional clusters, you must provide the exact region (e.g. us-central1).
    • SUBTYPE: an insight subtype, for example, DEPRECATION_K8S_1_23_CERTIFICATE. This limits the output to insights of a specified subtype. This flag is optional.

    Or, if you already have an insight ID, you can see details about the insight by making the following request:

    GET https://recommender.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/insightTypes/google.container.DiagnosisInsight/insights/INSIGHT
    

    Replace the INSIGHT with the value of insight ID, specified at the end of the URL from the name field for the insight.

    The response body includes details about what behavior GKE has detected with your cluster.

  2. View the list of recommendations for the clusters of a specific zone (for zonal clusters) or a specific region (for regional clusters):

    GET https://recommender.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/recommenders/google.container.DiagnosisRecommender/recommendations?filter=recommenderSubtype%20%3D%20SUBTYPE
    

    Replace SUBTYPEwith a recommender subtype, for example, DEPRECATION_K8S_1_23_CERTIFICATE. This limits the output to recommendations of a specified subtype. This flag is optional.

    Or, if you already have a recommendation ID, you can view the recommendation by making the following request:

    GET https://recommender.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/recommenders/google.container.DiagnosisRecommender/recommendations/RECOMMENDATION_ID
    

    Replace the RECOMMENDATION_ID with the saved value of associatedRecommendations.recommendation from the response body in Step 1.

    The response body provides guidance on what action you might need to take to optimize your cluster's usage of GKE.

Example: Insight

A deprecation insight includes details about the timing and amount of usage of a deprecated feature or API. The information included in the insight depends on for what type of deprecation the insight is providing information.

This is an example insight for the DEPRECATION_K8S_1_22_V1BETA1_API insight subtype:

  name: projects/<PROJECT-NUMBER>/locations/us-central1/insightTypes/google.container.DiagnosisInsight/insights/<INSIGHT-ID>
  observationPeriod: 2592000s
  severity: HIGH
  category: RELIABILITY
  stateInfo:
    state: ACTIVE
  insightSubtype: DEPRECATION_K8S_1_22_V1BETA1_API
  description: Your API clients have used deprecated APIs in the last 30 days that are removed in Kubernetes v1.22. Upgrading your cluster before migrating to the updated APIs supported by v1.22 could cause it to break. [Learn more](https://kubernetes.io/docs/reference/using-api/deprecation-guide/#v1-22).
  targetResources:
    - //container.googleapis.com/projects/<PROJECT-NUMBER>/locations/us-central1/clusters/c1
  associatedRecommendations:
    - recommendation: projects/<PROJECT-NUMBER>/locations/us-central1/recommenders/google.container.DiagnosisRecommender/recommendations/<RECOMMENDATION-ID>
  etag: '"2147dd8e1e302ed7"'
  lastRefreshTime: "2022-01-30T08:00:00Z"
  content:
  targetClusters:
    - clusterId: <CLUSTER-ID>
      clusterUri: //container.googleapis.com/projects/<PROJECT-NUMBER>/locations/us-central1/clusters/c1
  apiDeprecationInsight:
    - api: /apis/networking.k8s.io/v1beta1/Ingress
      stopServingVersion: 1.22
      deprecatedClientStats:
        userAgent: kubectl
        numberOfRequestsLast30Days: 288
        lastRequestTime: "2022-02-30T08:00:18Z"
    - api: /apis/rbac.authorization.k8s.io/v1beta1/ClusterRole
      stopServingVersion: 1.22
      deprecatedClientStats:
        userAgent: kubectl
        numberOfRequestsLast30Days: 126
        lastRequestTime: "2022-02-01T06:45:25Z"

Example: Recommendation

A deprecation recommendation includes a description of the deprecation with a link to the migration guide for the deprecation. You can also find these migration guides in the GKE feature deprecations table and Kubernetes API deprecations table.

This is an example recommendation for the DEPRECATION_K8S_1_22_V1BETA1_API recommendation subtype:

  name: projects/<PROJECT-NUMBER>/locations/us-central1/recommenders/google.container.DiagnosisRecommender/recommendations/<RECOMMENDATION-ID>
  description: Update manifests and API clients to use v1 API before upgrading to Kubernetes 1.22 by following the [instructions](https://cloud.google.com/kubernetes-engine/docs/deprecations/apis-1-22).
  primaryImpact:
    category: RELIABILITY
    reliabilityProjection:
      risks:
      - SERVICE_DISRUPTION
  priority: P2
  recommenderSubtype: DEPRECATION_K8S_1_22_V1BETA1_API
  stateInfo:
    state: ACTIVE
  targetResources:
    - //container.googleapis.com/projects/<PROJECT-NUMBER>/locations/us-central1/clusters/c1
  associatedInsights:
    - insight: projects/<PROJECT-NUMBER>/locations/us-central1/insightTypes/google.container.DiagnosisInsight/insights/<INSIGHT-ID>
  etag: '"4dc0f7b33594072f"'
  lastRefreshTime: "2022-01-30T08:00:00Z"
  content:
    overview:
      targetClusters:
        - clusterId: <CLUSTER-ID>
          clusterUri: //container.googleapis.com/projects/<PROJECT-NUMBER>/locations/us-central1/clusters/c1
      apiDeprecationRecommendation:
        - api: /apis/networking.k8s.io/v1beta1/Ingress
          apiReplacement: /apis/networking.k8s.io/v1/Ingress
          stopServingVersion: 1.22
        - api: /apis/rbac.authorization.k8s.io/v1beta1/ClusterRole
          apiReplacement: /apis/rbac.authorization.k8s.io/v1/ClusterRole
          stopServingVersion: 1.22

Export insights and recommendations to BigQuery

You can use BigQuery to export and analyze insights and recommendations for your entire organization. To learn more, see Export recommendations to BigQuery.

Dismiss or restore deprecation recommendations

If you don't want to see a deprecation recommendation for a cluster in the Google Cloud console anymore, dismiss it. You might do this, for example, to dismiss recommendations that apply to temporary, short-lived clusters.

If you dismiss the recommendation, it doesn't appear for any user viewing the cluster in the console. After you dismiss the recommendation, it doesn't appear again even if a user agent calls the deprecated API or feature in this subtype after dismissal.

However, the recommendation remains discoverable with the Google Cloud CLI and the Recommender API.

When you dismiss a recommendation, you only hide it for all users. Automatic upgrades remain paused until you migrate from the deprecated feature or API and GKE does not detect usage of the deprecated APIs for 30 consecutive days, or until the cluster's version reaches end of support. To learn more, see When does GKE resume automatic upgrades?.

Dismiss a recommendation

To dismiss the recommendation, follow the instructions for Viewing insights and recommendations in the Google Cloud console to open the sidebar panel, which has more details. In that panel, click the Dismiss button.

Restore a dismissed recommendation

You can restore a dismissed recommendation by following the instructions for Restoring a recommendation.

What's next