This page shows you how to view insights and recommendations about Kubernetes deprecations for your Google Kubernetes Engine (GKE) clusters. To learn more about how to manage Kubernetes deprecations with GKE, see GKE deprecations.
What are deprecation insights and recommendations?
When GKE detects that a cluster is using a Kubernetes feature or API that is deprecated and will be removed in an upcoming minor version, the following happens:
- Automatic upgrade to the upcoming minor version is paused. To learn more about how this works, see What happens when GKE pauses automatic upgrades.
- An insight and recommendation are generated so that you can assess and mitigate your cluster's exposure to the deprecation.
Deprecation insights and recommendations are available from Recommender, a service that provides insights and recommendations for using resources on Google Cloud. To learn more about how Recommender works with GKE, see Optimize your usage of GKE with insights and recommendations.
For the deprecations topic with Recommender:
- An insight explains that your cluster uses a feature or API that is deprecated and will be removed in an upcoming minor version.
- A recommendation provides guidance on what to do to mitigate your cluster's exposure to the deprecation.
For example, an insight might explain that your cluster is using a beta API that is deprecated and will be removed in the next minor version. The recommendation explains that you should migrate your workloads from using the beta APIs to using v1 APIs. Each recommendation includes a reference to a migration guide for the deprecated feature or API. This migration guide is also available in the corresponding table for Kubernetes deprecations.
When GKE generates an insight and recommendation pair, it covers
usage of any deprecated Kubernetes APIs or features in a particular
insight subtype for one cluster in your project. For
example, if any user agents call any of the APIs addressed by
the DEPRECATION_K8S_1_22_V1BETA1_API
subtype, one insight and recommendation
appear to recommend that you migrate this cluster from using these APIs. If
another cluster is also using these APIs, GKE generates another
insight and recommendation.
Use of deprecated Kubernetes features and APIs detected by GKE
Deprecation | Removed in GKE version | Insight subtype |
---|---|---|
TLS certificates signed with SHA-1 algorithm | 1.29 | DEPRECATION_K8S_SHA_1_CERTIFICATE |
Kubernetes 1.29 deprecated APIs | 1.29 | DEPRECATION_K8S_1_29_API |
Kubernetes 1.27 deprecated APIs | 1.27 | DEPRECATION_K8S_1_27_API |
Kubernetes 1.26 deprecated APIs | 1.26 | DEPRECATION_K8S_1_26_API |
Kubernetes 1.25 deprecated APIs | 1.25 | DEPRECATION_K8S_1_25_API |
PodSecurityPolicy | 1.25 | DEPRECATION_K8S_1_25_PODSECURITYPOLICY |
Docker-based node images | 1.24 | DEPRECATION_K8S_1_24_DOCKERSHIM |
X.509 Common Name field in webhook certificates | 1.23 | DEPRECATION_K8S_1_23_CERTIFICATE |
Kubernetes 1.22 deprecated APIs,
Kubernetes Ingress Beta APIs removed in GKE 1.23 |
1.22, 1.23 | DEPRECATION_K8S_1_22_V1BETA1_API |
Remember that for upcoming deprecations, you must independently assess your cluster environment's exposure and mitigate any issues.
Before you begin
Before you start, make sure you have performed the following tasks:
- Enable the Google Kubernetes Engine API. Enable Google Kubernetes Engine API
- If you want to use the Google Cloud CLI for this task,
install and then
initialize the
gcloud CLI. If you previously installed the gcloud CLI, get the latest
version by running
gcloud components update
.
Required roles
Ensure that you have the necessary permissions for Kubernetes insights and recommendations, either with the following basic roles or predefined roles:
View insights and recommendations with one of the following roles:
View and update the state of insights and recommendations (for example, dismiss a recommendation):
View insights and recommendations
You can view insights and recommendations with the Google Cloud CLI, in the Google Cloud console, or with the Recommender API.
Console
Go to the Google Kubernetes Engine page in the Google Cloud console:
Check the Notifications column for specific clusters to find insights.
Click on the insight to reveal more information. In the revealed sidebar panel, you can see details about this insight, including any associated recommendation.
gcloud
An insight typically has a corresponding recommendation. Insights are
fetched with gcloud recommender insights
and recommendations are fetched with gcloud recommender recommendations
.
View the list of insights for the clusters of a specific zone (for zonal clusters) or a specific region (for regional clusters):
gcloud recommender insights list \ --insight-type=google.container.DiagnosisInsight \ --location=LOCATION \ --project=PROJECT_ID \ --format=FORMAT \ --filter="insightSubtype:SUBTYPE"
Replace the following:
PROJECT_ID
: the Google Cloud project ID for the project where the cluster(s) are located.LOCATION
: the exact region or zone of the cluster. For zonal clusters, you must provide the exact zone (e.g.us-central1-c
). For regional clusters, you must provide the exact region (e.g.us-central1
).FORMAT
: change the output format to YAML. This flag is optional.SUBTYPE
: an insight subtype, for example,DEPRECATION_K8S_1_23_CERTIFICATE
. This limits the output to insights of a specified subtype. This flag is optional.
Or, if you already have an insight ID, you can see details about the insight by running the following command:
gcloud recommender insights describe INSIGHT \ --insight-type=google.container.DiagnosisInsight \ --location=LOCATION \ --project=PROJECT_ID \ --format=FORMAT
Replace the
INSIGHT
with the value of insight ID, specified at the end of the URL from thename
field for the insight.The output includes additional details about what behavior GKE has detected with your cluster.
View the list of recommendations for the clusters of a specific zone (for zonal clusters) or a specific region (for regional clusters):
gcloud recommender recommendations list \ --recommender=google.container.DiagnosisRecommender \ --location=LOCATION \ --project=PROJECT_ID \ --format=FORMAT \ --filter="recommenderSubtype:SUBTYPE"
Replace
SUBTYPE
with a recommender subtype, for example,DEPRECATION_K8S_1_23_CERTIFICATE
. This limits the output to recommendations of a specified subtype. This flag is optional.Or, if you already have a recommendation ID, you can view the recommendation by running the following command:
gcloud recommender recommendations describe RECOMMENDATION_ID \ --recommender=google.container.DiagnosisRecommender \ --location=LOCATION \ --project=PROJECT_ID \ --format=FORMAT
Replace the
RECOMMENDATION_ID
with the saved value ofassociatedRecommendations.recommendation
from the command output in Step 1.The output provides guidance on what action you might need to take to optimize your cluster's usage of GKE.
API
Each insight has a corresponding recommendation. Insights are fetched with REST Resource: projects.locations.insightTypes.insights and recommendations are fetched with REST Resource: projects.locations.recommenders.recommendations.
View the list of insights for the clusters of a specific zone (for zonal clusters) or a specific region (for regional clusters):
GET https://recommender.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/insightTypes/google.container.DiagnosisInsight/insights?filter=insightSubtype%20%3D%20SUBTYPE
Replace the following:
PROJECT_ID
: the Google Cloud project ID for the project where the cluster(s) are located.LOCATION
: the exact region or zone of the cluster. For zonal clusters, you must provide the exact zone (e.g.us-central1-c
). For regional clusters, you must provide the exact region (e.g.us-central1
).SUBTYPE
: an insight subtype, for example,DEPRECATION_K8S_1_23_CERTIFICATE
. This limits the output to insights of a specified subtype. This flag is optional.
Or, if you already have an insight ID, you can see details about the insight by making the following request:
GET https://recommender.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/insightTypes/google.container.DiagnosisInsight/insights/INSIGHT
Replace the
INSIGHT
with the value of insight ID, specified at the end of the URL from thename
field for the insight.The response body includes details about what behavior GKE has detected with your cluster.
View the list of recommendations for the clusters of a specific zone (for zonal clusters) or a specific region (for regional clusters):
GET https://recommender.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/recommenders/google.container.DiagnosisRecommender/recommendations?filter=recommenderSubtype%20%3D%20SUBTYPE
Replace
SUBTYPE
with a recommender subtype, for example,DEPRECATION_K8S_1_23_CERTIFICATE
. This limits the output to recommendations of a specified subtype. This flag is optional.Or, if you already have a recommendation ID, you can view the recommendation by making the following request:
GET https://recommender.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/recommenders/google.container.DiagnosisRecommender/recommendations/RECOMMENDATION_ID
Replace the
RECOMMENDATION_ID
with the saved value ofassociatedRecommendations.recommendation
from the response body in Step 1.The response body provides guidance on what action you might need to take to optimize your cluster's usage of GKE.
Example: Insight
A deprecation insight includes details about the timing and amount of usage of a deprecated feature or API. The information included in the insight depends on for what type of deprecation the insight is providing information.
This is an example insight for the DEPRECATION_K8S_1_22_V1BETA1_API
insight
subtype:
name: projects/<PROJECT-NUMBER>/locations/us-central1/insightTypes/google.container.DiagnosisInsight/insights/<INSIGHT-ID>
observationPeriod: 2592000s
severity: HIGH
category: RELIABILITY
stateInfo:
state: ACTIVE
insightSubtype: DEPRECATION_K8S_1_22_V1BETA1_API
description: Your API clients have used deprecated APIs in the last 30 days that are removed in Kubernetes v1.22. Upgrading your cluster before migrating to the updated APIs supported by v1.22 could cause it to break. [Learn more](https://kubernetes.io/docs/reference/using-api/deprecation-guide/#v1-22).
targetResources:
- //container.googleapis.com/projects/<PROJECT-NUMBER>/locations/us-central1/clusters/c1
associatedRecommendations:
- recommendation: projects/<PROJECT-NUMBER>/locations/us-central1/recommenders/google.container.DiagnosisRecommender/recommendations/<RECOMMENDATION-ID>
etag: '"2147dd8e1e302ed7"'
lastRefreshTime: "2022-01-30T08:00:00Z"
content:
targetClusters:
- clusterId: <CLUSTER-ID>
clusterUri: //container.googleapis.com/projects/<PROJECT-NUMBER>/locations/us-central1/clusters/c1
apiDeprecationInsight:
- api: /apis/networking.k8s.io/v1beta1/Ingress
stopServingVersion: 1.22
deprecatedClientStats:
userAgent: kubectl
numberOfRequestsLast30Days: 288
lastRequestTime: "2022-02-30T08:00:18Z"
- api: /apis/rbac.authorization.k8s.io/v1beta1/ClusterRole
stopServingVersion: 1.22
deprecatedClientStats:
userAgent: kubectl
numberOfRequestsLast30Days: 126
lastRequestTime: "2022-02-01T06:45:25Z"
Example: Recommendation
A deprecation recommendation includes a description of the deprecation with a link to the migration guide for the deprecation. You can also find these migration guides in the GKE feature deprecations table and Kubernetes API deprecations table.
This is an example recommendation for the DEPRECATION_K8S_1_22_V1BETA1_API
recommendation subtype:
name: projects/<PROJECT-NUMBER>/locations/us-central1/recommenders/google.container.DiagnosisRecommender/recommendations/<RECOMMENDATION-ID>
description: Update manifests and API clients to use v1 API before upgrading to Kubernetes 1.22 by following the [instructions](https://cloud.google.com/kubernetes-engine/docs/deprecations/apis-1-22).
primaryImpact:
category: RELIABILITY
reliabilityProjection:
risks:
- SERVICE_DISRUPTION
priority: P2
recommenderSubtype: DEPRECATION_K8S_1_22_V1BETA1_API
stateInfo:
state: ACTIVE
targetResources:
- //container.googleapis.com/projects/<PROJECT-NUMBER>/locations/us-central1/clusters/c1
associatedInsights:
- insight: projects/<PROJECT-NUMBER>/locations/us-central1/insightTypes/google.container.DiagnosisInsight/insights/<INSIGHT-ID>
etag: '"4dc0f7b33594072f"'
lastRefreshTime: "2022-01-30T08:00:00Z"
content:
overview:
targetClusters:
- clusterId: <CLUSTER-ID>
clusterUri: //container.googleapis.com/projects/<PROJECT-NUMBER>/locations/us-central1/clusters/c1
apiDeprecationRecommendation:
- api: /apis/networking.k8s.io/v1beta1/Ingress
apiReplacement: /apis/networking.k8s.io/v1/Ingress
stopServingVersion: 1.22
- api: /apis/rbac.authorization.k8s.io/v1beta1/ClusterRole
apiReplacement: /apis/rbac.authorization.k8s.io/v1/ClusterRole
stopServingVersion: 1.22
Export insights and recommendations to BigQuery
You can use BigQuery to export and analyze insights and recommendations for your entire organization. To learn more, see Export recommendations to BigQuery.
Dismiss or restore deprecation recommendations
If you don't want to see a deprecation recommendation for a cluster in the Google Cloud console anymore, dismiss it. You might do this, for example, to dismiss recommendations that apply to temporary, short-lived clusters.
If you dismiss the recommendation, it doesn't appear for any user viewing the cluster in the console. After you dismiss the recommendation, it doesn't appear again even if a user agent calls the deprecated API or feature in this subtype after dismissal.
However, the recommendation remains discoverable with the Google Cloud CLI and the Recommender API.
When you dismiss a recommendation, you only hide it for all users. Automatic upgrades remain paused until you migrate from the deprecated feature or API and GKE does not detect usage of the deprecated APIs for 30 consecutive days, or until the cluster's version reaches end of support. To learn more, see When does GKE resume automatic upgrades?.
Dismiss a recommendation
To dismiss the recommendation, follow the instructions for Viewing insights and recommendations in the Google Cloud console to open the sidebar panel, which has more details. In that panel, click the Dismiss button.
Restore a dismissed recommendation
You can restore a dismissed recommendation by following the instructions for Restoring a recommendation.
What's next
- To learn more about how deprecations work with GKE, see GKE deprecations.
- To learn more about the Recommender API, see Using the API - Recommendations.