This document lists production updates to Google Distributed Cloud. We recommend that Google Distributed Cloud developers periodically check this list for any new announcements.
You can see the latest product updates for all of Google Cloud on the Google Cloud page, browse and filter all release notes in the Google Cloud console, or programmatically access release notes in BigQuery.
To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly.
January 31, 2024
Security bulletin (all minor versions)
A security vulnerability, CVE-2024-21626, has been discovered in runc
where a user with permission to create Pods might be able to gain full access to the node filesystem.
For instructions and more details, see the GCP-2024-005 security bulletin.
June 27, 2023
Security bulletin (all minor versions)
A number of vulnerabilities have been discovered in Envoy, which is used in Anthos Service Mesh (ASM). These were reported separately as GCP-2023-002.
For more information, see the GCP-2023-016 security bulletin.
June 16, 2023
Security bulletin (all minor versions)
Two new security issues were discovered in Kubernetes where users may be able to launch containers that bypass policy restrictions when using ephemeral containers and either ImagePolicyWebhook (CVE-2023-2727) or the ServiceAccount admission plugin (CVE-2023-2728).
For more information, see the GCP-2023-014 security bulletin.
May 10, 2023
CentOS Linux 8 Support Deprecated
CentOS Linux 8 reached its end of life (EOL) on December 31st, 2021. We strongly recommend that you migrate to one of the other supported operating systems from Anthos clusters on bare metal. All support for CentOS is removed from Anthos clusters for bare metal release 1.17 (December 2023) and subsequent releases.
April 12, 2023
Kubernetes image registry redirect
As of March 21, 2023, traffic to k8s.gcr.io
is redirected to registry.k8s.io
, following the community announcement. This change is happening gradually to reduce disruption, and should be transparent for most Anthos clusters.
To check for edge cases and mitigate potential impact to your clusters, follow the step-by-step guidance in k8s.gcr.io Redirect to registry.k8s.io - What You Need to Know.
June 09, 2022
Release 1.9.8
Anthos clusters on bare metal 1.9.8 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.9.8 runs on Kubernetes 1.21.
Fixes:
The following container image security vulnerabilities have been fixed:
Known issues:
For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.
April 27, 2022
Release 1.9.7
Anthos clusters on bare metal 1.9.7 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.9.7 runs on Kubernetes 1.21.
Fixes:
- The following container image security vulnerabilities have been fixed:
Known issues:
For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.
April 26, 2022
Security bulletin (all minor versions)
Two security vulnerabilities, CVE-2022-1055 and CVE-2022-27666 have been discovered in the Linux kernel. Each can lead to a local attacker being able to perform a container breakout, privilege escalation on the host, or both. These vulnerabilities affect Linux operating systems supported by Anthos clusters on bare metal. For instructions and more details, see the GCP-2022-014 security bulletin.
April 12, 2022
Security bulletin (1.8, 1.9, and 1.10)
A security vulnerability, CVE-2022-23648, has been discovered in containerd's handling of path traversal in the OCI image volume specification. Containers launched through containerd's CRI implementation with a specially-crafted image configuration could gain full read access to arbitrary files and directories on the host.
For more information, see the GCP-2022-013 security bulletin.
March 23, 2022
Release 1.9.6
Anthos clusters on bare metal 1.9.6 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.9.6 runs on Kubernetes 1.21.
Fixes:
- The following container image security vulnerabilities have been fixed:
- CVE-2021-43824
- CVE-2021-43825
- CVE-2021-43826
- CVE-2021-21654
- CVE-2021-21655
- CVE-2021-23606
- CVE-2021-21657
- CVE-2021-21656
- CVE-2021-23635
- CVE-2022-23648
- CVE-2021-45960
- CVE-2021-3996
- CVE-2021-3995
- CVE-2021-45960
- CVE-2022-22823
- CVE-2022-22824
- CVE-2022-22822
- CVE-2022-23852
- CVE-2022-23990
- CVE-2021-43618
- CVE-2022-22825
- CVE-2022-22827
- CVE-2021-46143
- CVE-2022-22826
Known issues:
When you upgrade Anthos clusters on bare metal from a version with a security patch to the next minor release, we recommend that you upgrade to the highest patch version to ensure that you have the latest security fixes. Always review the release notes before upgrading so that you're aware of what has changed, including security fixes and known issues. Upgrading to a lower release version isn't supported.
For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.
February 28, 2022
Release 1.9.5
Anthos clusters on bare metal 1.9.5 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.9.5 runs on Kubernetes 1.21.
Fixes:
- The following container image security vulnerabilities have been fixed:
Known issues:
When you upgrade Anthos clusters on bare metal from a version with a security patch to the next minor release, we recommend that you upgrade to the highest patch version to ensure that you have the latest security fixes. Always review the release notes before upgrading so that you're aware of what has changed, including security fixes and known issues. Upgrading to a lower release version isn't supported.
For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.
February 25, 2022
Security bulletin (1.8, 1.9, and 1.10)
Envoy recently released multiple security vulnerability fixes. The vulnerabilities affect Anthos clusters on bare metal, because Envoy is used for Metrics Server.
For instructions and more details, see the GCP-2022-008 security bulletin.
February 04, 2022
Security bulletin (all minor versions)
A security vulnerability, CVE-2021-4034, has been discovered in pkexec
, a part of the Linux policy kit package (polkit), that allows an authenticated user to perform a privilege escalation attack. PolicyKit is generally used only on Linux desktop systems to allow non-root users to perform actions, such as rebooting the system, installing packages, restarting services, as governed by a policy.
For instructions and more details, see the GCP-2022-004 security bulletin.
January 27, 2022
Release 1.9.4
Anthos clusters on bare metal 1.9.4 is now available for download. To upgrade, see Upgrade Anthos on bare metal. Anthos clusters on bare metal 1.9.4 runs on Kubernetes 1.21.
Fixes:
- The following container image security vulnerabilities have been fixed:
Known issues:
When you upgrade Anthos clusters on bare metal from a version with a security patch to the next minor release, we recommend you upgrade to the highest patch version to ensure you have the latest security fixes. Always review the release notes before upgrading so you're aware of what has changed, including security fixes and known issues. Upgrading to a lower release version isn't supported.
For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.
December 22, 2021
Release 1.9.3
Anthos clusters on bare metal 1.9.3 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.9.3 runs on Kubernetes 1.21.
Fixes:
Fixed an issue in which cluster creation fails if a cluster has more than one control plane node, and the
HTTPS_PROXY
environment variable has been defined on one or more of the control plane nodes.Upgraded Kubernetes version from 1.21.4 to 1.21.5 to address an error in which pods become stuck in the
ContainerCreating
state becauselibcontainer
mistakenly throws a "unit already exists" error.The following container image security vulnerability has been fixed:
Known issues:
For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.
November 15, 2021
Release 1.9.2
Anthos clusters on bare metal 1.9.2 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.9.2 runs on Kubernetes 1.21.
Fixes:
Updated preflight checks to use strict mode for decoding YAML to catch issues with indentation and misplaced fields in the cluster configuration file.
Fixed an issue that caused
containerRuntime
to default todocker
, instead ofcontainerd
in certain uncommon situations.Fixed an issue where
node_filesystem
metrics report incorrect size in Cloud Monitoring for mount-points other than root.Fixed an issue that caused communication failures between Cloud Logging metadata agent and the Cloud Monitoring API when the root certificate authority (CA) on the host node isn't set up properly.
The following container image security vulnerabilities have been fixed:
Known issues:
For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.
October 29, 2021
Security bulletin (all minor versions)
The security community recently disclosed a new security vulnerability CVE-2021-30465 found in runc
that has the potential to allow full access to a node filesystem.
For more information, see the GCP-2021-011 security bulletin.
October 26, 2021
Release 1.9.1
Anthos clusters on bare metal 1.9.1 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.9.1 runs on Kubernetes 1.21.
Fixes:
Fixed
bmctl
to eliminate stack trace from error output.The following container image security vulnerabilities have been fixed:
Functionality changes:
- Updated the
bmctl reset cluster
command to prevent you from resetting an admin cluster if the admin cluster is managing user clusters. - Updated the
bmctl create cluster
command to block you from enabling the Anthos VM Runtime for admin clusters.
Known issues:
For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.
October 21, 2021
Security bulletin (all minor versions)
A security issue was discovered in the Kubernetes ingress-nginx controller, CVE-2021-25742. Ingress-nginx custom snippets allows retrieval of ingress-nginx service account tokens and secrets across all namespaces. For more information, see the GCP-2021-024 security bulletin.
October 04, 2021
Security bulletin (all minor versions)
A security vulnerability, CVE-2020-8561,
has been discovered in Kubernetes where certain webhooks can be made to
redirect kube-apiserver
requests to private networks of that API
server.
For more information, see the GCP-2021-021 security bulletin.
September 28, 2021
Release 1.9.0
Anthos clusters on bare metal 1.9.0 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.9.0 runs on Kubernetes 1.21.
Improved cluster lifecycle functionalities:
Preview: Added ability to reset individual nodes with the
bmctl reset node
command. To give access to the needed cluster configuration file, use the command with the-c
flag.Preview: Added ability to recover from HA control plane quorum loss with
bmctl restore --control-plane-node
command.Added
bmctl create ksa
command to create a Kubernetes Service Account (KSA) and generate a bearer token. To log in to the registered cluster, you can use the token in Cloud Console Kubernetes Engine > Clusters.Preview: Added
bmctl backup cluster
andbmctl restore cluster
commands to facilitate disaster recovery for clusters.
Introduced new troubleshooting capabilities:
Updated the
bmctl check cluster --snapshot
command to support uploading cluster diagnostic snapshots to a Cloud Storage bucket for review by Cloud Customer Care.Provided access to bootstrap cluster logs to help troubleshoot cluster creation or upgrade problems.
Preview: Added support for Node Problem Detector service on nodes for quick detection of common node problems.
Enhanced monitoring and logging:
GA: Cloud Audit Logs capability is now generally available and enabled by default. Audit logs are useful for investigating suspicious API requests and for collecting statistics. For more information, see Use Audit Logging.
Switched to new open telemetry-based metrics agents to improve reliability, ability to scale, and resource usage.
Improved networking capabilities:
GA: The multi-NIC capability to provide additional interfaces to your pods is now generally available.
Preview: Added the single root I/O virtualization (SR-IOV) container network interface (CNI) plugin for multi-NIC.
Added support to configure cluster Domain Name System (DNS) provider options, such as upstream nameservers, with the new
ClusterDNS
custom resource definition.
Enhanced security:
SELinux is now always enabled in the container runtime for CentOS and RHEL.
Preview: Enhanced the capability to rotate cluster certificate authorities (CAs). Updates include support for all cluster types, rotation of front-proxy and etcd CAs, and changes to the
bmctl
command syntax.Preview: Added Okta group support for authentication in Anthos Identity Service.
Functionality changes:
- Changed default container runtime to containerd,
containerRuntime: containerd
for new clusters. Customers can still choose Docker as the container runtime. Preview: Updated
bmctl
command,bmctl reset nodes --force
, to support force removal of control plane nodes with etcd membership cleanup.Added checks for cluster updates to verify access to cluster machines if changes to
loginUser
orsshKeyPrivatePath
are detected. If the checks pass, Anthos clusters on bare metal saves the secret in the cluster.Added new Anthos cluster control plane uptime dashboard in Cloud Monitoring with new metric
kubernetes.io/anthos/container/uptime
for component availability.Added new alerts for control plane components availability with new metric
kubernetes.io/anthos/container/uptime
to replace deprecated alerts with metrickubernetes.io/anthos/up
.
Fixes:
Added missing registry mirror package required for Cloud Audit Logs to the Registry Mirror.
Fixed issue with containerd not finding
crictl
due to/usr/local/bin
not being in the SSH user's PATH.Fixed flapping node readiness issues caused by an unhealthy Pod Lifecycle Event Generator (PLEG).
Fixed kernel support issue for Ubuntu 18.04 and 18.04.1 that prevented the
anetd
networking controller from working properly. Anthos clusters on bare metal release 1.9.0 works with all kernels supplied with supported distributions.
Known issues:
Control group v2 (cgroup v2) is not officially supported in Anthos clusters on bare metal release 1.9.0 and later. The presence of
/sys/fs/cgroup/cgroup.controllers
indicates that your system uses cgroup v2.Anthos Service Mesh v1.10 is incompatible with Anthos clusters on bare metal release 1.9.0 running on Red Hat Enterprise Linux (RHEL) when SELinux is enabled. If you want to use Anthos Service Mesh, you must disable SELinux or set it to permissive mode on the host.
For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.