This document lists production updates to Google Distributed Cloud (software only) for bare metal, formerly known as Google Distributed Cloud. Check this page periodically for any new announcements.
You can see the latest product updates for all of Google Cloud on the Google Cloud page, browse and filter all release notes in the Google Cloud console, or programmatically access release notes in BigQuery.
To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly.
October 01, 2024
Release 1.29.600-gke.108
Google Distributed Cloud for bare metal 1.29.600-gke.108 is now available for download. To upgrade, see Upgrade clusters. Google Distributed Cloud for bare metal 1.29.600-gke.108 runs on Kubernetes 1.29.
After a release, it takes approximately 7 to 14 days for the version to become available for installations or upgrades with the GKE On-Prem API clients: the Google Cloud console, the gcloud CLI, and Terraform.
If you use a third-party storage vendor, check the Ready storage partners document to make sure the storage vendor has already passed the qualification for this release of Google Distributed Cloud for bare metal.
Fixes:
- Fixed Cloud Audit Logging failure due to allowlisting issue with multiple project IDs.
The following container image security vulnerabilities have been fixed in 1.29.600-gke.108:
Critical container vulnerabilities:
High-severity container vulnerabilities:
Medium-severity container vulnerabilities:
Low-severity container vulnerabilities:
Known issues:
For information about the latest known issues, see Google Distributed Cloud for bare metal known issues in the Troubleshooting section.
September 12, 2024
Release 1.29.500-gke.163
Google Distributed Cloud for bare metal 1.29.500-gke.163 is now available for download. To upgrade, see Upgrade clusters. Google Distributed Cloud for bare metal 1.29.500-gke.163 runs on Kubernetes v1.29.7-gke.1200.
After a release, it takes approximately 7 to 14 days for the version to become available for installations or upgrades with the GKE On-Prem API clients: the Google Cloud console, the gcloud CLI, and Terraform.
If you use a third-party storage vendor, check the Ready storage partners document to make sure the storage vendor has already passed the qualification for this release of Google Distributed Cloud for bare metal.
The following container image security vulnerabilities have been fixed in 1.29.500-gke.163:
High-severity container vulnerabilities:
Medium-severity container vulnerabilities:
Low-severity container vulnerabilities:
Known issues:
For information about the latest known issues, see Google Distributed Cloud for bare metal known issues in the Troubleshooting section.
August 08, 2024
Release 1.29.400-gke.86
Google Distributed Cloud for bare metal 1.29.400-gke.86 is now available for download. To upgrade, see Upgrade clusters. Google Distributed Cloud for bare metal 1.29.400-gke.86 runs on Kubernetes 1.29.
If you use a third-party storage vendor, check the Ready storage partners document to make sure the storage vendor has already passed the qualification for this release of Google Distributed Cloud for bare metal.
- GA: Added support in version 1.29.400-gke.86 and higher for Red Hat Enterprise Linux (RHEL) version 9.2. For more information, see Select your operating system.
Fixes:
The following container image security vulnerabilities have been fixed in 1.29.400-gke.86:
Fixed the following vulnerabilities:
High-severity container vulnerabilities:
Medium-severity container vulnerabilities:
Low-severity container vulnerabilities:
Known issues:
For information about the latest known issues, see Google Distributed Cloud for bare metal known issues in the Troubleshooting section.
July 25, 2024
Release 1.29.300-gke.185
Google Distributed Cloud for bare metal 1.29.300-gke.185 is now available for download. To upgrade, see Upgrade clusters. Google Distributed Cloud for bare metal 1.29.300-gke.185 runs on Kubernetes 1.29.
If you use a third-party storage vendor, check the Ready storage partners document to make sure the storage vendor has already passed the qualification for this release of Google Distributed Cloud for bare metal.
Functionality changes:
Updated registry mirror support to allow you to specify a port for host addresses.
Updated Kubernetes audit logging to include request and response payloads from the Kubernetes API server for bare metal custom resources, such as
Cluster
,NodePool
,BareMetalMachine
, andBareMetalCluster
.
Fixes:
The following container image security vulnerabilities have been fixed in 1.29.300-gke.185:
High-severity container vulnerabilities:
Medium-severity container vulnerabilities:
Low-severity container vulnerabilities:
Known issues:
For information about the latest known issues, see Google Distributed Cloud for bare metal known issues in the Troubleshooting section.
July 03, 2024
Security bulletin (all minor versions)
A remote code execution vulnerability, CVE-2024-6387, was recently discovered in OpenSSH. The vulnerability exploits a race condition that can be used to obtain access to a remote shell, enabling attackers to gain root access. At the time of publication, exploitation is believed to be difficult and take several hours per machine being attacked. We are not aware of any exploitation attempts. This vulnerability has a Critical severity.
For mitigation steps and more details, see the GCP-2024-040 security bulletin.
June 27, 2024
Release 1.29.200-gke.243
Google Distributed Cloud for bare metal 1.29.200-gke.243 is now available for download. To upgrade, see Upgrade clusters. Google Distributed Cloud for bare metal 1.29.200-gke.243 runs on Kubernetes 1.29.
If you use a third-party storage vendor, check the Ready storage partners document to make sure the storage vendor has already passed the qualification for this release of Google Distributed Cloud for bare metal.
Functionality changes:
Updated registry mirror support to allow you to specify a port for host addresses.
Updated the networking preflight check to verify that either the
ip_tables
or thenf_tables
kernel module is available for loading, instead of being explicitly loaded.Added support for Red Hat Enterprise Linux 8.10 for Google Distributed Cloud software version 1.29.200-gke.243 and higher.
Fixes:
Fixed an issue where upgraded clusters didn't get label updates that match the labels applied for newly created clusters, for a given version.
Fixed an issue where service accounts created by using the
--create-service-accounts
flag with thebmctl create config
command don't have enough permissions.
The following container image security vulnerabilities have been fixed in 1.29.200-gke.243
High-severity container vulnerabilities:
Medium-severity container vulnerabilities:
Low-severity container vulnerabilities:
Known issues:
For information about the latest known issues, see Google Distributed Cloud for bare metal known issues in the Troubleshooting section.
May 28, 2024
Security bulletin (all minor versions)
A new vulnerability (CVE-2024-4323) has been discovered in Fluent Bit that could result in remote code execution. Fluent Bit versions 2.0.7 through 3.0.3 are affected.
Google Distributed Cloud software doesn't use a vulnerable version of Fluent Bit and is unaffected.
For more information, see the GCP-2024-031 security bulletin.
May 15, 2024
Release 1.29.100-gke.251
GKE on Bare Metal 1.29.100-gke.251 is now available for download. To upgrade, see Upgrade clusters. GKE on Bare Metal 1.29.100-gke.251 runs on Kubernetes 1.29.
If you use a third-party storage vendor, check the GDCV Ready storage partners document to make sure the storage vendor has already passed the qualification for this release of GKE on Bare Metal.
Added new API and IAM role requirements for Cloud Monitoring:
You must enable the
kubernetesmetadata.googleapis.com
API for your project and grant theroles/kubernetesmetadata.publisher
IAM role to the Logging and Monitoring service account (anthos-baremetal-cloud-ops
, when created automatically). Clusters use this API as an endpoint to send Kubernetes metadata to Google Cloud. The metadata is vital for cluster monitoring, debugging, and recovery. If you install your clusters behind a proxy, addkubernetesmetadata.googleapis.com
to the list of allowed connections.Due to changes in the way service accounts are checked, you must also grant the following IAM roles to the Logging and Monitoring service account:
roles/monitoring.viewer
roles/serviceusage.serviceUsageViewer
These API and IAM role requirements apply to both creating new 1.29 clusters and upgrading existing clusters to 1.29.
Functionality changes:
Added checks to validate the SSH client certificate file type before saving the certificate as a Secret.
Deprecated the
spec.gkeVersion
field inMachine
andBareMetalMachine
custom resources. After GKE on Bare Metal release 1.30, the value ofgkeVersion
isn't guaranteed to be reliable.Added preflight checks for available disk space in specific directories:
During cluster creation, the following directories are checked:
/
(the root directory) has at least 4 GiB of free space/var/log/fluent-bit-buffers
has at least 12 GiB of free space/var/opt/buffered-metrics
has at least 10016 MiB of free space
During a cluster upgrade, the following directory is checked:
/
(the root directory) has at least 2 GiB of free space
Fixes:
- Fixed an issue where the kubelet doesn't honor shortened, 1-second grace period for pod deletion during eviction-based draining.
The following container image security vulnerabilities have been fixed in 1.29.100-gke.251:
Medium-severity container vulnerabilities:
Known issues:
For information about the latest known issues, see GKE on Bare Metal known issues in the Troubleshooting section.
April 29, 2024
Release 1.29.0-gke.1449
GKE on Bare Metal 1.29.0-gke.1449 is now available for download. To upgrade, see Upgrade clusters. GKE on Bare Metal 1.29.0-gke.1449 runs on Kubernetes 1.29.
If you use a third-party storage vendor, check the GDCV Ready storage partners document to make sure the storage vendor has already passed the qualification for this release of GKE on Bare Metal.
Version 1.15 end of life: In accordance with the Version Support Policy, version 1.15 (all patch releases) of GKE on Bare Metal has reached its end of life and is no longer supported.
Added new API and IAM role requirements for Cloud Monitoring:
You must enable the
kubernetesmetadata.googleapis.com
API for your project and grant theroles/kubernetesmetadata.publisher
IAM role to the Logging and Monitoring service account (anthos-baremetal-cloud-ops
, when created automatically). Clusters use this API as an endpoint to send Kubernetes metadata to Google Cloud. The metadata is vital for cluster monitoring, debugging, and recovery. If you install your clusters behind a proxy, addkubernetesmetadata.googleapis.com
to the list of allowed connections.Due to changes in the way service accounts are checked, you must also grant the following IAM roles to the Logging and Monitoring service account:
roles/monitoring.viewer
roles/serviceusage.serviceUsageViewer
These API and IAM role requirements apply to both creating new 1.29 clusters and upgrading existing clusters to 1.29.
GA: Support GKE Identity Service v2 capability for an improved security flow when you authenticate with third-party identity solutions.
The GA offering of GKE Identity Service v2 has the following requirements and restrictions:
GKE Identity Service v2 now requires ports
11001
and11002
on the control plane load balancer nodes, instead of8443
and8444
. Ensure these ports are open and available before you upgrade a cluster to version 1.29.0-gke.1449 and higher. If the ports aren't open, upgrade preflight checks fail.GKE Identity Service v2 requires version 1.5.1 or higher of the Anthos Auth gcloud CLI component. If necessary, update the Anthos Auth component (
gcloud components update anthos-auth
). If you use the Google Cloud SDK, updating the SDK (gcloud components update
) to version 474.0.0 or later also updates the Anthos Auth component to the required version.GKE Identity Service v2 doesn't work with GKE on Bare Metal clusters with the following configurations:
Clusters with a single control plane node only.
Clusters that use control plane nodes for load balancing. That is, clusters that aren't configured with either a separate load balancing node pool or manual load balancing.
GA: Added support for skews of up to two minor versions for selective node pool upgrades.
GA: Added capability to pause and resume cluster upgrades.
GA: Maintenance mode now uses eviction-based draining for nodes, instead of taint-based draining. Eviction-based draining uses the Eviction API, which honors Pod Disruption Budgets (PDBs). Draining nodes this way provides better protection against workload disruptions.
Preview: Added support for node-level private registry configuration for workload images.
Preview: Added support for rolling back select node pool upgrades.
Preview: Added support for admin and hybrid clusters to manage multiple versions user clusters concurrently.
Preview: Added support for using an intermediate Certificate Authority (CA) as the cluster root CA.
Preview: Added support to route workload logs to a third-party custom Kafka destination. This capability isn't enabled by default. You enable this capability in the cluster
stackdriver
resource spec by adding theunmanagedKafkaOutputConfig
section. This section lets you specify the IP addresses of Kafka message brokers (brokers
), topic names (topics
), and keys to map the topics to partitions (topicKeys
).Improved command-line interface errors and error documentation.
Functionality changes:
GKE Identity Service v2 now sends extra parameters (
extraParams
) to your OIDC provider.Extra node viewing permissions are added for accounts specified with the
spec.clusterSecurity.authorization.clusterViewer.gcpAccounts
field in the Cluster resource.Added
Status.Available
field toBareMetalMachine
resources to indicate whether the machine is available.Updated preflight checks add a check for networking kernel modules (
ip_tables
ornp_tables
) and remove theiptables
package check.The Google plugin for the GKE Identity Service now caches the public keys based on
max-age
incache-control
header.
Fixes:
Fixed an issue where the kubelet doesn't honor shortened, 1-second grace period for pod deletion during eviction-based draining.
Fixed a cluster upgrade issue where the
lifecycle-controller-deployer
Pod was unable to migrate existing GKE on Bare Metal resources to the latest API version. This issue blocked upgrades to earlier version 1.28 releases.Fixed an issue with configuring a proxy for your cluster that required you to manually set
HTTPS_PROXY
andNO_PROXY
environment variables on the admin workstation.Fixed an issue where upgrades are blocked because
cluster-operator
can't delete stale, failing preflight check resources.Fixed an issue where the network check ConfigMap wasn't updated when nodes were added or removed.
The following container image security vulnerabilities have been fixed in version 1.29.0-gke.1449:
Critical container vulnerabilities:
High-severity container vulnerabilities:
Medium-severity container vulnerabilities:
Low-severity container vulnerabilities:
Known issues:
- Clusters that use bundled load balancing with BGP might have performance degradation as the total number of Services of type
LoadBalancer
approaches 2,000.
For information about the latest known issues, see GKE on Bare Metal known issues in the Troubleshooting section.