This page contains release notes for releases, features, and updates to Google Distributed Cloud Edge.
You can see the latest product updates for all of Google Cloud on the Google Cloud page, browse and filter all release notes in the Google Cloud console, or programmatically access release notes in BigQuery.
To get the latest product updates delivered to you, add the URL of this page to your
feed
reader, or add the feed URL directly: https://cloud.google.com/feeds/distributed-cloud-edge-release-notes.xml
March 14, 2024
This is a patch release of Google Distributed Cloud Edge (version 1.6.1).
The following new features have been introduced in this release of Distributed Cloud Edge:
Multi-rack deployments. Distributed Cloud Edge now supports aggregating the resources of multiple Distributed Cloud Edge Racks into a single zone. You can now create clusters that span nodes across multiple Distributed Cloud Edge Racks. A single multi-rack deployment supports one Distributed Cloud Edge Base Rack and up to 10 Distributed Cloud Edge Standalone Racks. For more information, see How Distributed Cloud Edge works.
Distributed Cloud Edge Base Rack. We are now shipping a new form factor of Distributed Cloud Edge Rack hardware, the Distributed Cloud Edge Base Rack. This form factor is a pair of existing Distributed Cloud Edge Standalone Rack hardware with the addition of four network switches that aggregate network traffic from up to 10 Distributed Cloud Edge Standalone Racks.
Prometheus integration. You can now use the Prometheus metrics solution to collect Distributed Cloud Edge metrics and workload metrics on local control plane clusters running in survivability mode. For more information, see Collect metrics with Prometheus.
Node labels. You can now assign unique labels to individual nodes when creating a node pool. For more information, see Create a node pool.
The following changes have been introduced in this release of Distributed Cloud Edge:
Cloud control plane clusters can no longer be created in subsequent releases of Distributed Cloud Edge. Distributed Cloud Edge version 1.6.1 is the last release of Distributed Cloud Edge in which you can create Cloud control plane clusters. Creation of cloud control plane clusters will be disabled in the next minor release of Distributed Cloud Edge. Existing cloud control plane clusters will continue to run workloads.
Release channel requirement for specifying cluster software versions. If you want to specify a Distributed Cloud Edge software version when creating a cluster, you must now set the cluster's release channel to
NONE
. If you do not specify a release channel or explicitly set it toREGULAR
, the cluster automatically upgrades to the latest version of Distributed Cloud Edge software and specifying a software version is not possible.
This release of Distributed Cloud Edge contains the following known issues:
Nodes can get stuck in
Ready,SchedulingDisabled
state after applying configuration changes. Applying or deleting theNodeSystemConfigUpdate
orSriovNetworkNodePolicy
resources can result in a node that's stuck in theReady, Scheduling Disabled
state after it reboots. To resolve this issue, see Troubleshoot Distributed Cloud Edge.Deleting clusters and node pools fails when a node is not ready. If a node in a cluster or node pool that you want to delete is in the
NotReady
state, the deletion can fail. Contact Google Support to remedy this condition.Nodes using Symcloud Storage report the file system as read-only after reboot. When multiple nodes that use Symcloud Storage reboot at once in a cluster, they can incorrectly mark the file system as read-only. Contact Google Support to remedy this condition.
Google Distributed Cloud Edge management software has been updated.
December 19, 2023
This is a minor release of Google Distributed Cloud Edge (version 1.6.0).
The following new features have been introduced in this release of Distributed Cloud Edge:
Distributed Cloud Edge Servers. You can now order Distributed Cloud Edge in sets of three clustered server machines without ToR switches in addition to the existing fully configured rack offering. These three-machine clusters connect directly to your local network.
Configuration 8 standalone server machines. Distributed Cloud Edge now offers a new hardware option, the Configuration 8 standalone server machine. You can order this hardware option in sets of three. Each Configuration 8 machine offers 16 processor cores (32 vCPUs), 64GB of RAM, and 1.6TB of SSD storage. The machine is housed in a half-depth 1U rackmount chassis.
Symcloud Storage support for virtual machine workloads. Distributed Cloud Edge now supports configuring virtual machine workloads with Symcloud Storage.
Locally cached system images. Distributed Cloud Edge clusters can now access system images while disconnected from Google Cloud. This allows Pods to transfer onto another node using locally cached system images.
Manage clusters using the
kubectl
tool while disconnected from Google Cloud. You can now contact Google Cloud support to request emergency credentials to authenticate to a Distributed Cloud Edge cluster and manage it using thekubectl
command-line tool from your local network while disconnected from Google Cloud.
The following changes have been introduced in this release of Distributed Cloud Edge:
Reduced outbound network traffic. Distributed Cloud Edge has reduced the amount of outbound data it sends to Google Cloud. For a typical 3-machine deployment, outbound traffic bandwidth is now below 4Mbps.
Nodes can now rejoin clusters after rebooting while disconnected from Google Cloud. When creating a local control plane cluster, you can now configure it so that when a node reboots while your Distributed Cloud Edge deployment is disconnected from Google Cloud, the node rejoins its cluster after the reboot is complete and resumes running its designated workloads. For more information, see Create a cluster. This is a preview-level feature.
November 03, 2023
This is a patch release of Google Distributed Cloud Edge (version 1.5.1).
The following changes have been introduced in this release of Distributed Cloud Edge:
Cluster software version upgrades for local control plane clusters. You can now trigger a software version upgrade on a local control plane cluster to a specific version of Distributed Cloud Edge software, starting with version 1.5.1. This feature is not available for Cloud control plane clusters. For instructions, see Upgrade the software version on a local control plane cluster.
Cluster software version pinning for local control plane clusters. You can now pin a local control plane cluster to a specific version of Distributed Cloud Edge software, starting with version 1.5.0. A cluster pinned to a specific version does not automatically upgrade when new Distributed Cloud Software becomes available. This feature is not available for Cloud control plane clusters. For instructions, see Create a cluster.
Cluster status. The
gcloud edge-cloud container describe
command now returns the operational status of a Distributed Cloud Edge cluster.
The following issues have been resolved in this release of Distributed Cloud Edge:
- CVE-2022-40982 "Downfall" remediation. The CVE-2022-40982 vulnerability, also known as "Downfall," has been patched.
This release of Distributed Cloud Edge contains the following known issues:
Cloud SDK version 450.0.0 or later is required. You must upgrade your Cloud SDK to version 450.0.0 or later to create local control plane clusters with Distributed Cloud Edge software version 1.5.0. Otherwise, creating such clusters will fail.
Node and machine labels are not applied when upgrading to Distributed Cloud Edge version 1.5.1. When upgrading to Distributed Cloud Edge version 1.5.1, system-required labels might not be applied to nodes and machines within existing node pools. To work around this issue, either modify the affected node pool to update its corresponding resource definition, or delete and re-add the affected nodes. For instructions, see Create and manage node pools.
September 07, 2023
This is a minor release of Google Distributed Cloud Edge (version 1.5.0).
The following features have been introduced in this release of Distributed Cloud Edge:
Bastion host support. Distributed Cloud Edge now allows you to set up one or more bastion host virtual machines. The bastion host feature allows Google support engineers to connect to your Distributed Cloud Edge deployment and work with you to diagnose and resolve issues. For more information, see Configure a bastion host. This is a preview-level feature.
Selectable cluster software versions. You now have the option to create a cluster running a specific version of Distributed Cloud Edge software, starting with version 1.5.0. For more information, see Create and manage clusters. This is a preview-level feature.
Container image registry access over secondary networks. Distributed Cloud Edge now allows you to specify the network interface in the
spec.containerRuntimeDNSConfig
field of theNodeSystemConfigUpdate
resource. This allows you to specify a container image registry IP/domain pair for a network interface other than the primary. For more information, seeNodeSystemConfigUpdate
resource. This is a preview-level feature.CMEK support for local control plane nodes. You can now configure Cloud KMS integration for storage on nodes running local control planes for Distributed Cloud Edge clusters. For more information, see Enable support for customer-managed encryption keys (CMEK) for local storage.
The following changes have been introduced in this release of Distributed Cloud Edge:
Survivability mode is now generally available. For more information, see Distributed Cloud Edge survivability mode. After your Distributed Cloud Edge deployment has been upgraded from version 1.4.0 to version 1.5.0, you must manually delete and recreate all local control plane clusters you have created with Distributed Cloud Edge version 1.4.0 or 1.4.1. Otherwise, unexpected behavior and data loss can occur. Clusters configured to use a cloud control plane continue to run normally after upgrading Distributed Cloud Edge to version 1.5.0.
Symcloud Storage integration is now generally available. For more information, see Configure Distributed Cloud Edge for Symcloud Storage.
Local control plane clusters now support virtual machines and GPU workloads. For more information, see Manage virtual machines and Manage GPU workloads.
Loadable SCTP kernel modules. Distributed Cloud Edge now configures the sctp kernel module as loadable. This allows you to load custom networking stacks into the kernel's user space. For more information, see SCTP kernel modules. This is a preview-level feature.
The following issues have been resolved in this release of Distributed Cloud Edge:
When creating a Cloud control plane cluster, creating a node pool that includes nodes that were previously part of a local control plane cluster no longer fails.
BGP sessions now properly recover when the associated network interface goes down and then comes back up.
This release of Distributed Cloud Edge contains the following known issues:
When creating a local control plane cluster, Distributed Cloud Edge instantiates dummy
BGPPeer
andBGPLoadBalancer
resources. You can ignore these resources.Distributed Cloud Edge does not support BGP peering to multiple VLANs within the same virtual router. You must set up a separate virtual router with a unique loopback IP addresses for each affected VLAN to allow concurrent BGP peering sessions.
June 30, 2023
This is a patch release of Google Distributed Cloud Edge (version 1.4.1).
The following changes have been introduced in this release of Distributed Cloud Edge:
- The IP addresses of local control plane endpoints are now accessible on your local network. You must ensure that your local network's security configuration prevents external access to those IP addresses.
The following issues have been resolved in this release of Distributed Cloud Edge:
- Resource utilization metrics that were previously not exported to Cloud Monitoring are now exported as expected.
- The status of the
kube-apiserver
mirrored Pods is no longer erroneously reported as "Pending."
May 19, 2023
This is a minor release of Google Distributed Cloud Edge (version 1.4.0).
The following features have been introduced in this release of Distributed Cloud Edge:
Survivability mode. Distributed Cloud Edge now allows you to create clusters with the Kubernetes control plane running locally on your Distributed Cloud Edge hardware. This improves the reliability of Distributed Cloud Edge when your connection to Google Cloud is intermittent. This is a Public Preview feature. For more information, see Distributed Cloud Edge survivability mode.
Symcloud Storage integration. You can now integrate Distributed Cloud Edge with Rakuten Symcloud Storage, a third-party storage abstraction solution that allows Pods to access local storage on different Distributed Cloud Edge nodes. This is a Public Preview feature. For more information, see Configure Distributed Cloud Edge for Symcloud Storage.
Enhanced rNDC security. Distributed Cloud Edge has replaced the
bond0
interface with thegdcenet0
interface that allows you to use the physical management network interface card for your application workloads while maintaining complete separation from Distributed Cloud Edge control and management traffic. You must manually reconfigure any existing network resources that reference thebond0
interface to use thegdcenet0
interface. For more information, see Upgrade CustomNetworkInterfaceConfig resources from Distributed Cloud Edge 1.3.0 to 1.4.0 and Upgrade NetworkAttachmentDefinition resources to Distributed Cloud Edge 1.4.0.Cloud Router reuse for VPN connections. When creating a VPN connection, Distributed Cloud Edge now automatically reuses any Cloud Router resource it has automatically created for a VPN connection. You can also specify a custom Cloud Router resource when creating a VPN connection. Existing VPN connections are not affected. For more information, see Manage VPN connections.
The following changes have been introduced in this release of Distributed Cloud Edge:
The cross-project VPN connection functionality is now generally available. For more information, see Manage cross-project VPN connections.
The default behavior of the
gcloud edge-cloud clusters get credentials
command has changed. The command now requires the `gke-gcloud-auth-plugin
plugin, which replaces the legacyin-tree-auth-plugin
plugin. For more information about thegke-gcloud-auth-plugin
plugin, see Important changes to Kubectl authentication are coming in GKE v1.26. You have the option to revert to the legacyin-tree-auth-plugin
plugin by setting theUSE_GKE_CLOUD_AUTH_PLUGIN
environment flag tofalse
.The Kubernetes control plane has been updated to version 1.25.5-gke.1001 for all clusters.
The Kubernetes container daemon (
containerd
) has been updated to version 1.6.6-gke.1 for remote control plane clusters and to 1.6.12-gke.0 for survivability mode clusters.The Kubernetes worker node agent (
kubelet
) has been updated to version 1.24.7.gke.1700 for remote control plane clusters and 1.25.5-gke.1001 for local control plane clusters.Distributed Cloud Edge now supports the ConfigSync feature of Anthos Config Management. Distributed Cloud Edge does not support any other Anthos features.
The following issues have been resolved in this release of Distributed Cloud Edge:
Distributed Cloud Edge now supports dynamic IPAM for multi-networking configurations.
Disabling the Anthos VM Runtime virtual machine subsystem no longer removes the
network-controller-manager
container. You can now disable the subsystem without affecting Distributed Cloud Edge networking features.
This release of Distributed Cloud Edge contains the following known issues:
BGP sessions do not recover when the associated network interface goes down and then comes back up.
In the
CustomNetworkInterfaceConfig
resource, setting theifname
field togdcenet0
while themasterInterface
field is also set togdcenet0
causes the resource to not apply to the cluster.When configuring a
CustomNetworkInterfaceConfig
resource, you must explicitly set the MTU size to be no greater than the MTU size of its parent network interface. Otherwise, unpredictable behavior might result.If you reboot a node running a local control plane workload for a local control plane cluster, the cluster loses its GKEConnect connection to GKEHub until the node fully starts up again. The workloads deployed on the cluster continue to run.
If you are creating a remote control plane plane cluster, creating a node pool using nodes that were previously part of a local control plane cluster might fail. If you encounter this issue, contact Google Support for assistance.
March 27, 2023
This is a patch release of Google Distributed Cloud Edge (version 1.3.1).
The following changes have been introduced in this release of Distributed Cloud Edge:
- The Kubernetes control plane has been updated to version 1.24.9-gke.2500.
- The Kubernetes container daemon (
containerd
) has been updated to version 1.6.6-gke.1. - The Kubernetes worker node agent (
kubelet
) has been updated to version 1.24.7-gke.5.
The following issues have been resolved in this release of Distributed Cloud Edge:
- Errors in the
NodeSystemConfigUpdate
custom resource definition that shipped with Distributed Cloud Edge 1.3.0 have been corrected. The outputs of the affected status fields are now accurate.
This release of Distributed Cloud Edge contains the following known issues:
- If you have enabled the Anthos VM Runtime virtual machine subsystem, disabling it removes the
network-controller-manager
service and its container. This renders Distributed Cloud Edge networking inoperable. To prevent this, keep the Anthos VM Runtime virtual machine subsystem enabled on your Distributed Cloud Edge deployment. If the subsystem has been disabled, re-enable it by following the steps in Enable the Anthos VM Runtime support on Distributed Cloud Edge to restore Distributed Cloud Edge networking to an operable state.
February 21, 2023
This is a minor release of Google Distributed Cloud Edge (version 1.3.0).
The following new features have been introduced in this release of Google Distributed Cloud Edge:
- Distributed Cloud Edge now exposes the Edge Network API, which allows you to configure the networking components of Distributed Cloud Edge. For more information, see How it works and Distributed Cloud Edge networking features.
The following changes have been introduced in this release of Distributed Cloud Edge:
- Getting information about a Machine resource now returns the version of the Distributed Cloud Edge cluster stack.
- You can now connect Distributed Cloud Edge clusters to a Virtual Private Cloud network in a Cloud project other than your Distributed Cloud Edge cluster project.
- When creating a cross-project VPN connection, you can no longer specify a VPC project service account. Distributed Cloud Edge now uses your cluster project service account.
January 06, 2023
This is a patch release of Google Distributed Cloud Edge (version 1.2.2).
The following changes have been introduced in this release of Distributed Cloud Edge:
- The NVIDIA Tesla T4 GPU driver has been updated to version 515.65.01.
- The NVIDIA Tesla T4 GPU resource name has been changed from
nvidia.com/gpu
tonvidia.com/gpu-pod-TESLA_T4
. If you have existing GPU-based container workloads, you must manually update their configuration to use the new resource name. For more information, see Configure a container to use GPU resources. - The Kubernetes worker node agent (
kubelet
) has been updated to version 1.23.5-gke.1505.
November 07, 2022
This is a minor release of Google Distributed Cloud Edge (version 1.2.0).
The following new features have been introduced in this release of Google Distributed Cloud Edge:
- Anthos VM Runtime replaces Kubevirt in Google Distributed Cloud Edge starting with this release. To continue using your existing virtual machines, you must shut them down and back them up before your Distributed Cloud Edge deployment is upgraded to release 1.2.0, and then re-create them as described in Manage virtual machines.
- A new Google Distributed Cloud Edge hardware configuration is available. This new configuration supports GPU-based workloads that run on NVIDIA Tesla T4 GPUs in both containers and virtual machines. To order a GPU-enabled configuration, see Order Google Distributed Cloud Edge. To learn more about running workloads on GPUs, see Manage GPU workloads.
- Google Distributed Cloud Edge now supports the following networking features:
- (Preview) Cross-project VPN Connections. To learn more, see Manage cross-project VPN Connections.
- (Preview) MacVLAN driver support for creating secondary network interfaces for Pods running containerized workloads. The MacVLAN driver is not supported on Pods running virtual machines. To learn more, see Configure a secondary network interface on a Pod using the MacVLAN driver.
- (Preview) Multi-network support for creating secondary network interfaces for Pods. To learn more, see Configure a secondary network interface on a Pod using Distributed Cloud Edge multi-networking.
- (Preview)
ClusterDNS
resource. To learn more, see ClusterDNS resource.
The following changes have been introduced in this release of Google Distributed Cloud Edge:
- Google Distributed Cloud Edge now ships with the NVIDIA Tesla T4 GPU driver version 470.63.01.
- The Network Function operator feature of Google Distributed Cloud Edge has been updated as follows. To learn more, see Network Function operator.
- The
NodeSystemConfigUpdate
resource now supports additionalsysctls
fields. - The
NodeSystemConfigUpdate
resource now supports fields for specifying the IP address lists and domain lists of private image registries. - The
CustomNetworkInterfaceConfig
resource no longer supports certain previously supported fields. - You can now scope both safe and unsafe
sysctls
parameters to a specific Pod or namespace using thetuning
Container Networking Interface (CNI) plug-in. - Webhook-level enforcement of valid field values is now in effect.
- The
- The Kubernetes control plane has been updated to version 1.23.5-gke.1505.
- The
coredns
service has been updated to version 1.8.6-gke.0.
The following issues have been resolved in this release of Google Distributed Cloud Edge:
- Google Distributed Cloud Edge nodes no longer become temporarily unresponsive due to excessive memory utilization.
September 23, 2022
This is a patch release of Google Distributed Cloud Edge (version 1.1.2).
The following changes have been introduced in this release of Google Distributed Cloud Edge:
cgroups
has been reverted tov1
to retain compatibility with legacy workloads.- The Kubernetes control plane has been updated to version 1.22.8-gke.204.
- The Kubernetes container daemon (
containerd
) has been updated to version 1.5.13-gke.0. - The Kubernetes worker node agent (
kubelet
) has been updated to version 1.22.8-gke.200.
August 26, 2022
This is a patch release of Google Distributed Cloud Edge (version 1.1.1).
The following changes have been introduced in this release of Google Distributed Cloud Edge:
- Google Distributed Cloud Edge worker nodes have been updated to Kubernetes 1.22.
The following issues have been resolved in this release of Google Distributed Cloud Edge:
- The SR-IOV interface no longer fails to start after a Google Distributed Cloud Edge worker node has been rebooted.
July 14, 2022
This is a minor release of Google Distributed Cloud Edge (version 1.1.0).
The following changes have been introduced in this release of Google Distributed Cloud Edge:
- The Kubernetes control plane has been updated to version 1.22.
The following issues have been resolved in this release of Distributed Cloud Edge:
- The Kubernetes control plane no longer becomes intermittently unavailable during Google Distributed Cloud Edge software updates.
- VPN connectivity between non-Anthos gateway nodes and Google Cloud Platform now works reliably.
This release of Distributed Cloud Edge contains the following known issues:
- Garbage collection intermittently fails to clean up terminated Pods.
May 25, 2022
This is a patch release of Google Distributed Cloud Edge (version 1.0.2).
The following changes have been introduced in this release of of Distributed Cloud Edge:
Configuring a maintenance window now controls the scheduling of software updates for the Kubernetes control plane and Kubernetes nodes.
You can now deploy KubeVirt virtual machines on Distributed Cloud Edge in unmanaged namespaces with support for the Containerized Data Importer (CDI) plug-in.
The following issues have been resolved in this release of Distributed Cloud Edge:
Intermittent VPN connection persistence after deletion has been resolved. You no longer need to manually check whether the VPN connection and its associated resources have been successfully deleted.
The
localpv-shared
Persistent Volume has been eliminated. You will no longer see this Persistent Volume on the filesystem of your Distributed Cloud Edge nodes.
This release of Distributed Cloud Edge contains the following known issues:
The NodePort Service is not supported. This release of Distributed Cloud Edge only supports the LoadBalancer and ClusterIP Kubernetes Services.
The Kubernetes control planes associated with Distributed Cloud Clusters can briefly go down during Distributed Cloud Cluster software updates.
A large number of webhook calls might cause the Konnectivity proxy to temporarily fail.
The metrics agents running on Distributed Cloud Edge nodes can accumulate a backlog of events and stall, preventing the capture of further metrics.
March 30, 2022
This is the General Availability release of Google Distributed Cloud Edge (version 1.0.0).
For information about the latest known issues, see Known issues in this release of Distributed Cloud Edge.