Release notes 1.13

Stay organized with collections Save and categorize content based on your preferences.

This document lists production updates to Anthos clusters on bare metal. We recommend that Anthos clusters on bare metal developers periodically check this list for any new announcements.

You can see the latest product updates for all of Google Cloud on the Google Cloud page, browse and filter all release notes in the Google Cloud console, or you can programmatically access release notes in BigQuery.

To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly: https://cloud.google.com/feeds/anthos-bare-metal-release-notes.xml

November 22, 2022

Release 1.13.2

Anthos clusters on bare metal 1.13.2 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.13.2 runs on Kubernetes 1.24.

Fixes:

  • Ensured the kubeadmconfig Secret is deleted when a Cluster API node is removed.
  • Added preflight check command (bmctl check preflight) that you can use when upgrading version 1.13 and higher clusters.
  • Updated the commands bmctl check preflight and bmctl create cluster so that they fail if worker or control-plane nodes have docker credentials in /root/.docker/config.json. (Anthos clusters on bare metal version 1.13 and higher can no longer use Docker Engine as a container runtime. All clusters must use the default container runtime containerd).
  • The following container image security vulnerabilities have been fixed:

Known issues:

For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

November 07, 2022

Security bulletin (1.11, 1.12, and 1.13)

A security vulnerability, CVE-2022-39278, has been discovered in Istio, which is used in Anthos Service Mesh, that allows a malicious attacker to crash the control plane.

For instructions and more details, see the Anthos clusters on bare metal security bulletin.

November 01, 2022

Cluster lifecycle improvements in 1.13 and later

Preview: You can use the Google Cloud console to create user clusters, delete user clusters, and to add and remove node pools from a user cluster. To explore the new feature, try out the tutorial Create an Anthos on bare metal user cluster on Compute Engine VMs using the console.

October 31, 2022

Release 1.13.1

Anthos clusters on bare metal 1.13.1 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.13.1 runs on Kubernetes 1.24.

Fixes:

Known issues:

For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

September 29, 2022

Release 1.13.0

Anthos clusters on bare metal 1.13.0 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.13.0 runs on Kubernetes 1.24.

The dockershim component in Kubernetes enables cluster nodes to use the Docker Engine container runtime. However, Kubernetes 1.24 removed the dockershim component. Since Anthos clusters on bare metal version 1.13 runs on Kubernetes 1.24, version 1.13 and higher clusters can no longer use Docker Engine as a container runtime. All clusters must use the default container runtime containerd.

Improved cluster lifecycle functionalities:

  • Upgraded from Kubernetes version 1.23 to 1.24:

    • Reverted some of the changes Kubernetes and the kubeadm tool made to certain labels and taints on control plane nodes. Changes were reverted so that older versions of Anthos clusters on bare metal remain supported. As a result, control plane nodes have the following labels and taints:

      • node-role.kubernetes.io/master label
      • node-role.kubernetes.io/control-plane label
      • node-role.kubernetes.io/master:NoSchedule taint
    • Upgraded from kubeadm.k8s.io/v1beta2 to kubeadm.k8s.io/v1beta3 since the former is deprecated.

    • Stopped automatic generation of Secret API objects containing service account tokens for every Service Account. For more information, see the LegacyServiceAccountTokenNoAutoGeneration section of the upgrade notes.

  • Breaking change: Version 1.12 clusters that use Docker Engine can upgrade to 1.13 only if the new container runtime is specified as containerd. Blocked the creation of new 1.13 clusters that use Docker Engine as the container runtime.

  • Preview: Added feature so that upgrades of an admin/hybrid/standalone cluster can proceed without a bootstrap cluster. Management of Anthos clusters on bare metal is now fully conformant to the Kubernetes Resource Model.

  • Added support of Red Hat Enterprise Linux (RHEL) 8.6.

  • Removed an erroneous CustomResourceDefinition (app.k8s.io.Application) from inclusion in the cluster creation process.

  • Fixed vulnerability to YAML injection by switching to safetext/yamltemplate.

  • GA: Added support for installing Anthos clusters on bare metal, using your own registry service, instead of gcr.io. For instructions and additional information, see Use a registry mirror to create clusters.

  • Eliminated false error messaging when the bmctl create cluster is run. The message erroneously reported an Invalid value in the spec.labels field of NodePool specifications.

  • Added feature so that resetting a user cluster doesn't require the cluster configuration file.

  • Reduced containerd disk usage by having containerd store just the uncompressed layers of an image rather than both the compressed and uncompressed layers.

  • Upgraded containerd to version 1.6.6.

Networking:

  • GA: Enabled Dynamic Flat IP with Border Gateway Protocol (BGP) support. This feature lets you configure flat mode using BGP in clusters by leveraging Network Gateway Group and BGP. In this mode the Pod's IP address is visible and routable without masquerading across multiple subdomains. Currently supports advertising IPv4 and IPv6 routes over IPv4 sessions.

  • GA: Added BGP-based Load Balancer support for IPv6. Added ability to disable the Bundled Ingress feature. Customers should disable this feature if they are using full Anthos Service Mesh (ASM) instead. (Bundled Ingress is unnecessary when full ASM is installed).

Observability:

  • Preview: Added support of multi-line parsing for Go and Java logs.

  • GA: Added support for Google Cloud Managed Service for Prometheus (GMP) for application metrics.

  • Refined kube-state-metrics so that only core metrics are collected by default.

Security:

  • GA: Added Google Groups support for Connect Gateway.

  • Switched distroless base image for Node Problem Detector.

  • Changed anet-operator/cilium-operator to run as non-root container.

  • Secured communication between metrics-server and api-server using the Transport Layer Security (TLS) protocol.

VM Runtime:

  • Fixed a memory leak in libvirt-go, which caused unbounded memory growth and risked crashing long-running VMs.

  • Provided guaranteed compute support so that customers can get Guaranteed Quality of Service (QoS)for the VM when needed.

  • Preview: Enabled Anthos VM to be allocated dedicated host cores. Each VM virtual core can be pinned to a dedicated host core.

  • Separated GPU installation and deletion logic. If only the container GPU workload is needed, customers can enable the GPU without having to enable VM Runtime.

  • Added support for the T4 GPU card.

  • Enabled automatic use of the VirtualMachineDisk name as the disk serial number. This change makes it easier for customers to identify the disk in the VM.

  • Enabled KubeVM cloud-init API and startup script API.

  • Added new CLI command (Virtctl) for resetting Windows VM password.

  • Fixed the following container image security vulnerability: CVE-2022-1798

  • Added feature that stops NVIDIA device plugins from crashing if a GPU card hasn't been allocated to a container.

  • Added support for automatic VM restarts after a configuration update. Previously, customers needed to stop the VM, apply the change, and then re-start the VM. To use the feature, set the autoRestartOnConfigurationChange flag to true in the VirtualMachine custom resource.

  • Improved the Kubernetes audit log of VM operations so that it contains detailed VM configuration and update information.

  • Fixed flooding of logs with cluster events that arise when a VM encounters disk I/O errors.

  • Added KubeVM roles. By binding with these roles, customers are granted permission to resources that manage VMs.

Known issues:

For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.