Google Cloud release notes

The following release notes cover the most recent changes over the last 60 days. For a comprehensive list of product-specific release notes, see the individual product release note pages.

You can also see and filter all release notes in the Google Cloud Console

To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly: https://cloud.google.com/feeds/gcp-release-notes.xml

June 11, 2021

Cloud Spanner

You can now find common queries for monitoring and troubleshooting on the Query page in the Cloud Console. This page now has query templates to help you to access these introspection system tables: Query Stats, Read Stats, Transaction Stats, Lock Stats, and Oldest active queries.

Google Kubernetes Engine

GKE Multi-cluster Services support for pod-specific addressing is now generally available.

Network Connectivity Center

If you use a Router appliance spoke to connect more than 1,000 VMs, you might be unable to establish BGP sessions between the router appliance instance and Cloud Router. The 1,000-VM limit includes any VMs that are accessible through VPC Network Peering.

Vertex AI

June 10, 2021

Compute Engine

NVIDIA® T4 GPUs are now available in the following additional regions and zones:

  • St. Ghislain, Belgium: europe-west1-b,c,d

For more information about using GPUs on Compute Engine, see GPUs on Compute Engine.

Google Kubernetes Engine

Volume snapshots is now generally available. Starting in GKE version 1.21 and later, you can now use v1 snapshots; v1beta1 snapshots will continue to operate as expected until further notice.

Committed use discounts are now generally available to purchase for Google Kubernetes Engine (Autopilot Mode).

Google Kubernetes Engine (Autopilot Mode) committed use discounts apply to all Autopilot Pod workload vCPU, memory, and ephemeral storage usage in the region in which you have committed. Google Kubernetes Engine (Autopilot Mode) committed use discounts do not apply to the cluster management fee or to GKE Standard mode compute nodes.

See the documentation for more details.

For GKE clusters running Windows Server node pools, you can see the version mapping between GKE versions and Windows Server versions for all available GKE versions by using a gcloud command. This feature is now available in preview.

For more details, see Use gcloud tool to get version mapping.

Identity and Access Management

The documentation for IAM role recommendations now has more detail about how insights are used to generate recommendations.

Memorystore for Redis

Added support for Upgrading the Redis version of an instance with the Google Cloud Console.

Released support for Redis version 6.x (Preview) on Memorystore for Redis. For more details, see Supported versions.

SAP on Google Cloud

SAP NetWeaver high-availability cluster documentation for SLES

A new load-balancer-based configuration guide for SAP NetWeaver high-availability clusters on SUSE Linux Enterprise Server (SLES) is available for use: HA cluster configuration guide for SAP NetWeaver on SLES.

June 09, 2021

Cloud Load Balancing

Network Load Balancing now supports load-balancing ESP (Encapsulating Security Payload) and ICMP (Internet Control Message Protocol) traffic. To handle these protocols, you specify the new L3_DEFAULT protocol on the load balancer's forwarding rule.

For details, see:

This feature is available in Preview.

Dataflow

Dataflow SQL now supports user-defined functions (UDFs) written using Java. For more information, see Dataflow SQL user-defined functions. This feature is in Preview.

Document AI

VPC Service Controls

Integration with Document AI VPC Service Controls is now generally available.

Google Kubernetes Engine

(2021-R19) Version updates

GKE cluster versions have been updated.

New versions available for upgrades and new clusters

The following Kubernetes versions are now available for new clusters and for opt-in control plane upgrades and node upgrades for existing clusters. For more information on versioning and upgrades, see GKE versioning and support and Upgrades.

No channel

Stable channel

  • Version 1.18.17-gke.1900 is now the default version in the Stable channel.
  • Version 1.18.17-gke.1901 is now available in the Stable channel.
  • Version 1.19.10-gke.1000 is now available in the Stable channel.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.18 to version 1.18.17-gke.1900 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.19 to version 1.19.10-gke.1000 with this release.

Regular channel

  • Version 1.19.10-gke.1600 is now available in the Regular channel.
  • Version 1.20.6-gke.1000 is now available in the Regular channel.
  • Version 1.19.9-gke.1400 is no longer available in the Regular channel.
  • Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.19 to 1.19.9-gke.1900 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.20 to 1.20.6-gke.1000 with this release.

Rapid channel

  • Version 1.20.6-gke.1400 is now the default version in the Rapid channel.
  • Version 1.21.1-gke.400 is now available in the Rapid channel.
  • Version 1.20.6-gke.1000 is no longer available in the Rapid channel.
  • Version 1.21.1-gke.100 is no longer available in the Rapid channel.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.20 to 1.20.6-gke.1400 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to 1.21.1-gke.400 with this release.

If you manually upgrade your cluster from 1.18 to 1.19 and the network tier configuration on an existing external network load balancer does not match the network tier annotation in the service spec (if unspecified, defaults to Premium), the load balancer will be deleted and recreated, and the network tier configuration will be enforced.

A domain-scoped project is not supported in GKE version 1.20. The cluster's CertificateSigningRequest will be denied when validating the DNS name and the nodes cannot join the cluster.

1.20 is now generally available

Kubernetes 1.20 is now generally available (GA). Before upgrading, read the Kubernetes 1.20 Release Notes especially the Urgent upgrade notes and Deprecations sections.

The node.k8s.io/v1beta1 RuntimeClass API has graduated to node.k8s.io/v1 with no changes. API clients and manifests should switch to using the node.k8s.io/v1 API after version 1.20. The node.k8s.io/v1beta1 API is deprecated and will no longer be served starting in version 1.25.

As of version 1.20, the kubelet no longer creates the target_path for NodePublishVolume in accordance with the CSI spec. If you have self-managed CSI drivers deployed in your cluster, ensure that they are idempotent and do any necessary mount creation or verification. For more information, see Kubernetes issue #88759.

Starting in version 1.20, timeouts on exec probes are honored, and default to 1 second if unspecified. If you have Pods using exec probes, ensure that they can easily complete in 1 second or explicitly set an appropriate timeout. For more information, see ConfigureProbes.

Non-deterministic treatment of objects with invalid ownerReferences was fixed in version 1.20. Run the kubectl-check-ownerreferences tool prior to upgrade to locate existing objects with invalid ownerReferences.

  • A namespaced object with an ownerReference to another namespaced object which does not exist in the same namespace is now consistently treated as having a missing owner and is deleted.

  • A cluster-scoped object with an ownerReference to a namespaced object is now consistently treated as having an unresolvable owner, and is ignored by the garbage collector.

  • Starting in version 1.20, when a namespace mismatch between a child and owner object is detected, an event with a reason code of OwnerRefInvalidNamespace is recorded.

The metadata.selfLink field, deprecated since version 1.16, is no longer populated in version 1.20. See Kubernetes issue #1164 for details. A related bug in the k8s.io/client-golibrary in the GetReference function was fixed in versions 0.15.9 or later, 0.16.4 or later, and 0.17.0 or later. Clients using the GetReference function should upgrade to one of those versions of client-go or newer in order to work correctly against an API Server running version 1.20 or later.

Reminder: Future beta API removals in versions 1.22 and 1.25

Kubernetes versions 1.22 and 1.25 will stop serving several deprecated beta APIs. It is recommended to begin migrating your clients and manifests to the stable replacement APIs now. More information is available in the OSS Kubernetes documentation.

VPC Service Controls

Integration with Document AI VPC Service Controls is now generally available.

Virtual Private Cloud

If you enable PROXY protocol for a Private Service Connect service attachment, the PROXY protocol header value was previously either 0xEA or 0xE0. Starting today, the value will always be 0xE0.

June 08, 2021

AI Platform Prediction

Runtime version 2.5 is now available. You can use runtime version 2.5 to serve online predictions with TensorFlow 2.5.1, scikit-learn 0.24.1, or XGBoost 1.4.0. Runtime version 2.5 does not support batch prediction.

See the full list of updated dependencies in runtime version 2.5.

Anthos clusters on VMware

Anthos clusters on VMware 1.5.4-gke.2 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.5.4-gke.2 runs on Kubernetes v.1.17.9-gke.4400. The supported versions that offer the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.7, 1.6, and 1.5.

Fixes

These security vulnerabilities have been fixed:

Fixed CVE-2021-25735 mentioned in the GCP-2021-003 Security Bulletin, CVE-2021-31535, and other medium and low vulnerability CVEs with fixes available.

Cloud Billing

Committed use discounts for Google Kubernetes Engine (GKE) are now Generally Available to purchase for workloads running on GKE Autopilot.

They provide discounted prices in exchange for your commitment to use a minimum level of resources for a specified term. The spend-based committed use discounts apply to all GKE Autopilot Pod workload CPU, memory, and ephemeral storage usage in the region in which you have committed. This gives you low, predictable costs, without the need to make any manual changes or updates yourself. This flexibility saves you time and helps you to save more by achieving high utilization rates across your commitments.

GKE Autopilot Mode commitments do not apply to the cluster management fee or to GKE Standard mode compute nodes.

See the documentation for more details.

Cloud VPN

You can check for VPN tunnel overutilization using the VPN tunnel utilization recommender. A recommender is a service in Google Cloud that provides usage recommendations for cloud resources.

Compute Engine

Generally available: You can configure how your regional managed instance group distributes instances across zones by using capacity-aware distribution shapes, which can automatically deploy instances to zones where capacity is available and optionally prioritize the use of reservations.

Preview: When rolling out configuration or application updates to a stateful or stateless managed instance group, use the minimum and most disruptive allowed actions to control disruption to your workload.

Dataproc

Custom image limitation: Currently, the following Dataproc image versions are the latest images that can be used as the base for custom images:

  • 1.3.89-debian10, 1.3.89-ubuntu18
  • 1.4.60-debian10, 1.4.60-ubuntu18
  • 1.5.35-debian10, 1.5.35-ubuntu18, 1.5.35-centos8
  • 2.0.9-debian10, 2.0.9-ubuntu18, 2.0.11-centos8
Migrate for Compute Engine

Transition the underlying OS used by Migrate for Compute Engine components (Manager, Cloud Extensions, Importers, and Exporters) to use Ubuntu Advantage.

Resource Manager

The Resource Settings API has entered general availability. You can use Resource Settings to centrally configure settings for your Google Cloud projects, folders, and organization. For more information, see Resource Settings overview.

June 07, 2021

Anthos clusters on VMware

Anthos clusters on VMware 1.6.3-gke.3 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.6.3-gke.3 runs on Kubernetes v1.18.18-gke.100. The supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.7, 1.6, and 1.5.

Fixes

These security vulnerabilities have been fixed:

Fixed CVE-2021-25735 mentioned in the GCP-2021-003 Security Bulletin, CVE-2021-31535, and other medium and low vulnerability CVEs with fixes available.

BigQuery

BigQuery now supports parameterized types. The following parameterized types are supported:

This feature is in Preview.

Cloud Asset Inventory

New resource types are now available.

The following resource types are now publicly available through the Export APIs (ExportAssets and BatchGetAssetsHistory) and the Feed API:

  • Cloud Redis
    • redis.googleapis.com/Instance
Cloud Composer

New versions of Cloud Composer images:

  • composer-1.17.0-preview.2-airflow-2.0.1
  • composer-1.16.6-airflow-1.10.15
  • composer-1.16.6-airflow-1.10.14 (default)
  • composer-1.16.6-airflow-1.10.12

You can now store values for the smtp_password Airflow configuration option in Secret Manager.

Increased the timeout for environment upgrade operations to support upgrades for databases up to 16 GB in size. If an upgrade operation times out and the Airflow database size is more than 10 GB, a warning message about the database size is generated.

Fixed memory issues that occurred while syncing files on machine types with more than 8 vCPUs.

DAG parsing and task processing in Airflow no longer fails because of incorrectly formatted Airflow logs. This happened due to a bug in Airflow log message formatting. Before this fix, errors related to sensor tasks with reschedule intervals shorter than scheduler processing time were not displayed.

(New environments only) Some log messages related to Airflow web server access were previously missing in Cloud Logging. This problem is fixed and these messages now appear in Cloud Logging.

(Available without upgrading) Updating environment labels now correctly overrides previous labels in billing reports.

Cloud Composer 1.10.4 has reached its end of full support period.

Cloud Functions

Cloud Functions now supports Ruby 2.6 and 2.7 at the General Availability release level.

Cloud SQL for MySQL

Cloud SQL now offers faster maintenance, with connectivity dropping for less than 60 seconds on average.

Cloud SQL for PostgreSQL

Cloud SQL now offers faster maintenance, with connectivity dropping for less than 60 seconds on average.

Cloud SQL for SQL Server

Cloud SQL now offers faster maintenance, with connectivity dropping for less than 120 seconds on average.

Cloud TPU

Cloud TPU now supports Tensorflow 2.5.0. For more information, see Tensorflow 2.5.0 Release Notes

Dataflow

Dataflow is now able to use workers, Dataflow Shuffle, Streaming Engine, FlexRS, and regional endpoints in zones in Melbourne (australia-southeast2).

Google Kubernetes Engine

You can now specify the default image type to use for new auto-provisioning node pools. See Using node auto-provisioning for more details.

Security Command Center

Security Command Center Legacy, previously known as Cloud Security Command Center, and Event Threat Detection Legacy have been permanently disabled.

To continue benefiting from Security Command Center, you must migrate your organizations to Security Command Center's free Standard tier or Premium tier. Event Threat Detection, a built-in service of Security Command Center, is available only in the Premium tier.

For information on upgrading to Security Command Center Standard or Premium, see Migrate from legacy Security Command Center products. To inquire about flexible pricing options for the Premium tier, complete our Premium inquiry form. You should receive a response within two US business days.

Workflows

String processing functions are now available in the text module of the Workflows standard library.

June 04, 2021

Artifact Registry

Maven, npm, and Python repositories are now in Preview.

Storage and network egress charges apply to all formats that are in Preview or are generally available.

Cloud Asset Inventory

Cloud Asset Inventory Console Preview is now publicly available. It enables you to see insights about Google Cloud footprint, details and history of resources, and provides powerful and easy filtering and search capabilities.

Cloud SQL for PostgreSQL

Both the Cloud SQL Java Connector and Cloud SQL Python Connector now support IAM Authentication for PostgreSQL.

Cloud Spanner

We are replacing the Insert a row and Edit a row data forms in the Cloud Console with pre-populated DML query templates on the Query page. These templates provide you more flexibility when adding and editing data. Learn More

Dialogflow

Dialogflow CX will have new pricing on September 1, 2021. For details, see the pricing documentation. In summary, the new pricing will be:

  • Text: $0.007/request
  • Audio: $0.06/minute
Google Kubernetes Engine

The security community recently disclosed a new security vulnerability CVE-2021-30465 found in runc that has the potential to allow full access to a node filesystem.

For more information, see the GCP-2021-011 security bulletin.

Virtual Private Cloud

The Private Service Connect Published Services tab in the Google Cloud Console now correctly displays service attachments. You can now view and manage service attachments using the Console, the gcloud command-line tool, or the API

When a Private Service Connect consumer endpoint is deleted, the service attachment details now correctly reflects this change.

June 03, 2021

Anthos GKE on AWS

Anthos clusters on AWS 1.7.2-gke.0 is now available.

Anthos clusters on AWS 1.7.2-gke.0 clusters run the following Kubernetes versions:

  • 1.16.15-gke.18500
  • 1.17.17-gke.8200
  • 1.18.18-gke.1500
  • 1.19.10-gke.1500

The Anthos clusters on AWS 1.7.2-gke.0 release addresses the following vulnerabilities:

Cloud Asset Inventory

New resource types are now available.

The following resource types are now publicly available through the Export APIs (ExportAssets and BatchGetAssetsHistory) and the Feed API:

  • Cloud Bigtable
    • bigtableadmin.googleapis.com/AppProfile
Cloud Run

Request timeouts up to 60 minutes are now at general availability (GA).

Compute Engine

N2D machine types are now available in us-west4-a , Las Vegas, Nevada. See VM instance pricing for details.

June 02, 2021

Anthos Anthos clusters on bare metal

Release 1.7.2

Anthos clusters on bare metal release 1.7.2 is now available. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.7.2 runs on Kubernetes 1.19.

Fixes:

  • Fixed CVE-2021-25735 that could allow node updates to bypass a Validating Admission Webhook. For more details, open the Anthos clusters on bare metal tab of the GCP-2021-003 security bulletin.
  • Resolved the bmctl snapshot command failure when the user creates a custom cluster namespace omitting "cluster-" prefix from the cluster config file. The prefix is no longer required for a custom cluster namespace.
  • Added webhook blocks to prevent users from modifying control plane node pool and load balancer node pool resources directly. Control plane and load balancer node pools for Anthos clusters on bare metal are specified in the cluster resource, using the spec.controlPlane.nodePoolSpec and spec.LoadBalancer.nodePoolSpec sections of the cluster config file respectively.
  • Fixed the cluster upgrade command, bmctl upgrade cluster, to prevent it from interfering with user-installed Anthos Service Mesh (ASM).

Functionality changes:

  • Updated the bmctl check snapshot command so that it includes certificate signing requests in the snapshot.
  • Changed the upgrade process to prevent node drain issues from blocking upgrades. The upgrade process triggers a node drain. Now, if the node drain takes longer than 20 minutes, the upgrade process carries on to completion even when the draining hasn't completed. In this case, the upgrade output reports the incomplete node drain. Excessive drain times signal a problematic with pods. You may need to restart problem pods.
  • Updated cluster creation process, bmctl create cluster, to display logged errors directly on the command line. Prior to this release, detailed error messages were only available in the log files.

Known issues:

  • Node logs from nodes with a dot (".") in their name are not exported to Cloud Logging. For workaround instructions, see Node logs aren't exported to Cloud Logging in Anthos clusters on bare metal known issues.

For information about the latest known issues, see Anthos clusters on bare metal known issues in the Troubleshooting section.

Cloud Data Loss Prevention

MEDICAL_TERM infoType detector is now available in all regions.

Config Connector

Config Connector 1.51.2 is now available.

Miscellaneous bug fixes.

Deep Learning Containers

M71 release

Deep Learning VM Images

M71 Release

  • Refreshed the Debian-10 images (Ubuntu images not refreshed in this release).
  • Upgraded TensorFlow Probability, TensorFlow I/O, and TensorFlow Estimator in TensorFlow 2.5 images.
  • Added support for a Post Startup script and provided status in guest attributes.
  • TensorFlow 2.x image names are now available in two formats: tf-xxx-2-y-zzz (the new standard format) tf2-xxx-2-y-zzz (the previous standard format). Image names in the previous standard format will be deprecated in a future release.
Traffic Director Transfer Appliance

Transfer Appliance offers the Transfer Appliance Cloud Setup Application. The application prompts for several settings, and uses the information you provide to configure your Google Cloud permissions, preferred Cloud Storage bucket, and Cloud KMS key for your transfer.

Virtual Private Cloud

Private Service Connect service attachment details always show a status of Accepted for consumer endpoints, even if they have a different status. The status is correctly displayed in the consumer endpoint details.

When a Private Service Connect consumer endpoint is deleted, the service attachment details do not reflect this change.

Updating a Private Service Connect service attachment using the PATCH API method requires that you provide all values in the request body, not just the values that you are updating. This affects Managing access requests for a service and Changing the connection preference for a service.

If you enable PROXY protocol for a Private Service Connect service attachment, the PROXY protocol header value might be 0xEA or 0xE0. After General Availability, the value will always be 0xE0.

If you publish a service using Private Service Connect, and the referenced load balancer does not have any backend VMs, all Private Service Connect endpoints in the consumer network might become unresponsive. Make sure that that all load balancers that are referenced by a service attachment have backend VMs.

If you want to create a Private Service Connect endpoint in a Shared VPC network, the endpoint must be created in the same project that contains the virtual machines (VMs) that send requests to the endpoint.

The Private Service Connect Published Services tab in the Google Cloud Console does not display service attachments. Use the gcloud command-line tool or the API to view and manage service attachments.

June 01, 2021

Chronicle

Chronicle Automated GCP Log Ingestion

Google Cloud customers can now send logs directly to their Chronicle account. Customers can send both Cloud Audit and Cloud DNS logs. See Ingesting GCP Logs in to Chronicle for more information.

Cloud Monitoring

A JSON editor has been integrated with the dashboard page. In addition to using the JSON editor to change the contents of the dashboard, you can save the current dashboard definition to a local system, and you can upload a dashboard definition to your Google Cloud project. For more information, see Managing dashboards through the Cloud Console.

Cloud SQL for MySQL

CloudSQL for MySQL now supports the MySQL flags expire_logs_days (for MySQL 5.6 and 5.7) and binlog_expire_logs_seconds (for MySQL 8.0). Note that if you enable point-in-time recovery, the expiration period of your binary logs will be determined by the lesser of your transaction log retention period and the value of these flags.

Cloud SQL for PostgreSQL

The logical replication and decoding functionality of PostgreSQL is available as a preview. These features enable logical replication workflows and change data capture workflows.

For more information, see Setting up logical replication and decoding.

Cloud SQL for PostgreSQL now supports the pg_similarity extension, which provides support for similarity queries in PostgreSQL.

Also, the default value for the database flag autovacuum_vacuum_cost_delay is changed to 2 milliseconds in PostgreSQL 9.6, 10 and 11.

The minor versions for various extensions have also been upgraded:

9.6 10 11 12 13
address_standardizer not avail 2.4.9 2.5.5 3.0.2 3.0.2
hll 2.14 2.14 2.14 2.14 2.14
pg_repack 1.4.6 1.4.6 1.4.6 1.4.6 1.4.6
pgaudit 1.1.3 1.2.3 1.3.2 1.4.1 no change
pglogical 2.3.3 2.3.3 2.3.3 2.3.3 2.3.3
pl/proxy 2.10.0 2.10.0 2.10.0 2.10.0 2.10.0
postgis 2.3.11 2.4.9 2.5.5 3.0.2 3.0.2
Cloud TPU

New Cloud TPU VMs make training your ML models on TPUs easier than ever

The new Cloud TPU VM architecture makes it easier than ever before to use our industry-leading TPU hardware. The Cloud TPU VMs provide direct access to TPU host machines, offering a new and improved user experience for developing and deploying TensorFlow, PyTorch, and JAX on Cloud TPUs. Instead of accessing Cloud TPUs remotely over the network, Cloud TPU VMs let you set up your own interactive development environment on each TPU host machine. Now you can write and debug an ML model line-by-line using a single TPU VM, and then scale it up on a Cloud TPU Pod slice to take advantage of the super-fast TPU interconnects. You have root access to every TPU VM you create, so you can install and run any code you wish in a tight loop with your TPU accelerators. You can use local storage, execute custom code in your input pipelines, and more easily integrate Cloud TPUs into your research and production workflows. Google supports Cloud TPU integrations with TensorFlow, PyTorch, and JAX, and you can even write your own integrations via a new libtpu shared library on the VM. For more information, see https://cloud.google.com/blog/products/compute/introducing-cloud-tpu-vms

Compute Engine

Preview: Access the Compute Engine API using Cloud Client Libraries built on our latest client library model. Updated client libraries are now available in the following languages:

  • Java
  • .NET
  • Node.js
  • PHP
  • Python
  • Ruby

For more information, see Compute Engine client libraries.

Dataproc

New sub-minor versions of Dataproc images: 1.3.91-debian10, 1.3.91-ubuntu18, 1.4.62-debian10, 1.4.62-ubuntu18, 1.5.37-centos8, 1.5.37-debian10, 1.5.37-ubuntu18, 2.0.11-centos8, 2.0.11-debian10, and 2.0.11-ubuntu18.

Image 1.3 - 2.0

  • All jobs now share a single JobthreadPool.

  • The number of Job threads in the Agent is configurable with the dataproc:agent.process.threads.job.min and dataproc:agent.process.threads.job.max cluster properties, defaulting to 10 and 100, respectively. The previous behavior was to always use 10 Job threads.

Image 2.0

  • Added snappy-jar dependency to Hadoop.
  • Upgraded versions of Python packages: nbdime 2.1 -> 3.0, pyarrow 2.0 -> 3.0, spyder 4.2 -> 5.0, spyder-kernels 1.10 -> 2.0, regex 2020.11 -> 2021.4.

Image 1.5 and 2.0

Image 1.3 - 2.0

  • SPARK-35227: Replace Bintray with the new repository service for the spark-packages resolver in SparkSubmit.

Image 2.0

  • Fixed the problem that the environment variable PATH was not set in YARN containers.

  • SPARK-34731: ConcurrentModificationException in EventLoggingListener when redacting properties.

May 28, 2021

Google Kubernetes Engine

1.21 available in the Rapid channel

Kubernetes version 1.21 is now available in the Rapid channel. Before upgrading, read the Kubernetes 1.21 Release Notes, especially the action required and deprecation sections.

1.21 Features

The following features are introduced in version 1.21:

CronJob (GA)

The CronJob API has graduated to General Availability (GA), bringing performance improvements and allowing scheduled jobs to be run using a stable API.

  • This resource is now available in the batch/v1 group/version.
  • The batch/v1beta1 group/version is deprecated, and will be removed in version 1.25. See the migration guide for details.

PodDisruptionBudget (GA)

The PodDisruptionBudget has graduated to GA, allowing pod evictions to be controlled using a stable API.

  • This resource is now available in the policy/v1 group/version.
  • The policy/v1beta1 group/version is deprecated, and will be removed in version 1.25. See the migration guide for details.

EndpointSlice (GA)

The EndpointSlice API has graduated to GA, bringing performance improvements over the v1 Endpoints API.

  • This more scalable API for service discovery is now enabled on all clusters and is promoted to discovery.k8s.io/v1.
  • The discovery.k8s.io/v1beta1 group/version is deprecated, and will be removed in version 1.25. See the migration guide for details.

Default namespace label (Beta)

Namespace API objects now have a kubernetes.io/metadata.name label matching their metadata.name field to allow selecting any namespace by its name using a label selector. This can be used for objects which select namespaces by label, such as admission webhooks and network policies.

Bound service account token volumes (Beta)

  • The API credentials injected into containers at /var/run/secrets/kubernetes.io/serviceaccount/token are now time-limited, auto-refreshed, and invalidated when the containing pod is deleted.
  • By default, injected tokens are given an extended lifetime so they remain valid even after a new refreshed token is provided. The metric serviceaccount_stale_tokens_total and the audit annotation authentication.k8s.io/stale-token can be used to monitor for workloads that depend on the extended lifetime and are continuing to use tokens even after a refreshed token is provided to the container.
  • Clients should reload the token from disk periodically (once per minute is recommended) to ensure they use the refreshed token. k8s.io/client-go version 11.0.0+ and 0.15.0+ reload tokens automatically.

In Kubernetes 1.21, newly provisioned PersistentVolumes by gce-pd will use the topology.kubernetes.io/zone GA label instead of the failure-domain.beta.kubernetes.io/zone beta label.

1.21 New Beta and Stable APIs

The following Stable APIs are new in 1.21:

  • batch/v1 CronJob
  • policy/v1 PodDisruptionBudget
  • discovery.k8s.io/v1 EndpointSlice

The following Beta APIs are new in 1.21:

  • storage.k8s.io/v1beta1 CSIStorageCapacity

1.21 Deprecated APIs

The following APIs are deprecated in the 1.21 release:

  • PodSecurityPolicy
    • policy/v1beta1 PodSecurityPolicy
    • Deprecated in 1.21 with removal targeted for version 1.25.
  • The following Beta versions of newly graduated APIs will be removed in 1.25 in favor of GA versions:
    • discovery.k8s.io/v1beta1 EndpointSlice
    • policy/v1beta1 PodDisruptionBudget
    • batch/v1beta1 CronJob
  • The following Beta versions of previously graduated APIs will be removed in 1.22 in favor of GA versions:
    • admissionregistration.k8s.io/v1beta1, MutatingWebhookConfiguration
    • admissionregistration.k8s.io/v1beta1, ValidatingWebhookConfiguration
    • apiextensions.k8s.io/v1beta1, CustomResourceDefinition
    • apiregistration.k8s.io/v1beta1, APIService
    • authentication.k8s.io/v1beta1, TokenReview
    • authorization.k8s.io/v1beta1, LocalSubjectAccessReview
    • authorization.k8s.io/v1beta1, SelfSubjectAccessReview
    • authorization.k8s.io/v1beta1, SubjectAccessReview
    • certificates.k8s.io/v1beta1, CertificateSigningRequest
    • coordination.k8s.io/v1beta1, Lease
    • extensions/v1beta1, Ingress
    • networking.k8s.io/v1beta1, Ingress
    • networking.k8s.io/v1beta1, IngressClass
    • rbac.authorization.k8s.io/v1beta1, ClusterRole
    • rbac.authorization.k8s.io/v1beta1, ClusterRoleBinding
    • rbac.authorization.k8s.io/v1beta1, Role
    • rbac.authorization.k8s.io/v1beta1, RoleBinding
    • scheduling.k8s.io/v1beta1, PriorityClass
    • storage.k8s.io/v1beta1, CSIDriver
    • storage.k8s.io/v1beta1, CSINode
    • storage.k8s.io/v1beta1, StorageClass
    • storage.k8s.io/v1beta1, VolumeAttachment

(2021-R18) Version updates

GKE cluster versions have been updated.

New versions available for upgrades and new clusters

The following Kubernetes versions are now available for new clusters and for opt-in control plane upgrades and node upgrades for existing clusters. For more information on versioning and upgrades, see GKE versioning and support and Upgrades.

No channel

  • Version 1.19.9-gke.1900 is now the default version.
  • Version 1.18.18-gke.1700 is now available.
  • Version 1.19.10-gke.1700 is now available.
  • Version 1.18.17-gke.100 is no longer available.
  • Version 1.19.8-gke.1600 is no longer available.

Stable channel

  • Version 1.18.17-gke.1200 is now the default version in the Stable channel.
  • Version 1.18.17-gke.1900 is now available in the Stable channel.
  • Version 1.17.17-gke.4900 is no longer available in the Stable channel.
  • Version 1.17.17-gke.5400 is no longer available in the Stable channel.
  • Version 1.18.17-gke.700 is no longer available in the Stable channel.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.18 to version 1.18.17-gke.1200 with this release.

Regular channel

  • Version 1.19.9-gke.1900 is now the default version in the Regular channel.

Rapid channel

  • Version 1.20.6-gke.1400 is now available in the Rapid channel.
  • Version 1.21.1-gke.100 is now available in the Rapid channel.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to 1.21.1-gke.100 with this release.

GKE clusters running version 1.18 or later now support container native Cloud DNS (available in Preview). Cloud DNS can be used as the in-cluster DNS provider instead of kube-dns.

May 27, 2021

Anthos clusters on VMware

Anthos clusters on VMware 1.7.2-gke.2 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.7.2-gke.2 runs on Kubernetes 1.19.10-gke.1602.

The supported versions that offer the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.7, 1.6, and 1.5.

The Ubuntu node image shipped in version 1.7.2 is qualified with the CIS (Center for Internet Security) L1 Server Benchmark.

Fixes:

An admin cluster upgrade may fail due to an expired front-proxy-client certificate on the admin control plane node. Make sure that the certificate is not expired, and recreate it if needed. See: Renew an expired certificate.

Cloud Data Fusion

Cloud Data Fusion version 6.4.1 is now available. To upgrade, see Upgrading instances and pipelines. This release is in parallel with the CDAP 6.4.1 release.

In Cloud Data Fusion version 6.4.1, Replication supports the Datetime data type in BigQuery targets. You can now read and write to tables that contain Datetime fields.

Fixed in 6.4.1 (for more information, see the CDAP release note):

  • Fixed an issue that caused pipelines with aggregations and Decimal fields to fail with an exception.

  • Fixed the Join Condition Type so that it is displayed in the Joiner plugin for pipelines that were upgraded from versions before 6.4.0.

  • Fixed Wrangler so that pipelines fail when there is an error. In Wrangler 6.2 and above, there was a backwards-incompatible change where pipelines did not fail if there was an error and were instead marked as complete.

  • Fixed an issue that prevented new previews from being scheduled after the preview manager had been stopped ten times.

  • Fixed an issue while writing non-null values to a nullable field in BigQuery.

  • Fixed an issue in the BigQuery plugins to correctly delete temporary storage buckets.

  • Fixed an issue in the BigQuery sink that caused pipelines to fail when the input schema was not provided.

  • Fixed an issue in the BigQuery sink that caused pipelines to fail or give incorrect results.

  • Fixed an issue that caused pipelines to fail when a Pub/Sub source Subscription field was a macro.

Cloud Spanner

We've enhanced the experience for creating, updating, and deleting schemas in the Cloud Console. On a database's Overview page you'll now find a Write DDL link to the DDL editor where you can perform all these activities.

Config Connector

Config Connector 1.51.1 is now available

Miscellaneous bug fixes.

Kf

Prevent panic in reconcilers when a Space is not found

Memorystore for Memcached

Added support for the Reserved Memory configuration for Memorystore for Memcached. For more information, see Memory management best practices.

May 26, 2021

Anthos Config Management

Hierarchy Controller has been updated to use HNC v0.8.0.

Increased reconciler memory limit to 300Mi.

The output of the nomos hydrate command does not pass nomos vet and cannot be synced using Config Sync without modifying the output. To work around this, we recommend removing the following annotations: configmanagement.gke.io/cluster-name , configmanagement.gke.io/source-path and removing label configsync.gke.io/declared-version from the output so that the output can be successfully synced.

The nomos hydrate command attempts to connect to the API Server even if --no-api-server-check is passed. This behavior can be safely ignored in CI as if the CLI is unable to connect to the API Server it will not produce errors resulting from being unable to connect.

Cloud Bigtable Cloud Load Balancing

Starting May 15, 2021, a newly-created custom static route using a next hop forwarding rule of an internal TCP/UDP load balancer will forward all protocol traffic, not just TCP and UDP traffic.

If a route created before May 15, 2021 is still in operation on August 14, 2021, it will automatically be migrated to forward all protocol traffic starting August 15, 2021. If you don't want to wait until then, you can enable forwarding of traffic for all protocols by creating new routes and deleting the old ones.

For more information, see Processing of TCP, UDP, and other protocol traffic.

Compute Engine

Preview: Disable simultaneous multithreading (SMT) on VMs. For more information, see Disabling simultaneous multithreading.

Datastream

Datastream is a serverless and easy-to-use change data capture (CDC) and replication service. It allows you to synchronize data across heterogeneous databases and applications reliably, and with minimal latency and downtime.

Datastream supports streaming from Oracle and MySQL databases into Cloud Storage. The service offers streamlined integration with Dataflow templates to power up-to-date materialized views in BigQuery for analytics, replicate your databases into Cloud SQL or Spanner for database synchronization, or leverage the event stream directly from Cloud Storage to realize event-driven architectures.

Benefits of Datastream include:

  • Being serverless so there are no resources to provision or manage, and the service scales up and down automatically, as needed, with minimal downtime.
  • Easy-to-use setup and monitoring experiences that achieve super-fast time-to-value.
  • Integration across the best of Google Cloud data services' portfolio for data integration across Datastream, Dataflow, Data Fusion, Pub/Sub, BigQuery, and more.
  • Synchronizing and unifying data streams across heterogeneous databases and applications.
  • Security, with private connectivity options and the security you expect from Google Cloud.
  • Being accurate and reliable, with transparent status reporting and robust processing flexibility in the face of data and schema changes.
  • Supporting multiple use cases, including analytics, database replication, and synchronization for migrations and hybrid-cloud configurations, and for building event-driven architectures.

Documentation for Datastream includes a quickstart, conceptual content, how to use this service through the user interface, REST API calls, and gcloud, an API tutorial, and reference, support, and resource-related information. Click here to access the documentation.

Network Connectivity Center

The Cloud documentation now includes a list of partners whose solutions are integrated with Network Connectivity Center.

Resource Manager

The process for migrating a project from one organization to another has released into general availability. To make it easier to see the impact a project migration will have on your organization, you can use the Cloud Asset Inventory Analyze Move API to get a detailed report before performing a move. For more information, see Migrating projects and Analyze project move.

SAP on Google Cloud

GA: Google Cloud monitoring agent for SAP HANA, version 2

Version 2.0 of the monitoring agent for SAP HANA is now generally available. V2.0 represents a complete refactoring of the monitoring agent for SAP HANA. A new Cloud Monitoring dashboard template for SAP HANA data is now also available for use with V2.0.

For more information, see Monitoring agent for SAP HANA V2.0 planning guide.

May 25, 2021

BigQuery BI Engine

The free trial period for BigQuery BI Engine's SQL interface has been extended to July 15th, 2021. You must enroll to participate in the preview. With this feature, BI Engine now interacts with popular BI tools such as Looker, Tableau, and more, by means of an interactive SQL interface.

Compute Engine

Generally Available: Enable nested virtualization directly when creating a VM. For more information, see Nested virtualization overview.

Google Cloud VMware Engine

Added security bulletin for the VMware Engine response to VMware security advisory VMSA-2021-0010.

Network Connectivity Center

You can now use the Cloud Console to create hubs and spokes in Network Connectivity Center.

May 24, 2021

Cloud Asset Inventory

New resource types are now available.

The following resource types are now publicly available through the Export APIs (ExportAssets and BatchGetAssetsHistory) and the Feed API:

  • Artifact Registry
    • artifactregistry.googleapis.com/Repository
Config Connector

Config Connector 1.51.0 is now available

Added field spec.basic.conditions[].devicePolicy.osConstraints[].requireVerifiedChromeOs to AccessContextManagerAccessLevel

Added field spec.externalDataConfiguration.hivePartitioningOptions.requirePartitionFilter to BigQueryTable

Added field spec.initialGroupConfig to CloudIdentityGroup

Added field spec.initialSize to ComputeNodeGroup

Added field spec.maintenanceWindow to ComputeNodeGroup

Added field spec.replication.userManaged.replicas[].customerManagedEncryption to SecretManagerSecret

Added field spec.encryptionConfig to SpannerDatabase

Memorystore for Redis

Added support for specifying an IP address range for the private service access connection mode. For more information, see Custom ranges with private services access.

Secret Manager

The Secret Manager SLA has been updated.

Security Command Center

Security Command Center Premium has launched project- and folder-level roles in general availability. The feature lets you grant users Identity and Access Management (IAM) roles for specific folders and projects. You have more granular control over who can access what resources throughout your organization. For more information, see Access control.

You must be a Security Command Center Premium customer to use this feature. Security Command Center Standard continues to support granting roles only at the organization level. To subscribe to Security Command Center Premium, contact your sales representative or fill out our inquiry form.

Security Command Center now supports two versions of CIS Benchmarks for Google Cloud Platform Foundation:

  • CIS Google Cloud Computing Foundations Benchmark v1.1.0 (CIS Google Cloud Foundation 1.1)
  • CIS Google Cloud Computing Foundations Benchmark v1.0.0 (CIS Google Cloud Foundation 1.0)

For more information about supported compliance standards, see Detectors and compliance.

Security Health Analytics, a built-in service of Security Command Center, has expanded the number of detectors in the Standard tier. The Standard tier, which is free of charge, now includes the following detectors:

  • LEGACY_AUTHORIZATION_ENABLED: Legacy Authorization is enabled on Google Kubernetes Engine (GKE) clusters.
  • OPEN_CISCOSECURE_WEBSM_PORT: A firewall is configured to have an open CISCOSECURE_WEBSM port that allows generic access.
  • OPEN_DIRECTORY_SERVICES_PORT: A firewall is configured to have an open DIRECTORY_SERVICES port that allows generic access.
  • OPEN_TELNET_PORT: A firewall is configured to have an open TELNET port that allows generic access.
  • PUBLIC_COMPUTE_IMAGE: A Compute Engine image is publicly accessible.

For a complete list of detectors in the Standard tier, see Pricing. For detailed information about all Security Health Analytics detectors, see Vulnerabilities findings.

Speech-to-Text

Speech-to-Text now supports Spoken Punctuation and Spoken Emoji as Preview features. See the documentation for details.

May 21, 2021

Anthos clusters on VMware

In Anthos clusters on VMware 1.7, logs are sent to the parent project of your logging-monitoring service account. That is, logs are sent to the parent project of the service account specified in the stackdriver.serviceAccountKeyPath field of your cluster configuration file. The value of stackdriver.projectID is ignored. This issue will be fixed in an upcoming release.

As a workaround, view logs in the parent project of your logging-monitoring service account.

Cloud Composer

New versions of Cloud Composer images:

  • composer-1.17.0-preview.1-airflow-2.0.1
  • composer-1.16.5-airflow-1.10.15
  • composer-1.16.5-airflow-1.10.14 (default)
  • composer-1.16.5-airflow-1.10.12

Error messages about PyPI package conflicts now contain links to corresponding cluster build logs.

Cloud Composer 1.10.3 has reached its end of full support period.

Google Kubernetes Engine

Network Policy Logging is generally available (GA). Note that Network Policy Logging requires Dataplane V2.

May 20, 2021

Anthos Config Management

If Syncing from multiple repositories is enabled on a private GKE cluster, it's required to add a firewall rule to allow port 8676.

Anthos clusters on VMware

In version 1.7.1, the stackdriver-log-forwarder starts to consume significantly increasing memory after a period of time, and the logs show an excessive number of OAuth 2.0 token requests. Follow these steps to mitigate this issue.

App Engine standard environment Java
  • Updated Java SDK to version 1.9.89.
  • Upgraded to Jetty version 9.4.41.v20210516.
  • Stopped releasing Maven artifact appengine-api-labs-1.9.xx.jar. Last release is 1.9.88.
BigQuery

BigQuery GIS now supports loading geography data from newline-delimited GeoJSON files. This feature is generally available (GA). For more information, see Loading GeoJSON data.

BigQuery GIS now supports the following functions. These functions are generally available (GA).

These functions return a point of a linestring geography as a point geography.

Cloud Asset Inventory

Policy Analyzer now supports evaluations on time-based conditions. See the user guide for more information.

Asset Insights are now available. See the user guide for more information.

Cloud Build

Upgraded to Docker server version 20.10.6.

Cloud DNS Dataproc

You can customize the Conda environment during cluster creation using new Conda-related cluster properties. See Using Conda-related cluster properties.

Added validation for clusters created with Dataproc Metastore services to determine compatibility between the Dataproc image's Hive version and the DPMS service's hive version

Google Kubernetes Engine

In GKE version 1.20 and later, audit logging does not occur for Binary Authorization fail open events.

May 19, 2021

Anthos Service Mesh

Anthos Service Mesh 1.6 is no longer supported. For more information see Supported versions.

BigQuery

BigQuery now supports the ability to rename tables using SQL. See ALTER TABLE RENAME TO. This feature is generally available (GA).

Cloud Key Management Service

The Cloud KMS and Cloud HSM SLA has been updated.

Cloud SQL for MySQL

Cloud SQL supports the preview version of the out-of-disk recommender. This feature proactively generates recommendations that helps you reduce the risk of downtime that might be caused by your instances running out of disk space. These recommendations can be applied when a Cloud SQL instance is trending towards the storage limit.

For information about pricing, prerequisites, and instructions for how to view the out-of-disk recommender, see Cloud SQL out of disk recommender.

Cloud SQL for PostgreSQL

Cloud SQL supports the preview version of the out-of-disk recommender. This feature proactively generates recommendations that helps you reduce the risk of downtime that might be caused by your instances running out of disk space. These recommendations can be applied when a Cloud SQL instance is trending towards the storage limit.

For information about pricing, prerequisites, and instructions for how to view the out-of-disk recommender, see Cloud SQL out of disk recommender.

Cloud SQL for SQL Server

Cloud SQL supports the preview version of the out-of-disk recommender. This feature proactively generates recommendations that helps you reduce the risk of downtime that might be caused by your instances running out of disk space. These recommendations can be applied when a Cloud SQL instance is trending towards the storage limit.

For information about pricing, prerequisites, and instructions for how to view the out-of-disk recommender, see Cloud SQL out of disk recommender.

Compute Engine

Generally Available: You can now create VM instances with V100, A100, and T4 GPUs that support network bandwidths of up to 100 Gbps. See Using network bandwidths of up to 100 Gbps.

Google Kubernetes Engine

(2021-R17) Version updates

GKE cluster versions have been updated.

New versions available for upgrades and new clusters

The following Kubernetes versions are now available for new clusters and for opt-in control plane upgrades and node upgrades for existing clusters. For more information on the Kubernetes versioning scheme, see Versioning.

No channel

  • Version 1.17.17-gke.8200 is now available.
  • Version 1.18.18-gke.1100 is now available.
  • Version 1.19.10-gke.1600 is now available.
  • Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.17 to version 1.18.17-gke.700 with this release.
  • Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.18 to version 1.18.17-gke.700 with this release.

Stable channel

  • Version 1.18.17-gke.700 is now the default version in the Stable channel.
  • Version 1.18.17-gke.1200 is now available in the Stable channel.
  • Version 1.18.17-gke.100 is no longer available in the Stable channel.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.17 to version 1.18.17-gke.700 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.18 to version 1.18.17-gke.700 with this release.

Regular channel

  • Version 1.19.9-gke.1900 is now available in the Regular channel.
  • Version 1.18.17-gke.700 is no longer available in the Regular channel.

Rapid channel

  • Version 1.20.6-gke.1000 is now the default version in the Rapid channel.
  • Version 1.19.9-gke.1900 is no longer available in the Rapid channel.
  • Version 1.19.10-gke.1000 is no longer available in the Rapid channel.
  • The following control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded with this release:

For GKE clusters running 1.18.18-gke.1200 or later, Ingress Controller only syncs NEGs that were created by the controller. Custom named NEGs that were created outside of the controller will no longer be synced.

Migrate for Anthos

Removed from the legacy PV-based Migrate for Anthos versions a Webhook that was simplifying the definition of Migrate for Anthos pods. This Webhook was not being used in any subsequent versions, including the latest 1.6 and 1.7 releases.

162275866: When generating migration artifacts, you no longer see the following error:

Error: failed to update vgenerateartifactsflow.kb.io

Traffic Director

Traffic Director security service with GKE is now available in Public Preview. This provides the following:

  • Authentication and encryption using transport layer security (TLS) and mutual TLS (mTLS) for both Traffic Director with Envoy and proxyless gRPC applications. Server TLS policies and client TLS policies control whether services need to prove their identities to each other and use encrypted communication channels.

  • Authorization, based on characteristics of the client and the request. Authorization policies control whether a service is permitted to access another service, and which actions are allowed. Authorization is currently available only for Traffic Director with Envoy.

May 18, 2021

BigQuery ML

The CREATE MODEL statement for training AutoML Tables models is now generally available (GA). AutoML Tables enable you to automatically build state-of-the-art machine learning models on structured data at massively increased speed and scale. For more information, see CREATE MODEL statement for training AutoML Tables models.

Cloud Run for Anthos

Events for Cloud Run for Anthos version 0.21.0-gke.108 is now available for the following GKE minor versions:

  • 1.19
  • 1.20
  • 1.21
Vertex AI

AI Platform (Unified) is now Vertex AI.

Vertex AI has added support for custom model training, custom model batch prediction, custom model online prediction, and a limited number of other services in the following regions:

  • us-west1
  • us-east1
  • us-east4
  • northamerica-northeast1
  • europe-west2
  • europe-west1
  • asia-southeast1
  • asia-northeast1
  • australia-southeast1
  • asia-northeast3

Vertex AI now supports forecasting with time series data for AutoML tabular models, in Preview. You can use forecasting to predict a series of numeric values that extend into the future.

Vertex Pipelines is now available in Preview. Vertex Pipelines helps you to automate, monitor, and govern your ML systems by orchestrating your ML workflow.

Vertex Model Monitoring is now available in Preview. Vertex Model Monitoring enables you to monitor model quality over time.

Vertex Feature Store is now available in Preview. Vertex Feature Store provides a centralized repository for organizing, storing, and serving ML features.

Vertex ML Metadata is now available in Preview. Vertex ML Metadata lets you record the metadata and artifacts produced by your ML system so you can analyze the performance of your ML system.

Vertex Matching Engine is now available in Preview. Vertex Matching Engine enables vector similarity search.

Vertex TensorBoard is now available in Preview. Vertex TensorBoard enables you to track, visualize, and compare ML experiments.

May 17, 2021

Anthos Service Mesh

1.9.5-asm.2, 1.8.6-asm.3, and 1.7.8-asm.8 are now available.

This release fixes the following security vulnerabilities:

For details on upgrading Anthos Service Mesh, refer to the following upgrade guides:

Anthos Service Mesh uses a proxy that is based on OSS Envoy. The Envoy version that the Anthos Service Mesh proxy uses differs by Anthos Service Mesh version, as follows:

Anthos clusters on bare metal

Release 1.6.3

Anthos clusters on bare metal release 1.6.3 is now available. To upgrade, see Upgrading Anthos clusters on bare metal. Anthos clusters on bare metal 1.6.3 runs on Kubernetes 1.18.

Fixed:

Known issues:

When you upgrade Anthos clusters on bare metal from a version with a security patch to the next minor release, we recommend you upgrade to the highest patch version to ensure you have the latest security fixes. Always review the release notes before upgrading so you're aware of what has changed, including security fixes and known issues. Upgrading to a lower release version isn't supported.

For information about the latest known issues, see Anthos clusters on bare metal known issues in the Troubleshooting section.

Config Connector

Config Connector version 1.50.0 is now available.

Resource CRDs are now using apiextensions.k8s.io/v1. The minimum required Kubernetes version for using Config Connector v1.50.0 and above is Kubernetes 1.16. This change is in preparation for the removal of apiextensions.k8s.io/v1beta1 in Kubernetes 1.22.

Fixed the issue that Project creation failed if spec.resourceID was set. (Issue #462)

Fixed the issue that Storage resources couldn't be deleted if the referenced StorageBucket was deleted first. (Issue #463)

Fixed the IAM resource references in go-client. (Issue #413)

Google Cloud VMware Engine

VMware Engine nodes are now available in the following additional region:

  • Mumbai, India, APAC (asia-south1)
Google Kubernetes Engine

The UpgradeAvailableEvent notification is now generally available.

May 15, 2021

Chronicle

Archive Rules

You can now archive rules specified for the Detection Engine. Archiving a rule hides the security data related to that rule (and all of its versions) without actually deleting the rule. See Archive rules for more information.

May 14, 2021

Cloud Storage

XML API multipart uploadsPreview launched.

Dataflow

You can now enable logging of human-readable hot keys. For more information, see the hot key entry in Pipeline options.

Deep Learning Containers

M70 Release

  • Added TensorFlow Enterprise 2.5 containers. Note this is an Enterprise version but not a Long Term Support (LTS) version.
Deep Learning VM Images

M70 Release

  • Added TensorFlow Enterprise 2.5 images. Note this is an Enterprise version but not a Long Term Support (LTS) version.
Dialogflow

Preview launch of Twilio telephony integration.

Identity and Access Management

You can now use the Google Cloud Console to manage workload identity federation. For details, see the documentation for your identity provider:

Secret Manager

Secret Manager now supports etags for optimistic concurrency control. This feature is available in Preview.

See Etags to learn more.

May 13, 2021

Anthos Anthos Config Management

Anthos Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 9b5e4cf).

A bug in Anthos Config Management 1.7.0 which broke nomos hydrate --no-api-server-check has been fixed.

The Config Sync admission webhook in Anthos Config Management 1.7.0 would block requests when a managed resource in the cluster copied annotations to another resource.

Config Sync container images are now correctly updated when Anthos Config Management is upgraded.

A bug in Anthos Config Management 1.7.0 which caused nomos status to return errors when both unstructured repos and Hierarchy Controller were being used has been fixed.

Cloud Billing

Committed use discounts are now available for public preview to purchase for Cloud Run. They provide discounted prices in exchange for your commitment to use a minimum level of resources for a specified term. The spend-based committed use discounts apply to all aggregated Cloud Run CPU, memory, and request usage in a region, giving you low, predictable costs when your code is running in one of the supported container ecosystems.

Cloud Run commitments do not apply to networking changes.

See the documentation for more details.

Cloud Composer

Preview: Cloud Composer supports Airflow 2. For more information about transferring from environments with Airflow 1 to Airflow 2, see Migrate environments to Airflow 2.

Airflow 2.0.1 is available in Cloud Composer images.

You can now break down costs associated with particular Cloud Composer environments. User labels that you assign to your environments now appear in billing reports.

New versions of Cloud Composer images:

  • composer-1.17.0-preview.0-airflow-2.0.1
  • composer-1.16.4-airflow-1.10.15
  • composer-1.16.4-airflow-1.10.14 (default)
  • composer-1.16.4-airflow-1.10.12

For new Cloud Composer environments with Airflow 2, SMTP configuration properties for Airflow have new default values:

  • smtp_user is set to an empty value by default.
  • smtp_password is set to an empty value by default.
  • smtp_mail_from is set to a default value used by Airflow.

Improved the error message that is generated when the specified service account does not have enough permissions to run Airflow workloads.

Added troubleshooting information to error messages generated on Airflow web server deployment failures.

GKE clusters of new Cloud Composer environments use Container-Optimized OS with Containerd (cos_containerd) image type.

Kerberos client (krb5-user) package is pre-installed in Cloud Composer container images.

Some environment operations that failed because of networking problems are now retried instead of failing.

Database passwords are now redacted in error messages that appear in Composer Agent logs.

Error messages about dependency conflicts that happen when installing Python packages are now correctly reported.

When an environment upgrade fails because of package dependency conflicts, the error message contains detailed information about the conflict.

Compute Engine

Preview: You can use OS configuration management to deploy and automate software configurations on your virtual machine (VM) instances using gcloud command-line and OS Config API.

With the release of OS configuration management (preview), you can now rollout policies from the Cloud console, control the rollout pace, use more VM filter options, and view compliance reports. For more information, see OS configuration management (preview).

Datastore Deep Learning Containers

M69 Release

  • Updated cuDNN from 8.0.4 to 8.0.5.
Deep Learning VM Images

M69 Release

  • Migrated Collection Agent to Cloud Monitoring version 2.
Traffic Director

Fixed an issue where the Services user interface would display a warning if a service had a mix of healthy backend groups (x out of x healthy endpoints) and empty backend groups (0 out of 0 healthy endpoints). Now, services that have a mix of healthy backend groups and empty backend groups are shown as healthy.

May 12, 2021

Cloud DNS

Configuring Cloud DNS scopes is now available in Preview.

Cloud Debugger

Cloud Debugger has updated the configuration file naming and keywords that you use to block access to sensitive data. For the updated configuration, see Hiding sensitive data.

Cloud Monitoring

Cloud Monitoring is introducing metrics scopes. For a Google Cloud project, its metrics scope defines the projects whose metrics the project can view and monitor:

  • When you create a project, its metrics scope is set to self.
  • You can modify a project's metrics scope to include other Google Cloud projects, or to include AWS accounts. For more information, see Viewing metrics for multiple projects.
  • A Google Cloud project can be included in multiple metrics scopes.

For more information about metrics scopes, see Configuring your project for Cloud Monitoring.

The replacement of Cloud Monitoring Workspaces with metrics scopes is complete.

All of your existing Cloud Monitoring Workspaces have been migrated to the new data model.

Cloud Run

Committed use discounts are now available for Cloud Run . (Available in public preview.)

Customer managed encryption keys are now available for use with Cloud Run. (Available in public preview.)

You can now use Binary authorization with Cloud Run to enforce policy-based deployment of Cloud Run services. (Available in public preview.)

Recommender now provides recommendations for securing Cloud Run services by creating dedicated service accounts. (Available in public preview.)

Cloud Run now provides UI, command line, and YAML support for referencing Secret Manager Secrets. (Available in public preview.)

Compute Engine

N2 machines are now available in the following regions and zones:

  • Osaka, Japan: asia-northeast2-a,b,c
  • Seoul, South Korea: asia-northeast3-a,b,c

See VM instance pricing for details.

Google Kubernetes Engine

(2021-R16) Version updates

GKE cluster versions have been updated.

New versions available for upgrades and new clusters

The following Kubernetes versions are now available for new clusters and for opt-in control plane upgrades and node upgrades for existing clusters. For more information on the Kubernetes versioning scheme, see Versioning.

No channel

  • Version 1.19.9-gke.1400 is now the default version.
  • Version 1.17.17-gke.7800 is now available.
  • Version 1.19.10-gke.1000 is now available.
  • The following versions are no longer available:
    • 1.18.15-gke.1501
    • 1.18.15-gke.1502
    • 1.18.16-gke.1201
    • 1.18.16-gke.2100
    • 1.18.16-gke.300
    • 1.18.16-gke.302
    • 1.18.16-gke.502
  • The following control planes and nodes with auto-upgrade enabled will be upgraded with this release:

Stable channel

  • Version 1.18.17-gke.700 is now available in the Stable channel.
  • The following control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded with this release:

Regular channel

  • Version 1.19.9-gke.1400 is now the default version in the Regular channel.
  • Version 1.18.17-gke.100 is no longer available in the Regular channel.
  • Version 1.19.8-gke.1600 is no longer available in the Regular channel.
  • Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.18 to version 1.19.9-gke.1400 with this release.

Rapid channel

  • Version 1.19.10-gke.1000 is now available in the Rapid channel.
  • Version 1.20.6-gke.1000 is now available in the Rapid channel.
  • Version 1.20.5-gke.2000 is no longer available in the Rapid channel.
  • The following control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded with this release:

Dataplane V2 is generally available in newly created clusters using GKE versions 1.20.6-gke.700 and later.

The GKE Gateway controller, Google Cloud's implementation of the Gateway API, is available in Preview in GKE version 1.20 and later. See Deploying Gateways for how to expose applications using Gateway.

In GKE version 1.20 and later, the GKE Gateway controller introduces the new gateway.networking.x-k8s.io resource. This is similar but different from the gateway.networking.istio.io resource. This may cause the kubectl get gateway command to return the incorrect Gateway resource unless the fully qualified resource name is used. To avoid seeing unexpected results when using kubectl, see Kubernetes Gateways and Istio Gateways.

The Istio project recently disclosed a new security vulnerability (CVE-2021-31920) affecting Istio. For more information, see the GCP-2021-006 security bulletin.

Secret Manager

Secret Manager integration with Cloud Run

Cloud Run now provides UI, command line, and YAML support for using secrets. This feature is available in Preview.

May 11, 2021

Anthos clusters on VMware

A recently discovered vulnerability, CVE-2021-31920, affects Istio in respect to its authorization policies. Istio contains a remotely exploitable vulnerability where an HTTP request with multiple slashes or escaped slash characters can bypass Istio authorization policy when path-based authorization rules are used. While Anthos clusters on VMware uses an Istio Gateway object for network ingress traffic into clusters, authorization policies are not a supported or intended use case for Istio as part of the Anthos clusters on VMware prerequisites. For more details, refer to the Istio security bulletin.

BigQuery

Updated version of ODBC driver for BigQuery includes bug fixes and install guide improvements.

Updated version of JDBC driver for BigQuery includes bug fixes, service account keyfile support, connection property enhancements, and BigQuery client library updates.

Cloud Asset Inventory

New resource types are now available.

The following resource types are now publicly available through the Export APIs (ExportAssets and BatchGetAssetsHistory) and the Feed API:

  • Cloud Bigtable
    • bigtableadmin.googleapis.com/Backup
Cloud Bigtable

The Cloud Bigtable documentation on schema design for time series data has been updated with an emphasis on recommended design patterns.

Cloud Run for Anthos

CVE-2021-31920 affects Istio, a component used by Cloud Run for Anthos. The CVE specifically impacts Istio's path-based AuthorizationPolicy configurations.

To ensure that your Cloud Run for Anthos clusters are not affected by the CVE, see the security best practices guide to learn more about mitigating this vulnerability.

Dataflow

Dataflow Shuffle is now the default mode for all batch pipelines.

May 10, 2021

BigQuery

BigQuery now supports the following SQL query clauses and operators:

This feature is in Preview.

Cloud Bigtable

You can now use IAM conditions to define and enforce conditional access control for Cloud Bigtable instances, clusters, and tables. This feature is generally available.

Cloud Billing

Cloud Billing Reports now show the target budget amount when you open the report from a budget

In the Cloud Billing Console, Billing Budgets are linked to the Billing Reports page. If you open the Reports page from a Budget, the budget's scopes are used to set the report's filters and the report opens displaying the costs tracked by the budget. Additionally, the budget's target amount appears in the report chart as a red, dashed line, helping you to visualize the budget amount in the report while you are analyzing the specific, budget-related costs. You can open the cost report from the list of budgets, or from a budget's cost trend chart.

For more details about how budgets and cost reports are linked, see Viewing a budget in your report.

Compute Engine

N2D machines are now available in Tokyo asia-northeast1-c. See VM instance pricing for details.

Identity and Access Management

The ability to attach service accounts to resources in other projects is now generally available.

Istio on Google Kubernetes Engine

Google Support does not provide support for Istio installations. For more information, see the Istio support statement.

Workflows

Workflows is HIPAA compliant.

May 07, 2021

Cloud Bigtable

New guidance is available to help you schedule Cloud Bigtable backups using Cloud Scheduler, Pub/Sub, and Cloud Functions.

Cloud Interconnect

Cloud Interconnect support for GRE traffic is available in General Availability. For more information, see the Cloud Interconnect overview.

Cloud VPN

Cloud VPN support for GRE traffic is available in General Availability. For more information, see the Cloud VPN overview.

Speech-to-Text

The Speech-to-Text model adaptation feature is now a GA feature. See the model adaptation concepts page for more information about using this feature.

Traffic Director

gRPC's observability features can now be used with services that use Traffic Director, including monitoring and tracing metrics that help you solve issues with your deployment. For more details, see Observability with proxyless gRPC applications.

Proxyless gRPC applications can now use these advanced traffic management features:

  • Circuit breaking
  • Fault injection
  • Max stream duration

For complete information, see Setting up proxyless gRPC services with advanced traffic management

Virtual Private Cloud

GRE support for VPC networks is now available in General Availability.

May 06, 2021

Anthos clusters on VMware

The Envoy and Istio projects recently announced several new security vulnerabilities (CVE-2021-28683, CVE-2021-28682, and CVE-2021-29258) that could allow an attacker to crash Envoy.

For more information, see the GCP-2021-004 security bulletin.

Anthos clusters on bare metal

The Envoy and Istio projects recently announced several new security vulnerabilities (CVE-2021-28683, CVE-2021-28682, and CVE-2021-29258) that could allow an attacker to crash Envoy.

For more information, see the GCP-2021-004 security bulletin.

Cloud Bigtable

Cloud Bigtable now provides a Cloud Monitoring metric that reports the amount of logical storage bytes that a backup is using. The metric is backup/bytes_used, and it includes information about the source table and storage type.

Cloud Logging

The Logs Explorer Histogram offers new time controls, including zooming and scrolling, to give you more in-depth analysis of your logs data. For details, see Analyzing logs using time controls.

Google Kubernetes Engine

You can now enable and configure OS Login for private GKE clusters and nodes. This feature is enabled for private GKE clusters running node pool versions 1.20.5 or later.

The Envoy and Istio projects recently announced several new security vulnerabilities ( CVE-2021-28683, CVE-2021-28682, and CVE-2021-29258) that could allow an attacker to crash Envoy.

For more information, see the GCP-2021-004 security bulletin.

VPC Service Controls

General availability for the following integration:

May 05, 2021

Anthos clusters on VMware

Anthos clusters on VMware 1.7.1-gke.4 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.7.1-gke.4 runs on Kubernetes 1.19.7-gke.2400.

The supported versions that offer the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.7, 1.6, and 1.5.

If you upgrade the admin cluster before you upgrade the associated user clusters within the same minor version, such as from 1.7.0 to 1.7.1, the user control-planes will be upgraded together with the admin cluster. This applies even if you use the flag --force-upgrade-admin. This behavior, in versions 1.7.0 and later, is different from versions 1.6 and earlier, and is expected behavior.

Fixes:

  • Fixed a bug, so that the hardware version of a virtual machine is determined based on the ESXi host apiVersion instead of the host version. When host ESXi apiVersion is at least 6.7U2, VMs with version vmx-15 are created. Also, the CSI preflight checks validate the ESXi host API version instead of the host version.

  • Fixed a bug, so that if vSphereCSIDisabled is set to true, Container Storage Interface (CSI) preflight checks do not run when you execute commands such as gkectl check-config or create loadbalancer or create cluster.

  • Fixed CVE-2021-3444, CVE-2021-3449, CVE-2021-3450, CVE-2021-3492, CVE-2021-3493, and CVE-2021-29154 on the Ubuntu operating system used by the admin workstation, cluster nodes, and Seesaw.

  • Fixed a bug where attempting to install or upgrade GKE on-prem 1.7.0 failed with an "/STSService/ 400 Bad Request" when the vCenter is installed with the external platform services controller. Installations where the vCenter server is a single appliance are not affected. Note that VMware deprecated the external platform services controller in 2018.

  • Fixed a bug where auto repair failed to trigger for unhealthy nodes if the cluster-health-controller was restarted while a previously issued repair was in progress.

  • Fixed a bug so that the command gkectl diagnose snapshot output includes the list of containers and the containerd daemon log on Container-Optimized OS (COS) nodes.

  • Fixed a bug that caused gkectl update admin to generate an InternalFields diff unexpectedly.

  • Fixed the issue that the stackdriver-log-forwarder pod was sometimes in crashloop because of fluent-bit segfault.

Cloud Data Fusion

There is an issue in the BigQuery sink plugin version 0.17.0, which causes data pipelines to fail or give incorrect results. This issue is resolved in BigQuery sink plugin version 0.17.1. For more information, see the Cloud Data Fusion Troubleshooting page.

Cloud Monitoring

Cloud Monitoring has added new ways to interact with charts. You can now select a range of lines displayed on chart, shift the time axis by using your pointer, and have new controls to expand the chart around a specific point in time. Charts displaying distribution data include 50th, 95th, and 99th percentile lines as an optional overlay. For more information, see Exploring charted data.

Deep Learning Containers

M68 Release

  • Upgraded R containers from 3.6 to 4.0.
  • Added xai-tabular-widget onto all TensorFlow containers.
  • Miscellaneous bug fixes and updates.
Deep Learning VM Images

M68 Release

  • Upgraded R Images from 3.6 to 4.0.
  • Added xai-tabular-widget onto all TensorFlow images.
  • Miscellaneous bug fixes and updates.
SAP on Google Cloud

Updated SAP HANA certification of the 6 TB m2-megamem-416 machine type

For OLAP workloads, the SAP certification of the Compute Engine 6 TB m2-megamem-416 machine type now includes:

  • Scale-out configurations up to 16 nodes.
  • Compute Engine persistent disks for storage in scale-up or scale-out configurations.

For more information, see Certified Compute Engine VMs for SAP HANA.

Security Command Center

Security Command Center Premium has launched Continuous Exports for Pub/Sub in general availability. The feature simplifies the process of creating a NotificationConfig and automates the export of new findings to Pub/Sub.

You must be a Security Command Center Premium customer to use the feature. Security Command Center Standard continues to support one-time exports. To subscribe to Security Command Center Premium, contact your sales representative or fill out our inquiry form.

Security Health Analytics, a built-in service of Security Command Center, has launched a new detector, PUBSUB_CMEK_DISABLED, in general availability. The detector, available to Security Command Center Premium customers, identifies Pub/Sub topics that are not encrypted with customer-managed encryption keys (CMEK). For more information, see the PUBSUB_SCANNER table in Vulnerabilities findings.

Event Threat Detection, a built-in service of Security Command Center, has launched a new detector in general availability. Discovery: Service Account Self-Investigation detects when a service account credential is used to investigate the roles associated with that same service account. For more information on detectors, see Event Threat Detection conceptual overview.

Documentation

VPC Service Controls

Beta stage support for the following integration:

May 04, 2021

Cloud Healthcare API

The defaultSearchHandlingStrict field in the projects.locations.datasets.fhirStores.FhirStore resource is now available in the v1 version of the Cloud Healthcare API.

Cloud Load Balancing

Zonal NEGs (with GCE_VM_IP network endpoints) can now be used as backends for internal TCP/UDP load balancers. For more information on this type of zonal NEG, see Zonal NEGs overview. For instructions on how to set up an internal TCP/UDP load balancer with a zonal NEG backend, see Setting up Internal TCP/UDP Load Balancing with zonal NEGs

This feature is in General Availability.

Cloud Monitoring

The Query Editor for Monitoring Query Language (MQL) has been reimplemented. In addition to autocompletion and error detection, it now supports code folding and a find-and-replace capability. For more information, see Using the Query Editor.

Cloud Run for Anthos

Starting in Cloud Run for Anthos versions 0.21 and later, the new default progress deadline for deployments is up to 10 minutes. For example, it can take 10 mins before a bad revision will reach a failed state. To specify a different deadline, see Configuring progress deadlines.

Config Connector

Config Connector version 1.49.1 is now available.

Miscellaneous bug fixes.

Google Kubernetes Engine

(2021-R15) Version updates

GKE cluster versions have been updated.

New versions available for upgrades and new clusters

The following Kubernetes versions are now available for new clusters and for opt-in control plane upgrades and node upgrades for existing clusters. For more information on the Kubernetes versioning scheme, see Versioning.

No channel

  • Version 1.18.17-gke.100 is now the default version.
  • Version 1.17.17-gke.7200 is now available.
  • The following versions are no longer available:
    • 1.16.15-gke.12500
    • 1.16.15-gke.14800
    • 1.17.17-gke.1101
    • 1.17.17-gke.1500
    • 1.17.17-gke.2800
    • 1.17.17-gke.3000
  • The following control planes and nodes with auto-upgrade enabled will be upgraded with this release:

Stable channel

  • Version 1.18.17-gke.100 is now the default version in the Stable channel.
  • Version 1.17.17-gke.5400 is now available in the Stable channel.
  • The following versions are no longer available in the Stable channel:
    • 1.17.17-gke.3700
    • 1.18.16-gke.2100
  • The following control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded with this release:

Regular channel

  • Version 1.18.17-gke.100 is now the default version in the Regular channel.
  • The following versions are now available in the Regular channel:
  • Version 1.18.16-gke.2100 is no longer available in the Regular channel.
  • The following control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded with this release:

Rapid channel

  • Version 1.19.9-gke.1900 is now the default version in the Rapid channel.
  • Version 1.19.9-gke.1400 is no longer available in the Rapid channel.
  • The following control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded with this release:
Pub/Sub Lite

Pub/Sub Lite is now available in the following regions:

  • Hong Kong (asia-east2)
  • Tokyo (asia-northeast1)
  • Osaka (asia-northeast2)
  • Seoul (asia-northeast3)
  • Mumbai (asia-south1)
  • Jakarta (asia-southeast2)
  • Warsaw (europe-central2)
  • Montreal (northamerica-northeast1)
  • Sao Paulo (southamerica-east1)
  • Northern Virginia (us-east4)
  • Salt Lake City (us-west3)
  • Las Vegas (us-west4)

For the full list of available regions, see Pub/Sub Lite locations.

Video Intelligence API

The following features are available in the Video Intelligence API version v1:

Face detection: Locate faces within a video, and identify attributes such as glasses being worn. Learn more

Person detection: Locate people in a video, and identify attributes and 2D landmarks. Learn more

This GA launch brings significant quality improvement to both features.

May 03, 2021

Artifact Registry

Artifact Registry now supports audit logging for container images in Cloud Audit Logs.

Cloud Bigtable

The ability to restore from a Cloud Bigtable backup to a different instance is now generally available. This feature enhancement lets you use backups for a wider variety of use cases.

Cloud Logging

You can now add custom fields in the Logs Explorer to better analyze logs and refine your queries. For more information, see Adding fields to Log fields pane .

Cloud Monitoring

The Inventory tab on the Cloud Monitoring VM Instances dashboard now offers the ability to filter and sort the instance table by any combination of columns. In addition, new health scorecards report a variety of metrics and statistics related to the health and status of your VMs and agents.

Cloud Run

By default, the memory allocated to each container instance of a new service is 512MiB. The new default applies to new services. Existing services retain their allocated memory.

You can now use Identity-aware Proxy with Cloud Run to use identity and context to guard access to your applications. (Available in public preview.)

Compute Engine

Generally available: Create virtual machines for high performance computing (HPC) workloads using the HPC VM image.

Google Kubernetes Engine

The kubelet graceful node shutdown feature is now enabled on preemptible and GPU accelerator nodes running versions 1.20.5-gke.500 or later.

Vertex AI

April 30, 2021

Anthos GKE on AWS

Anthos clusters on AWS 1.7.1-gke.1 is now available.

Anthos clusters on AWS 1.7.1-gke.1 clusters run the following Kubernetes versions:

  • 1.16.15-gke.17300
  • 1.17.17-gke.7000
  • 1.18.18-gke.300
  • 1.19.9-gke.900

The Anthos clusters on AWS 1.7.1-gke.1 patch release addresses the following security vulnerabilities:

Anthos clusters on bare metal

Anthos clusters on bare metal release 1.7.1 is now available. To upgrade, see Upgrading Anthos clusters on bare metal. Anthos clusters on bare metal 1.7.1 runs on Kubernetes 1.19.

Functionality changes:

  • Customers can now take cluster snapshots regardless of whether the admin cluster control plane is running. This is helpful for diagnosing installation issues.
  • Deploying Anthos clusters on bare metal with SELinux is now fully supported on supported versions of Redhat Enterprise Linux. This applies for new installations of Anthos clusters on bare metal cases only.
  • User cluster creation with bmctl supports credential inheritance from the admin cluster by default. Credential overrides for the user cluster can be specified in the config file during cluster creation.

Fixes:

  • (Updated May 12, 2021) Fixed CVE-2021-28683, CVE-2021-28682, CVE-2021-29258. For more details, see the GCP-2021-004 security bulletin.
  • Fixed potential stuck upgrade from 1.6.x to 1.7.0. The bug was caused by a rare race condition when the coredns configmap failed to be backed up and restored during the upgrade.
  • Fixed potential missing GKE connect agent during installation due to a rare race condition.
  • Fixed issue that prevented automatic updates to the control plane load balancer config when adding/removing node(s) from the control plane node pool.
  • Addressed problem with syncing NodePool taints and labels that resulted in deletion of pre-existing items. Syncs will now append, update, or delete items that are added by taints and labels themselves only.

Known issues:

  • Upgrading the container runtime from containerd to Docker will fail in Anthos clusters on bare metal release 1.7.1. This operation is not supported while the containerd runtime option is in preview.
  • bmctl snapshot command fails when the user creates a custom cluster namespace omitting cluster- prefix from the cluster config file. To avoid this issue, the cluster namespace should follow the cluster-$CLUSTER_NAME naming convention.

For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

Assured Workloads for Government

Assured Workloads now provides support for CJIS and FedRAMP High, and a more streamlined provisioning experience for some compliance regimes. For more information, see the Assured Workloads documentation.

BigQuery

BigQuery now supports the following data definition language (DDL) statements:

This feature is in GA.

Cloud Asset Inventory

New resource types are now available.

The following resource types are now publicly available through the Export APIs (ExportAssets and BatchGetAssetsHistory) and the Feed API:

  • Cloud Monitoring
    • monitoring.googleapis.com/AlertPolicy
  • Cloud Filestore
    • file.googleapis.com/Backup
Cloud SQL for SQL Server

The following version upgrade applies to Cloud SQL for SQL Server:

  • SQL Server 2017 is upgraded from 14.0.3257.3 to 14.0.3370.1

If you use maintenance windows, the new version will be available after your maintenance update. For information about maintenance windows, and to manage maintenance updates, see Finding and setting maintenance windows.

Config Connector

Config Connector version 1.49.0 is now available.

Hierarchical reference field is optional for BigQueryDataset, ComputeDisk, Folder, and Project (Fixes a follow-up issue in #349).

April 29, 2021

Binary Authorization

Binary Authorization now supports Continuous Validation. See Continuous Validation documentation.

Cloud Composer

New versions of Cloud Composer images:

  • composer-1.16.3-airflow-1.10.15
  • composer-1.16.3-airflow-1.10.14 (default)
  • composer-1.16.3-airflow-1.10.12

When Airflow configuration is updated, an erroneous log message about a web server update failure no longer appears in logs.

Fixed problems with execution date in environment health monitoring when Airflow uses a custom time zone.

Cloud Composer versions 1.8.3 to 1.10.2 have reached their end of full support period.

Compute Engine

Preview: With the introduction of OS inventory management v2.0, you can now query the OS Config API to get inventory and vulnerability report data for your VMs in a specific zone, see OS inventory management.

You can now create extreme persistent disks in certain regions. With consistently high performance for both random access workloads and bulk throughput, extreme persistent disks are designed for high-end database workloads.

For more information, see Extreme persistent disks.

Google Kubernetes Engine

For GKE clusters with Windows Server nodes, node names will now be limited to 15-characters to allow for Active Directory joining.

Fixes for the following GKE Autopilot clusters issues are rolling out to the Rapid release channel:

  • Pods with a priority lower than -10 would not trigger scale up.
  • Pod anti-affinity might cause overscaling.

April 28, 2021

Cloud Load Balancing

Internal TCP/UDP Load Balancing now supports session affinity for the UDP protocol. This feature is available in General Availability.

Compute Engine

C2 machines are available in the following regions and zones:

  • Osaka asia-northeast2-a

See VM instance pricing for details.

April 27, 2021

Access Approval

Google Kubernetes Engine is supported by Access Approval in Preview stage.

Cloud Spanner is supported by Access Approval in GA stage.

App Engine standard environment Go

Automatic scaling elements min_instances and min_idle_instances will now only apply to versions of a service that have been configured to receive traffic. This change is to reduce unexpected billing due to instances running old versions that are not intended to receive traffic.

App Engine standard environment Java

Automatic scaling elements min_instances and min_idle_instances will now only apply to versions of a service that have been configured to receive traffic. This change is to reduce unexpected billing due to instances running old versions that are not intended to receive traffic.

App Engine standard environment Node.js

Automatic scaling elements min_instances and min_idle_instances will now only apply to versions of a service that have been configured to receive traffic. This change is to reduce unexpected billing due to instances running old versions that are not intended to receive traffic.

App Engine standard environment PHP

Automatic scaling elements min_instances and min_idle_instances will now only apply to versions of a service that have been configured to receive traffic. This change is to reduce unexpected billing due to instances running old versions that are not intended to receive traffic.

App Engine standard environment Python

Automatic scaling elements min_instances and min_idle_instances will now only apply to versions of a service that have been configured to receive traffic. This change is to reduce unexpected billing due to instances running old versions that are not intended to receive traffic.

App Engine standard environment Ruby

Automatic scaling elements min_instances and min_idle_instances will now only apply to versions of a service that have been configured to receive traffic. This change is to reduce unexpected billing due to instances running old versions that are not intended to receive traffic.

Channel Services

The create, delete, get, list, and patch Customer APIs can now use an alternate parent binding to specify the customer's Channel Partner. The returned resource name follows the format accounts/*/customers/* regardless of the parent binding.

Added LICENSE_CAP_CHANGED to the list of EntitlementEvent.Type.ENUM_VALUES to deliver notifications for a new Pub/Sub event type.

Cloud Build

Webhook triggers are now generally available. Learn more about using webhook triggers to build repos hosted on Gitlab, Bitbucket Cloud, and Bitbucket Server.

Users can now run manual triggers on a schedule. For more information, see Scheduling builds.

Cloud Logging

You can now install the Cloud Logging agent, Cloud Monitoring agent, and Ops Agent on VMs running OpenSUSE Leap versions 15, 15.1, and 15.2.

Cloud Monitoring

You can now install the Cloud Logging agent, Cloud Monitoring agent, and Ops Agent on VMs running OpenSUSE Leap versions 15, 15.1, and 15.2.

Cloud Storage

You can now compose objects using source objects that were encrypted with Cloud KMS keys.

Compute Engine

N2D machines are available in the following regions and zones:

  • Osaka asia-northeast2-c
  • Montréal northamerica-northeast1-a,c
  • Finland europe-north1-a,b,c

See VM instance pricing for details.

Config Connector

Config Connector version 1.48.0 is now available.

ComputeDisk added support for projectRef

Added go-clients for GKEHubMembership and CloudIdentityGroup

Google Kubernetes Engine

(2021-R14) Version updates

GKE cluster versions have been updated.

New versions available for upgrades and new clusters

The following Kubernetes versions are now available for new clusters and for opt-in control plane upgrades and node upgrades for existing clusters. For more information on the Kubernetes versioning scheme, see Versioning.

No channel

Stable channel

  • Version 1.17.17-gke.4900 is now available in the Stable channel.
  • Version 1.18.17-gke.100 is now available in the Stable channel
  • Version 1.18.16-gke.302 is no longer available in the Stable channel.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.18 to version 1.18.16-gke.2100 with this release.

Regular channel

  • Version 1.18.16-gke.2100 is now the default version in the Regular channel.
  • Version 1.18.17-gke.100 is now available in the Regular channel.
  • Version 1.18.16-gke.502 is no longer available in the Regular channel.
  • Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.17 to version 1.18.16-gke.2100 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.18 to version 1.18.16-gke.2100 with this release.

Rapid channel

  • Version 1.19.9-gke.1400 is now the default version in the Rapid channel.
  • Version 1.19.9-gke.1900 is now available in the Rapid channel.
  • Version 1.20.5-gke.2000 is now available in the Rapid channel.
  • Version 1.19.9-gke.700 is no longer available in the Rapid channel.
  • Version 1.20.5-gke.1300 is no longer available in the Rapid channel.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.18 to version 1.19.9-gke.1400 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.19 to version 1.19.9-gke.1400 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.20 to version 1.20.5-gke.2000 with this release.

Multi-Instance GPU on GKE is available in Preview.

Vertex AI

Vizier is now available in preview. Vizier is a feature of AI Platform (Unified) that you can use to perform black-box optimization. You can use Vizier to tune hyperparameters or optimize any evaluable system.

April 26, 2021

Cloud Run for Anthos

Cloud Run for Anthos on Google Cloud version 0.21.0-gke.0 is now available for the following GKE minor versions:

  • 1.19
  • 1.20
  • 1.21

Events for Cloud Run for Anthos version 0.20.0-gke.108 is now available for the following GKE minor versions:

  • 1.19
  • 1.20
  • 1.21
Cloud Translation

Document Translation for Cloud Translation - Advanced (v3) is now available in Preview. Document Translation supports the DOCX, PPTX, XLSX, and PDF file formats. For more information, see Translate documents.

Dialogflow

Preview launch of the following languages in Dialogflow ES:

  • Bengali
  • Filipino
  • Finnish
  • Malay
  • Marathi
  • Romanian
  • Sinhala
  • Tamil
  • Telugu
  • Vietnamese

April 23, 2021

Chronicle

Supported Data Sets

Chronicle can now ingest and parse data from the following additional systems and services:

  • Aruba Airwave
  • Blue Coat Proxy
  • Brocade ServerIron ADX
  • CIS Albert Alerts
  • Cisco Application Control Engine
  • Cisco Email Security
  • Cisco NX-OS
  • Citrix StoreFront
  • Cofense Triage
  • Comodo
  • Fidelis Network
  • FireEye NX
  • Honeyd
  • Kemp Load Balancer
  • Kyriba Treasury Management
  • Microsoft Intune
  • MySQL
  • Palo Alto Networks Cortex XDR
  • Red Canary EDR
  • ServiceNow CMDB
  • Symantec VIP Enterprise Gateway
  • Tanium Discover
  • Tripwire File Integrity Monitoring
Cloud Healthcare API

The reference patterns document provides sample code and technical reference guides for common Cloud Healthcare API use cases.

Cloud SQL for PostgreSQL

The following PostgreSQL minor versions are now available. If you use maintenance windows, you might not yet have the minor version. In this case, you will see the new minor version once your maintenance update occurs. To find your maintenance window or manage maintenance updates, see Finding and setting maintenance windows.

  • 9.6.20 is upgraded to 9.6.21.
  • 10.15 is upgraded to 10.16.
  • 11.10 is upgraded to 11.11.
  • 12.5 is upgraded to 12.6.
  • 13.1 is upgraded to 13.2.

For more information about the content of these minor versions, please see the PostgreSQL release notes.

Config Connector

Config Connector version 1.47.0 is now available.

Added support CloudIdentityGroup and GKEHubMembership

Added resourceID support for Project resource

Fixed the issue of acquiring ComputeBackendService with iap configuration (GitHub #304)

Dataproc

Announcing Dataproc Confidential Compute: Dataproc clusters now support Compute Engine Confidential VMs.

New sub-minor versions of Dataproc images: 1.3.89-debian10, 1.3.89-ubuntu18, 1.4.60-debian10, 1.4.60-ubuntu18, 1.5.35-centos8, 1.5.35-debian10, 1.5.35-ubuntu18, 2.0.9-centos8, 2.0.9-debian10, and 2.0.9-ubuntu18.

Image 1.4

Image 1.5

  • CentOS only: adoptopenjdk is set as the default Java environment.

Image 1.5 and 2.0

  • Updated Oozie version to 5.2.1
  • The Jupyter optional component now uses the "GCS" subdirectory as the initial working directory when you open the JupyterLab UI.

April 22, 2021

Cloud Composer

New versions of Cloud Composer images:

  • composer-1.16.2-airflow-1.10.15
  • composer-1.16.2-airflow-1.10.14 (default)
  • composer-1.16.2-airflow-1.10.12

Airflow 1.10.10 is no longer included in Cloud Composer images.

When a GKE authorization error occurs during an environment operation, the GKE error message is reported and the operation fails immediately.

When an environment operation fails during the installation of PyPI packages, error messages generated by pip are now correctly reported.

When Airflow uses a non-UTC time zone, manually triggered DAGs are executed at correct times now. The monitoring panel displays the correct environment health status.

A deprecation message is now displayed for the xcom_push argument of KubernetesPodOperator.

Cloud Scheduler

The Cloud Scheduler Console UI now has support for three additional options:

  • Headers for HTTP and App Engine targets
  • Message attributes for Pub/Sub targets
  • Retry config for all targets
Kf

Allow long-running source uploads.

Traffic Director

Fixed an issue that caused unexpected behavior when handling malformed HTTP requests.

VPC Service Controls

General Availability release of Ingress and egress rules for VPC Service Controls.

April 21, 2021

BigQuery

BigQuery supports changing an existing non-clustered table to a clustered table and vice versa. You can also update the set of clustered columns of a clustered table. This feature was first documented in October 2020 but was not included in a release note. For more information, see Modifying clustering specification.

Cloud Logging

You can now provision and manage the Cloud Logging agent on Windows using Ansible. For more information, refer to the Ansible Role for Cloud Ops documentation.

Google Kubernetes Engine

See GKE release schedule for information on the current versions rollout and support schedule. See Versioning for details on the GKE version suppport and life cycle.

April 20, 2021

Anthos GKE on AWS

The Kubernetes project recently announced a new security vulnerability, CVE-2021-25735, that could allow node updates to bypass a Validating Admission Webhook. For more details, see the GCP-2021-003 security bulletin.

Anthos Service Mesh

1.9.3-asm.2, 1.8.5-asm.2, 1.7.8-asm.1, and 1.6.14-asm.2 are now available.

Fixes the security issue, ISTIO-SECURITY-2021-003, with the same fixes as Istio 1.9.3. These fixes were also backported to the specified Anthos Service Mesh versions.

This release updates the envoy versions for the following Anthos Service Mesh versions:

For details on upgrading Anthos Service Mesh, refer to the following upgrade guides:

Adding multiple private clusters from different projects into a single Mesh on GKE is now available as a generally available (GA) feature.

Adding multiple private clusters from different projects into a single Mesh on GKE is now available as a public preview feature.

Anthos clusters on VMware

The Kubernetes project recently announced a new security vulnerability, CVE-2021-25735, that could allow node updates to bypass a Validating Admission Webhook. For more details, see the GCP-2021-003 security bulletin.

Anthos clusters on bare metal

The Kubernetes project recently announced a new security vulnerability, CVE-2021-25735, that could allow node updates to bypass a Validating Admission Webhook. For more details, see the GCP-2021-003 security bulletin.

App Engine standard environment Go

Build environment variables support is now available in preview.

App Engine standard environment Java

Build environment variables support is now available in preview.

App Engine standard environment Node.js

Build environment variables support is now available in preview.

App Engine standard environment PHP

Build environment variables support is now available in preview.

App Engine standard environment Python

Build environment variables support is now available in preview.

App Engine standard environment Ruby

Build environment variables support is now available in preview.

Cloud Healthcare API

Resource indexing will now complete before the service sends asynchronous notifications such as Pub/Sub notifications. This ensures that services receiving notifications through Pub/Sub can assume that the resource is searchable when the notification is received.

Dialogflow

Preview launch of Change history and Auto sync in Dialogflow CX.

Google Kubernetes Engine

(2021-R13) Version updates

GKE cluster versions have been updated.

New versions available for upgrades and new clusters

The following Kubernetes versions are now available for new clusters and for opt-in control plane upgrades and node upgrades for existing clusters. For more information on the Kubernetes versioning scheme, see Versioning.

No channel

Stable channel

  • Version 1.17.17-gke.3700 is now the default version in the Stable channel.
  • Version 1.18.16-gke.2100 is now available in the Stable channel.
  • Version 1.17.17-gke.3000 is no longer available in the Stable channel.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.16 to version 1.17.17-gke.3700 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.17 to version 1.17.17-gke.3700 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.18 to version 1.18.16-gke.302 with this release.

Regular channel

  • Version 1.18.16-gke.2100 is now available in the Regular channel.
  • Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.19 to version 1.19.8-gke.1600 with this release.

Rapid channel

  • Version 1.19.9-gke.700 is now the default version in the Rapid channel.
  • Version 1.19.9-gke.1400 is now available in the Rapid channel.
  • Version 1.20.5-gke.1300 is now available in the Rapid channel.
  • Version 1.19.9-gke.100 is no longer available in the Rapid channel.
  • Version 1.20.5-gke.800 is no longer available in the Rapid channel.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.18 to version 1.19.9-gke.700 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.19 to version 1.19.9-gke.700 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.20 to version 1.20.5-gke.1300 with this release.

The Kubernetes project recently announced a new security vulnerability, CVE-2021-25735, that could allow node updates to bypass a Validating Admission Webhook. For more details, see the GCP-2021-003 security bulletin.

Istio on Google Kubernetes Engine

1.6.14-gke.1 is now available.

Fixes the security issue, ISTIO-SECURITY-2021-003, with the same fixes as Istio 1.9.3. These fixes were also backported to the specified Istio on Google Kubernetes Engine versions.

April 19, 2021

API Keys API

API Keys API in Preview.

BigQuery ML

BigQuery ML is introducing new ARIMA_PLUS models and deprecating the ARIMA model type. While the underlying modeling technique has not changed, the following improvements are now available in ARIMA_PLUS:

Cloud Billing

Budget API now supports configurable budget time periods, beyond monthly budgets

Using the Cloud Billing Budget API to manage your budgets, you can now specify the time period of the budget. Prior to this update, you could only configure a budget to monitor costs incurred during a calendar month. Using the usage_period filter that is available in the Cloud Billing Budget API, you can configure the budget time period to a CalendarPeriod or a CustomPeriod, allowing you to create budgets to monitor time frames beyond the default calendar month, such as a quarter, a year, or a custom date range that you specify.

At this time, budgets configured with a non-monthly time period can only be viewed and managed using the Cloud Billing Budget API. Non-monthly budgets are not yet visible in the Budgets page in the Cloud Console.

For more information on using the Cloud Billing Budget API, see Get started with the Cloud Billing Budget API.

Cloud Functions

Cloud Functions has added support for a new runtime, PHP 7.4, in Preview.

Cloud Functions now supports the following runtimes at the General Availability release level:

Cloud Monitoring

Cloud Monitoring Workspaces are changing. Over the next few weeks, new capabilities are being deployed:

  • A Cloud Monitoring Workspace will be created automatically for a Google Cloud project. This change replaces the manual creation process.
  • The restriction that you can view the metrics for a project from only one Workspace is being eliminated. You'll be able to view the metrics for a project from multiple Workspaces.
  • Navigation to a Workspace that manages metrics from multiple projects is changing. For information on this change, see Navigating to a Workspace.
Cloud Trace

Cloud Trace announces that the OpenTelemetry library for Python is now generally available. For information about configuring your Python application to use Open Telemetry, see Python and OpenTelemetry.

Compute Engine

N2 VMs are now available in the following regions and zones:

  • Mumbai asia-south1-a,b
  • Jakarta asia-southeast2-a,b,c

See VM instance pricing for details.

Deep Learning Containers

M67 Release

  • Added Horovod to TensorFlow GPU containers.
  • Regular package refreshment and bug fixes.
Deep Learning VM Images

M67 Release

  • GPU support added for Beam Notebooks.
  • Added Horovod to TensorFlow GPU Deep Learning VMs.
  • Regular package refreshment and bug fixes.
Dialogflow

Dialogflow CX now supports the us-west1 (US, Oregon) region.

Google Kubernetes Engine

Due to GKE Autopilot restrictions on the kubelet API surface, the Datadog Agent is not operating correctly on Autopilot mode clusters.

Network Intelligence Center

Network Topology is Generally Available.

Resource Manager

The Resource Manager v3 API has been released into general availability. For more information, see the API reference documentation.

SAP on Google Cloud

File sharing options for SAP on Google Cloud: New guidance has been published to help you determine the best file sharing option for your SAP deployments on Google Cloud.

For more information, see File sharing solutions for SAP on Google Cloud.

April 16, 2021

AI Platform Prediction

Runtime version 2.4 is now available. You can use runtime version 2.4 to serve online predictions with TensorFlow 2.4.1, scikit-learn 0.24.0, or XGBoost 1.3.1. Runtime version 2.4 does not support batch prediction.

See the full list of updated dependencies in runtime version 2.4.

Cloud Monitoring

The Cloud Operations for GKE monitoring dashboard now allows you to manage and display service-level objectives (SLOs) that you define for your applications. For more information, see the Managing SLOs section of the Observing your GKE clusters guide.

Compute Engine

N2D machines are available in the following regions and zones:

  • Montréal northamerica-northeast1-b
  • Osaka asia-northeast2-a,b

See VM instance pricing for pricing details.

Config Connector

Config Connector version 1.46.0 is now available.

cnrm-resource-stats-recorder container now binds to hostPort 48797 rather than 8888 (fixes GitHub issue #449)

Go Client now uses a pointer type or allows for a built-in nil value for spec fields that are optional. (fixes GitHub issue #426)

BigQueryDataset add support for projectRef

ContainerCluster supports enableAutopilot, enableL4IlbSubsetting, and privateIpv6GoogleAccess.

ContainerNodePool supports disabling autoscaling by setting min and max node counts to 0 (fixes GitHub issue #437)

SecretManagerSecretVersion now requires the secretData field.

Added observedGeneration field to status for resources, enabling compatibility with kstatus (fixes GitHub issue #410]{:.external})

Dataproc

Added the ability to stop and start high-availability clusters.

Fixed a bug where scale-down update cluster requests failed due to quota validation if the user project was over a quota limit.

Dialogflow

Preview launch of the Dialogflow CX Phone Gateway integration.

April 15, 2021

Access Approval

Access Transparency logs associated with an approval request can be viewed on the Access Approval UI.

Cloud Composer

New versions of Cloud Composer images:

  • composer-1.16.1-airflow-1.10.15
  • composer-1.16.1-airflow-1.10.14 (default)
  • composer-1.16.1-airflow-1.10.12
  • composer-1.16.1-airflow-1.10.10

If an environment's service account does not have required permissions for a requested operation, Cloud Composer generates an actionable error message. The operation fails faster in this case.

Fixed a bug that caused environment update and upgrade operations to fail with errors related to GKE cluster endpoints, instead of the actual root cause. This problem affected environments with installed custom PyPI packages.

Cloud Monitoring

Compute Engine's Instance Groups Monitoring tab now includes charts for your managed instance groups. Charted metrics include group size, CPU utilization, disk I/O, and more. You can select the time window for the charts and view the corresponding logs from the integrated logs viewer panel. You can also use the links on each chart to create alerting policies or to analyze the data in Metrics Explorer.

The Cloud Operations for GKE monitoring dashboard now includes a column called Error logs that displays the number of error logs associated with an entity based on the selected time range. You can also select which columns to display in the tables. For more information, see the Configuring the dashboard tables section of the Observing your GKE clusters guide.

Compute Engine

You can now see additional metrics for your managed instance groups from the Instance Groups Monitoring tab. Metrics include: group size, CPU utilization, disk I/O, and more. Use the time range picker to select the time window for the charts and view the corresponding logs from the integrated logs viewer panel. Follow the links on each chart to create alerts or to analyze the details in the Cloud Operations Metrics Explorer.

Dataproc Metastore

The asynchronous workflows logs now have labels that appear in Cloud logging.

You no longer need to manually override metastore.expression.proxy to use PartitionProxyForMetastore in Hive 3.1.2.

Memorystore for Redis

Added new Memorystore for Redis region: Warsaw (europe-central2).

SAP on Google Cloud

SAP HANA high-availability configurations on Red Hat: If you configured a RHEL HA cluster for SAP HANA before April 15, 2021 by following the Google Cloud documentation, you need to modify the location constraints of your cluster fencing devices to avoid possible race conditions during failovers.

To see the updated documentation to correct the issue, see Set up fencing, step 1.b.

Vertex AI

The Python client library for AI Platform (Unified) is now called the AI Platform (Unified) SDK. With the release of version 0.7 (Preview), the AI Platform (Unified) SDK provides two levels of support. The high-level aiplatform library is designed to simplify common data science workflows by using wrapper classes and opinionated defaults. The lower-level aiplatform.gapic library remains available for those times when you need more flexibility or control. Learn more.

April 14, 2021

App Engine standard environment Go

Serverless VPC Access support for Shared VPC is now generally available.

Serverless VPC Access support for Shared VPC is now generally available.

App Engine standard environment Java

Serverless VPC Access support for Shared VPC is now generally available.

Serverless VPC Access support for Shared VPC is now generally available.

App Engine standard environment Node.js

Serverless VPC Access support for Shared VPC is now generally available.

App Engine standard environment PHP

Serverless VPC Access support for Shared VPC is now generally available.

App Engine standard environment Python

Serverless VPC Access support for Shared VPC is now generally available.

Serverless VPC Access support for Shared VPC is now generally available.

App Engine standard environment Ruby

Serverless VPC Access support for Shared VPC is now generally available.

Cloud Asset Inventory

New resource types are now available.

The following resource types are now publicly available through the Export APIs (ExportAssets and BatchGetAssetsHistory) and the Feed API:

  • Cloud AI Platform (Unified)
    • aiplatform.googleapis.com/BatchPredictionJob
    • aiplatform.googleapis.com/CustomJob
    • aiplatform.googleapis.com/DataLabelingJob
    • aiplatform.googleapis.com/Dataset
    • aiplatform.googleapis.com/Endpoint
    • aiplatform.googleapis.com/HyperparameterTuningJob
    • aiplatform.googleapis.com/Model
    • aiplatform.googleapis.com/SpecialistPool
    • aiplatform.googleapis.com/TrainingPipeline
Cloud Run

Cloud Run is now available in europe-central2 (Warsaw)

Dialogflow

The "Auto-preview changes" option was removed from the Dialogflow ES Google Assistant integration.

Google Cloud Armor

Managed Protection Plus subscribers are also eligible to receive reactive or proactive DDoS response support from Google's DDoS mitigation experts to help triage and mitigate ongoing attacks, as well as DDoS bill protection to provide credits for some bill spikes caused by increased GCP usage as a result being target by a DDoS attack.

For more information, see the public docs.

Google Kubernetes Engine

(2021-R12) Version updates

GKE cluster versions have been updated.

New versions available for upgrades and new clusters

The following Kubernetes versions are now available for new clusters and for opt-in control plane upgrades and node upgrades for existing clusters. For more information on the Kubernetes versioning scheme, see Versioning.

No channel

Stable channel

  • Version 1.17.17-gke.3000 is now the default version in the Stable channel.
  • Version 1.17.17-gke.3700 is now available in the Stable channel.
  • Version 1.17.17-gke.2800 is no longer available in the Stable channel.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.16 to version 1.17.17-gke.3000 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.17 to version 1.17.17-gke.3000 with this release.

Regular channel

  • Version 1.19.8-gke.1600 is now available in the Regular channel.
  • Version 1.18.16-gke.302 is no longer available in the Regular channel.

Rapid channel

  • Version 1.19.9-gke.100 is now the default version in the Rapid channel.
  • Version 1.19.9-gke.700 is now available in the Rapid channel.
  • Version 1.20.5-gke.800 is now available in the Rapid channel.
  • Version 1.19.8-gke.2000 is no longer available in the Rapid channel.
  • Version 1.20.5-gke.101 is no longer available in the Rapid channel.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.18 to version 1.19.9-gke.100 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.19 to version 1.19.9-gke.100 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.20 to version 1.20.5-gke.800 with this release.

1.19 GA

GKE version 1.19 is now generally available (GA).

Before upgrading to 1.19, read the Kubernetes 1.19 Release Notes especially the Urgent upgrade notes.

See below for notable changes and features in version 1.19.

The basic authentication method is no longer available starting with Kubernetes version 1.19. GKE clusters also no longer support basic authentication as they gradually upgrade to Kubernetes version 1.19. Basic authentication has been disabled by default for new GKE clusters since GKE version 1.12 and its usage has been discouraged in the Hardening your cluster's security guide. Migrate away from basic authentication before your cluster control planes are upgraded to Kubernetes version 1.19 to ensure your API clients can continue accessing the API server. To learn more about recommended authentication methods in GKE, see Authenticating to the Kubernetes API Server.

Admission webhooks and custom resource conversion webhooks must use serving certificates that contain the server name in a subjectAltName extension. Server names in the certificate CommonName will not be honored in future versions.

kube-proxy now uses EndpointSlices by default.

With the release of GKE node version 1.19, the Container-Optimized OS with Docker (cos) variant is deprecated. Please migrate to the Container-Optimized OS with Containerd (cos_containerd) variant, which is now the default GKE node image. For instructions, see Containerd images.

Seccomp General Availability (GA)

Seccomp (secure computing mode) support for Kubernetes has graduated to General Availability (GA). This feature can be used to increase the workload security by restricting the system calls for a Pod (applies to all containers) or individual containers.

A new seccompProfile field is added to Pod and Container securityContext objects, starting in Kubernetes version 1.19.

securityContext:
  seccompProfile:
    # "Unconfined", "RuntimeDefault", or "Localhost"
    type: Localhost
    # only necessary if type == Localhost
    localhostProfile: my-profiles/profile-allow.json

The alpha seccomp annotations seccomp.security.alpha.kubernetes.io/pod and container.seccomp.security.alpha.kubernetes.io/... are deprecated in favor of the GA API field. The alpha annotations will not be honored in Kubernetes versions 1.22 and later.

Prepare for transition

If you are currently using Seccomp annotations on Pods or Containers, you should identify and transition workloads using the annotations to set the API fields before version 1.21 is released on GKE (approximately in June 2021). No change on PodSecurityPolicy is required, as it supports both annotation and field seccomp profiles. You can perform the following recommended steps:

Locate Seccomp annotation usages

In your Kubernetes manifest files, search for "seccomp.security.alpha.kubernetes.io/pod" and "container.seccomp.security.alpha.kubernetes.io/".

Add or update securityContext fields

Based on your annotation usage, add or update (if securityContext already exists) the securityContext field in the Pod or Container spec. The annotations can be left in place, but must match the securityContext API field.

Current annotation usage Add or update securityContext
seccomp.security.alpha.kubernetes.io/pod In the Pod's securityContext, add the seccompProfile field.
container.seccomp.security.alpha.kubernetes.io/container-name In the container-name container's securityContext, add the seccompProfile field.

Set values for seccompProfile

The type field of seccompProfile corresponds to the annotation value, and localhostProfile field corresponds to the path following localhost annotation value.

Current annotation value seccompProfile value
unconfined
seccompProfile:
 type: Unconfined
runtime/default or docker/default
seccompProfile:
 type: RuntimeDefault
localhost/path/to/profile.json
seccompProfile:
 type: Localhost
 localhostProfile: path/to/profile.json

More resources

The widely used Ingress API has graduated to general availability in Kubernetes 1.19. The v1beta1 Ingress API is deprecated, and will no longer be served in versions 1.22 and later. Before version 1.21, identify and transition clients and manifests using the v1beta1 Ingress API to use networking.k8s.io/v1.

Clusters with Google Cloud's operations suite enabled can use the following query to identify clients that access the Ingress v1beta1 APIs:

resource.type="k8s_cluster"
resource.labels.cluster_name="$CLUSTER_NAME"
protoPayload.authenticationInfo.principalEmail:("system:serviceaccount" OR "@")
protoPayload.request.apiVersion=("extensions/v1beta1" OR "networking.k8s.io/v1beta1")
protoPayload.request.kind="Ingress"
NOT ("kube-system")

Identify and transition clients and manifests using the v1beta1 Ingress APIs to use networking.k8s.io/v1 before version 1.21 is released on GKE (approximately in June 2021), then verify no clients are using the v1beta1 API during the version 1.21 timeframe. Workloads using the v1beta1 APIs need to be upgraded before your cluster is upgraded to GKE 1.22.

To migrate manifests to networking.k8s.io/v1, perform the following:

  1. Rename the spec.backend field (if specified) to spec.defaultBackend.
  2. Rename each backend.serviceName field to backend.service.name.
  3. Rename each numeric backend.servicePort field to backend.service.port.number.
  4. Rename each string backend.servicePort field to backend.service.port.name.
  5. Specify a pathType field for each defined path. Options are Prefix, Exact, and ImplementationSpecific. To match the undefined v1beta1 behavior, use ImplementationSpecific.

As an example, to migrate this v1beta1 manifest to v1:

Original v1beta1 manifest Equivalent networking.k8s.io/v1 manifest
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: example
spec:
  backend:
    serviceName: default-backend
    servicePort: 80
  rules:
  - http:
      paths:
      - path: /testpath
        backend:
          serviceName: test
          servicePort: 80
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example
spec:
  defaultBackend:
    service:
      name: default-backend
      port:
        number: 80
  rules:
  - http:
      paths:
      - path: /testpath
        pathType: ImplementationSpecific
        backend:
          service:
            name: test
            port:
              number: 80

CertificateSigningRequest v1 API

The CertificateSigningRequest API has graduated to certificates.k8s.io/v1 in Kubernetes 1.19. The v1beta1 CertificateSigningRequest API is deprecated and will no longer be served in version 1.22 and later.

Clusters with Google Cloud's operations suite enabled can use the following query to identify clients that access the CertificateSigningRequest v1beta1 APIs:

resource.type="k8s_cluster"
resource.labels.cluster_name="$CLUSTER_NAME"
protoPayload.authenticationInfo.principalEmail:("system:serviceaccount" OR "@")
protoPayload.request.apiVersion="certificates.k8s.io/v1beta1"
NOT ("kube-system")

Identify and transition clients and manifests using the v1beta1 CertificateSigningRequest API to use certificates.k8s.io/v1 before version 1.21 is released on GKE (approximately in June 2021), then verify no clients are using the v1beta1 API during the version 1.21 timeframe. Workloads using the v1beta1 API need to be upgraded before your cluster is upgraded to GKE version 1.22.

Differences between the v1beta1 and v1 API are as follows:

  • For API clients requesting certificates:
    • spec.signerName is now required, and requests for kubernetes.io/legacy-unknown are not allowed to be created via the certificates.k8s.io/v1 API.
    • spec.usages is now required, may not contain duplicate values, and must only contain known usages.
  • For API clients approving or signing certificates:
    • status.conditions may not contain duplicate types.
    • status.conditions[*].status is now required.
    • status.certificate must be PEM-encoded, and must contain only CERTIFICATE blocks.

Admission webhooks and custom resource conversion webhooks using invalid serving certificates that do not contain the server name in a subjectAltName extension cannot be contacted by the Kubernetes API server in 1.19 prior to version 1.19.9-gke.400. This will be resolved in version 1.19.9-gke.400, and automatic upgrades from 1.18 to 1.19 will not begin until this issue is resolved. However, affected webhooks should work to correct their serving certificates in order to work correctly with Kubernetes version 1.22 and later.

Service API objects with more than 100 ports do not work correctly with EndpointSlices (https://issue.k8s.io/99382). This will be resolved in version 1.19.9-gke.600, and automatic upgrades from 1.18 to 1.19 will not begin until this issue is resolved.

Migrate for Compute Engine

Google Cloud Console UI

End-to-end migration experience in Google Cloud Console including: Dashboard, Source inventory, Migrations managements, VM groups, and Targets.

To access the UI:

  1. Open the Migrate for Compute Engine page in the Google Cloud Console.

  2. In the upper-right corner, select Try the new version to open the Google Cloud Console to the 5.0 UI.

Migration primitives

Migration primitives controlling VM migration journey, which includes:

  • Replication - Initiate replication based migration, control periodical replication cycle schedule.

  • Test-Clone - Test a clone of migrating VM in Google Cloud with no disruptions on source VM to reduce migration risk.

  • Cut-Over - Cutting over to Google Cloud process with minimized downtime to migrating VM.

See VM Migration lifecycle for more.

VM groups

Group migration operations to enable you to manage and execute mass migration sprints.

See Mass migration with groups for more.

Seamless OS adaptation

Seamless OS adaptation of migrating VMs to prepare OS to run in Compute Engine (such as network settings) and deploy Compute Engine agents for seamless day 2 integrations with Compute Engine services.

See Adapting VMs to run on Google Cloud for more.

Compute Engine Targets

Migration to n Google Cloud target projects and flexible configuration of migrating VM target details (such as instance type, disk type, and network settings).

See Configuring the target for a migrated VM for more.

vSphere Source

Agentless migration of vSphere source environment utilizing Migrate Connector appliance deployed in source.

See On-premises VMware to Compute Engine migrations for more.

VM utilization reports

To help you determine the optimal settings for the Compute Engine target, Migrate for Compute Engine lets you create a source VM utilization report. This report displays information about resource allocation and utilization for the source VMs deployed on vCenter.

See Creating a source VM utilization report for more.

Virtual Private Cloud

Access to Google APIs and services using Private Service Connect is now available in General Availability.

Using non-RFC 1918 addresses for Private Service Connect endpoints results in unexpected costs due to a billing issue. To prevent this issue, avoid using non-RFC 1918 IP addresses and instead use RFC 1918 IP addresses for Private Service Connect endpoints. If you are affected by this issue, contact your account team for remediation.