Google Cloud release notes

The following release notes cover the most recent changes over the last 60 days. For a comprehensive list of product-specific release notes, see the individual product release note pages.

You can also see and filter all release notes in the Google Cloud Console or you can programmatically access release notes in BigQuery.

To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly: https://cloud.google.com/feeds/gcp-release-notes.xml

September 24, 2021

Deep Learning Containers

Starting with the M80 image release, all environments will include JupyterLab 3.x by default. To continue using an existing environment's JupyterLab 1.x version, disable auto-upgrade (if enabled) and do not manually upgrade the environment to a new environment version. To create new instances using older images that have JupyterLab 1.x installed, see creating specific versions of instances.

M80 release

  • Updated JupyterLab from 1.x to 3.x.
  • Added Jupytext.
Deep Learning VM Images

Starting with the M80 image release, all environments will include JupyterLab 3.x by default. To continue using an existing environment's JupyterLab 1.x version, disable auto-upgrade (if enabled) and do not manually upgrade the environment to a new environment version. To create new instances using older images that have JupyterLab 1.x installed, see creating specific versions of instances.

M80 release

  • Updated JupyterLab from 1.x to 3.x.
  • Added Jupytext.
  • Deep Learning VM Images in Cloud Marketplace have been updated. They were not updated in the last release.
Google Kubernetes Engine

GKE versions 1.18.20-gke.5100 and later fix the issue with v1beta1 of the Backendconfig API, where a Cloud Armor security policy was inadvertently deleted from the backend Service of an Ingress resource.

For more information, see Kubernetes issue #1508 and the Ingress Known issues page.

GKE clusters running node pools that use Docker might experience containers restarting every time Docker restarts.

The following versions are affected:

  • GKE 1.20 versions lower than 1.20.9-gke.2100
  • GKE 1.21 versions lower than 1.21.3-gke.1600

To fix this issue, either use Containerd or upgrade your nodes to version:

  • For GKE 1.20: 1.20.9-gke.2100 or higher
  • For GKE 1.21: 1.21.3-gke.1600 or higher
Pub/Sub Lite

Pub/Sub Lite reservations allow you to reserve and share throughput capacity among multiple topics in a region. - For more information, see Creating and managing Lite reservations.

Vertex AI

Vertex Matching Engine is generally available (GA).

September 23, 2021

AI Platform Training

Pre-built PyTorch containers for PyTorch 1.9 are available for training. You can use these containers to train with CPUs, GPUs, or TPUs.

Anthos Config Management

Config Sync supports rendering Kustomize configurations and Helm charts in multi-repo mode. The Git repository must have a kustomization.yaml file in the root of the sync directory to trigger the rendering process. To learn more, see Use a repo with Kustomize configurations and Helm charts.

The nomos hydrate command supports rendering unstructured source format and it supports rendering Kustomize configurations or Helm charts.

The nomos vet command supports rendering and it supports rendering Kustomize configurations or Helm charts. It provides a --keep-output flag to preserve the rendered output.

Config Sync ignores validating and applying any resource configuration in the Git repo with the annotation config.kubernetes.io/local-config: "true".

When encountering KNV1021: UnknownObjectError, Config Sync applies other resources that aren't affected by this error.

Updated Config Sync CPU requests to fit inside a default GKE cluster and for better resource utilization.

We strongly recommend that all Config Sync users enable multi-repo mode. It provides you with additional features and gives you the flexibility to sync to a single repository, or multiple repositories. If you are using kubectl to install and manage Config Sync, you can enable multi-repo mode by setting spec.enableMultiRepo: true in your ConfigManagement object. For more details, see Syncing from multiple repositories.

The Anthos Config Management operator is now installed into the config-management-system namespace rather than the kube-system namespace. If you are running custom monitoring or installation processes you need to update those processes. For specific instructions, see Manually installing Config Sync and Policy Controller with kubectl.

In nomos versions earlier than 1.9.0, the nomos status command reports an incorrect status for clusters using an Anthos Config Management version of 1.9.0 or later. Before upgrading to Anthos Config Management 1.9.0 or later, download the latest nomos CLI tool.

Fixed the issue causing the reconciler image version not getting updated, when upgrading from Anthos Config Management version 1.6.2. This was caused by an immutable label added in Anthos Config Management 1.6.2 and removed in 1.7.0.

Anthos clusters on VMware

Anthos clusters on VMware 1.7.4-gke.2 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.7.4-gke.2 runs on Kubernetes v1.19.12-gke.2101.

The supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.8, 1.7, and 1.6.

Fixes:

  • Fixed high-severity CVE-2021-3711.
  • Fixed CVE-2021-25741 mentioned in the GCP-2021-018 security bulletin.
  • Fixed the Istio security vulnerabilities listed in the GCP-2021-016 security bulletin.
  • Fixed the issue that gkeadm tries to set permissions for the component access service account when --auto-create-service-accounts=false.
Cloud Asset Inventory

The following resource types are now publicly available through the Export APIs (ExportAssets and BatchGetAssetsHistory) and the Feed API:

  • Vertex AI
    • aiplatform.googleapis.com/ModelDeploymentMonitoringJob
Cloud Composer

Cloud Composer 1.17.1 release started on September 23, 2021. Get ready for upcoming changes and features as we roll out the new release to all regions. This release is in progress at the moment. Listed changes and features might not be available in some regions yet.

Cloud Composer supports Privately used public IP addresses in Preview. This feature is available in new environments.

Cloud Composer images with Airflow 2 now use Python 3.8.12.

Changes in the preinstalled apache-airflow-backport-providers-google package for Airflow 1.10.15:

  • CloudDataFusionStartPipelineOperator can be run in async mode.
  • Added a new sensor, DatafusionPipelineStateSensor.
  • Fixes for the success_states and pipeline_timeout parameters in CloudDataFusionStartPipelineOperator.

The GRPC_POLLING_STRATEGY environment variable is set to epoll1 by default. This fix is a workaround for a bug introduced in grpcio 1.31.

(New environments only) The FluentD environment component (composer-fluentd) no longer breaks if it was manually turned off for more than 30 days.

(New environments only) The airflow-monitoring pod is restarted instead of being marked as unhealthy when the GKE control plane IP changes.

New versions of Cloud Composer images:

  • composer-1.17.1-airflow-2.1.2
  • composer-1.17.1-airflow-2.0.2
  • composer-1.17.1-airflow-1.10.15 (default)
  • composer-1.17.1-airflow-1.10.14
  • composer-1.17.1-airflow-1.10.12
  • composer-2.0.0-preview.2-airflow-2.1.2
  • composer-2.0.0-preview.2-airflow-2.0.2

Cloud Composer 1.12.0 has reached its end of full support period.

Cloud Load Balancing

Internal TCP/UDP Load Balancing now allows you to configure a connection tracking policy for the load balancer's backend service. A connection tracking policy introduces the following new properties to let you customize your load balancer's connection tracking behavior:

  • Tracking mode
  • Connection persistence on unhealthy backends
  • Idle timeout

To learn about how connection tracking works, see Traffic distribution.

This feature is available in Preview.

Compute Engine

Generally Available: Use patch alerting to monitor the patch jobs running in your environment. For more information, see Monitoring patch jobs.

September 22, 2021

Chronicle

The Linux Forwarder has been enhanced to support load balancing and high-availability. This enables you to deploy the forwarder in an environment where a Layer 4 load balancer is installed between syslog data sources and forwarder instances.

Cloud CDN

Cloud CDN now supports custom named cookies and headers in the cache key, to enable A/B (multivariate) testing, canarying, and similar scenarios. Allowlisting of query parameters is now also enabled for backend buckets, to allow for cache busting. These features are available in Preview.

For details, see the caching documentation.

Cloud Storage

Object listing is no longer impacted when performing large-scale object deletion.

  • Previously, object listing performance could be degraded for up to several days when deleting millions of objects at once in a bucket.
Compute Engine

Preview: You can now access installer properties for your Windows applications by using OS inventory management. For more information, see OS inventory management.

For information on setting up and using OS inventory management, see Viewing operating system details.

Google Cloud VMware Engine

Beginning in the middle of October 2021, VMware Engine will upgrade the VMware stack from version 7.0 Update 1 to 7.0 Update 2 and the NSX-T stack from version 3.0 to 3.1.2. Users affected by this upgrade will receive an email with planned maintenance dates and times.

For details about the upgrade and steps to prepare, see Service announcements.

September 21, 2021

Anthos clusters on VMware

Anthos clusters on VMware 1.8.3-gke.0 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.8.3-gke.0 runs on Kubernetes v1.20.9-gke.701.

The supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.8, 1.7, and 1.6.

Fixes:

  • Fixed high-severity CVE-2021-3711.
  • Fixed CVE-2021-25741 mentioned in the GCP-2021-018 security bulletin.
  • Fixed the Istio security vulnerabilities listed in the GCP-2021-016 security bulletin.
  • Fixed the issue that gkeadm tries to set permissions for the component access service account when --auto-create-service-accounts=false.

In versions 1..8.0-1.8.3, the gkectl create-config admin/cluster command panics with the message panic: invalid version: "latest". As a workaround, use gkectl create-config admin/cluster --gke-on-prem-version=$DESIRED_CLUSTER_VERSION. Replace DESIRED_CLUSTER_VERSION with the desired version.

Anthos on bare metal

Release 1.8.4

Anthos clusters on bare metal 1.8.4 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.8.4 runs on Kubernetes 1.20.

Fixes:

The following container image security vulnerabilities have been fixed: - CVE-2021-3711 - CVE-2021-3712 - CVE-2021-20305 - CVE-2021-33560

Known issues:

When you upgrade Anthos clusters on bare metal from a version with a security patch to the next minor release, we recommend you upgrade to the highest patch version to ensure you have the latest security fixes. Always review the release notes before upgrading so you're aware of what has changed, including security fixes and known issues. Upgrading to a lower release version isn't supported.

For information about the latest known issues, see Anthos clusters on bare metal known issues in the Troubleshooting section.

BigQuery

When saving query results from the Cloud Console to a CSV file, the available download size is now 10 MB. Previously the limit was 16,000 rows. Also, you can now download tables with nested and repeated data to CSV files.

Config Connector

Config Connector 1.62.0 is now available.

Added Age and Healthy columns for the kubectl get tabular outputs of ConfigConnector and ConfigConnectorContext resources.

Miscelleanous bug fixes.

Dataflow

Dataflow now uses Zonal DNS for worker resources. This enables Dataflow to offer higher reliability guarantees around Internal DNS registration.

Google Cloud VMware Engine

Added security bulletin for the VMware Engine response to VMware security advisory VMSA-2021-0020.

Vertex AI

Vertex Vizier is generally available (GA).

September 20, 2021

Anthos Service Mesh

1.9.8-asm.6 and 1.10.4-asm.14 are now available.

These patch releases fix a potential memory leak in the control plane.

Anthos on bare metal

Release 1.7.4

Anthos clusters on bare metal release 1.7.4 is now available for download. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.7.4 runs on Kubernetes 1.19.

Fixes:

  • Fixed vulnerability CVE-2021-25741 that might allow users to create a container with subpath volume mounts to access files and directories outside of the volume, including on the host filesystem. This vulnerability affects all clusters created or upgraded with Anthos clusters on bare metal release 1.7.0. For more information, see the GCP-2021-018 security bulletin.

  • Updated the Kubernetes patch version to address the following container image security vulnerabilities:

Known issues:

When you upgrade Anthos clusters on bare metal from a version with a security patch to the next minor release, we recommend you upgrade to the highest patch version to ensure you have the latest security fixes. Always review the release notes before upgrading so you're aware of what has changed, including security fixes and known issues. Upgrading to a lower release version isn't supported.

For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

Security bulletin (1.7 and 1.8)

A security issue was discovered in Kubernetes, CVE-2021-25741, where a user may be able to create a container with subpath volume mounts to access files and directories outside of the volume, including on the host filesystem. This vulnerability affects all clusters created or upgraded with Anthos clusters on bare metal 1.7.x and 1.8.x releases, specifically 1.7.3 and earlier and 1.8.2 and earlier.

To fix this vulnerability, upgrade your Anthos clusters to version 1.7.4 or 1.8.3. For more information, see the GCP-2021-018 security bulletin.

App Engine standard environment Go

Previously, Legacy API calls made from the App Engine standard environment after the request had finished would immediately return with an error. API calls after the request has finished are now allowed. These API calls are billed according to the standard rates.

App Engine standard environment Java

Previously, Legacy API calls made from the App Engine standard environment after the request had finished would immediately return with an error. API calls after the request has finished are now allowed. These API calls are billed according to the standard rates.

App Engine standard environment PHP

Previously, Legacy API calls made from the App Engine standard environment after the request had finished would immediately return with an error. API calls after the request has finished are now allowed. These API calls are billed according to the standard rates.

App Engine standard environment Python

Previously, Legacy API calls made from the App Engine standard environment after the request had finished would immediately return with an error. API calls after the request has finished are now allowed. These API calls are billed according to the standard rates.

Cloud Storage

Object Versioning can now be managed in the Cloud Console.

Google Cloud Deploy

Google Cloud Deploy is available in Preview.

Identity and Access Management

The IAM documentation now refers to the identities that can be granted access to a resource as principals. Previously, these identities were known as members.

This change does not affect the REST API, the client libraries, or the flags for the gcloud command-line tool.

The reference documentation for predefined roles now uses a new format that is easier to browse.

Migrate for Compute Engine

Compute Engine now supports the deployment of migrated workloads to sole-tenant nodes. A sole-tenant node is a Compute Engine server that is dedicated to hosting only your project's VMs.

See Migrating individual VMs for more information on sole tenancy.

Network Intelligence Center

Connectivity to Cloud VPN and Cloud Interconnect is now generally available in Network Topology. Google Cloud users can use Network Topology to audit their networking configuration and troubleshoot issues related to the hybrid connectivity to and from their on-premises networks.

Resource Manager

The Organization Policy Service v2 API reference documentation is now available. For more information, see the API reference documentation.

Video Intelligence API

The CELEBRITY_RECOGNITION model will undergo an upgrade to a newer version over the next 90 days. The API interface and client library will remain same as the previous version. The API follows the same Service Level Agreement (SLA). You have 30 days from this release date to test the new model. To do so, specify "builtin/latest" in the model field of the Feature object while requesting image annotation. After the end of this 30-day period, the new version will be promoted to the default model and accessible as "builtin/stable". Going forward, the original model will still be available for another 60 days using "builtin/legacy". If you encounter problems with this upgrade, contact the Video Intelligence API engineering team by submitting a ticket in the private issue tracker.

September 17, 2021

Anthos clusters on VMware

A security issue was discovered in Kubernetes, CVE-2021-25741, where a user may be able to create a container with subpath volume mounts to access files and directories outside of the volume, including on the host filesystem. For more information, see the GCP-2021-018 security bulletin.

BigQuery

BigQuery now supports the following geospatial data functions:

  • ST_EXTERIORRING: Returns a linestring geography that corresponds to the outermost ring of a polygon geography.

  • ST_INTERIORRINGS: Returns an array of linestring geographies that corresponds to the interior rings of a polygon geography.

  • ST_ANGLE: Returns the angle between two intersecting lines.

  • ST_AZIMUTH: Returns the azimuth of a line segment formed by two points.

  • ST_NUMGEOMETRIES: Returns the number of geometries in a geography.

  • ST_GEOMETRYTYPE: Returns the Open Geospatial Consortium (OGC) geometry type that describes a geography as a string.

These functions are generally available (GA).

Cloud Billing

Cost breakdown report now supports new filters and report sharing

In the Cloud Billing Console Cost breakdown report, you can now select the costs you want to analyze using the Time range and other report filters, such as projects, services, and SKUs.

For detailed insights behind the results of your cost breakdown report, view the Reports page. The cost breakdown report is linked to the Cloud Billing Reports page; the link uses the same time range and report filters you configure on your cost breakdown report. When you open the Reports page from your cost breakdown report, the report opens displaying the same totals as the cost breakdown report.

Along with the new report filters, the cost breakdown report now supports URL bookmarking and sharing. As you configure your cost breakdown report by setting the time range and other filters, the cost breakdown URL updates to include your selections. You can save your report settings by bookmarking the URL. You can share the cost breakdown report by copying the URL.

For more details about the cost breakdown report and using the new report filters and sharing feature, see the documentation.

Dataproc

Updated August 19, 2021 release notes with cluster creation Failure Action feature.

Google Cloud Deploy

Resource names, such as release name, are now validated for conformance with AIP-122. If you created any Google Cloud Deploy resources with names that don't conform, those resources might not work.

Google Kubernetes Engine

(2021-R29) Version updates

GKE cluster versions have been updated.

New versions available for upgrades and new clusters

The following Kubernetes versions are now available for new clusters and for opt-in control plane upgrades and node upgrades for existing clusters. For more information on versioning and upgrades, see GKE versioning and support and Upgrades.

No channel

Stable channel

  • Version 1.19.13-gke.701 is now the default version in the Stable channel.
  • Version 1.19.13-gke.1200 is now available in the Stable channel.
  • Version 1.20.9-gke.1000 is now available in the Stable channel.
  • The following versions are no longer available in the Stable channel:
    • 1.18.20-gke.901
    • 1.18.20-gke.3001
    • 1.19.12-gke.2101
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.19 to 1.19.13-gke.701 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.20 to 1.20.9-gke.1000 with this release.

Regular channel

  • Version 1.20.9-gke.1001 is now the default version in the Regular channel.
  • Version 1.20.10-gke.301 is now available in the Regular channel.
  • Version 1.20.9-gke.701 is no longer available in the Regular channel.
  • Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.20 to 1.20.9-gke.1001 with this release.

Rapid channel

  • Version 1.21.4-gke.301 is now the default version in the Rapid channel.
  • Version 1.21.4-gke.1801 is now available in the Rapid channel.
  • Version 1.21.3-gke.2001 is no longer available in the Rapid channel.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to 1.21.4-gke.301 with this release.

(2021-R29) Version updates

  • Version 1.20.9-gke.1001 is now the default version in the Regular channel.
  • Version 1.20.10-gke.301 is now available in the Regular channel.
  • Version 1.20.9-gke.701 is no longer available in the Regular channel.
  • Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.20 to 1.20.9-gke.1001 with this release.

(2021-R29) Version updates

  • Version 1.21.4-gke.301 is now the default version in the Rapid channel.
  • Version 1.21.4-gke.1801 is now available in the Rapid channel.
  • Version 1.21.3-gke.2001 is no longer available in the Rapid channel.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to 1.21.4-gke.301 with this release.

(2021-R29) Version updates

  • Version 1.19.13-gke.701 is now the default version in the Stable channel.
  • Version 1.19.13-gke.1200 is now available in the Stable channel.
  • Version 1.20.9-gke.1000 is now available in the Stable channel.
  • The following versions are no longer available in the Stable channel:
    • 1.18.20-gke.901
    • 1.18.20-gke.3001
    • 1.19.12-gke.2101
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.19 to 1.19.13-gke.701 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.20 to 1.20.9-gke.1000 with this release.

(2021-R29) Version updates

September 16, 2021

Anthos clusters on VMware

Anthos clusters on VMware 1.6.5-gke.0 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.6.5-gke.0 runs on Kubernetes 1.18.20-gke.4501.

The supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.8, 1.7, and 1.6.

Fixes:

BigQuery ML

BigQuery ML documentation has been updated with the following improvements:

Dataproc Metastore

For new projects, Dataproc Metastore service creations with CMEK enabled fail if a service without CMEK enabled has never been created before.

To work around this issue, create a service without CMEK enabled first.

Google Kubernetes Engine

In GKE versions 1.21.0-gke.1500 and later, VPC-native is the default network mode during cluster creation. To create a routes-based cluster, you can use the --no-enable-ip-alias flag:

gcloud container clusters create CLUSTER_NAME --no-enable-ip-alias

For Autopilot clusters, starting with GKE version 1.21.3-gke.900:

  • Users can also create mutating webhooks. However, Autopilot modifies the mutating webhooks objects to add a namespace selector which excludes the resources in managed namespaces (currently, kube-system) from being intercepted. Additionally, webhooks which specify one or more of following resources (and any of their sub-resources) in the rules, will be rejected:

    - group: ""
      resource: nodes
    - group: ""
      resource: persistentvolumes
    - group: certificates.k8s.io
      resource: certificatesigningrequests
    - group: authentication.k8s.io
      resource: tokenreviews
    
  • The SYS_PTRACE capability is allowed in user workloads.

  • Gatekeeper is no longer used in Autopilot policy enforcement, letting users install their own Gatekeeper instances.

When downgrading Autopilot clusters versions 1.21 to the older minor versions, the cluster might intermittently become unavailable. Once the downgrade is complete, the cluster will be available.

Identity and Access Management Identity-Aware Proxy

Security bulletin c2agxr12ne

Certain Google Cloud load balancers routing to an Identity-Aware Proxy enabled Backend Service could have been vulnerable to an untrusted party under limited conditions.

For details, see GCP-2021-020

Kf

Improved kf doctor reliability for Anthos on-prem clusters.

Fixed an error that can occur during the initialization of the subresource API.

Virtual Private Cloud

Enabling or disabling PROXY protocol after a Private Service Connect service attachment is created now correctly changes the configuration.

September 15, 2021

Anthos GKE on AWS

A security issue was discovered in Kubernetes, CVE-2021-25741, where a user may be able to create a container with subpath volume mounts to access files & directories outside of the volume, including on the host filesystem. For more information, see the GCP-2021-018 security bulletin.

Cloud Composer

Airflow 2 in Cloud Composer is now generally available (GA).

HA Scheduler in Cloud Composer is now generally available (GA).

We plan to stop offering Airflow 1.10.14 and 1.10.12 in new versions of Cloud Composer. Starting from December 2021, new releases of Cloud Composer will support only Airflow 1.10.15 and Airflow 2.x versions.

We plan to switch new versions of Cloud Composer with Airflow 1.10.15 to Python 3.8. Starting from November 2021, new Cloud Composer images with Airflow 1.10.15 will use Python 3.8 instead of Python 3.6. Please check for PyPI package compatibility before upgrading your environment to a version with Python 3.8.

Java Runtime in Airflow workers and schedulers is updated from version 8 to version 11.

The default value for the visibility-timeout Airflow configuration option, which determines the amount of time after which the task is re-picked by another worker, is increased from 6 hours to 7 days. This change fixes a problem with long-running tasks, when two Airflow workers attempt to write to one log file at the same time, causing tasks to fail with a logging exception.

Airflow has its own system for controlling task health, which is not dependent on visibility-timeout. If required, you can override the value of this configuration option for your environment.

If an Airflow configuration option is blocked in the image version specified for an upgrade operation, and this option has an override in your environment, the upgrade operation is rejected.

In Airflow UI, menu items for Configuration and DAG dependencies pages are now correctly displayed for users with the Op role.

New versions of Cloud Composer images:

  • composer-1.17.0-airflow-2.1.2
  • composer-1.17.0-airflow-2.0.2
  • composer-1.17.0-airflow-1.10.15 (default)
  • composer-1.17.0-airflow-1.10.14
  • composer-1.17.0-airflow-1.10.12
  • composer-2.0.0-preview.1-airflow-2.1.2
  • composer-2.0.0-preview.1-airflow-2.0.2

Airflow 2.1.1 is no longer included in Cloud Composer images.

Cloud Functions

Cloud Functions now supports PHP 7.4 at the General Availability release level.

Google Cloud Deploy

The 3-part cluster specification is no longer supported in target configuration. The only accepted format is now as follows:

gke:
  cluster: projects/[project_name]/locations/[location]/clusters/[cluster_name]

The promoteRelease API is removed. Users can call releases.rollouts.create to promote a release through the API.

In the release resource, the archive_uri output field is replaced with artifact_uri. This reflects the fact that the Skaffold configuration and rendered manifest are no longer stored together as a tar file in a Google Cloud Storage bucket. They are now stored as files in GCS, in a folder corresponding to each render operation.

SAP on Google Cloud

Google Cloud monitoring agent for SAP NetWeaver, version 2.0 is generally available

Version 2.0 of the monitoring agent for SAP NetWeaver is now generally available (GA). For Linux, version 2.0 of the monitoring agent provides a simpler installation and upgrade path that conforms to the standard OS-based package management. For Windows, only the service name is changed to google-sapnetweavermonitoring-agent.

The information collected by version 2.0 of the monitoring agent for SAP NetWeaver is the same as version 1.0.

For more information, see SAP NetWeaver Planning Guide.

Vertex AI

Vertex Explainable AI is generally available (GA).

September 14, 2021

Anthos Service Mesh

1.9.8-asm.3 and 1.10.4-asm.9 are now available.

These patch releases:

  • Introduced a rate limit to improve control plane availability under load spikes.
  • Fixed a memory leak and proxy count issue in the control plane.
Cloud SQL for MySQL

Cloud SQL for MySQL now supports custom formatting controls for CSVs. For more information on how to select custom characters for field delimiters, quotes, escapes, and other characters in admin exports and imports, see our documentation.

Cloud SQL for PostgreSQL

Cloud SQL for PostgreSQL has enhanced the support for multiline log entries in postgres.log. Before, when a log entry spanned multiple lines, each line was recorded as a separate entry in Cloud Logging. The lines are now recorded as a single entry in Cloud Logging for ease of query and processing.

Cloud SQL for PostgreSQL now supports custom formatting controls for CSVs. For more information on how to select custom characters for field delimiters, quotes, escapes, and other characters in admin exports and imports, see our documentation.

Cloud Shell

Cloud Shell is available directly in the Google Cloud documentation.

You can use this feature to activate Cloud Shell in the documentation and run sample code in the terminal on the page. For more information, see Launching within documentation.

The following list summarizes known issues that you might encounter:

  • You can only activate Cloud Shell in the documentation when you're using Chrome desktop browsers (version 74 or higher).
  • If Cloud Shell is activated and you open a site search result, the browser asks if you want to leave the site and then closes Cloud Shell.
  • If Cloud Shell is activated and you open a URL that redirects you to a different URL, your Cloud Shell session restarts.
Google Kubernetes Engine

With GKE versions 1.21.4-gke.30 and later, users can create ServiceAttachment resources to provision Private Service Connect (PSC) for internal LoadBalancer Services. This feature is available in Preview.

Multi-cluster Ingress now supports SSL policies and HTTPS redirects using the FrontendConfig resource. This feature is generally available in GKE versions 1.17.13-gke.2600 and later.

Security Command Center

Event Threat Detection, a built-in service of Security Command Center Premium, has launched new detectors in public preview.

The following detectors monitor your Google Workspace and Cloud Audit logs and alert you when external members are added to privileged Google Groups—groups that are granted sensitive IAM roles and permissions:

  • Credential Access: Privileged Group Joinability Risk: Detects when Google Groups are changed to be accessible to the general public
  • Persistence: IAM Anomalous Group Grant: Detects when sensitive roles are granted to privileged Google Groups with external members
  • Credential Access: External Member In Privileged Group: Detects when an external member is added to a privileged Google Group

The following detectors monitor your Admin Activity logs and alert you to suspicious changes in Compute Engine instances:

  • Persistence: Compute Engine Admin Added SSH Key: Detects modification of the Compute Engine instance metadata ssh key value on established instances
  • Persistence: Compute Engine Admin Added Startup Script: Detects modification of the Compute Engine instance metadata startup script value on established instances

The Persistence: IAM Anomalous Grant detector is enhanced and detects when sensitive roles are granted to users and service accounts.

For more information on Event Threat Detection findings, see Rules. To learn how Event Threat Detection monitors changes in Google Groups and defines sensitive roles, see Unsafe Google Group changes.

Virtual Private Cloud

Full control over which protocols are mirrored by Packet Mirroring is now available in General Availability.

Workflows

Call logging is available in Preview.

September 13, 2021

Cloud Run

You can now configure Cloud Run services to have CPU allocated for the entire lifetime of container instances. Pricing depends on the CPU allocation configuration. (Available in public preview.)

Cloud Run for Anthos

Preview: Installing Cloud Run for Anthos as an Anthos feature is now available as a Preview. Currently available for new clusters only.

This preview of Cloud Run for Anthos installs as an Anthos fleet component and requires Anthos Service Mesh. Learn more.

Compute Engine

Generally Available: NVIDIA® T4 GPUs are now available in the following additional regions and zones:

  • Las Vegas, Nevada,: us-west4-a,b
  • Los Angeles, California: us-west2-b,c

For more information about using GPUs on Compute Engine, see GPUs on Compute Engine.

Dataproc

New sub-minor versions of Dataproc images: 1.4.71-debian10, 1.4.71-ubuntu18, 1.5.46-centos8, 1.5.46-debian10, 1.5.46-ubuntu18, 2.0.20-centos8, 2.0.20-debian10, 2.0.20-ubuntu18

Added support for enabling/disabling Ubuntu Snap daemon with cluster property dataproc:dataproc.snap.enabled. The default value is "true". If set to "false", pre-installed Snap packages in the image won't be affected, but auto refresh will be disabled. Applies to all Ubuntu images.

HIVE-21018: Grouping/distinct on more than 64 columns should be possible. Applies to 2.0 images.

Eventarc

Eventarc can be configured for data location and is supported as a resource location.

Google Kubernetes Engine

GKE versions 1.19.14-gke.301 and later fix the issue with v1beta1 of the Backendconfig API, where a Cloud Armor security policy was inadvertently deleted from the backend Service of an Ingress resource.

For more information, see Kubernetes issue #1508 and the Ingress Known issues page.

Vertex AI Workflows

Connectors are now generally available (GA).

September 10, 2021

Cloud Asset Inventory

New resource types are now available.

The following resource types are now publicly available through the Analyze Policy APIs (AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning):

  • Container
    • k8s.io/Node
    • k8s.io/Pod
    • k8s.io/Namespace
    • rbac.authorization.k8s.io/Role
    • rbac.authorization.k8s.io/RoleBinding
    • rbac.authorization.k8s.io/ClusterRole
    • rbac.authorization.k8s.io/ClusterRoleBinding
Cloud SQL for MySQL

The Cloud SQL out-of-disk recommender is now generally available. This feature proactively generates recommendations that help you reduce the risk of downtime that might be caused by your instances running out of disk space.

Cloud SQL for PostgreSQL

The Cloud SQL out-of-disk recommender is now generally available. This feature proactively generates recommendations that help you reduce the risk of downtime that might be caused by your instances running out of disk space.

Cloud SQL for PostgreSQL now supports the min_wal_size flag. For more information about this flag, see the Cloud SQL for PostgreSQL flags documentation.

Cloud SQL for SQL Server

The Cloud SQL out-of-disk recommender is now generally available. This feature proactively generates recommendations that help you reduce the risk of downtime that might be caused by your instances running out of disk space.

Config Connector

Config Connector 1.61.0 is now available

Added the secuirtySettings field to ComputeBackendService

Added jitter to resource reconciliation reenqueue period to smooth out the traffic pattern

Fixed a bug in BigqueryJob that generates unexpected diff for 'kms_key_name'

Notebooks

Due to a recent change, the iam.serviceAccounts.actAs permission on the specified service account for the notebook instance is required for users to continue to have access to their notebook instances. The Google internal Inverting Proxy server that provides access to notebook instances now verifies that this permission is present before allowing users access to the JupyterLab URL. The JupyterLab URL this update covers is: *.notebooks.googleusercontent.com This update only applies to notebook instances in Single User mode and verifies that the assigned single user is authorized to execute code inside the notebook instance. Notebook instances running in Service Account or Project Editor mode already perform this verification via the Inverting Proxy server.

Vertex AI

Vertex Model Monitoring is generally available (GA).

When you perform custom training, you can access Cloud Storage buckets by reading and writing to the local filesystem. This feature, based on Cloud Storage Fuse, is available in Preview.

September 09, 2021

Cloud Billing

Cloud Billing Budgets & alerts now support configurable budget time periods, beyond monthly budgets

In the Cloud Billing Console Budgets & alerts settings, you can now specify the time period of your budgets. Using the Time range settings now available to budgets in the Cloud Console, you can configure the budget's time range to a calendar period or a custom date range, allowing you to create budgets to monitor spend for time frames beyond the default calendar month, such as a quarter, a year, or a custom date range that you specify.

With this update, you can create, view, and manage all budgets (monthly and non-monthly) in the Budgets & alerts page in the Cloud Console or by using the Cloud Billing Budget API.

For more information on budgets and alerts, see Create, edit, or delete budgets and budget alerts.

Cloud Composer

Cloud Composer 2 is available in Preview.

Cloud Composer 2 brings environments that scale automatically based on the demands of your workflows. For more information about Cloud Composer 2, see Major versions of Cloud Composer, Environment scaling, and Pricing pages in the documentation.

Cloud Composer 2 uses the following Cloud Composer images:

  • composer-2.0.0-preview.0-airflow-2.1.2
  • composer-2.0.0-preview.0-airflow-2.1.1
  • composer-2.0.0-preview.0-airflow-2.0.2
Cloud Functions Cloud Key Management Service

Cloud KMS now provides a library that conforms to the PKCS #11 standard, which enables working with existing applications that use the PKCS #11 API. See Library for PKCS #11 to learn more.

Cloud Monitoring

You can now collect JVM metrics from the Ops Agent, starting with version 2.2.0. For more information, see Monitoring third-party applications: JVM.

Deep Learning Containers

M79 release

  • Updated Pytorch 1.9 containers (they were not refreshed in the last release).
  • Updated Theia IDE (experimental) containers.
  • Node.js is pinned to >=12.14.1,<13.
  • M79 is the last release version that has JupyterLab 1.x installed. For the next release (M80), JupyterLab will be upgraded to 3.x for all Deep Learning VM Images, Deep Learning Containers, and Notebooks.
  • Fixed a bug in which the home folder in custom container VMs was owned by the root instead of Jupyter.
Deep Learning VM Images

M79 release

  • Updated Pytorch 1.9 images (they were not refreshed in the last release).
  • Updated Theia IDE (experimental) images.
  • Node.js is pinned to >=12.14.1,<13.
  • M79 is the last release version that has JupyterLab 1.x installed. For the next release (M80), JupyterLab will be upgraded to 3.x for all Deep Learning VM Images, Deep Learning Containers, and Notebooks.
  • Deep Learning VM Images in Cloud Marketplace have not been updated. They are planned to be refreshed during the next release.
  • Fixed a bug in which the home folder in custom container VMs was owned by the root instead of Jupyter.
Firestore Google Kubernetes Engine

The managed Filestore CSI driver for GKE is now available in GKE versions 1.21 and later to provision and manage Filestore instances for GKE workloads.

Network Intelligence Center

Firewall Insights now provides comprehensive analysis of whether your firewall rules are overly permissive. Through overly permissive rule insights, which are now in public preview, Firewall Insights identifies rules and attributes that could be made more strict and secure.

Overly permissive rule insights include the following:

  • Allow rules with no hits
  • Allow rules with unused attributes
  • Allow rules with overly permissive IP address or port ranges

Firewall Insights uses Firewall Rules Logging to identify these rules. It uses machine learning to predict future usage of overly permissive rules.

By default, the product analyzes the past six weeks when it identifies overly permissive rules. However, you can choose a different observation period.

For more information about overly permissive rule insights, see the Firewall Insights overview. For details about how to enable overly permissive rules, see Using Firewall Insights.

September 08, 2021

Artifact Registry

Maven, npm, and Python repositories are now generally available.

Storage and network egress charges apply to all formats that are in Preview or are generally available.

BigQuery

Deleting the metadata for a specific job using the bq command-line tool is now generally available (GA).

Session support for BigQuery is now in Preview. With sessions:

  • You can associate your SQL activities in a session across scripts and multi-statement transactions in BigQuery with a unique session identifier.
  • You can use session variables (for example, default timezone or dataset) and temporary tables throughout the life of the session and also across scripts and transactions
  • When you enable sessions, all actions performed across multiple sessions can be viewed using the SESSION_ID column now available in jobs INFORMATION_SCHEMA views.
Cloud Healthcare API

The Healthcare Natural Language API is generally available (GA).

Cloud SQL for MySQL

Cloud SQL for MySQL now allows you to specify mysqldump options during migration from external servers. For more information, see Configuring Cloud SQL to replicate from an external server and Using a managed import to set up replication from external databases.

Compute Engine

Preview: You can now review OS vulnerability report data, which is collected by VM Manager, from the Security Command Center. This feature is available for Security Command Center premium tier users. For more information, see View vulnerability report data.

Dataproc

The following previously released sub-minor versions of Dataproc images included a bug where the dataproc user account was broken. This prevented some Dataproc services from functioning properly, which resulted in features being unavailable. In particular, this prevented Jupyter from running in clusters with Personal Cluster Authentication enabled.

These sub-minor versions have been rolled back, and can only be used when updating existing clusters that already use them:

  • 1.4.66-debian10, 1.4.66-ubuntu18
  • 1.4.67-debian10, 1.4.67-ubuntu18
  • 1.5.41-centos8, 1.5.41-debian10, 1.5.41-ubuntu18
  • 1.5.42-centos8, 1.5.42-debian10, 1.5.42-ubuntu18
  • 2.0.15-centos8, 2.0.15-debian10, 2.0.15-ubuntu18
  • 2.0.16-centos8, 2.0.16-debian10, 2.0.16-ubuntu18
Google Kubernetes Engine

Several gcloud flags used to configure which logs and metrics are collected are deprecated and replaced with new flags. See Deprecated Configuration Parameters for a list of the deprecated logging and monitoring flags as well as the equivalent values for the new --logging and --monitoring flags.

Kf

Kf for Anthos on-prem (Vsphere) is now available on the Public Previews page.

Addressed a potential panic in the kf build-logs command.

Changed flag and manifest validation for route and task fields so manifest routes are ignored when creating tasks.

September 07, 2021

Cloud Build

Build triggers support for buildpacks is now generally available. To learn more, see Creating and managing build triggers.

Cloud Load Balancing

Cloud Load Balancing now supports load-balancing traffic to endpoints that extend beyond Google Cloud, such as on-premises data centers and other public clouds that you can reach using hybrid connectivity.

Hybrid load balancing is supported by the following load balancers:

  • External HTTP(S) Load Balancing
  • Internal HTTP(S) Load Balancing
  • TCP Proxy and SSL Proxy Load Balancing

For details, see Hybrid load balancing overview.

This feature is available in Preview.

Dataflow

Dataflow now supports Shielded VM workers.

Dataproc

Added additional messages to the error messages for networking and IAM errors when creating a new cluster.

Google Kubernetes Engine

The R28 release notes were updated on September 24, 2021 with the following additions:

No channel

Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.18 to 1.19.12-gke.2101 with this release.

Stable channel

Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.18 to 1.19.12-gke.2101 with this release.

(2021-R28) Version updates

GKE cluster versions have been updated.

New versions available for upgrades and new clusters

The following Kubernetes versions are now available for new clusters and for opt-in control plane upgrades and node upgrades for existing clusters. For more information on versioning and upgrades, see GKE versioning and support and Upgrades.

No channel

Stable channel

  • Version 1.19.12-gke.2101 is now the default version in the Stable channel.
  • The following control plane and node versions are now available in the Stable channel:
  • The following versions are no longer available in the Stable channel:
    • 1.18.20-gke.3000
    • 1.19.12-gke.2100
    • 1.19.13-gke.700
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.18 to 1.19.12-gke.2101 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.19 to 1.19.12-gke.2101 with this release.

Regular channel

  • Version 1.20.9-gke.701 is now the default version in the Regular channel.
  • The following control plane and node versions are now available in the Regular channel:
  • The following versions are no longer available in the Regular channel:
    • 1.20.9-gke.700
    • 1.20.9-gke.1000
  • Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.20 to 1.20.9-gke.701 with this release.

Rapid channel

  • Version 1.21.3-gke.2001 is now the default version in the Rapid channel.
  • The following control plane and node versions are now available in the Rapid channel:
  • Version 1.21.3-gke.2000 is no longer available in the Rapid channel.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to 1.21.3-gke.2001 with this release.

Two security vulnerabilities, CVE-2021-33909 and CVE-2021-33910, have been discovered in the Linux kernel that can lead to an OS crash or an escalation to root by an unprivileged user. This vulnerability affects all GKE node operating systems (COS and Ubuntu).

For more information, see the GCP-2021-017 security bulletin.

This note was updated on September 24, 2021. Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.18 to 1.19.12-gke.2101 with this release.

(2021-R28) Version updates

This note was updated on September 24, 2021. Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.18 to 1.19.12-gke.2101 with this release.

(2021-R28) Version updates

  • Version 1.19.12-gke.2101 is now the default version in the Stable channel.
  • The following control plane and node versions are now available in the Stable channel:
  • The following versions are no longer available in the Stable channel:
    • 1.18.20-gke.3000
    • 1.19.12-gke.2100
    • 1.19.13-gke.700
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.18 to 1.19.12-gke.2101 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.19 to 1.19.12-gke.2101 with this release.

(2021-R28) Version updates

  • Version 1.20.9-gke.701 is now the default version in the Regular channel.
  • The following control plane and node versions are now available in the Regular channel:
  • The following versions are no longer available in the Regular channel:
    • 1.20.9-gke.700
    • 1.20.9-gke.1000
  • Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.20 to 1.20.9-gke.701 with this release.

(2021-R28) Version updates

  • Version 1.21.3-gke.2001 is now the default version in the Rapid channel.
  • The following control plane and node versions are now available in the Rapid channel:
  • Version 1.21.3-gke.2000 is no longer available in the Rapid channel.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to 1.21.3-gke.2001 with this release.
Memorystore for Redis

Redis version 6.x is now Generally Available on Memorystore for Redis.

Security Command Center

VM Manager vulnerability reports, which are in preview, are now available in Security Command Center Premium. The reports identify vulnerabilities in operating systems installed on Compute Engine virtual machines, including Common Vulnerabilities and Exposures (CVEs).

For more information on integrating VM Manager with Security Command Center, see VM Manager.

Workflows

Support for callback endpoints is available in Preview.

September 06, 2021

Dataproc Metastore

The default Dataproc Metastore service creation version is changed to Hive 3.1.2.

September 05, 2021

Migrate for Compute Engine

Added support for overriding the default license type to explicitly specify a license type of PAYG or BYOL.

See Configuring the target for a migrated VM for more information.

September 03, 2021

Anthos clusters on VMware

Anthos clusters on VMware 1.7.3-gke.6 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.7.3-gke.X runs on Kubernetes v1.19.12-gke.1100

The supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.8, 1.7, and 1.6.

Fixes:

  • Fixed the Ubuntu user password expiration issue. This is a required fix for customers running 1.7.2 or 1.7.3-gke.2. Either use the suggested workaround to fix this issue, or upgrade to get this fix.

  • Fixed the issue that the stackdriver-log-forwarder pod was sometimes in crashloop because of fluent-bit segfault.

Cloud Functions

Cloud Functions has added support for a new runtime, Go 1.16, at the Preview release level.

Cloud Key Management Service

Re-importing previously destroyed keys is now supported in Cloud KMS.

Cloud Monitoring

Cloud Monitoring now lets you configure how long Monitoring waits to close an incident when observations stop arriving. For more information, see Closing incidents.

Cloud Talent Solution Job Search

Added a new KeywordMatchMode field to support more keyword matching options.

Added more DiversificationLevel configuration options.

Cloud VPN

Added Terraform examples to automate HA VPN gateway creation:

Config Connector

Config Connector 1.60.0 is now available.

Added support for ComputeFirewallPolicy resource.

Fixed the error when deleting the ConfigConnectorContext object. (Issue #523)

September 02, 2021

Cloud Data Fusion

Preview: Cloud Data Fusion version 6.5.0 is now available. This version is a Preview. This release is in parallel with the CDAP 6.5.0 release.

Features in 6.5.0:

  • Preview: Cloud Data Fusion now supports role-based access control (RBAC). This gives administrators fine-grained access control over what users can do at the namespace level.

  • Preview: Cloud Data Fusion now supports customer-managed encryption keys (CMEK), which provide user encryption control over the data written to Google internal resources in tenant projects, and data written by Cloud Data Fusion pipelines.

  • Preview: Cloud Data Fusion Instance Admins can now create, view, duplicate, delete, import, and export connections from the Pipeline Studio, Wrangler, or the Namespace Admin page. A connection stores sensitive data, such as user credentials and host information, needed to connect to data sources. For more information, see Managing connections.

  • Preview: Transformation pushdown is now available. It helps you efficiently design and execute ELT workloads by pushing join transformations down to BigQuery. It gives users that prefer ELT in BigQuery access to the same visual experience that ETL users get in Cloud Data Fusion, without needing to maintain complex SQL scripts. When you enable Transformation pushdown, Cloud Data Fusion executes Join operations in BigQuery (instead of Apache Spark). All other stages in a pipeline are executed using Spark. For pipelines that perform multiple complex joins, BigQuery can execute these joins operations faster than Spark.

  • Preview: Dataproc cluster reuse is now available. It can be used to speed up pipeline run startup by reusing clusters from previous runs.

Changes in 6.5.0:

  • In version 6.5.0, Spark 3 is the new default engine used when using Cloud Data Fusion Preview and when running pipelines on Dataproc clusters. After an instance is upgraded to version 6.5.0, any new or upgraded pipeline that uses a Dataproc profile without an explicit image version will use the latest Dataproc image 2.0 that has Spark 3.1 bundled. For more information, see Upgrade notes for Spark 3.

  • Added support for labels in the Dataproc provisioner.

  • Added Shielded VMs to the configuration settings for the Dataproc provisioner. For more information, see the CDAP documentation.

  • Added authorization checks for preferences, logging, compute profiles, and metadata endpoints.

  • Added support to search for tables based on schema name when you select tables for a Replication job.

  • Added additional trace logging in the authorization flow for debugging.

  • Added support for BIGNUMERIC data type for BigQuery target in replication.

  • Behavior change: MySQL, Oracle, Postgres, and SQL Server batch sources, sinks, actions, and pipeline alerts are now installed by default as system plugins. Previously, these plugins were available in the Hub as user plugins.

Fixed in 6.5.0 preview version (for more information, see the CDAP release note):

  • Fixed an issue in Replication that caused jobs to fail if more than 1000 tables were selected for replication.

  • Fixed an issue that caused replication jobs to hang when there were too many Delete or DDL events.

  • Fixed an issue that caused Wrangler to ignore all the other columns other than the given column when parsing Excel files.

  • Fixed Wrangler to fail pipelines upon error. In Wrangler 6.2 and above, there was a backwards-incompatible change where pipelines did not fail if there was an error and instead were marked as completed.

  • Improved resilience of TMS.

  • Fixed an issue that caused File Source Plugin validation to fail when there was a macro in the Format field.

You can create connections for Database, MySQL, Oracle, PostgreSQL, and SQL Server sources, but the plugin properties do not include Use Connection. This means that you cannot reference a connection in a database source plugin. For more information, see Known issues: Database connections.

Cloud SQL for MySQL

Cloud SQL for MySQL now supports using a custom import to set up replication from large external databases. To use this replication option, see Configuring Cloud SQL to replicate from an external server and Using a custom import to set up replication from large external databases.

Datastore

Added DATA_READ and DATA_WRITE Data Access audit logs. See Firestore in Datastore mode audit logging information. This feature is available in Preview.

Firestore

Added DATA_READ and DATA_WRITE Data Access audit logs. See Firestore audit logging information. This feature is available in Preview.

Google Kubernetes Engine

Multi-Instance GPU on GKE is is now generally available.

Network Connectivity Center

You can now create Router appliance spokes by using the Google Cloud Console. For more information, see Working with hubs and spokes.

Workflows

Support for iterating over a sequence of numbers or through a collection of data is generally available (GA).

September 01, 2021

AI Platform Prediction

Runtime version 2.6 is now available. You can use runtime version 2.6 to serve online predictions with TensorFlow 2.6.0, scikit-learn 0.24.2, or XGBoost 1.4.2. Runtime version 2.6 does not support batch prediction.

See the full list of updated dependencies in runtime version 2.6.

Anthos GKE on AWS

Anthos clusters on AWS aws-1.8.2-gke.2 is now available.

Anthos clusters on AWS aws-1.8.2-gke.2 clusters run the following Kubernetes versions:

  • 1.17.17-gke.15800
  • 1.18.20-gke.4800
  • 1.19.14-gke.600
  • 1.20.10-gke.600

The supported versions also offer the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on AWS 1.8.

Cloud Asset Inventory

New resource types are now available.

The following resource types are now publicly available through the Analyze Policy APIs (AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning):

  • Logging
    • logging.googleapis.com/LogBucket
    • logging.googleapis.com/LogSink
    • logging.googleapis.com/LogMetric
Cloud Build

VPC Service Controls support for build triggers is now available in the preview release stage. This feature enables users to use build triggers in projects in the VPC Service Controls perimeter. For instructions, see Using VPC Service Controls.

Compute Engine

Generally available: When deleting VMs from a managed instance group, you can flag the operation to continue even if some instances were already deleted or if other instance validation errors occur.

Dialogflow

New Dialogflow CX agent roles are introduced for granular control of agent resources.

Eventarc

Support for Cloud Storage triggers is now available in Preview.

August 31, 2021

Anthos clusters on VMware

Anthos clusters on VMware 1.8.2-gke.11 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.8.2-gke.11 runs on Kubernetes 1.20.9-gke.701.

The supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.8, 1.7, and 1.6.

Starting from version 1.8.2, Anthos clusters on VMware uses cert-manager instead of Istio Citadel for issuing TLS certificates used by metrics endpoints.

Fixes:

  • Fixed the Ubuntu user password expiration issue. You must get this fix. Either use the suggested workaround to fix this issue, or upgrade to get this fix.
  • Enhanced the admin cluster upgrade logic to prevent the admin cluster state (that is, the admin master data disk) from being lost in those cases when the disk is renamed or migrated accidentally.
  • Fixed the issue that the GKE connect-register service account key is printed in the klog in 1.8.0 and 1.8.1 when users run gkectl update cluster to update the GKE connect spec, such as to register an existing user cluster.
  • Fixed issue that when ESXi hosts were unavailable in the vCenter cluster (such as when disconnected from vCenter or in maintenance mode), the Cluster API controller and cluster health controllers would crash loop, and the gkectl diagnose cluster command would crash.
  • Fixed the issue that an admin cluster upgrade might be blocked indefinitely if admin node machines are upgraded before the new Cluster API controller is ready.
  • Fixed the issue that the onprem-user-cluster-controller might leak vCenter sessions over time.

  • Fixed the issue that the gateway IP was assigned to a Windows Pod, which made it unable to have network connectivity.

  • Fixed CVE-2021-33909 and CVE-2021-33910 on Ubuntu and COS.

HPA with custom metrics doesn't work in version 1.8.2 due to the migration from Istio to cert-manager for the monitoring pipeline. Customers using the HPA custom metrics with the monitoring pipeline should wait for a future release that will include this fix.

BigQuery

An updated version of ODBC driver for BigQuery is now available that includes enhancements.

Cloud Logging

You can now collect nginx metrics and logs from the Ops Agent, starting with version 2.1.0. For more information, see Monitoring third-party applications: nginx.

Cloud Monitoring

You can now collect nginx metrics and logs from the Ops Agent, starting with version 2.1.0. For more information, see Monitoring third-party applications: nginx.

Cloud Spanner

The R2DBC driver for Cloud Spanner is available in Preview. This driver lets you connect to Cloud Spanner from fully reactive applications.

Compute Engine

Generally available: You can now reference the latest available image in a public image family for a specific zone. This feature improves zonal fault tolerance for your workflows during Google image updates.

Dataflow

Dataflow Prime is now available in Preview.

SAP on Google Cloud

SAP HANA Fast Restart and Compute Engine M2 machines speed memory-error recovery

Compute Engine recently updated M2 VM types so that they can keep running when uncorrectable memory errors occur by using memory-poisoning recovery. Except for the block affected by the error, SAP HANA Fast Restart can then maintain all data in memory, which significantly reduces restart time because only the affected block is reloaded.

For more information, see Memory-error recovery with Fast Restart on Compute Engine VMs.

Transcoder API

All client library code samples updated to v1 of the API.

Delete operations for jobs and job templates now return a 404 resource not found error if the specified resource name does not exist. Previously, these delete operations failed silently in this case. To enable the previous behavior, set the allowMissing query parameter to true when sending a request to delete a job or job template.

Encryption support (and its associated documentation) is temporarily unavailable.

August 30, 2021

Access Approval

Access Approval supports Speaker ID in Preview stage.

BigQuery

Exporting table data in Parquet format is now generally available (GA).

Cloud Composer

Airflow 2.1.2 is available in Cloud Composer images.

(Airflow 2) Cloud Composer now supports the stable Airflow REST API. The stable Airflow REST API is enabled by default.

(Airflow 2) Cloud Composer now uses a custom authentication backend for authentication in the stable Airflow REST API. The default value of the [api]auth_backend Airflow configuration option is changed to airflow.composer.api.backend.composer_auth. The default role for new users that authenticate through the stable Airflow REST API is defined by the [api]composer_auth_user_registration_role Airflow configuration option, which is set to Op by default.

New versions of Cloud Composer images:

  • composer-1.16.16-airflow-1.10.12
  • composer-1.16.16-airflow-1.10.14
  • composer-1.16.16-airflow-1.10.15 (default)
  • composer-1.17.0-preview.12-airflow-2.0.2
  • composer-1.17.0-preview.12-airflow-2.1.1
  • composer-1.17.0-preview.12-airflow-2.1.2
Cloud Functions

Cloud Functions adds support for setting a minimum number of instances, available at the Preview release level. For more information, see the blog post.

Cloud Monitoring

The VM Instances page features enhanced scorecards for VM health. The new scorecards now include both "maintenance" and "system" events that might affect your VMs and agents, along with other metrics and statistics about the health of your VMs. The filtering and sorting of the Inventory table have also been enhanced.

Cloud SQL for PostgreSQL

Cloud SQL for PostgreSQL support for pglogical, native logical replication, wal2json and test_decoding is now generally available.

Cloud SQL for PostgreSQL support for Automatic IAM database authentication is now generally available. See Automatic IAM database authentication.

Cloud Spanner

In the Cloud Console, a database's Query page now supports multiple query tabs so you no longer have to clear one query to create and run another. Additionally, you can enter multiple query and DML statements in a single query tab. When you do so, the Results and Explanation subtabs let you choose which statement's results or query plan you want to view. See A tour of the query editor for details.

Added support for changing the leader region location of a Cloud Spanner database.

Added support for the JSON data type. For more information, see Working with JSON data.

Dataproc

New sub-minor versions of Dataproc images: 1.4.70-debian10, 1.4.70-ubuntu18, 1.5.45-centos8, 1.5.45-debian10, 1.5.45-ubuntu18, 2.0.19-centos8, 2.0.19-debian10, 2.0.19-ubuntu18

Backported SPARK-34295: Added a new spark.yarn.kerberos.renewal.excludeHadoopFileSystemsconfiguration option.

Image 2.0:

OOZIE-3599: Upgraded Jetty version to 9.4.

Dataproc Metastore

Hive version 3.1.2 will become the default Dataproc Metastore service creation version in 1 week on September 6, 2021.

Google Kubernetes Engine

GKE Autoscaling profiles are now generally available.

Traffic Director

Traffic Director deployed with proxyless gRPC can now use the advanced traffic management features retry and session affinity.

Vertex AI

You can now use a pre-built container to perform custom training with TensorFlow 2.6 and PyTorch 1.9.

August 27, 2021

Access Transparency

Document AI is supported by Access Transparency in GA stage.

Anthos on bare metal

Release 1.8.3

Anthos clusters on bare metal 1.8.3 is now available. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.8.3 runs on Kubernetes 1.20.

Breaking changes:

In Anthos clusters on bare metal release 1.8.0, we added a kernel version requirement for Ubuntu 18.04. We required a Linux kernel version of 4.17.0 or later. Anthos clusters on bare metal release 1.8.3 again supports all Linux kernel versions that ship with Ubuntu 18.04 and 20.04 distributions. As a result of this change, however, the egress NAT gateway feature that was provided for Preview in release 1.8.0 does not work with Anthos clusters on bare metal release 1.8.3.

Features:

  • Preview: Anthos Identity Service now works with Anthos clusters on bare metal to support LDAP authentication methods in addition to OIDC. You can use AIS with Microsoft Active Directory without the need for provisioning Active Directory Federation Services. For more information, see Setting up Anthos Identity Service with LDAP.

  • Preview: Anthos Metadata Agent replaces Stackdriver Metadata Collector and collects more accurate and usable metadata for Kubernetes resources. When you configure logging and monitoring, you need to enable the Config Monitoring for Ops API and grant the opsconfigmonitoring.resourceMetadata.writer IAM role to your logging-monitoring service account. If Anthos clusters on bare metal is installed behind a proxy, your proxy server must also allow connections to opsconfigmonitoring.googleapis.com.

  • Added preflight checks to verify that specific APIs are enabled for your Google Cloud project. Preflight checks return an error if any of the following APIs aren't enabled for your project:

    • anthos.googleapis.com
    • anthosaudit.googleapis.com
    • anthosgke.googleapis.com
    • cloudresourcemanager.googleapis.com
    • gkeconnect.googleapis.com
    • gkehub.googleapis.com
    • iam.googleapis.com
    • opsconfigmonitoring.googleapis.com
    • logging.googleapis.com
    • monitoring.googleapis.com
    • stackdriver.googleapis.com

    To enable these APIs when you create a cluster configuration file, use the --enable-apis flag with the bmctl create config command. For an example that uses the --enable-apis flag, see Create an admin cluster config with bmctl.

  • Added preflight checks for the following machine requirements:

    • Minimum supported Linux kernel version
    • Minimum required CPU
    • Minimum required RAM

Fixes:

  • Fixed the following container image security vulnerabilities:
  • Fixed cluster creation and cluster update failures for nodes running CentOS or Red Hat Enterprise Linux (RHEL) with both SELinux and Cloud Audit Logs enabled.
  • Fixed Transmission Control Protocol (TCP) connection leakage issue.
  • Fixed an issue that prevented cert-manager from issuing ACME certificates over HTTP due to ImagePullBackOff errors.

Changes:

  • The Kubevirt version used for working with VM-based workloads is now v0.43.0-gke.3.
  • The bootstrap cluster is deleted when a cluster upgrade completes without errors.

For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

Cloud Billing

The detailed usage cost data export to BigQuery is now generally available (GA).

The detailed export includes all of the data fields from the standard usage cost data export, along with additional fields that provide resource-level cost data. The resource-level cost data available in the detailed export is limited to Compute Engine resources, such as virtual machines or SSDs that generate service usage.

In your BigQuery dataset, the detailed usage cost data is exported to a table named gcp_billing_export_resource_v1_<BILLING_ACCOUNT_ID>.

The following are the newly available fields:

  • resource (Struct) - The fields that describe the structure and value of information relevant to service resources (like a virtual machine or a SSD) that generate service usage.
  • resource.global_name (String) - A globally unique service identifier for the resource that generated relevant usage.
  • resource.name (String) - A service-specific identifier for the resource that generated relevant usage. This can be input generated by the user. Note, the first full day of data with this field is August 11, 2021.

The daily cost detail data export to BigQuery is now the standard usage cost data export.

To understand the differences between the standard and detailed usage cost data exports to BigQuery, see the documentation.

Cloud SQL for PostgreSQL

Cloud SQL for PostgreSQL now supports the following flags:

  • huge_pages
  • shared_buffers
  • wal_buffers

For more information about these flags, see the Cloud SQL for PostgreSQL flags documentation.

Identity and Access Management

Managing Google Groups from the Cloud Console is now generally available.

Secret Manager

Cloud Audit Logs and Platform Logs are now available directly in the Secret Manager UI. See the Secret Manager page to learn more.

August 26, 2021

Anthos Config Management

kube-rbac-proxy has been removed since Hierarchy Controller does not expose any sensitive metrics, and kube-rbac-proxy is no longer actively maintained.

Fixed the issue causing a root or namespace reconciler to fail to be updated after switching from the auth type from none|gcenode|gcpserviceaccount to other types.

Fixed the issue causing Config Sync not to report sync errors when it fails to access Git repos.

Config Sync supports storing HTTPS/HTTP proxy credentials inside the git-creds Secret, using https_proxy or http_proxy as a key, to avoid exposing these credentials as plaintext.

Fixed the issue causing Config Sync not to pick up the latest schema of the CustomResourceDefinitions synced from Git repos.

Config Sync provides a way for users to override some system values:

  • Use the spec.override.resources field of a RootSync or RepoSync object to override the resource limits for the reconciler container and the git-sync container.
  • Use the spec.override.gitSyncDepth field of a RootSync or RepoSync object to override the number of git commits to fetch from the git repository.
  • Set the spec.git.noSSLVerify field of a RootSync or RepoSync object to true to disable Git SSL certificate verification.

Anthos Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 07e2fd0).

Cloud Asset Inventory

The Policy Analyzer page in the Cloud Console has been updated. To learn how to use the new UI, see Analyzing IAM policies.

Cloud Functions

Cloud Functions has added support for a new runtime, Node 16, at the Preview release level.

Dialogflow

During the week of September 6, 2021, two new Dialogflow IAM permissions will become effective: dialogflow.fulfillments.get and dialogflow.fulfillments.update . If you use custom roles to grant access to the Dialogflow ES console and API, make sure to add these permissions to your custom roles to allow to create and update Dialogflow ES Fulfillment in the Dialogflow ES console and via the API. If you don't use custom roles, no action is required.

August 25, 2021

Compute Engine

Generally available: You can now collect core dumps for uses such as debugging of unresponsive VMs. For more information, see Collecting core dumps.

Virtual Private Cloud

Private Service Connect service attachment deletions are now logged in Cloud Logging.

August 24, 2021

Anthos Service Mesh

1.10.4-asm.6 is now available.

This patch release contains the fixes for the security vulnerabilities listed in GCP-2021-016. For details on upgrading Anthos Service Mesh, refer to the following upgrade guides:

The asmcli script is now available in preview. With this script you can install and upgrade Anthos Service Mesh on GKE and On-premises. For more information, see About the asmcli.

Google-managed data plane is now available in preview as a part of managed Anthos Service Mesh. Google-managed data plane helps you upgrade data plane proxies automatically. For more information see Configure managed Anthos Service Mesh.

Anthos Service Mesh for Compute Engine VMs now uses gcloud commands and supports Google-managed control planes. For more information, see Add Compute Engine virtual machines to Anthos Service Mesh.

The Istio project recently disclosed a series of CVEs that can expose Anthos Service Mesh to remotely exploitable vulnerabilities. For more information, see the security bulletin.

Anthos Service Mesh now supports skip-version upgrades for single-project clusters on GKE running versions 1.7 and higher. This means you can now upgrade 1.7 and 1.8 installations directly to 1.10. For more information, see Upgrading Anthos Service Mesh to the latest version.

1.9.8-asm.1 is now available.

This patch release contains the fixes for the security vulnerabilities listed in GCP-2021-016. For details on upgrading Anthos Service Mesh, refer to the following upgrade guides:

1.8.6-asm.8 is now available.

This patch release contains the fixes for the security vulnerabilities listed in GCP-2021-016. For details on upgrading Anthos Service Mesh, refer to the following upgrade guides:

1.7.8-asm.10 is now available.

This patch release contains the fixes for the security vulnerabilities listed in GCP-2021-016. For details on upgrading Anthos Service Mesh, refer to the following upgrade guides:

Cloud Asset Inventory

New resource types are now available.

The following resource types are now publicly available through the Analyze Policy APIs (AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning):

  • Secret Manager
    • secretmanager.googleapis.com/Secret
    • secretmanager.googleapis.com/SecretVersion
Cloud Functions

Cloud Functions offers a native integration with Secret Manager, available at the Preview release level. For more information, see the blog post.

Cloud Run

Deploying to Cloud Run from source code is now at General Availability (GA).

Cloud TPU

Cloud TPU team just released TF-2.3.4, TF-2.4.3 and TF-2.5.1 on Cloud TPUs. The TensorFlow release notes for these releases are shown below.

Dataproc Metastore

Fixed the issue causing metadata changes introduced through imports and backups to not be reflected in Data Catalog due to broken batch sync.

Google Kubernetes Engine

Identity Service for GKE (Preview) is available. Identity Service for GKE extends existing identity solutions for authentication into GKE clusters by supporting OpenID Connect (OIDC). For more information, see Authenticating with Identity Service for GKE.

You can now enable Google Virtual NIC in a new GKE cluster on GPU nodes. For more information, see Using Google Virtual NIC.

Istio on Google Kubernetes Engine

1.4.10-gke.17 is now available. This patch release contains the fixes for the security vulnerabilities listed in GCP-2021-016. For more information see Upgrading 1.4 Istio to the latest patch release.

1.6.14-gke.5 is now available. This patch release contains the fixes for the security vulnerabilities listed in GCP-2021-016. For more information, see Upgrading operator based 1.6 Istio to the latest patch release.

The Istio project recently disclosed a series of CVEs that can expose Istio on GKE to remotely exploitable vulnerabilities. For more information, see the security bulletin.

Vertex AI

The following tools for creating embeddings to use with Vertex Matching Engine are available in Preview:

August 23, 2021

BigQuery

BigQuery Admin Resource Charts are now generally available (GA) for reservation users, enabling administrators to more easily monitor and troubleshoot their BigQuery environment. They provide visibility into key metrics such as slot consumption, job concurrency, job execution time, job errors, and bytes processed across the entire organization.

BigQuery Slot Estimator is now in Preview for reservation users. This tool analyzes slot utilization data to help administrators estimate the right number of slots to purchase, and provides insights on how job performance might be impacted by adding or reducing slot capacity for the entire organization or specific reservations.

Cloud Billing

Proportional attribution for spend-based committed use discounts is now generally available (GA).

Proportional attribution applies the subscription fees from your committed use discounts to the projects in your Cloud Billing account, directly in proportion to the amount of eligible credit consumed by each project. Any subscription fees that are not attributed to a project are charged at the Cloud Billing account level.

Starting from August 2021, all spend-based commitments you purchase for any of your Cloud Billing accounts use proportional attribution by default. If you purchased spend-based commitments before then, you can request that they be converted from account to proportional attribution.

To understand proportional attribution for your spend-based commitments and how to enable it, see the documentation.

Cloud Composer

The default value for the [core]parallelism Airflow configuration option is set to match the maximum worker concurrency in Airflow. Before this change, some tasks could be present in the queue but not executed, and then fail after reaching the timeout.

Non-error output of CloudSQL Proxy instances is routed to stdout instead of stderr. Regular CloudSQL Proxy output is no longer marked as errors in container logs.

(Airflow 2.1.1) Airflow configuration options that are deprecated in Airflow 2 are now blocked. In Airflow 2, these options are located in a different configuration section, but Airflow still uses the deprecated location as a fallback.

(Airflow 2) The airflow_db connection used by Airflow to connect to the Airflow database now correctly uses PostgreSQL connection type for Airflow 2 environments.

New versions of Cloud Composer images:

  • composer-1.16.15-airflow-1.10.12
  • composer-1.16.15-airflow-1.10.14
  • composer-1.16.15-airflow-1.10.15 (default)
  • composer-1.17.0-preview.11-airflow-2.0.2
  • composer-1.17.0-preview.11-airflow-2.1.1

Cloud Composer 1.11.3 has reached its end of full support period.

Cloud Load Balancing

Added Terraform examples to automate load balancer configuration:

Dataproc

New sub-minor versions of Dataproc images: 1.4.69-debian10, 1.4.69-ubuntu18, 1.5.44-centos8, 1.5.44-debian10, 1.5.44-ubuntu18, 2.0.18-centos8, 2.0.18-debian10, and 2.0.18-ubuntu18.

Configured YARN ResourceManager to use port 8554 and Druid to use port 17071 for JMX Remote RMI port.

Google Cloud Armor

Google Cloud Armor now has rate-based throttling and ban rules that enable you to limit requests from clients. These rules help you protect your applications from a large volume of requests that flood your instances and block access for legitimate users.

Network Connectivity Center

The Spoke Admin role now includes the following permissions:

  • networkconnectivity.hubs.get
  • networkconnectivity.hubs.list

For full details about Network Connectivity Center permissions, see Access control.

Virtual Private Cloud

August 21, 2021

Cloud SQL for MySQL

Cloud SQL now supports IAM Conditions.

You can use IAM Conditions to define and enforce conditional, attribute-based access control for Google Cloud resources, including Cloud SQL instances. See Overview of IAM Conditions for more information.

Cloud SQL for PostgreSQL

Cloud SQL now supports IAM Conditions.

You can use IAM Conditions to define and enforce conditional, attribute-based access control for Google Cloud resources, including Cloud SQL instances. See Overview of IAM Conditions for more information.

Cloud SQL for SQL Server

Cloud SQL now supports IAM Conditions.

You can use IAM Conditions to define and enforce conditional, attribute-based access control for Google Cloud resources, including Cloud SQL instances. See Overview of IAM Conditions for more information.

August 20, 2021

Cloud Key Management Service

MAC keys are now supported by Cloud KMS. See Creating and validating MAC digital signatures to learn more.

Cloud KMS now supports a configurable variable soft deletion window for cryptographic keys.

You can now retrieve random bytes from the random number generator in Cloud HSM. See Generating random bytes to learn more.

Cloud Spanner

Cloud Spanner now creates dedicated backup jobs to take backups instead of using an instance's server resources. As a result, backup time is reduced and backup operations do not affect instance performance.

Views are now supported in Cloud Spanner databases. Use views to provide logical data-modeling to applications, to centralize query definitions and simplify maintenance, and to ensure stability of query definitions across schema changes. Learn more.

Dataproc Metastore Document AI

Managing processor versions

You can now switch between different versions of a processor. For more information, see Managing processor versions.

New processor versions

We have added new versions of the following processors:

  • Bank statement parser: improved model quality
  • Pay slip parser: improved model quality and extraction of three additional fields: net_pay, net_pay_ytd, and employee_account_number.

New Lending DocAI processors

The following Lending DocAI (LDAI) processors are now available in limited Preview:

  • 1065 parser
  • 1099-NEC parser
  • 1099-R parser
  • 1120 parser
  • 1120-S parser
  • SSA-1099 parser

Additionally, the LDAI Document Splitter and Classifier has been updated to support the new LDAI processors as well as the following processors:

  • US Driver License Parser
  • US Passport Parser

Human in the Loop (HITL) support for Lending DocAI processors

The following Lending DocAI processors now support Human in the Loop (HITL):

  • 1003 parser
  • 1040 Parser
  • 1040 Schedule C parser
  • 1040 Schedule E parser
  • 1099-DIV parser
  • 1099-G parser
  • 1099-INT parser
  • 1099-MISC parser
  • Bank Statement parser
  • Pay Stub parser
  • W2 parser
  • W9 parser

Knowledge Graph support

The following processors now support Knowledge Graph enrichment:

  • Bank Statement
  • Pay Slip
  • W2 Parser
  • W9 Parser
Google Cloud Deploy

Cloud Deploy now supports configuring custom execution environments for render and deploy activity. You can choose the default pool or a private pool, default or custom service account, and the default or alternative Cloud Storage bucket.

Google Cloud VMware Engine

Preview: VMware Engine integration with Google Cloud's operations suite using a standalone metrics and logs agent. The agent brings syslog messages and metrics from vCenter and vSAN to Google Cloud's operations suite, where you can set up your own dynamic alerts on over 50 metrics and leverage pre-built dashboards.

For details about this feature, see Setting up Cloud Monitoring with a standalone agent.

Google Kubernetes Engine

(2021-R27) Version updates

GKE cluster versions have been updated.

New versions available for upgrades and new clusters

The following Kubernetes versions are now available for new clusters and for opt-in control plane upgrades and node upgrades for existing clusters. For more information on versioning and upgrades, see GKE versioning and support and Upgrades.

No channel

  • Version 1.20.8-gke.2100 is now the default version.
  • The following control plane and node versions are now available:
  • The following control plane versions are no longer available:
    • 1.18.20-gke.501
    • 1.19.9-gke.1900
    • 1.19.10-gke.1000
    • 1.19.10-gke.1001
    • 1.19.10-gke.1601
    • 1.19.10-gke.1701
    • 1.19.11-gke.1701
    • 1.19.11-gke.2101
    • 1.19.12-gke.700
    • 1.19.12-gke.900
    • 1.19.12-gke.1100
    • 1.20.8-gke.700
  • Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.18 to 1.18.20-gke.901 with this release.
  • Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.19 to 1.19.12-gke.2100 with this release.

Stable channel

There are no new releases in the Stable channel.

Regular channel

  • Version 1.20.8-gke.2100 is now the default version in the Regular channel.
  • Version 1.20.9-gke.700 is now available in the Regular channel.
  • Version 1.20.8-gke.900 is no longer available in the Regular channel.
  • Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.20 to 1.20.8-gke.2100 with this release.

Rapid channel

  • Version 1.20.8-gke.2100 is now the default version in the Rapid channel.
  • The following control plane and node versions are now available in the Rapid channel:
  • The following versions are no longer available in the Rapid channel:
    • 1.20.8-gke.2100
    • 1.21.3-gke.900
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.20 to 1.20.8-gke.2100 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to 1.21.3-gke.901 with this release.

For GKE clusters running Windows Server node pools, you can proactively receive updates about new GKE versions and the Windows OS versions they use by subscribing to UpgradeAvailableEvent notifications. This feature is now available in Preview.

(2021-R27) Version updates

There are no new releases in the Stable channel.

(2021-R27) Version updates

  • Version 1.20.8-gke.2100 is now the default version in the Regular channel.
  • Version 1.20.9-gke.700 is now available in the Regular channel.
  • Version 1.20.8-gke.900 is no longer available in the Regular channel.
  • Control planes and nodes with auto-upgrade enabled in the Regular channel will be upgraded from version 1.20 to 1.20.8-gke.2100 with this release.

(2021-R27) Version updates

  • Version 1.20.8-gke.2100 is now the default version in the Rapid channel.
  • The following control plane and node versions are now available in the Rapid channel:
  • The following versions are no longer available in the Rapid channel:
    • 1.20.8-gke.2100
    • 1.21.3-gke.900
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.20 to 1.20.8-gke.2100 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to 1.21.3-gke.901 with this release.

(2021-R27) Version updates

  • Version 1.20.8-gke.2100 is now the default version.
  • The following control plane and node versions are now available:
  • The following control plane versions are no longer available:
    • 1.18.20-gke.501
    • 1.19.9-gke.1900
    • 1.19.10-gke.1000
    • 1.19.10-gke.1001
    • 1.19.10-gke.1601
    • 1.19.10-gke.1701
    • 1.19.11-gke.1701
    • 1.19.11-gke.2101
    • 1.19.12-gke.700
    • 1.19.12-gke.900
    • 1.19.12-gke.1100
    • 1.20.8-gke.700
  • Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.18 to 1.18.20-gke.901 with this release.
  • Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.19 to 1.19.12-gke.2100 with this release.

August 19, 2021

BigQuery

Cloud Spanner federated queries are now generally available (GA).

Channel Services

The ImportCustomer method is now generally available.

Cloud DNS

Managing routing policies in Cloud DNS is available in preview.

Config Connector

Config Connector 1.59.0 is now available

Added networkConfig field into ContainerNodePool

Added processingUnits field into SpannerInstance

config-connector CLI supports IAMPartialPolicy as an IAM output format

Fixed the issue where ComputeInstance fails reconciliation if metadata is set outside KCC (Issue #524)

Dataproc

Added support for Dataproc Metastore in three recently turned up regions: .europe-west1, northamerica-northeast1, and asia-southeast1.

Users can now help assure the successful creation of a cluster by automatically deleting any failed primary workers (the master(s) and at least two primary workers must be successfully provisioned for cluster creation to succeed). To delete any failed primary workers when you create a cluster:

  1. Using gcloud: Set the gcloud dataproc clusters create --action-on-failed-primary-workers flag to "DELETE".

  2. Using the Dataproc clusters.create API: Set the actionOnFailedPrimaryWorkers field to "DELETE".

Dataproc issues a warning message if the staging or test bucket name contains an underscore.

Google Kubernetes Engine

A simplified GKE API for configuring which logs and metrics are collected and sent to Cloud Logging and Cloud Monitoring is now available. The gcloud container clusters create and gcloud container clusters update commands now support the --logging and --monitoring flags.

For example, to collect both system and workload logs in an existing cluster, use gcloud container clusters update --logging=SYSTEM,WORKLOAD. Or, to create a new cluster with no metrics collected, use gcloud container clusters create --monitoring=NONE.

See a complete list of available logs and available metrics.

These flags are available in Google Cloud SDK version 352.0.0 and later.

August 18, 2021

Cloud Asset Inventory

Starting September 30, 2021, the fields that are bytes in the export output for both Cloud Storage and BigQuery and in real-time notification will be encoded using the web-safe base64 algorithm. They are currently encoded using the base64 algorithm.

Cloud Build

Cloud Build approvals are now generally available, allowing users to configure triggers that only execute a build when granted approval by a set of users. To learn how to create a trigger gated by approval, see Creating and managing triggers. To learn how to approve a build, see Approving builds.

Cloud Healthcare API

An interactive version of the Cloud Healthcare API quickstart is now available in the Cloud Console. Access the quickstart.

Cloud Logging

You can now use Chef to install and manage the Google Cloud operations suite agents across your fleet of Linux and Windows VMs. For more information, refer to the Chef Integration documentation.

Cloud Monitoring

You can now use Chef to install and manage the Google Cloud operations suite agents across your fleet of Linux and Windows VMs. For more information, refer to the Chef Integration documentation.

Deep Learning Containers

M78 Release

  • Updated TensorFlow Enterprise patch version 2.3.3 to 2.3.4.

TensorFlow Enterprise 2.5

  • TensorFlow Enterprise 2.5 Deep Learning Containers are now deprecated.
Deep Learning VM Images

M78 Release

  • Updated TensorFlow Enterprise patch version 2.3.3 to 2.3.4.
  • Fixed a bug that prevented users from exporting a notebook as a PDF.
  • Fixed a bug that caused some users to be unable to SSH into their host machines.

TensorFlow Enterprise 2.5

  • TensorFlow Enterprise 2.5 Deep Learning VM images are now deprecated.
Google Kubernetes Engine

GKE clusters running node pools that use containerd might experience IP leak issues and exhaust all Pod IPs on a node. A Pod scheduled on an affected node shows an error message similar to the following:

failed to allocate for range 0: no IP addresses available in range set: 10.48.131.1-10.48.131.62

For more information about the issue, see containerd issue #5438 and issue #5768.

For workarounds to mitigate this issue, see the Known issues section in containerd node images.

Tensorflow Enterprise

TensorFlow Enterprise 2.3 has been updated to 2.3.4 from 2.3.3.

TensorFlow Enterprise 2.5 Deep Learning VM images and Deep Learning Containers are now deprecated.

August 17, 2021

App Engine standard environment Java

Upgraded Jetty to version 9.4.43.v20210629

App Engine standard environment Node.js

The NodeJS 16 runtime for App Engine standard environment is now available in Public Preview.

Cloud SQL for MySQL

MySQL 5.7.33 has been upgraded to 5.7.34.

Cloud SQL for PostgreSQL

Cloud SQL now offers faster maintenance, with connectivity dropping for less than 30 seconds on average.

Cloud Shell

Cloud Code Extension updated to v1.13.1

Updates includes a simplified combined Kubernetes Explorer and GKE Explorer experience, Managed Cloud SDK performance improvements, as well as improved Cloud Run local development webview. Review the Cloud Code release notes for a complete list of features/updates/bug fixes.

Cloud Shell Editor is now built with Theia 1.16.0

Review the Theia release notes for a complete list of features/updates/bug fixes.

Cloud Shell Editor Built-in Extensions Updated to 1.45.1

All built-in VS Code Extensions used by the Cloud Shell Editor have been updated to 1.45.1 to incorporate upstream changes.

The Flutter SDK now comes preinstalled in Cloud Shell.

Visit the Flutter Docs or simply enter flutter in Cloud Shell to get started.

.Net 5.0 now comes preinstalled in Cloud Shell.

Checkout the dotnet-five documentation for more details.

Cloud Spanner

Released Query Optimizer version 3. Query Optimizer v3 is currently set to off by default in production.

Eventarc

Support for 17 new regions. See Eventarc locations.

Google Kubernetes Engine

An issue was identified with v1beta1 of the BackendConfig API, where a Cloud Armor security policy was inadvertently deleted from the backend Service of an Ingress resource on the following affected GKE versions:

  • 1.18.19-gke.1400 and later
  • 1.19.10-gke.700 and later
  • 1.20.6-gke.700 and later

To fix this issue, use v1 of the BackendConfig API, or update your clusters to one of the following GKE versions:

  • 1.20.9-gke.900 and later
  • 1.21.1-gke.2700 and later

For more information, see Kubernetes issue #1508 and the Ingress Known issues page.

Kf

Kf CLI can now override the manifest.yaml app name.

When deleting resources, Kf will now delete any sub resources before returning success on the deletion request.

Migrate for Anthos and GKE

Version 1.8 added the initial support for the preview release of the enhanced runtime, which lets you deploy containers to GKE Autopilot clusters and to Cloud Run. This release adds the following new features to the preview:

  • You no longer set an annotation in the migration plan to enable the enhanced runtime. Instead, you now set v2kServiceManager.
  • The environment variable HC_GAMMA_RUNTIME has been renamed to HC_V2K_SERVICE_MANAGER.
  • The prestart and poststart entries in the config.yaml file now automatically populated.
  • Added support to the config.yaml file that lets you specify environment variables at the global level or at the application level.
  • Added logging support that lets you customize log data written to Cloud Logging.

See Enhanced runtime for more on these new features

Version 1.8 added the initial support for the preview release of the fit assessment tool. The fit assessment tool for version 1.8.1 adds new functionality, including:

  • Ability to collect data for a Windows VM
  • Ability to remotely collect data for Linux and Windows VMs using VMware tools
  • Ability to remotely collect data over SSH

See Using the fit assessment tool for more.

When you generate the migration artifacts, Migrate for Anthos and GKE now generates the new logs.yaml file from the migration plan. This file contains the list of log files detected on the source VM. You can now edit the logs.yaml file to configure logging and the data written to Cloud Logging.

See Customizing log data written to Cloud Logging for more.

Added support for specifying connection strings when migrating a Windows workload. Connection strings define a connection from the migrated container workload to a .NET Framework data provider.

See Setting connection strings for a data provider for more.

The cos-runtime option to the migctl setup install command has been renamed to runtime.

179171930: A migrated container workload can now be deployed to a cluster running GKE 1.20 and later.

Before you run your migrated workloads, you must install migctl with runtime support for Container-Optimized OS nodes on your cluster:

migctl setup install --runtime

See Deploying a Linux workload to a target cluster for more information.

166014117 : The documentation has been updated to describe how to delete the migration to free up the source VM after a successful migration. See Deleting a migration for more.

183082390: The collection script used by the Linux discovery tool uses service --status-all to query system V services. This call no longer takes an arbitrary amount of time to return.

194186514: When using Anthos clusters on AWS as the processing cluster to perform migrations of AWS workloads, if you have insufficient credentials to create an ECR repository, sometimes the migration succeeds. However, the ECR repository is not created.

Workaround: Update your credentials, then recreate and retry the migration.

197206783: The user credentials passed to the mfit discover ssh ... command must be the credentials of the root user on the VM. Running the command as a non-root user executes the command successfully, but only collects a small part of the data required for a full assessment.

Virtual Private Cloud

If you are using Private Service Connect endpoints to access services in another VPC network, deleting an endpoint no longer fails if you try to delete multiple endpoints in a short period of time.

August 16, 2021

Anthos on bare metal

Release 1.7.3

Anthos clusters on bare metal 1.7.3 is now available. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.7.3 runs on Kubernetes 1.19.

Fixes:

The following container image security vulnerabilities have been fixed:

Known issues:

When you upgrade Anthos clusters on bare metal from a version with a security patch to the next minor release, we recommend you upgrade to the highest patch version to ensure you have the latest security fixes. Always review the release notes before upgrading so you're aware of what has changed, including security fixes and known issues. Upgrading to a lower release version isn't supported.

For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

Cloud Data Fusion

SQL Server source plugin version 1.5.5 is now available. This version fixes a NullPointerException bug that occurs in version 1.5.4. Versions 1.5.4 and above support the Datetime data type. In versions 1.5.3 and earlier, if you had a Datetime column in your SQL Server source, it mapped to the Timestamp data type. Upgrades to version 1.5.4 are backwards incompatible, but upgrades to version 1.5.5 are compatible. For more information, see Troubleshooting and the CDAP SQL Server Batch Source.

Compute Engine

Preview: Manually live migrate VMs from one host to another. For more information, see Manually live migrate sole-tenant VMs.

Google Cloud Armor

Google Cloud Armor integration with reCAPTCHA Enterprise is now available in Public Preview. See the Cloud Armor bot management overview and the Overview of reCAPTCHA Enterprise for WAF and Google Cloud Armor integration.

reCAPTCHA Enterprise

reCAPTCHA Enterprise for WAF and Google Cloud Armor integration is now available in Public Preview. For more information, see the Overview of reCAPTCHA Enterprise for WAF and Google Cloud Armor integration and Cloud Armor bot management overview.

August 13, 2021

Anthos on bare metal

Release 1.6.4

Anthos clusters on bare metal 1.6.4 is now available. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.6.4 runs on Kubernetes 1.18.

Fixes:

The following container image security vulnerabilities have been fixed:

Known issues:

When you upgrade Anthos clusters on bare metal from a version with a security patch to the next minor release, we recommend you upgrade to the highest patch version to ensure you have the latest security fixes. Always review the release notes before upgrading so you're aware of what has changed, including security fixes and known issues. Upgrading to a lower release version isn't supported.

For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

App Engine standard environment Go

The Go 1.16 runtime for App Engine standard environment is now available in Public Preview.

Dataproc

New sub-minor versions of Dataproc images: 1.4.68-debian10, 1.4.68-ubuntu18, 1.5.43-centos8, 1.5.43-debian10, 1.5.43-ubuntu18, 2.0.17-centos8, 2.0.17-debian10, and 2.0.17-ubuntu18.

Upgrade Flink to version 1.12.5 in image 2.0.

HIVE-2527: Fixed slow Hive partition deletion for Cloud Object Stores with expensive ListFiles.

Fixed Jupyter startup on Personal Auth clusters on all images.

August 12, 2021

Cloud Composer

Cloud Composer is now compliant with Access Transparency on the GA level.

(Airflow 2.1.1) Downgraded colorlog dependency from 5.0.1 to 4.8.0

(Airflow 2.1.1) downgraded google-cloud-memcache dependency from 1.1.0 to 1.0.0

New versions of Cloud Composer images:

  • composer-1.16.14-airflow-1.10.12
  • composer-1.16.14-airflow-1.10.14
  • composer-1.16.14-airflow-1.10.15 (default)
  • composer-1.17.0-preview.10-airflow-2.0.2
  • composer-1.17.0-preview.10-airflow-2.1.1

Cloud Composer 1.11.2 has reached its end of full support period.

Cloud Monitoring

Cloud Monitoring now supports creating an uptime check against a Kubernetes LoadBalancer Service hosted on GKE. For more information, see Managing uptime checks.

Cloud TPU

Cloud TPU now supports Tensorflow 2.6.0. For more information, see Tensorflow 2.6.0 Release Notes.

In TF 2.6.0, TensorFlow has introduced a new version of the TF/XLA bridge using the MLIR compiler infrastructure. The MLIR bridge is enabled by default. To explicitly disable it at runtime, add the following code snippet to your model's code:

tf.config.experimental.disable_mlir_bridge()

Config Connector

Config Connector 1.58.1 is now available.

Miscellaneous bug fixes.

Google Kubernetes Engine

(2021-R26) Version updates

GKE cluster versions have been updated.

New versions available for upgrades and new clusters

The following Kubernetes versions are now available for new clusters and for opt-in control plane upgrades and node upgrades for existing clusters. For more information on versioning and upgrades, see GKE versioning and support and Upgrades.

No channel

  • The following control plane and node versions are now available:
  • The following control plane versions are no longer available:
    • 1.17.17-gke.3700
    • 1.17.17-gke.4400
    • 1.17.17-gke.4900
    • 1.17.17-gke.5400
    • 1.17.17-gke.6000
    • 1.17.17-gke.6700
    • 1.17.17-gke.7200
    • 1.17.17-gke.7800
    • 1.17.17-gke.8200
  • Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.20 to 1.20.8-gke.900 with this release.

Stable channel

  • Version 1.19.12-gke.2100 is now the default version in the Stable channel.
  • Version 1.18.20-gke.901 is now available in the Stable channel.
  • Version 1.19.11-gke.2101 is no longer available in the Stable channel.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.18 and version 1.19 to 1.19.12-gke.2100 with this release.

Regular channel

Version 1.20.8-gke.2100 is now available in the Regular channel.

Rapid channel

  • The following control plane and node versions are now available in the Rapid channel:
  • The following control plane and node versions are no longer available in the Rapid channel:
    • 1.20.9-gke.700
    • 1.21.3-gke.100
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to 1.21.3-gke.900 with this release.

(2021-R26) Version updates

  • The following control plane and node versions are now available:
  • The following control plane versions are no longer available:
    • 1.17.17-gke.3700
    • 1.17.17-gke.4400
    • 1.17.17-gke.4900
    • 1.17.17-gke.5400
    • 1.17.17-gke.6000
    • 1.17.17-gke.6700
    • 1.17.17-gke.7200
    • 1.17.17-gke.7800
    • 1.17.17-gke.8200
  • Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.20 to 1.20.8-gke.900 with this release.

(2021-R26) Version updates

  • Version 1.19.12-gke.2100 is now the default version in the Stable channel.
  • Version 1.18.20-gke.901 is now available in the Stable channel.
  • Version 1.19.11-gke.2101 is no longer available in the Stable channel.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.18 and version 1.19 to 1.19.12-gke.2100 with this release.

(2021-R26) Version updates

Version 1.20.8-gke.2100 is now available in the Regular channel.

(2021-R26) Version updates

  • The following control plane and node versions are now available in the Rapid channel:
  • The following control plane and node versions are no longer available in the Rapid channel:
    • 1.20.9-gke.700
    • 1.21.3-gke.100
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to 1.21.3-gke.900 with this release.

August 11, 2021

Access Transparency

Cloud Composer is supported by Access Transparency in GA stage.

Deep Learning Containers

M77 release

TensorFlow Enterprise 2.6.0 is now available and includes Long Term Version Support.

Deep Learning VM Images

M77 release

TensorFlow Enterprise 2.6.0 is now available and includes Long Term Version Support.

Dialogflow

Dialogflow CX GA (generally available) launch of Experiments to compare the performance of flow versions to a control version while handling live traffic.

Google Cloud Deploy

Deployment targets are now shareable among multiple delivery pipelines. See Target definitions for more information.

Security Command Center

Event Threat Detection, a built-in service of Security Command Center Premium, has launched new detectors to protect your Google Workspace domains in general availability. The detectors identify suspicious activities in member accounts and your Admin Console, including leaked passwords, attempted account breaches, settings changes, and possible government-backed attacks. For more information, see Event Threat Detection overview.

Container Threat Detection, a built-in service of Security Command Center Premium, has launched a new detector, Malicious Script Executed, in general availability. The detector uses natural language processing to evaluate bash scripts and determine if they are malicious. For more information, see Container Threat Detection overview

Security Command Center findings now include two new attributes that provide additional information about the type of finding and the activity that triggered it. The attributes include the following:

  • Indicator: displayed as indicator. This is an indicator of compromise (IoC), or artifact, observed on a network or in an operating system that, with high confidence, indicates a computer intrusion.
  • Finding Class: displayed as findingClass. Indicates the type of finding. The following list includes finding classes and their descriptions:
    • Threat: unwanted or malicious activity
    • Vulnerability: a potential weakness in software that increases risk to the confidentiality, integrity, and availability of your resources
    • Misconfiguration: a potential weakness in a resource's configuration that increases risk
    • Observation: a security observation provided for informational purposes

To learn more about findings, see the Findings tab in Using the Security Command Center dashboard.

Documentation

  • Security Command Center documentation now includes a page that maps resource type formats between Cloud Asset Inventory and Security Command Center. The services use different naming conventions for resource types. For more information, see Resource type formats in Security Command Center.
Tensorflow Enterprise

TensorFlow Enterprise 2.6 is now available with LTS.

August 10, 2021

Cloud Functions

The use of private worker pools is now available for building your functions.

VPC Service Controls

General availability for the following integration:

August 09, 2021

AI Platform Training

You can use TPU Pods for training. This feature is available in Preview.

Anthos clusters on VMware

Anthos clusters on VMware 1.7.3-gke.2 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.7.3-gke.2 runs on Kubernetes 1.19.12-gke.1100.

The supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.8, 1.7, and 1.6.

Fixes:

  • These security vulnerabilities have been fixed: CVE-2021-3520, CVE-2021-33909, and CVE-2021-33910.

  • Fixed the issue that the /etc/cron.daily/aide` script uses up all existing space in /run, causing a crashloop in Pods.

  • Fixed the issue that admin cluster upgrade may fail due to an expired front-proxy-client certificate on the admin control plane node.

Cloud Asset Inventory

New resource types are now available.

The following resource types are now publicly available through the Analyze Policy APIs (AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning):

  • Spanner
    • spanner.googleapis.com/Database
    • spanner.googleapis.com/Instance
    • spanner.googleapis.com/Backup
  • Document AI
    • documentai.googleapis.com/HumanReviewConfig
    • documentai.googleapis.com/LabelerPool
    • documentai.googleapis.com/Processor

New resource types are now available to allowlisted users.

The following resource types are now available to allowlisted users through the Export APIs (ExportAssets and BatchGetAssetsHistory) and the Feed API. To join the allowlist, follow these instructions.

  • Firebase
    • firebase.googleapis.com/FirebaseProject
    • firebase.googleapis.com/FirebaseAppInfo
Cloud Build

GitHub Enterprise triggers are now generally available. Users can build repositories from GitHub Enterprise, including on-premises instances. For more information, see Creating GitHub Enterprise triggers.

Cloud Monitoring

When you create a line chart on a dashboard, you can now specify whether the selected metric is charted against the left Y-axis or the right Y-axis. This feature lets you display different metrics with different scales on the same chart. For an API example, see Dashboard with an XyChart and a threshold.

Cloud SQL for MySQL

The Cloud SQL Admin API v1 is now generally available. It is and will continue to be compatible with the v1beta4 version. There is no requirement to migrate from v1beta4 to the v1 Admin API.

Cloud SQL for PostgreSQL

The Cloud SQL Admin API v1 is now generally available. It is and will continue to be compatible with the v1beta4 version. There is no requirement to migrate from v1beta4 to the v1 Admin API.

Cloud SQL for SQL Server

The Cloud SQL Admin API v1 is now generally available. It is and will continue to be compatible with the v1beta4 version. There is no requirement to migrate from v1beta4 to the v1 Admin API.

Dataproc

New sub-minor versions of Dataproc images: 1.4.67-debian10, 1.4.67-ubuntu18, 1.5.42-centos8, 1.5.42-debian10, 1.5.42-ubuntu18, 2.0.16-centos8, 2.0.16-debian10, and 2.0.16-ubuntu18.

SPARK-28290: Fixed an issue where Spark History Server failed to serve because of a wild card certificate in the 1.4 and 1.5 images.

Resource Manager

You can now use the Cloud Console UI to manage your organization policies with tags. For more information, see Setting an organization policy with tags.

Tools for Eclipse

1.8.5

Fixed an issue where avatar images of logged-in accounts were not being displayed.

August 06, 2021

BigQuery ML

The principal component analysis (PCA) model is now available for preview. For more information, see CREATE MODEL statement for PCA models and the PCA details in the end-to-end user journey.

Cloud Asset Inventory

New resource types are now available.

The following resource types are now publicly available through the Analyze Policy APIs (AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning):

  • Hub
    • gkehub.googleapis.com/Membership
  • Managed Service For Microsoft Active Directory
    • managedservices.googapis.com/Domain
  • API Gateway
    • apigateway.googleapis.com/Api
    • apigateway.googleapis.com/ApiConfig
    • apigateway.googleapis.com/Gateway
Compute Engine

Generally available: The Observability tab on Compute Engine's VM instance details page includes a category for process metrics. You can use the new charts and reports to troubleshoot the behavior of processes running on your VMs.

Preview: You can now use the Slurm-GCP workload manager to create clusters that are based on the HPC virtual machine (VM) image and comply to the Intel Select Solution for Simulation and Modeling criteria. For more information, see Creating Intel Select Solution HPC clusters.

August 05, 2021

Anthos clusters on VMware

Anthos clusters on VMware 1.6.4-gke.7 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.6.4-gke.7 runs on Kubernetes 1.18.20-gke.2900.

The supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.8, 1.7, and 1.6.

Fixes:

  • These security vulnerabilities have been fixed: CVE-2021-3520, CVE-2021-33909, and CVE-2021-33910.

  • Fixed the issue that admin cluster upgrade may fail due to an expired front-proxy-client certificate on the admin control plane node.

Cloud Build

Users can now configure triggers to use a particular service account, available at the preview release stage. To learn more, see Configuring user-specified service accounts.

Cloud Composer

When upgrading your environment to a new version, you can now check if PyPI packages installed in your environment have any conflicts with preinstalled packages in the new Cloud Composer image.

(Airflow 2) By default, Airflow performs two retries for a failed task. The default value for the core-default_task_retries Airflow configuration option is 2.

Deleting a DAG that has queued tasks no longer causes worker pods to restart.

New versions of Cloud Composer images:

  • composer-1.16.13-airflow-1.10.12
  • composer-1.16.13-airflow-1.10.14
  • composer-1.16.13-airflow-1.10.15 (default)
  • composer-1.17.0-preview.9-airflow-2.0.2
  • composer-1.17.0-preview.9-airflow-2.1.1

Cloud Composer 1.11.1 has reached its end of full support period.

Cloud Monitoring

A new UI for creating alerting policies is available in Preview. This interface offers fine-grained control over the selection of the metrics used in alerting conditions. See Managing Alerting Policies for more information.

The Processes tab in the VM instances page is now Generally Available. This tab adds charts for process metrics to the charts provided by the existing CPU, Memory, Disk, and Network tabs.

Compute Engine

N2 VMs are now available in all three zones in Warsaw, Poland europe-central2-a,b,c. See VM instance pricing for details.

Dataproc Metastore

Hive version 3.1.2 will become the default Dataproc Metastore service creation version in 5 weeks on September 6, 2021.

Google Kubernetes Engine

GKE Multi Cluster Ingress is now available through standalone per-Pod pricing in addition to Anthos licensing for all GKE release channels.

SAP on Google Cloud

High-availability cluster configuration guidance has changed

To minimize the likelihood of timing errors during a restart after failover, the guidance for setting up Pacemaker clusters on Linux for SAP has changed to include setting a short delay in the startup of Corosync after a failover.

For more information, see:

August 04, 2021

Cloud Spanner

Cloud Spanner now has an end-to-end latency guide. This guide describes the high-level components involved in a Cloud Spanner API request, and explains how to extract, capture, and visualize latencies associated with these components to know the source of the latencies.

August 03, 2021

Artifact Registry

Apt and Yum repositories are now in Preview.

Storage and network egress charges apply to all formats that are in Preview or are generally available.

BigQuery

BigQuery now supports the ALTER COLUMN SET DATA TYPE data definition language (DDL) statement. This feature is generally available (GA).

BigQuery now supports the following data definition language (DDL) statement:

This feature is generally available (GA).

BigQuery is now available in the Toronto (northamerica-northeast2) region.

BigQuery BI Engine

BigQuery BI Engine is now available in the Toronto (northamerica-northeast2) region.

BigQuery Data Transfer Service

BigQuery Data Transfer Service is now available in the Toronto (northamerica-northeast2) region.

BigQuery ML

BigQuery ML is now available in the Toronto (northamerica-northeast2) region.

Cloud Asset Inventory

New resource types are now available.

The following resource types are now publicly available through the Export APIs (ExportAssets and BatchGetAssetsHistory) and the Feed API:

  • Service Management
    • servicemanagement.googleapis.com/ManagedService
Cloud Bigtable

Cloud Bigtable is now available in the northamerica-northeast2 (Toronto) region.

Cloud Run

The following new region is now available: northamerica-northeast2.

Cloud SQL for MySQL

Support for northamerica-northeast2-a,b,c (Toronto) region.

Cloud SQL for PostgreSQL

Support for northamerica-northeast2-a,b,c (Toronto) region.

Cloud SQL for SQL Server

Support for northamerica-northeast2-a,b,c (Toronto) region.

Cloud Spanner

Cloud Spanner regional instances can now be created in Toronto (northamerica-northeast2).

Added support for changing instance configuration (Preview).

Cloud Storage

Toronto region (northamerica-northeast2) launched.

Cloud VPN

Cloud VPN is now available in region northamerica-northeast2 (Toronto, Canada).

Pricing is available on the Cloud VPN pricing page.

Compute Engine

Toronto, Ontario, Canada northamerica-northeast2-a,b,c region has launched with E2, N2, N1 virtual machine (VM) instances in all three zones. See VM instance pricing for details.

Disks, snapshots, and images are available in Toronto, Ontario, Canada northamerica-northeast2 in all three zones. See Disks and image pricing for details.

Generally available: You can update the descriptions of your managed instance groups by using the API or gcloud tool.

Dataflow

Dataflow is now able to use workers, Dataflow Shuffle, Streaming Engine, FlexRS, and regional endpoints in zones in Toronto (northamerica-northeast2).

Dataproc

Dataproc is now available in the northamerica-northeast2 region (Toronto).

Filestore

Filestore is available in the northamerica-northeast2 (Toronto) region. See Regions and zones.

Google Kubernetes Engine

The northamerica-northeast2 region in Toronto is now available.

(2021-R25) Version updates

GKE cluster versions have been updated.

New versions available for upgrades and new clusters

The following Kubernetes versions are now available for new clusters and for opt-in control plane upgrades and node upgrades for existing clusters. For more information on versioning and upgrades, see GKE versioning and support and Upgrades.

No channel

  • The following control plane and node versions are now available:
  • The following control plane versions are no longer available:
    • 1.18.17-gke.1901
    • 1.18.19-gke.1701
    • 1.18.19-gke.2101
  • Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.18 to version 1.18.20-gke.501 with this release.

Stable channel

  • Version 1.18.20-gke.900 is now the default version in the Stable channel.
  • Version 1.18.20-gke.501 is no longer available in the Stable channel.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.18 to version 1.18.20-gke.900 with this release.

Regular channel

There are no new releases in the Regular release channel.

Rapid channel

  • Version 1.20.8-gke.900 is now the default version in the Rapid channel.
  • Version 1.20.9-gke.700 is now available in the Rapid channel.
  • Version 1.21.3-gke.900 is now available in the Rapid channel.
  • Version 1.20.8-gke.700 is no longer available in the Rapid channel.
  • Version 1.21.2-gke.600 is no longer available in the Rapid channel.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.20 to version 1.20.8-gke.900 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to version 1.21.3-gke.100 with this release.

(2021-R25) Version updates

  • The following control plane and node versions are now available:
  • The following control plane versions are no longer available:
    • 1.18.17-gke.1901
    • 1.18.19-gke.1701
    • 1.18.19-gke.2101
  • Control planes and nodes with auto-upgrade enabled will be upgraded from version 1.18 to version 1.18.20-gke.501 with this release.

(2021-R25) Version updates

  • Version 1.18.20-gke.900 is now the default version in the Stable channel.
  • Version 1.18.20-gke.501 is no longer available in the Stable channel.
  • Control planes and nodes with auto-upgrade enabled in the Stable channel will be upgraded from version 1.18 to version 1.18.20-gke.900 with this release.

(2021-R25) Version updates

There are no new releases in the Regular release channel.

(2021-R25) Version updates

  • Version 1.20.8-gke.900 is now the default version in the Rapid channel.
  • Version 1.20.9-gke.700 is now available in the Rapid channel.
  • Version 1.21.3-gke.900 is now available in the Rapid channel.
  • Version 1.20.8-gke.700 is no longer available in the Rapid channel.
  • Version 1.21.2-gke.600 is no longer available in the Rapid channel.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.20 to version 1.20.8-gke.900 with this release.
  • Control planes and nodes with auto-upgrade enabled in the Rapid channel will be upgraded from version 1.21 to version 1.21.3-gke.100 with this release.
Memorystore for Memcached

Added new Memorystore for Memcached region: Toronto (northamerica-northeast2).

Memorystore for Redis

Added new Memorystore for Redis region: Toronto (northamerica-northeast2).

Pub/Sub

Pub/Sub Lite is now available in northamerica-northeast2 (Toronto).

Secret Manager

Secret Manager is now available in northamerica-northeast2 (Toronto). See Secret Manager locations for more information.

Virtual Private Cloud

For auto mode VPC networks, added a new subnet 10.188.0.0/20 for the Toronto northamerica-northeast2 region. For more information, see Auto mode IP ranges.

August 02, 2021

BigQuery

An updated version of JDBC driver for BigQuery is now available that includes Enhancements & New Features.

Channel Services

This release includes the new ListProvisionableCloudIdentityTypes method.

ListProvisionableCloudIdentityTypes returns the Google Workspace customer types you can create for a given domain, and shows if they require a transfer.

Added extra information to clarify which fields are not required for team customers.

Cloud Logging

You can now use Puppet to install and manage the Google Cloud operations suite agents across your fleet of Linux and Windows VMs. For more information, refer to the Puppet Integration documentation.

Cloud Monitoring

The API to manage the metrics scope of a Google Cloud project is now in Preview. For more information, see Manage metrics scopes with the API.

You can now use Puppet to install and manage the Google Cloud operations suite agents across your fleet of Linux and Windows VMs. For more information, refer to the Puppet Integration documentation.

Cloud NAT

Cloud NAT rules is available in Preview. NAT rules let you create access rules that define how Cloud NAT is used to connect to the internet. NAT rules support source NAT based on destination address.

Cloud Translation

Removed the Phrase-Based Machine Translation (PBMT) model. For requests that specify the PBMT model, Cloud Translation uses the Neural Machine Translation (NMT) model instead.

Compute Engine

Preview: You can now share reservations of Compute Engine zonal resources between multiple projects. Learn about shared reservations and creating a shared reservation.

Config Connector

Config Connector 1.58.0 is now available.

Added support for MonitoringMetricDescriptor resource.

CloudBuildTrigger: added webhookConfig and pubsubConfig options for triggers.

Added a list of resources which have service-generated resource IDs.

Added limited support for the cnrm.cloud.google.com/state-into-spec annotation, which allows merge and absent values to merge GCP state into the spec field or not, respectively.

Currently only supported for BigQueryDataset.

Dataproc

1.3 images are no longer supported and will not receive new releases.

New sub-minor versions of Dataproc images: 1.4.66-debian10, 1.4.66-ubuntu18, 1.5.41-centos8, 1.5.41-debian10, 1.5.41-ubuntu18, 2.0.15-centos8, 2.0.15-debian10, and 2.0.15-ubuntu18.

In image 2.0, set the mapreduce.fileoutputcommitter.algorithm.version=2 property in Spark. This makes Spark commit algorithm version consistent with prior Dataproc image versions.

Deep Learning Containers

M76 Release

Deep Learning VM Images

M76 Release

Filestore

Filestore now has a new Enterprise tier, which lets you create 1–10 TiB regional instances that can scale up and down with your storage needs.

Identity and Access Management

You can now use Activity Analyzer to see when your service accounts and keys were last used to call a Google API. This feature is in Preview.

Memorystore for Redis

Enabled cross region access for Memorystore for Redis.

SAP on Google Cloud

Google Cloud Connector for SAP LaMa is generally available

The Google Cloud Connector for SAP Landscape Management 3.0, enterprise edition, is now generally available (GA).

For more information, see the Connector for SAP Landscape Management planning guide.

Transcoder API

Added new quotas in order to group requests into 3 buckets: mutation, get, and list. More information can be found at Quotas and limits.

Vertex AI

Vertex Pipelines is available in the following regions:

  • us-east1 (South Carolina)
  • europe-west2 (London)
  • asia-southeast1 (Singapore)

See all the locations where Vertex Pipelines is available.

August 01, 2021

Cloud Storage

Cloud Storage now has bandwidth quotas for egress to other Google services.

July 30, 2021

Cloud Logging

Cloud Logging now lets you control access to individual log entry fields using field-level access control. To learn more, see Field-level access control.

Cloud Talent Solution Job Search

Added WALKING and CYCLING as commute options in the CommuteFilter.

Removes companyName requirement for use of ListJobs API.

Increased the number of characters supported in custom ranking expressions.

Dataproc Metastore

Dataproc Metastore is available in the following regions:

  • asia-southeast1 (Singapore)
  • europe-west1 (Belgium)
  • northamerica-northeast1 (Montréal)

For more information, see Locations.

Dataproc Metastore backups and restores support cross-service restoration of metadata.

Document AI

The Invoice Parser now extracts a new field invoice_type that indicates the type of the input document.

Kf

New features may now start in preview and GA with the next Kf release.

New Early Access program (currently private preview).

Automated assessment and migration of your existing CF foundation.

[PREVIEW] NFS broker automatically configures NFS mounts on your cluster for Apps to bind to.

[PREVIEW] Schedule Tasks to run at recurring intervals specified using the unix-cron format.

[PREVIEW] Support for Anthos clusters on VMware via the Early Access program.

Minor reliability improvements.

Resolved race condition that could cause the first brokered service instance to fail.

Support for ASM 1.10.

Managed Service for Microsoft Active Directory

Support for audit logging is now generally available.

VPC Service Controls

General availability for the following integration:

July 29, 2021

Anthos GKE on AWS

Anthos clusters on AWS aws-1.8.1-gke.1 is now available.

Anthos clusters on AWS aws-1.8.1-gke.1 clusters run the following Kubernetes versions:

  • 1.17.17-gke.13600
  • 1.18.20-gke.2600
  • 1.19.13-gke.300
  • 1.20.9-gke.300

This release contains fixes for the following security vulnerabilities:

Anthos clusters on AWS now requires kubectl version 1.17 or higher and terraform version v0.14.3 or higher.

Anthos on bare metal

Release 1.8.2

Anthos clusters on bare metal 1.8.2 is now available. To upgrade, see Upgrading Anthos on bare metal. Anthos clusters on bare metal 1.8.2 runs on Kubernetes 1.20.

Features:

  • Preview: Added capability to rotate cluster certificate authorities (CAs) for user clusters. For instructions on using the bmctl cluster credentials command to rotate cluster CAs, see Rotate user cluster certificate authority.

  • Preview: Added support for AppArmor with Anthos clusters on bare metal. You don't need to disable AppArmor on Ubuntu as a prerequisite for installation. When you create new 1.8.2 clusters or upgrade clusters to version 1.8.2, you can enable AppArmor either before or after you upgrade.

Fixes:

  • Fixed CVE-2021-3520 vulnerability related to a flaw in lz4, which provides support for LZ4 a lossless compression algorithm. The flaw impacts availability, but has potential to impact confidentiality and integrity as well.

  • Fixed bmctl operation failures that occur for some Ubuntu 20.04 LTS distributions with a more recent Linux kernel, including GCP Ubuntu 20.04 LTS images on the 5.8 kernel. For more information about this issue and a workaround, see Ubuntu 20.04 LTS and bmctl.

  • Fixed OpenStack support for user clusters. In prior releases, cluster creation fails for user type clusters when the baremetal.cluster.gke.io/external-cloud-provider: "true" annotation is added to the cluster configuration file.

  • Fixed PATH environment issues for executing commands as a non-root user. For more information, see Known Issues.

  • Fixed an issue that caused user cluster resets (bmctl reset cluster) to get stuck while deleting namespaces.

  • Fixed out-of-memory (OOM) conditions related to Connect Agent memory usage that resulted in pod failures.

  • Fixed issue that blocked snapshots for clusters configured for passwordless SUDO capability for machine login (nodeAccess.loginUser: <login user name>).

  • Fixed issue that blocked some 1.7.x version admin, hybrid, or standalone clusters from upgrading to the 1.8 minor release. This issue affected some clusters that were updated by applying changes from an updated cluster configuration file.

  • Fixed Address Resolution Protocol (ARP) table issue for high-availability (HA) deployments that blocked upgrades from completing.

Functionality changes:

  • Expanded snapshots to include resource usage metrics to improve troubleshooting and support. Added metrics include the output of ip neigh, kubectl top nodes, and kubectl top pods commands.

For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.

Cloud Composer

Airflow 2.1.1 is available in Cloud Composer images.

Fixed the problem with environment health metrics not being exported in VPC Service Controls environments because of airflow-monitoring component restarts.

Fixed the problem with environment health metrics not being exported because of an incorrect environment variable value in the airflow-monitoring component.

(Available without upgrading) The Require OS Login (compute.requireOsLogin) organization policy constraint is now checked before an environment is created. If this policy is enabled, the environment creation operation fails immediately with an error message. Before this change, the operation failed on a timeout.

New versions of Cloud Composer images:

  • composer-1.16.12-airflow-1.10.12
  • composer-1.16.12-airflow-1.10.14
  • composer-1.16.12-airflow-1.10.15 (default)
  • composer-1.17.0-preview.8-airflow-2.0.2
  • composer-1.17.0-preview.8-airflow-2.1.1

Cloud Composer 1.11.0 has reached its end of full support period.

Cloud Key Management Service

Cloud EKM now supports Artifact Registry, Logs Router in Cloud Logging, and Cloud Spanner. For more information, see Cloud External Key Manager.

Cloud Load Balancing

Cloud Monitoring now provides a new predefined dashboard called External HTTP(S) Load Balancers. The new dashboard provides powerful visualizations to help you understand and troubleshoot connectivity issues on your external HTTP(S) load balancers.

For details, see HTTP(S) Load Balancing logging and monitoring.

Cloud Logging

Cloud Logging now supports the asia-east2 and europe-central2 regions. For a full list or regions, see Regionalization.

Cloud Monitoring

Monitoring Query Language (MQL) no longer requires you to use strict form when you create MQL-based alerting policies by using the Monitoring API. You can now provide queries in concise form. The query is stored as you provide it; concise queries are not converted to strict form.

Cloud SQL for PostgreSQL
  • Cloud SQL for PostgreSQL now supports the following flags:
    • tcp_keepalives_count
    • tcp_keepalives_idle
    • tcp_keepalives_interval

For more information about these flags, see the Cloud SQL for PostgreSQL flags documentation.

Dataproc Metastore

There is an issue with Dataproc Metastore to Data Catalog batch sync. Metadata changes introduced through imports and backups will not be reflected in Data Catalog until batch sync is restored.