Google Cloud release notes

The following release notes cover the most recent changes over the last 60 days. For a comprehensive list, see the individual product release note pages .

You can see the latest product updates for all of Google Cloud on the Google Cloud release notes page.

To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly: https://cloud.google.com/feeds/gcp-release-notes.xml

January 15, 2021

AI Platform Training

AI Platform Training now provides pre-built PyTorch containers for PyTorch 1.6.

In addition to training with CPUs or GPUs, you can use one of the PyTorch 1.6 containers to perform PyTorch training with a TPU.

Cloud Build

Regionalized builds from Cloud Functions and App Engine deployments are now visible in the Cloud Build History UI. To learn more, see Viewing build results.

Cloud Spanner

You can now run SQL queries to retrieve lock statistics to investigate lock conflicts in your database.

Dataproc

Announcing the Beta release of Dataproc Service Account Based Secure Multi-tenancy, which allows you to share a cluster with multiple users. With secure multi-tenancy, users can submit interactive workloads to the cluster with isolated user identities.

New sub-minor versions of Dataproc images: 1.3.81-debian10, 1.3.81-ubuntu18, 1.4.52-debian10, 1.4.52-ubuntu18, 1.5.27-centos8, 1.5.27-debian10, 1.5.27-ubuntu18, 2.0.0-RC23-debian10, and 2.0.0-RC23-ubuntu18.

Image 2.0 preview:

  • Upgraded Spark to version 3.1.0 RC1.

  • Upgraded Zeppelin to version 0.9.0.

  • Upgraded Cloud Storage Connector to version 2.2.0.

  • Upgraded JupyterLab to version 3.0.

The gcloud_dataproc_personal_cluster.py tool for the personal auth beta is no longer supported for new images. It will be replaced by an equivalent set of commands in an upcoming gcloud release.

Network Intelligence Center

The Network Topology graph now includes a checkbox, Show connections for child nodes only on focus, to display only the traffic paths between top-level entities, such as regions. When this checkbox is selected, you can still view the traffic paths between lower-level entities by selecting or holding the pointer over the lower-level entities.

Recommendations AI

Recommendations AI has migrated to the Retail API, which is now generally available.

The Recommendations Engine API (service endpoint https://recommendationengine.googleapis.com) and this documentation set remain available, but they will no longer be updated. We recommend migrating your recommendations to the Retail API (service endpoint https://retail.googleapis.com). See the new documentation:

Retail API

Recommendations AI is now generally available.

This product has migrated to the Retail API from the Recommendations Engine API.

The previous API (service endpoint https://recommendationengine.googleapis.com) and its documentation set remain available, but they will no longer be updated. If you used the previous API while it was in beta, we recommend migrating your recommendations to the Retail API (service endpoint https://retail.googleapis.com).

See the new documentation:

January 14, 2021

BigQuery

BigQuery's Cloud Console UI has been updated with many usability improvements for analyzing data, including multi-tab navigation, a new resource panel, and a new SQL editor. These updates are in Preview. For more information, see Using the Cloud Console.

Cloud Logging

Cloud Logging now lets you share your saved queries with other users of a project. To learn more, go to the Shared queries section on the Building queries page.

Cloud Spanner

Query statistics now includes information about queries that failed, queries that timed out, and queries that were canceled by the user.

Managed Service for Microsoft Active Directory

Managed Microsoft AD now supports audit logging. This feature is in the Preview stage.

January 13, 2021

Cloud Composer
  • Preview: You can now restart the Airflow web server using the command gcloud beta composer environments restart-web-server or the Beta API.
Cloud Functions

Cloud Functions has added support for a new runtime, Node 14, in Preview.

Cloud Functions has added support for a new runtime, Python 3.9, in Preview.

Cloud SQL for MySQL

Cloud SQL now exposes the metric database/memory/total_usage. This metric provides visibility into the database working set (including buffer cache). You can find this metric in the Metrics explorer within the Monitoring dashboard.

For more information about database/memory/total_usage, see Cloud SQL Metrics.

Cloud SQL for PostgreSQL

Cloud SQL now exposes the metric database/memory/total_usage. This metric provides visibility into the database working set (including buffer cache). You can find this metric in the Metrics explorer within the Monitoring dashboard.

For more information about database/memory/total_usage, see Cloud SQL Metrics.

Cloud SQL for SQL Server

Cloud SQL now exposes the metric database/memory/total_usage. This metric provides visibility into the database working set (including buffer cache). You can find this metric in the Metrics explorer within the Monitoring dashboard.

For more information about database/memory/total_usage, see Cloud SQL Metrics.

January 12, 2021

Anthos Service Mesh

1.6.14-asm.0 is now available.

This patch release contains the same bug fixes that are in Istio 1.6.14. For details on upgrading Anthos Service Mesh, refer to the following upgrade guides:

Cloud Composer

GA: Network ACL support for the Airflow web server is now generally available.

Preview: Composer can now be configured to use Customer-managed encryption keys (CMEK).

  • New versions of Cloud Composer images: composer-1.13.4-airflow-1.10.9, composer-1.13.4-airflow-1.10.10, and composer-1.13.4-airflow-1.10.12. The default is composer-1.13.4-airflow-1.10.10. Upgrade your Cloud SDK to use features in this release.
  • If environment creation succeeds when the requester did not have the iam.serviceAccounts.actAs permission on the service account, Composer will now generate a warning in the audit log entry.
  • Error handling for files synchronization between buckets has been improved.
  • You can now set the machine type for the Airflow web server and Cloud SQL database using the to v1 Composer API.
  • Preview: Added support for the Airflow Role-Based Access Control (RBAC) UI for Airflow version 1.10.10 or newer and Python 3. You can enable the Airflow RBAC UI by setting the [webserver]rbac=True Airflow configuration override.
  • Fixed an issue where connections were inserted into the Airflow database twice.
  • The Airflow UI will now always send requests to HTTPS addresses.
Dataproc

Added support for user configuration of Compute Engine Shielded VMs in a Dataproc Cluster.

Migrate for Compute Engine

#171638373: General stability improvements.

#171638373: Fixed Windows adaptation issue when boot partition and Windows partition were on different volumes.

Performance improvement during detach phase.

#175196444: Fixed Windows adaptation issue with network interface detection.

#174330790: Linux adaptations now archive ifcfg-* scripts to avoid Network Manager conflicts with iSCSI boot.

Security fixes applied.

January 11, 2021

AI Platform (Unified)

The default boot disk type for virtual machine instances used for custom training has changed from pd-standard to pd-ssd. Learn more about disk types for custom training and read about pricing for different disk types.

If you previously used the default disk type for custom training and want to continue training with the same disk type, make sure to explicitly specify the pd-standard boot disk type when you perform custom training.

Cloud Build

Users can now specify their own service accounts for Cloud Build to run builds. For more information, see User-specified service accounts.

Cloud DNS Cloud Functions

Cloud Functions has added support for a new runtime, Ruby, in Preview. This runtime supports Ruby 2.6 and Ruby 2.7.

Cloud Interconnect

Support for 1500 MTU for Cloud Interconnect is now available in General Availability.

Compute Engine

You can now create N2D VM instances in us-east4-c Northern Virginia. See VM instance pricing for details.

Service Directory Virtual Private Cloud

Support for 1500 MTU for Cloud Interconnect is now available in General Availability.

January 08, 2021

Anthos Config Management

Config Sync unintentionally started using the absolute path in the file system with spec.git.policyDir. This has no effect on Config Sync running on the cluster, but breaks validation when running nomos vet manually against hierarchical repositories. The issue will be corrected in 1.6.1.

Dataproc

Added support for new persistent disk type, pd-balanced.

New sub-minor versions of Dataproc images: 1.3.80-debian10, 1.3.80-ubuntu18, 1.4.51-debian10, 1.4.51-ubuntu18, 1.5.26-centos8, 1.5.26-debian10, 1.5.26-ubuntu18, 2.0.0-RC22-debian10, and 2.0.0-RC22-ubuntu18.

Image 2.0 preview:

  • Upgraded Delta Hive connector to version 0.2.0.
  • Upgraded Flink to version 1.12.0.
  • Updated Iceberg to version 0.10.0.

Image 2.0 preview:

HIVE-21646: Tez: Prevent TezTasks from escaping thread logging context

Dataproc Metastore

Dataproc Metastore Cloud Logging and Monitoring is unavailable. The issue will be fixed shortly.

January 07, 2021

VPC Service Controls

General availability for the following integration:

January 06, 2021

AI Platform (Unified) Cloud Data Loss Prevention

Hybrid Jobs are now available for inspecting external data sources.

Config Connector

Config Connector version 1.34.0 is now available.

Added support for IAM Member References. This allows users to create an IAMPolicyMember that references another resource as the IAM member (e.g. IAMServiceAccount, LoggingLogSink). For more information, see the memberFrom field in the IAMPolicyMember reference documentation. Support for IAM Member References is added only to IAMPolicyMember, not IAMPolicy.

Added support for the GameServicesRealm resource.

Added IAM support for ComputeDisk.

Added cacheMode, clientTtl, defaultTtl, maxTtl, negativeCaching, negativeCachingPolicy, serveWhileStale, and customResponseHeaders fields to ComputeBackendBucket.

Added customTimeBefore, daysSinceCustomTime, daysSinceNoncurrentTime, and noncurrentTimeBefore fields to StorageBucket.

Allow for IAMPolicy, IAMPolicyMember, and IAMAuditConfig to reference resources in other namespaces.

Added support for UpdateFailed, DeleteFailed, DependencyNotFound, and DependencyNotReady events to IAMPolicy, IAMPoicyMember, IAMAuditConfig.

Allow for Project and Folder resources to be migrated across folders and organizations by updating the folder-id/organization-id annotation. Only folder-to-folder or organization-to-organization migrations are allowed; folder-to-organization migrations or vice versa are not yet supported.

January 05, 2021

Cloud Composer

In an upcoming Cloud Composer version release, DAG Serialization will be enabled by default when creating new Cloud Composer environments.

Traffic Director

Traffic Director now supports TCP-based services in Preview. This brings service discovery, global load balancing, failover and many other Traffic Director capabilities to your non-HTTP services. See the setup guide to get started and the target proxies documentation for helpful background information.

December 23, 2020

Cloud Monitoring

Alerting is now Generally Available for Monitoring Query Language (MQL). For more information, see Alerting policies with MQL.

December 22, 2020

Filestore

IP-based access control is now generally available.

December 21, 2020

BigQuery

BigQuery standard SQL now supports the BigNumeric data type for high-precision computations. The BigNumeric data type is in Preview.

Cloud SQL for PostgreSQL

IAM database authentication for Cloud SQL for PostgreSQL is now generally available. To get started using IAM database authentication, see the Overview of Cloud SQL IAM database authentication.

December 18, 2020

Cloud Run

Cloud Run now allows you to restrict ingress of your Cloud Run services.

You can now allocate up to 8GiB of memory to your Cloud Run services.

December 17, 2020

AI Platform (Unified)

AI Platform (Unified) now stores and processes your data only in the region you specify for most features. Learn more.

Anthos GKE on AWS

GKE on AWS 1.6.0-gke.3 is now available.

GKE on AWS 1.6.0-gke.3 clusters run the following Kubernetes versions:

  • 1.16.15-gke.5300
  • 1.17.9-gke.6400
  • 1.18.10-gke.900

To upgrade your clusters, perform the following steps:

  1. Upgrade your Management service to 1.6.0-gke.1.
  2. Upgrade your user clusters to a supported Kubernetes version.

GKE on AWS now supports Kubernetes 1.18.

The Kubernetes 1.18 version includes CoreDNS 1.7.1 and Cluster Autoscaler 1.18.

GKE on AWS now supports mounting AWS Elastic File System file systems without having to install a driver.

You can now specify an AWS KMS alias in your anthos-gke.yaml instead of a KMS ARN.

You can now use custom DNS hostnames in your VPC by setting enableDnsHostnames to false

Cluster state synchronizations between the management service and S3 now use HTTPS.

Cloud Billing

Start using the Reports page and Cost Table in the Cloud Console for product-level cost details or subaccounts

Beginning with your January 2021 invoice or statement (available in February 2021), to simplify the format, we are removing all cost details from your invoice and statement documents, including product-level costs and costs by subaccounts (for Resellers). To view all of the cost details on your invoice or statement, in the Cloud Console, access the downloadable Cost Table report. The Cost Table report includes the product-level cost and cost by subaccounts (for Resellers), along with additional details you may need, such as costs by projects, services, SKU IDs, and labels. You can also analyze your usage costs using the Reports page.

For guidance on using these reports, see:

Cloud SQL for MySQL

In Cloud SQL for MySQL, parallel replication is generally available for improving replication performance.

Cloud SQL for PostgreSQL

Cloud SQL has expanded support for PostgreSQL extensions. Three additional PostgreSQL extensions are now available:

  • dblink
  • ip4r
  • prefix

For additional information, see PostgreSQL extensions.

The following PostgreSQL minor versions have been upgraded:

  • PostgreSQL 9.6.18 is upgraded to 9.6.19.
  • PostgreSQL 10.13 is upgraded to 10.14.
  • PostgreSQL 11.8 is upgraded to 11.9.
  • PostgreSQL 12.3 is upgraded to 12.4.
Cloud Spanner

A new multi-region instance configuration is now available in Europe - eur6 (Netherlands/Frankfurt/Zurich).

A new multi-region instance configuration is now available in North America - nam12 (Iowa/Northern Virginia/Oregon/Oklahoma).

Compute Engine

The m1-node-96-1433 sole-tenant node type is now Generally Available.

Dataproc

New sub-minor versions of Dataproc images: 1.3.79-debian10, 1.3.79-ubuntu18, 1.4.50-debian10, 1.4.50-ubuntu18, 1.5.25-centos8, 1.5.25-debian10, 1.5.25-ubuntu18, 2.0.0-RC21-debian10, and 2.0.0-RC21-ubuntu18.

Image 2.0 preview:

Changed the default value of Spark SQL property spark.sql.autoBroadcastJoinThreshold to 0.75% of executor memory.

Fixed SPARK-32436: Initialize numNonEmptyBlocks in HighlyCompressedMapStatus.readExternal

Image 1.4-1.5:

Fixed a NullPointerException in a primary worker shuffle when the BypassMergeSortShuffleWriter is used when some output partitions are empty.

Images 1.5-2.0 preview:

Fixed ZOOKEEPER-1936: Server exits when unable to create data directory due to race condition.

Fixed a bug where Dataproc agent logs had separate entries for exception stack trace in StackDriver.

Identity and Access Management

You can now attach service accounts to resources in other projects. This feature is available in Preview.

Memorystore for Redis

Added support for TLS encryption on Memorystore for Redis.

December 16, 2020

AI Platform Deep Learning Containers

Added TensorFlow 2.4 Deep Learning Containers images.

AI Platform Deep Learning VM Image

M60 release

  • Added TensorFlow 2.4 Deep Learning VM Images
AI Platform Prediction

You can now configure AI Platform Prediction to automatically scale prediction nodes for model versions that use GPUs for online prediction.

Previously, you could only configure manual scaling for model versions that use GPUs. Now, you can choose between automatic and manual scaling.

Using automatic scaling with GPUs is available in preview.

Anthos Service Mesh

1.8.1-asm.5 is now available.

Multi-cluster support for GKE on-prem Beta

Anthos Service Mesh now supports multi-cluster meshes when running on GKE on-prem. For more information, see Add clusters to Anthos Service Mesh on-prem.

New flags for the install_asm script

The install_asm script was enhanced to provide you with more granular control over the changes that the script makes on your project and GKE on Google Cloud cluster. For more information, see the Enablement flags section in the documentation for the script.

Third-party add-ons removed from all profiles

The Prometheus, Grafana, and Kiali add-ons were removed from all Anthos Service Mesh profiles. For information on why the add-ons were removed, see Reworking our Addon Integrations. Installation of these third-party add-ons was removed from the 1.8 IstioOperator API, which means that they can't be installed with the istioctl install command. For information on installing a demo version of the add-ons, see Integrating with third-party add-ons.

Note that by default, metrics are still exported to Prometheus in the asm-multicloud profile. You can optionally enable metrics export to Prometheus in the asm-gcp-multiproject profile.

Anthos Service Mesh 1.8 isn't supported on Anthos attached clusters and GKE on AWS

Anthos Service Mesh 1.8 currently isn't supported on Anthos attached clusters (Microsoft AKS and Amazon EKS) and GKE on AWS (Amazon EC2). Anthos Service Mesh 1.7 and 1.6 are supported for these environments. For more information, see the following guides:

Reduced permissions required for installation

The permissions required for installation have been scaled back. Testing has shown that the Project Editor role can be replaced with more granular roles. For the complete list, see Permissions required to install Anthos Service Mesh.

BigQuery Data Transfer Service

BigQuery Data Transfer Service is now fully integrated with VPC Service Controls, and can be protected using a service perimeter. Please refer to VPC-SC supported products page for more info.

Cloud Billing

Recommendations for Compute Engine committed use discounts are now Generally Available. Recommendations provide you opportunities to optimize your compute costs by analyzing your VM spending trends and recommending committed use discount contracts. For understanding and purchasing committed use discount recommendations, see the documentation.

Cloud Composer

Preview: A new Logs tab has been added to the Environment details page.

Cloud Logging

Logs regionalization is now generally available. You can set the region in which you want to store your logs data. For information about this feature, refer to the Regionalization documentation.

Cloud Monitoring

The dashboard editor that lets you create and edit all dashboard widget types, including gauges, scorecards, and text boxes, is now Generally Available. With this editor, you can quickly configure dashboard widgets by using Basic Mode, you can access all aggregation options with Advanced Mode, and you can use Monitoring Query Language when you select MQL Mode. When you set the dashboard layout to mosaic mode, you can resize and reposition widgets. For more information, see Custom dashboards.

Cloud NAT

The ability to enable or disable Endpoint-Independent Mapping for your gateway is available in General Availability.

Cloud Run

You can now build and deploy source code to Cloud Run using a single command: gcloud beta run deploy --source .

Cloud SQL for PostgreSQL

Cloud SQL for PostgreSQL now supports the effective_cache_size flag.

Cloud Tasks

The relationship between your queues and your App Engine app has changed. If your queues only manage tasks with HTTP Targets, you no longer need to have an enabled App Engine app. For more information, see Managing the Cloud Tasks queue transition.

Compute Engine

Compute-optimized (C2) machines are now available in Montréal, in all three zones , northamerica-northeast1-a,b,c. For pricing, see VM instance pricing.

Google Cloud Armor

Google Cloud Armor Managed Protection Plus tier is now available in public preview.

Virtual Private Cloud

Access to Google APIs and services using Private Service Connect is now available in Preview.

DNS peering for private services access is now available in General Availability.

December 15, 2020

Cloud Build

Users can now create manual triggers to run builds at a specified time. To learn more about how to schedule your builds, see Scheduling your build.

Compute Engine

Preview: Accelerator-optimized (A2) machine types are now available in the following three regions:

  • Iowa, North America: us-central1-a,c
  • Netherlands, Europe: europe-west4-a,b
  • Singapore, APAC: asia-southeast1-c

Preview: NVIDIA® A100 GPUs are now available in the following three regions:

  • Iowa, North America: us-central1-a,c
  • Netherlands, Europe: europe-west4-a,b
  • Singapore, APAC: asia-southeast1-c

    For more information, see GPUs on Compute Engine.

Dataproc

Announcing the Beta release of the Dataproc cluster Stop/Start.

Announcing the General Availability (GA) release of the Dataproc Workflow Timeout feature, which allows users to set a timeout on their graph of jobs and automatically cancel their workflow after a specified period.

Dialogflow

GA (general availability) launch of Dialogflow CX.

CX Regionalization expanded to multiple regions globally.

CX Analytics for agent activity statistics.

CX Prebuilt agents for common agent use cases.

CX Customer-managed encryption keys (CMEK) to manage your own Dialogflow data encryption keys.

CX Security settings to control data redaction and data retention.

CX DTMF input for telephony partner integrations.

CX Parameter redaction to redact end-user parameter data from logs.

Google Cloud Armor

Third-party named IP address lists are now in general availability. Note that when Google Cloud Armor Managed Protection Plus tier is in general availability, your ability to use third-party named IP address lists will be affected by which Managed Protection tier your projects are in.

The following new WAF rules have been added in public preview:

  • Method enforcement
  • Scanner detection
  • Protocol attack
  • PHP injection attack
  • Session fixation
Virtual Private Cloud

The ability to connect VM interfaces other than nic0 to a Shared VPC is now available in Preview. This feature presently only works with individual VM instances, not with instance templates or managed instance groups.

December 14, 2020

Cloud Bigtable

Key Visualizer diagnostic messages are visible to all Cloud Bigtable customers. Review the message descriptions to learn how diagnostic messages can help you troubleshoot your Cloud Bigtable tables.

Cloud Billing

Cloud Billing Reports page now allows you to save your report views.

The Cloud Billing Reports in the Google Cloud Console allows you to view and visualize your Google Cloud spend over time. You can filter and break down your usage by different dimensions, including: time range, projects, products, SKUs, labels, and subaccounts. Prior to this update, if you wanted to save your filter settings, your only options were to bookmark or make a copy of your report's URL. To offer a better user experience, you can now save your custom report views and access your saved views.

For information on the saved views feature, refer to Saving and sharing report views in the Cloud Billing documentation.

Cloud CDN

Cache modes, TTL overrides and custom response headers are now supported on backend buckets and backend services, and are now Generally Available.

Cache modes allow Cloud CDN to automatically cache static content types, including web assets like CSS, JavaScript and fonts, as well as image and video content.

TTL overrides support fine-tuning how long Cloud CDN caches your responses, and custom response headers introduce a new {cdn_cache_status} variable that is populated with the cache status response.

The Google Terraform provider also supports these latest Cloud CDN features, including cache modes, TTL overrides, and custom response headers. Refer the documentation for compute_backend_bucket and compute_backend_service for how to configure and use the new features with Terraform.

Cloud Composer
  • New versions of Cloud Composer images: composer-1.13.3-airflow-1.10.9, composer-1.13.3-airflow-1.10.10, and composer-1.13.3-airflow-1.10.12. The default is composer-1.13.3-airflow-1.10.10. Upgrade your Cloud SDK to use features in this release.
  • Composer will now fail faster when the network settings in Private IP environments prohibit the download of publicly stored Python packages.
  • Composer Agent error messages are now more descriptive.
  • Composer will now check whether the Artifact Registry API is enabled during updates (if it is required).
Cloud Run

Cloud Run container instances can now process up to 250 concurrent requests, see Configuring maximum concurrency. The default is still 80.

Cloud TPU

Cloud TPU now supports Shared VPC

Shared VPC allows an organization to connect resources from multiple projects to a common VPC network to communicate with each other securely and efficiently using internal IPs from that network. This release enables connecting to Cloud TPU Nodes from Shared VPC networks.

Cloud Vision

OCR On-Prem General Availability (GA) release

OCR On-Prem is now generally available for approved customers. OCR On-Prem enables easy integration of Google image text recognition technologies into your on-premises solution.

For more information, refer to the product documentation. Approved customers can also view the marketplace entry .

Dataproc Metastore

The public Preview release of Dataproc Metastore is now available.

Legacy Dataproc Metastore services created during private Preview (prior to December 14, 2020 at 12:00 PM Pacific Standard Time) will be automatically deleted on January 29, 2021.

The Thrift endpoints of legacy services will continue to function normally, but certain pre-existing functionality such as metadata imports will cease to work. Furthermore, new features (including those announced on December 14, 2020) and bugfixes will not be available to legacy services.

To ensure you receive the newest features, patches, and stability, we strongly recommend you recreate legacy Dataproc Metastore services. Since the new metadata export feature is not available for legacy services, if you need help migrating metadata from a legacy service, the Dataproc Metastore team will be happy to assist you with a manual migration.

Please contact dataproc-metastore-support@google.com with any questions or to request help migrating metadata.

Google Cloud VMware Engine

All new VMware Engine private clouds now deploy with VMware vSphere version 7.0 and NSX-T version 3.0. Existing private clouds will be upgraded to vSphere version 7.0 and NSX-T version 3.0 over a period of time in December 2020 and January 2021.

See Service announcements for more details on the contents of this upgrade.

Increased maximum number of nodes in a private cloud cluster to 32. This change applies to new clusters. Existing clusters can be expanded up to 32 nodes after the upgrade to vSphere 7.0 version.

When VMware Engine replaces a failed node, node customizations now transfer from the failed node to the replacement node. Customizations include vSphere labels, vSphere custom attributes, vSphere tags, and any affinity and anti-affinity rules.

VMware Engine now advertises routes learned from a VPC to your VMware Engine private cloud network, and advertises routes learned from your private cloud to a VPC. This allows network communication between Google Cloud resources and private cloud resources.

Identity and Access Management

You can now use Cloud Monitoring to check when your service accounts and service account keys were used. This feature is generally available.

VPC Service Controls

Preview support for the following integration:

December 11, 2020

Dataflow

Workers now use the Java 11 runtime.

December 10, 2020

Anthos Anthos Config Management

Anthos Policy Controller now includes additional policies covering many of the CIS Kubernetes Benchmark 1.5.1 controls. To learn more, see the Constraint template library.

Anthos Policy Controller has been updated to include a more recent build of OPA Gatekeeper (hash: 1de87b6).

Support for Git submodules has been fixed in this version.

Anthos clusters on VMware

Anthos clusters on VMware 1.6.0-gke.7 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.6.0-gke.7 clusters run on Kubernetes 1.18.6-gke.6600.

Note: The fully supported versions offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware are 1.6, 1.5, and 1.4.

Users can use a credential configuration file with gkeadm (credential.yaml), which is generated during running the gkeadm create config command, to improve security by removing credentials from admin-ws-config.yaml.

Node Problem Detector and Node Auto Repair automatically detect and repair additional failures, such as Kubelet-API server connection loss (an OSS issue) and long-lasting DiskPressure conditions.

Preview: Repair administrator master VM failures by using the new command, gkectl repair admin-master.

Preview: Secrets Encryption for user clusters using Thales Luna Network HSM Devices.

Preview: Service Account Key Rotation in gkectl for Usage Metering, Cloud Audit Logs, and Google Cloud's operations suite service accounts.

Anthos Identity Service enables dynamic configuration changes for OpenID Connect (OIDC) configuration without needing to recreate user clusters.

Google Cloud's operations suite support for bundled Seesaw load balancing:

Metrics and logs of bundled Seesaw load balancers are now uploaded to Google Cloud through Google Cloud's operations suite to provide the best observability experience.

Cloud Audit Logs

Offline buffer for Cloud Audit Logs: Audit logs are now buffered on disk if not able to reach Cloud Audit Logs and can withstand at least 4 hours of network outage.

CSI volume snapshots

The CSI snapshot controllers are now automatically deployed in user clusters, enabling the users to create snapshots of persistent volumes and restore the volumes' data by provisioning new volumes from these snapshots.

Functionality changes:

  • Gkectl diagnose cluster and snapshot enhancements:

    • Added a --log-since flag to gkectl diagnose snapshot. Users can use it to collect logs of containers and nodes within a relative time duration in the snapshot.

    • Replaced the --seed-config flag with the --config flag in the gkectl diagnose cluster command. Users can use this command with the seed configuration to rule out the VIP issue and provide more debugging information of the cluster.

    • Added more validations in gkectl diagnose cluster.

  • Added iscsid support: Qualified storage drivers that previously required additional steps benefit from the default iscsi service deployment on the worker nodes.

Breaking changes:

Fixes:

  • Security fix: Resolve credential file references when only a subset of credentials are specified by reference.

  • Fixed vSphere credential update when CSI storage is not enabled.

  • Fixed a bug in Fluent Bit in which the buffer for logs might fill up node disk space.

Known issues:

  • gkectl update reverts your edits on clientconfig CR in 1.6.0. We strongly suggest that customers back up the clientconfig CR after every manual change.

  • Kubectl describe CSINode and gkectl diagnose snapshot might sometimes fail due to the OSS Kubernetes issue on dereferencing nil pointer fields.

  • The OIDC provider doesn't use the common CA by default. You must explicitly supply the CA certificate.

BigQuery

Updated version of Magnitude Simba ODBC driver includes bug fixes and enhancements such as support for BigNumeric data and improved driver logic.

Updated version of Magnitude Simba JDBC driver includes bug and security fixes and enhancements such as support for Java 11, SSL trust store, BigNumeric data, and version-agnostic fully-qualified class names.

Cloud Asset Inventory

Filestore resource type now available

The following Filestore resource type is now publicly available through the Cloud Asset APIs.

  • file.googleapis.com/Instance
Compute Engine

Preview: You can configure how your regional managed instance group distributes instances across zones by using capacity-aware distribution shapes, which can automatically deploy instances to zones where capacity is available and optionally prioritize the use of reservations.

You can migrate a VM instance from one network to another. This feature is Generally available.

Dataflow

Dataflow now supports custom containers as a Preview offering.

Workflows

Workflows launched a visualization feature. The Google Cloud Console now displays a visualization of the workflow during editing.

December 09, 2020

AI Platform Prediction

Runtime version 2.3 is now available. You can use runtime version 2.3 to serve online predictions with TensorFlow 2.3.1, scikit-learn 0.23.2, or XGBoost 1.2.1. Runtime version 2.3 does not support batch prediction.

See the full list of updated dependencies in runtime version 2.3.

AI Platform Training

Runtime version 2.3 is now available. You can use runtime version 2.3 to train with TensorFlow 2.3.1, scikit-learn 0.23.2, or XGBoost 1.2.1. Runtime version 2.3 supports training with CPUs, GPUs, or TPUs.

See the full list of updated dependencies in runtime version 2.3.

Cloud Asset Inventory

OS inventory management resource type now available

The following OS inventory management resource type is now publicly available through the Cloud Asset APIs.

  • compute.googleapis.com/Instance

This resource type provides information on the operating system, installed packages, and available package updates for a Compute Engine VM instance.

Cloud Composer
Cloud Load Balancing

Health check logging is now available in General Availability.

Compute Engine

Preview: Schedule-based autoscaling for managed instance groups lets you improve the availability of your workloads by scheduling capacity ahead of anticipated load.

GA: You can now access OS inventory data from Cloud Asset Inventory. For more information, see OS inventory and Cloud Asset Inventory integration.

GA: Per-group metrics let you autoscale a zonal managed instance group based on any Cloud Monitoring metric—for example, a Pub/Sub queue size or custom metrics from your application.

Config Connector

Config Connector version 1.33.0 is now available.

Added support for the ComputeProjectMetadata resource

Added resourceID field to ServiceUsageService and StorageNotification

Added computeResponseHeaders field to ComputeBackendService

Added maintenancePolicy.maintenanceExclusion field to ContainerCluster

Added description and disabled fields to LoggingLogSink

DataflowJobs can now be acquired via name

Added IAM support to BigtableTable

December 08, 2020

Cloud CDN

The Google Terraform provider now supports the latest Cloud CDN features, including cache modes, TTL overrides, and custom response headers.

Refer to the documentation for the compute_backend_bucket and compute_backend_service for how to configure and use the new features with Terraform.

Cloud Logging

Cloud Logging calculates the system logs-based metrics byte_count and log_entry_count on stored logs only, unlike user-defined logs-based metrics which are calculated on both stored and excluded logs. For more information, see System logs-based metrics.

This change is currently rolling out and affects all users after December 11, 2020.

Cloud Run for Anthos

Events for Cloud Run for Anthos version 0.17.5-gke.103 is now available for the following GKE minor version:

  • 1.18
  • 1.19
Dataproc

Restartable jobs: Added the ability for users to specify the maximum number of total failures when a job is submitted.

Image 2.0 preview

  • Using the n1-standard-1 machine type is no longer supported.

  • Changed default values of Spark SQL properties:

    • spark.sql.adaptive.enabled=true
    • spark.sql.autoBroadcastJoinThreshold =< 2% of executor memory.

The Dataproc Metastore Service is now available in the us-east4, europe-west2, asia-northeast1, and australia-southeast1 regions in addition to the existing us-central1 region.

New sub-minor versions of Dataproc images: 1.3.78-debian10, 1.3.78-ubuntu18, 1.4.49-debian10, 1.4.49-ubuntu18, 1.5.24-debian10, 1.5.24-ubuntu18, 2.0.0-RC20-debian10, and 2.0.0-RC20-ubuntu18.

Image 1.5:

Memorystore for Redis

Support for Redis AUTH on Memorystore for Redis is now Generally Available.

Private Catalog

Private Catalog launches an updated Cloud Console experience for cloud admins. The updates include more options for managing access control, sharing catalogs, and bulk editing solutions.

VPC Service Controls Workflows

Workflows is now available in the following regions:

  • asia-southeast1 (Singapore)
  • europe-west4 (Netherlands)

December 07, 2020

App Engine standard environment Java
  • Updated Java SDK to version 1.9.84.
  • Fixed missing com.google.appengine.repackaged.com.google.api.client.googleapis.extensions.appengine.auth.oauth2.AppIdentityCredential$AppEngineCredentialWrapper class in the appengine-remote-api.jar.
Cloud Asset Inventory

Cloud Bigtable resource type now available

The following Cloud Bigtable resource types are now publicly available through the Cloud Asset APIs.

  • bigtableadmin.googleapis.com/Cluster
  • bigtableadmin.googleapis.com/Instance
  • bigtableadmin.googleapis.com/Table
Cloud Data Loss Prevention

Added whole document classification support with the following infoType detectors:

  • DOCUMENT_TYPE/FINANCE/REGULATORY
  • DOCUMENT_TYPE/FINANCE/SEC_FILING
  • DOCUMENT_TYPE/HR/RESUME
  • DOCUMENT_TYPE/LEGAL/BLANK_FORM
  • DOCUMENT_TYPE/LEGAL/BRIEF
  • DOCUMENT_TYPE/LEGAL/COURT_ORDER
  • DOCUMENT_TYPE/LEGAL/LAW
  • DOCUMENT_TYPE/LEGAL/PLEADING
  • DOCUMENT_TYPE/R&D/PATENT
  • DOCUMENT_TYPE/R&D/SOURCE_CODE
  • DOCUMENT_TYPE/R&D/SYSTEM_LOG
  • DOCUMENT_TYPE/R&D/DATABASE_BACKUP
Cloud Logging

In the Logs Explorer, you can now stream your log entries in real time as Cloud Logging ingests them. To learn more, see Streaming logs.

Cloud Spanner

Cloud Spanner supports a new statement hint, LOCK_SCANNED_RANGES, allowing you to request an exclusive lock on a set of ranges scanned by a transaction.

Cloud Vision

Confidence score field addition for TEXT_DETECTION

You can now provide the flag TextDetectionParams.enable_text_detection_confidence_score to a TEXT_DETECTION request to get a confidence score for response information.

Virtual Private Cloud

Packet Mirroring direction control is now available in General Availability.

DNS peering for private services access is now available in Preview.

December 04, 2020

Cloud Monitoring

Slack notification channels: All notification channels created before November 20 have been fixed, and new notification channels will be created correctly. Notification channels created between November 21 and December 3 need to be manually updated, as described in Adding the Monitoring app to a Slack channel.

Cloud Vision

LABEL_DETECTION model upgrade

The latest LABEL_DETECTION model announced on October 16, 2020 has been promoted to the default model. The original model will still be available for another 60 days using "builtin/legacy".

December 03, 2020

Cloud Asset Inventory

Cloud TPU resource type now available

The following Cloud TPU resource type is now publicly available through the Cloud Asset APIs.

  • tpu.googleapis.com/Node
Cloud Composer
  • New versions of Cloud Composer images: composer-1.13.2-airflow-1.10.9, composer-1.13.2-airflow-1.10.10, and composer-1.13.2-airflow-1.10.12. The default is composer-1.13.2-airflow-1.10.10. Upgrade your Cloud SDK to use features in this release.
  • You can now set web server network access control using the v1 Composer API.
  • New metrics have been added to monitor web server CPU and memory usage:
    • CPU usage time
    • CPU reserved cores
    • Memory bytes used
    • Memory quota
  • During environment creation and updates, Composer will now verify whether you have chosen a region compliant with any location restriction organization policies. Error reporting has also been improved in cases where location restrictions cause environment updates to fail.
  • Composer versions 1.8.1 and 1.8.2 have been deprecated.
Google Cloud Armor

The Google Cloud Armor documentation set has been reorganized. Key updates include:

December 01, 2020

Config Connector

Config Connector version 1.32.0 is now available.

Added the resourceID field to Folder, BigQueryTable, BigQueryJob, and BigQueryDataset. (Issue #147 and #128)

Added the customResponseHeaders field to ComputeBackendService.

Added the maintenancePolicy.maintenanceExclusion field to ContainerCluster.

Added the description and disabled fields to LoggingLogSink.

Added "ORC" as a new available value to the CRD description of externalDataConfiguration.sourceFormat field in BigQueryTable.

Fixed the bug that the Bigtable Garbage Collection Policy can't be created via the Config Connector BigQueryGCPolicy resource. (Issue #300)

Security Command Center

Container Threat Detection, a built-in service of Security Command Center Premium, is now in general availability. Read these notes to learn about updates, usability improvements, and new features. See our blog post, Monitor and secure your containers with new Container Threat Detection, to learn more.

Container Threat Detection now supports Google Kubernetes Engine (GKE) versions on the Stable channel. There are currently no plans to add support for GKE version 1.14.

Activation latency for newly created clusters has been improved.

A bug that blocked some information from appearing in the the process section of Added Library Loaded findings is fixed.

A bug that blocked the proper display of the resource name for regional clusters in Added Library Loaded findings is fixed.

Container Threat Detection documentation includes updated information about compatibility with GKE and Virtual Private Cloud.

Read Using Container Threat Detection for more information.

November 30, 2020

Anthos on bare metal

Anthos on bare metal is generally available

Anthos on bare metal is a deployment option to run Anthos on physical or virtual servers, deployed on an operating system provided by you, without a hypervisor layer. Anthos on bare metal ships with built-in networking, lifecycle management, diagnostics, health checks, logging, and monitoring. Anthos on bare metal supports CentOS, Red Hat Enterprise Linux (RHEL), and Ubuntu—all validated by Google. With Anthos on bare metal, you can use your company's standard hardware and operating system images, taking advantage of existing investments, which are automatically checked and validated against Anthos infrastructure requirements.

Anthos on bare metal is available today, with either subscription or pay-as-you-go pricing. Anthos on bare metal lets you leverage existing investments in hardware, OS, and networking infrastructure. The minimum system requirement to run Anthos on bare metal is 2 nodes with a minimum total of 4 cores, 32 GB RAM, and 128 GB of disk space with no specialized hardware. The setup lets you run Anthos on bare metal on almost any infrastructure.

Anthos on bare metal uses a "bring your own operating system" model. It runs atop physical or virtual instances, and supports Red Hat Enterprise Linux 8.1/8.2, CentOS 8.1/8.2, or Ubuntu 18.04/20.04 LTS. Anthos provides overlay networking and L4/L7 load balancing. You can also integrate with your own load balancer such as F5 and Citrix. For storage, you can deploy persistent workloads using CSI integration with your existing infrastructure.

You can deploy Anthos on bare metal using one of the following deployment models:

  • A standalone model lets you manage every cluster independently. This is a good choice when running in an edge location or if you want your clusters to be administered independent of one another.
  • The multiple-cluster model lets central IT teams manage a fleet of clusters from a centralized cluster, called the admin cluster. This is more suitable if you want to build automation or tooling, or if you want to delegate the lifecycle of clusters to individual teams without sharing sensitive credentials such as SSH keys or Google Cloud service account details.

Like with all Anthos environments, a bare metal cluster has a thin, secure connection back to Google Cloud called Connect. After it's installed in your clusters, you can centrally view, configure, and monitor your clusters from the Google Cloud Console.

Anthos on bare metal, which is part of the Anthos 1.6 release, provides the following features and capabilities:

  • Kubernetes 1.18
  • Ubuntu/RHEL/CentOS support
  • Standalone and multiple-cluster architecture
  • In-place upgrades (minor and major)
  • Overlay networking, Ingress (L7), integrated load balancing (L4, L2-Mode)
  • Manual load balancing (F5, Citrix)
  • Installs behind proxy support
  • Preflight and health checks
  • Node maintenance mode
  • Cloud Monitoring and Cloud Logging
  • ACM, ASM, identity, hub or connect, billing, and pay-as-you-go
  • NVIDIA GPU support
  • Scales to 500 nodes
  • Virtual machine management (Kubevirt) preview

November 29, 2020

Config Connector

Config Connector version 1.31.1 is now available

Miscellaneous fixes and improvements

November 25, 2020

Cloud Monitoring

If you created Slack notification channels after November 20, 2020, your channels are not receiving notifications. For information about resolving this issue, see Adding the Monitoring app to a Slack channel.

Istio on Google Kubernetes Engine

Upgrading the cluster to GKE versions 1.17 and higher causes the built-in ingress gateway to be unavailable for approximately 5 minutes during the upgrade process. We recommend installing and managing separate user-defined gateways to avoid this issue, as described in Adding gateways.

SAP on Google Cloud

New SAP certifications: For SAP NetWeaver, the following Compute Engine virtual machine types that use the AMD CPU platform are certified by SAP:

  • n2d-highmem-48
  • n2d-highmem-64
  • n2d-highmem-80
  • n2d-highmem-96
  • n2d-standard-48
  • n2d-standard-64
  • n2d-standard-80
  • n2d-standard-96

For more information, see N2D general-purpose machine types.

You can now automate the deployment of SAP HANA in a SUSE Linux Enterprise Server high-availability (HA) cluster that uses the recommended TCP internal load balancer implementation for the virtual IP address.

For more information, see Automated deployment of Linux high-availability clusters for SAP HANA.

The sap_hana_ha/template.yaml file that is provided by Google Cloud to deploy SAP HANA in a SLES high-availability cluster that uses a virtual IP address (VIP) with an alias-IP implementation is deprecated.

As a replacement, use the new sap_hana_ha_ilb/template.yaml file that uses a TCP internal load balancer for the VIP, as described in Automated SAP HANA HA deployment on SLES with load-balancer VIP implementation.

The deprecated template and the corresponding documentation, Automated SAP HANA SLES HA deployment with alias-IP VIP implementation, will continue to be available for at least 12 months from today.

November 24, 2020

Cloud Run for Anthos

Starting in Jan 2021, Cloud Run for Anthos will support only the v1 version of the API: serving.knative.dev/v1

The following v1alpha1 and v1beta1 API versions have been deprecated in Knative v0.19.0 and no longer supported:

  • Service versions serving.knative.dev/v1alpha1 and serving.knative.dev/v1beta1.

  • Route versions serving.knative.dev/v1alpha1 and serving.knative.dev/v1beta1.

  • Revision versions serving.knative.dev/v1alpha1 and serving.knative.dev/v1beta1.

  • Configuration versions serving.knative.dev/v1alpha1 and serving.knative.dev/v1beta1.

If you use YAML to deploy your services, you should migrate to the serving.knative.dev/v1 API before Jan 2021.

If you use the gcloud command-line tool, make sure you have the latest version by running: gcloud components update

Compute Engine

New sole-tenant node types:

  • GA:

    • c2-node-60-240
    • m1-node-160-3844
    • m2-node-416-11776
    • n2-node-80-640
    • n2d-node-224-896
  • Beta:

    • m1-node-96-1433
Dialogflow

The Dialogflow CX test cases feature is now launched and documented.

Identity and Access Management

IAM Conditions: Starting on February 26, 2021, if a permission check encounters an unsupported attribute in a conditional role binding, it will never interpret that part of the condition as granting access.

To prevent access issues, limit the scope of conditions when necessary, especially if a condition checks the resource.name attribute.

November 23, 2020

BigQuery ML

BigQuery ML integration with AI Platform for Boosted Tree models is now generally available (GA). For more information, see the following documentation:

BigQuery ML integration with AI Platform for Deep Neural Network (DNN) models is now generally available (GA). For more information, see CREATE MODEL statement for Deep Neural Network (DNN) models.

Exporting BigQuery ML models to Cloud Storage and using them for online prediction is now generally available (GA). For more information, see Exporting models and the EXPORT MODEL statement.

Cloud Composer

Cloud Composer is now available in Los Angeles (us-west2).

Config Connector

Config Connector version 1.31.0 is now available

Added support for the ComputeTargetGRPCProxy resource

Added support for the ResourceManagerLien resource

Fixed issue where IAMPolicyMember and IAMPolicy resources cannot be deleted if an invalid configuration is applied (such as referencing a non-existent resource)

Fixed issue where notificationConfig.pubsub.topicRef was not usable

Google Cloud VMware Engine

Beginning in the middle of December 2020, VMware Engine will upgrade the VMware stack from version 6.7 to 7.0 and the NSX-T stack from version 2.5 to 3.0. Users affected by this upgrade will receive an email with planned maintenance dates and times.

For details about the upgrade and steps to prepare, see Service announcements.

November 20, 2020

Cloud Asset Inventory

Cloud Composer resource type now available

The following Cloud Composer resource type is now publicly available through the Cloud Asset APIs.

  • composer.googleapis.com/Environment
Cloud Composer

Composer version 1.12.5 or newer: File synchronization between buckets in Domain restricted sharing compliant environments has been improved. If your environment was created before November 20, 2020 you must upgrade your environment to access this feature

Compute Engine Dialogflow

The Dialogflow Messenger integration is now implemented for Dialogflow CX.

Agent Validation is now implemented for Dialogflow CX.

Network Intelligence Center

Firewall Insights is available in General Availability.

November 19, 2020

Cloud Build

Users can now transfer Cloud Build metrics to BigQuery through the BigQuery notifier, a new open-source notifier.

Cloud Functions

Cloud Functions has added support for a new runtime, .NET, in Preview. This runtime is based on .NET Core 3.1.

Cloud SQL for MySQL

Cloud SQL for MySQL now supports retention settings for automated backups. You can configure retention of your automated backups for shorter or longer periods (1 to 365 days). The default setting remains 7 days.

Cloud SQL for MySQL now supports retention settings for point-in-time recovery. You have the option of retaining logs, used for point-in-time recovery, for shorter periods (1 to 7 days). The default setting is 7 days.

Cloud SQL for PostgreSQL

Cloud SQL for PostgreSQL now supports retention settings for automated backups. You can configure retention of your automated backups for shorter or longer periods (1 to 365 days). The default setting remains 7 days.

Cloud SQL for PostgreSQL now supports retention settings for point-in-time recovery. You have the option of retaining logs, used for point-in-time recovery, for shorter periods (1 to 7 days). The default setting is 7.

Cloud SQL for SQL Server

Cloud SQL for SQL Server now supports retention settings for automated backups. You can configure retention of your automated backups for shorter or longer periods (1 to 365 days). The default setting iremains 7 days.