Google Cloud release notes

The following release notes cover the most recent changes over the last 30 days. For a comprehensive list, see the individual product release note pages .

You can see the latest product updates for all of Google Cloud on the Google Cloud release notes page.

To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly: https://cloud.google.com/feeds/gcp-release-notes.xml

October 26, 2020

BigQuery

BigQuery standard SQL now supports the SUBSTRING function. This function is generally available (GA).

Cloud Healthcare API

A new configuration field enable_full_index, has been added for HL7v2 stores. This field enables indexing for all HL7v2 message fields so that you can search on any message field using a new generic filter.

Cloud Monitoring

Enhancements to the pre-configured Compute Engine VM Instances dashboard. The inventory table now includes a Logging Agent Status column, and the Logging agent can be installed by using a UI workflow from the table.

Compute Engine

N2D Machine types are now available in London, europe-west2-a,b. See VM instance pricing for details.

N2D Machine types are now available in Eemshaven, Netherlands, europe-west4-a. See VM instance pricing for details.

October 23, 2020

Anthos

Anthos 1.5.1 is now available.

Updated components:

Anthos GKE on-prem

Anthos GKE on-prem 1.5.1-gke.8 is now available. To upgrade, see Upgrading GKE on-prem. GKE on-prem 1.5.1-gke.8 clusters run on Kubernetes 1.17.9-gke.4400.

Binary Authorization for GKE on-prem Preview is now available:

This release enables customers to generate credential configuration templates by using the gkectl create-config credential command.

Published the best practices for how to set up GKE on-prem components for high availability and how to recover from disasters.

Published the best practices for creating, configuring, and operating GKE on-prem clusters at large scale.

Fixes:

  • Fixed cluster creation issue when Cloud Run is enabled.
  • Fixed the false positive error in docker registry preflight check where REGISTRY_ADDRESS/NAMESPACE might be mistakenly used as the registry address to store the certs on a test VM, causing authentication errors.
Cloud Composer
  • New versions of Cloud Composer images: composer-1.12.4-airflow-1.10.6, composer-1.12.4-airflow-1.10.9, and composer-1.12.4-airflow-1.10.10. The default is composer-1.12.4-airflow-1.10.9. Upgrade your Cloud SDK to use features in this release.
  • The following Composer environment database metrics are now available in Cloud Monitoring: CPU usage, CPU cores, CPU utilization, memory usage, memory quota, memory utilization, disk usage, disk quota, disk utilization.
  • Worker liveness checks now also check whether the log syncing process is running.
  • Improved configuration synchronization with the Airflow web server, fixing an issue with loading logs.
  • The Airflow database will no longer disconnect after 30 seconds of query runtime.
  • Worker health checks will no longer overload the Airflow database.
Cloud Healthcare API

It is now possible to use `` to escape special characters in FHIR resources.

Cloud Run

Eventarc is now available in public preview.

Cloud Spanner

A new multi-region instance configuration is now available in North America - nam7 (Iowa/North Virginia/Oklahoma).

Dataproc

Decreased the minimum allowed value of Dataproc Scheduled Deletion LifecycleConfig.idleDeleteTtl (Dataproc API) and --max-idle flag (gcloud command-line tool) from 10 minutes to 5 minutes.

New sub-minor versions of Dataproc images: 1.3.74-debian10, 1.3.74-ubuntu18, 1.4.45-debian10, 1.4.45-ubuntu18, 1.5.20-debian10, 1.5.20-ubuntu18, 2.0.0-RC16-debian10, and 2.0.0-RC16-ubuntu18.

2.0 preview image versions:

Sole-tenant node cluster create or update requests to use preemptible secondary workers or attach autoscaling policies that create preemptible secondary workers are now correctly rejected.

All image versions:

  • Fixed a bug where files uploaded to Cloud Storage through the JupyterLab UI were incorrectly base64 encoded.

1.4 and 1.5 image versions:

  • SPARK-32708: Fixed SparkSQL query optimization failure to reuse exchange with DataSourceV2.

October 22, 2020

Access Context Manager

Access levels now support checking the Storage encryption (allowedEncryptionStatuses), Require admin approval (requireAdminApproval) and Require corp owned device (requireCorpOwned) attributes of requests originating from mobile devices.

BigQuery

The ST_GEOGFROMGEOJSON and ST_GEOGFROMTEXT geographic functions support a new make_valid parameter. If set to TRUE, the function attempts to correct polygon issues when importing geography data.

The ST_GEOGFROMTEXT function also supports a new planar parameter. If set to TRUE, the function treats imported WKT geometries as having planar edges.

These new function parameters are in Beta.

Updated version of Magnitude Simba ODBC driver includes improvements to performance, logging, OpenSSL support, and bug fixes.

Cloud Logging

Cloud Logging now calculates logs-based metrics from both ingested and excluded logs. In other words, you can now calculate logs-based metrics from logs without ingesting them into a Logs Bucket.

This change started rolling out October 18, 2020 and will finish rolling out October, 30 2020.

For more information, see Overview of logs-based metrics.

Logs Views are now available in Preview. Using Logs Views, you can control who has access to the logs within your Logs Buckets. For more information on this feature, refer to the Managing Logs Views guide.

Dataproc

Announcing the Alpha release of the Dataproc Persistent History Server, which provides a UI to view job history for jobs run on active and deleted Dataproc clusters.

October 21, 2020

Cloud Data Fusion

In Cloud Data Fusion versions before 6.2, there is a known issue where pipelines get stuck during execution. Stopping the pipeline results in the following error: Malformed reply from SOCKS server. To fix this, delete the Dataproc cluster, and then update the memory settings in the compute profile.

October 20, 2020

Cloud Load Balancing

For HTTP requests, the httpRequest.remoteIp and httpRequest.serverIp fields can include port information. For example 10.0.0.1:80.

Cloud Logging

Recent queries is now generally available (GA). To learn more, go to Recent queries.

October 19, 2020

BigQuery

BigQuery Audit Logs stopped using the following checks for redacting resource names for cross-project access and caller identities: The bigquery.jobs.create permission check and the internal setting for a project domain. Please review the documentation at Caller identities and resource names.

Cloud Logging

In the Logs Explorer you can now download your logs in JSON and CSV to your computer, Google Drive, or view them in a new tab. To learn more, see Downloading logs.

Cloud Run

Cloud Run is now available in the following regions:

  • asia-east2 (Hong Kong)
  • asia-northeast3 (Seoul, South Korea)
  • asia-southeast2 (Jakarta)
  • asia-south1 (Mumbai, India)
  • europe-west2 (London, UK)
  • europe-west3 (Frankfurt, Germany)
  • europe-west6 (Zurich, Switzerland)
  • southamerica-east1 (Sao Paulo, Brazil)

You can now purchase a custom domain via Cloud Domains using the Cloud Run user interface.

Compute Engine

Memory-optimized M1 machine types are available in Frankfurt europe-west3-a,b,c. Memory-optimized M2 machine types are available in Frankfurt, europe-west3-a,b. See VM instance pricing for details.

Dataproc Istio on Google Kubernetes Engine

A fix for a known issue where custom resources created in the istio-system namespace were deleted when upgrading from GKE 1.16 to 1.17 and 1.18 is available in R33.

Upgrade to one of the following unaffected versions:

  • 1.17.12-gke.1501 and higher
  • 1.18.9-gke.1501 and higher

The issue only occurs during upgrades, so new clusters created in earlier versions are also unaffected.

Pub/Sub

Pub/Sub message ordering is now available in GA.

October 16, 2020

Cloud Bigtable

A tutorial is now available that demonstrates how to send a Cloud Bigtable read request using a Cloud Functions HTTP(S) request.

Cloud Vision

LABEL_DETECTION model upgrade

The LABEL_DETECTION model will undergo an upgrade over the next 90 days to a newer version. The API interface and client library will be the same as with the previous version. The API follows the same Service Level Agreement.

Please note that you have 30 days from today to test the new model by specifying "builtin/latest" in the model field of the Feature object while requesting image annotation. At the end of that period, it will be promoted to the default model accessible as "builtin/stable". After that event, the original model will still be available for another 60 days using "builtin/legacy".

If you encounter problems with this upgrade, please contact Vision API engineering team by submitting a ticket in the private issue tracker.

Dataproc Document AI

Document AI Preview released

The following beta and preview features are available in API version v1beta3:

  • General processors: Document OCR (Optical Character Recognition), form parser, and document splitter.
  • Lending processors: W9, 1040, W2, 1099-MISC, and 1003 parsers, as well as lending document splitter & classifier.
Identity and Access Management

Credential Access Boundaries are now generally available. Use Credential Access Boundaries to downscope the permissions that a short-lived credential can use to access a Cloud Storage bucket.

October 15, 2020

Cloud Bigtable

The steps to create a new Cloud Bigtable instance and edit an existing instance have been streamlined and improved in the Google Cloud Console.

Cloud Billing

Discount sharing for committed use discounts is now Generally Available. With discount sharing enabled, you can apply your purchased commitments across multiple projects within a single Cloud Billing account. Discount sharing helps you minimize the overhead of managing each of your commitments individually and provides increased flexibility so that you can use the compute options that best suit your needs, while also increasing cost predictability.

Cloud Composer
  • New versions of Cloud Composer images: composer-1.12.3-airflow-1.10.6, composer-1.12.3-airflow-1.10.9, and composer-1.12.3-airflow-1.10.10. The default is composer-1.12.3-airflow-1.10.9. Upgrade your Cloud SDK to use features in this release.
  • Cloud Build logs from the tenant project are now published in the Composer logs. They are available under the log name build-log-webserver.
  • Airflow DAG processor manager logs are now published in the Composer logs. They are available under the log name dag-processor-manager.
  • If an update operation fails, links to the specific Cloud Build log will now be included in the error message.
  • Compatibility with Domain Restricted sharing has been improved. Upgrading your environment to the newest version of Composer can now enable or disable its compatibility with Domain Restricted Sharing based on your organization policy.
  • Setting or updating the machine type of the Airflow web server or Cloud SQL instance in Composer versions that don't support this feature (older than composer-1.7.2) will now return an error instead of failing silently.
  • Environments will now fall back to the in-cluster build when PyPI package installation fails due to Cloud Build unavailability under your VPC Service Controls configuration.
  • Airflow 1.10.10:
    • Fixed an issue with async DAG bag loading.
    • Task instance details will now render properly, even if there are no DAG runs related to the instance.
Cloud Operations Suite

The Ops Agent is now available in Preview. Ops Agent provides a single agent to collect logs and metrics on Compute Engine instances.

Note that the Ops Agent uses new configuration files that are not compatible with the standalone Cloud Monitoring and Cloud Logging agents.

Cloud Run

You can now specify a minimum number of container instances to be kept warm and ready to serve requests, for services requiring reduced latency and fewer cold starts.

Cloud Spanner

A new multi-region instance configuration is now available in North America - nam9 (North Virginia/Iowa/South Carolina/Oregon).

Compute Engine

Support for 1500 MTU in VPC networks is now Generally available.

Config Connector

Support export sub-command in the config-connector CLI

Add support for the AccessContextManagerServicePerimeter resource

Add support for Folder-level IAM Audit Configs

Fix deadLetterTopicRef in the PubSubSubscription resource (Issue #281)

Identity and Access Management

If a role binding in an IAM policy refers to a deleted member (for example, deleted:user:tamika@example.com?uid=123456789012345678901), you can now add role bindings for a newly created member with the same name (in this case, user:tamika@example.com). The role bindings always apply to the newly created member.

For details, see the documentation for policies with deleted members.

Virtual Private Cloud

Support for 1500 MTU in VPC networks is now available in General Availability.

October 14, 2020

BigQuery

Dynamic SQL is now generally available (GA). Dynamic SQL lets you generate and execute SQL statements dynamically at runtime. For more information, see EXECUTE IMMEDIATE.

BigQuery standard SQL now supports the following new functions. These functions are generally available (GA).

BigQuery now supports the following new statements. These statements are generally available (GA).

BigQuery standard SQL now supports DATE arithmetics operators.

BigQuery now supports Unicode table names. For more information, see Table naming.

Queries can now have duplicate column names.

Cloud VPN

Classic VPN partial deprecation

Starting on October 31, 2021, you will no longer be able to do the following:

  • Create new Classic VPN tunnels using static routing (route based or policy based) that connect to another Classic VPN gateway
  • Create new Classic VPN tunnels using static routing (route based or policy based) that connect a Google Cloud Virtual Private Cloud (VPC) network to another cloud provider's network
  • Create new Classic VPN tunnels using dynamic routing (all configurations)

You can continue to create the following types of connections and get support for them:

  • VPN tunnels using static routing from Classic VPN gateways to on-premises VPN gateways and from on-premises VPN gateways to Classic VPN gateways
  • VPN tunnels using static routing from a Classic VPN gateway to and from a Compute Engine virtual machine (VM) acting as a VPN gateway

Although Google will not proactively disable existing connections on the deprecation date, deprecated Classic VPN configurations will no longer receive regular updates or maintenance.

For more information, see the Classic VPN partial deprecation page for a video tutorial and documentation to help you migrate, as soon as possible, to our more reliable High Availability Cloud VPN solution.

Compute Engine

Compute-optimized (C2) machine types are now available in the following regions and zones:

  • Finland: europe-north1-a,b,c
  • Seoul: asia-northeast3-a,b,c

See VM-instance-pricing for details.

Identity Platform

Sign in with Apple is now supported.

October 13, 2020

Anthos Service Mesh

1.4.10-asm.19 is now available

You can now allow an experimental feature to exceed 4GB of memory usage.

Cloud Domains

Cloud Domains is available in Preview. Cloud Domains enables you to search, register, and manage domain names with Google Cloud.

Cloud Functions

In runtimes that use buildpacks you can now configure aspects of your build by setting build configuration variables. See Using Environment Variables for more information. In Preview.

Cloud Logging

We've renamed the Logs Viewer (Preview) to the Logs Explorer. The Logs Explorer offers a robust set of tools for analyzing your logs data and is now the default viewer for Cloud Logging. To learn more, see Using the Logs Explorer.

The Logs Viewer (Classic) is now called the Legacy Logs Viewer. It will continue to be available and maintained until March 2021, but won't be actively developed further.

Cloud Run

You can now control egress traffic from a service and route all outbound requests to your VPC network. This allows you to configure a static outbound IP address by leveraging Cloud NAT.

Cloud Run for Anthos
  • Cloud Run for Anthos on Google Cloud version 0.17.2-gke.1 is now available for the following GKE minor version:

    1.16

  • Fixes the security issue, ISTIO-SECURITY-2020-010 for Cloud Run for Anthos on Google Cloud clusters running on 1.15+ k8s version.

Cloud Spanner

CHECK constraints is now generally available, allowing you to define a boolean expression on the columns of a table and require that all rows in the table satisfy the expression. For more information, see Creating and managing check constraints.

Generated columns support is now generally available, allowing you to define columns that are computed from other columns in a row. For more information, see Creating and managing generated columns.

Cloud Talent Solution Job Search

Cloud Talent Solution has launched the v4 version of the API. Migrate to Cloud Talent Solution v4 by October 14, 2021 to continue using Cloud Talent Solution.

As of today Cloud Talent Solution versions v3, v3p1beta1, and v4beta1 are deprecated. Deprecated means that these versions will continue to work until October 13, 2021, at which time these versions will be shut down. Migrate to Cloud Talent Solution v4 by October 14, 2021 to continue using Cloud Talent Solution.

When using orderBy to order job search results by distance_from from the search location, equidistant jobs from the center of the search location will be tie-broken based on each job's relevance to the search keywords. Previously, jobs in this scenario weren't primarily tie-broken based on each job's relevance to the search keywords.

When using the EmploymentType as part of HistogramQuery, facet counts for CONTRACTOR no longer also include facet counts for CONTRACT_TO_HIRE.

HistogramQuery facet counts no longer differ from the number of jobs returned when filtering search results by a given facet.

CTS has made improvements to the handling of accented characters and gendered terms in job titles and search keywords.

Dataproc

New sub-minor versions of Dataproc images: 1.3.72-debian10, 1.3.72-ubuntu18, 1.4.43-debian10, 1.4.43-ubuntu18, 1.5.18-debian10, 1.5.18-ubuntu18, 2.0.0-RC14-debian10, and 2.0.0-RC14-ubuntu18.

Storage Transfer Service

October 12, 2020

Anthos GKE on AWS

GKE on AWS 1.5.0 supports volume snapshots.

Cloud Logging

Cloud Logging has deprecated the following two logs-based metrics related to exclusions:

  • logging.googleapis.com/excluded_log_entry_count
  • logging.googleapis.com/excluded_byte_count

Cloud Logging will stop populating these metrics on October 1, 2021.

Cloud Run

You can now allocate 4 vCPUs to container instances of Cloud Run services.

Cloud SQL for MySQL

Cloud SQL now offers "deny maintenance periods". With deny maintenance periods, you can prevent automatic maintenance from occurring during a specific time period. For example, the end-of-year holiday season is a time of peak load that requires heightened focus on infrastructure stability for many retail businesses. By setting a deny maintenance period from mid-October to mid-January, these businesses can prevent planned upgrades from Cloud SQL during their busiest time of year.

Cloud SQL for PostgreSQL

Cloud SQL for PostgreSQL now offers IAM database authentication to help you better monitor and manage access for users and service accounts to databases. This feature allows users and service accounts to use IAM credentials to log into PostgreSQL instances. To learn more about how IAM database authentication works, see the Overview of Cloud SQL IAM database authentication. To configure an instance, see Configuring instances for IAM database authentication. To create users or service accounts, see Creating and managing users that use IAM database authentication.

Cloud SQL now offers "deny maintenance periods". With deny maintenance periods, you can prevent automatic maintenance from occurring during a specific time period. For example, the end-of-year holiday season is a time of peak load that requires heightened focus on infrastructure stability for many retail businesses. By setting a deny maintenance period from mid-October to mid-January, these businesses can prevent planned upgrades from Cloud SQL during their busiest time of year.

Database auditing in Cloud SQL for PostgreSQL is available through the open-source pgAudit extension. Using this extension, you can selectively record and track SQL operations performed against a given database instance.

The pgAudit extension helps you configure many of the logs often required to comply with government, financial, and ISO certifications.

Cloud SQL for SQL Server

Cloud SQL now offers "deny maintenance periods". With deny maintenance periods, you can prevent automatic maintenance from occurring during a specific time period. For example, the end-of-year holiday season is a time of peak load that requires heightened focus on infrastructure stability for many retail businesses. By setting a deny maintenance period from mid-October to mid-January, these businesses can prevent planned upgrades from Cloud SQL during their busiest time of year.

Compute Engine

N2 machine types are now available in the following four regions and zones:

  • Las Vegas: us-west4-a,b,c
  • Montréal: northamerica-northeast1-a,b,c
  • Finland: europe-north1-a
  • Hong Kong: asia-east2-a,b,c

For pricing details, see VM instance pricing.

Migrate for Compute Engine

Support added for migration of VMs from vSphere configured with CSM firmware type setting.

Resource Manager

You can now customize who receives notifications from GCP with Essential Contacts. This feature is available in preview. For more information, see Managing contacts for notifications.

October 11, 2020

Cloud Run

When a container instance needs to be shut down, it now receives a SIGTERM signal. If handled, CPU is allocated for up to 10 seconds before the container is shut down.

October 09, 2020

Identity and Access Management

The documentation now provides details about service agents for all publicly available services. A service agent is a special type of service account that is created and managed by Google, and is used by Google Cloud services to access your resources.

Pub/Sub

Pub/Sub Lite is now available in GA.

October 08, 2020

Cloud Billing

Cloud Billing budget settings have been updated to support credits by credit type. We have added all possible Cloud Billing credit types to the budget scope, allowing you to set your budget amount to include or exclude specific credits by type, such as promotional credits, committed use discounts, and free tiers. Previously, the credits setting was an optional checkbox when setting the budget amount — the Include credits in cost option — and not the granular options now available in the budget scope. The previous credits checkbox setting could only include either all of the available credits or none of the credits.

For budgets that were set up before the granular credits budget scope feature became available:

  • In the budget amount, if you had enabled the Include credits in cost option, then all credits are included in the cost calculation.
  • In the budget amount, if you had deselected the Include credits in cost option, then none of the credits are included in the cost calculation.

To implement the new credits scope feature in existing budgets, edit the budget's credit settings.

Read more about credits and budget scope options in our documentation.

Cloud Healthcare API

The Consent Management API is available in beta.

Cloud Load Balancing

External HTTP(S) Load Balancing is now supported for App Engine, Cloud Functions, and Cloud Run services. To configure this, you will need to use a new type of network endpoint group (NEG) called a Serverless NEG.

This feature is now available in General Availability.

Cloud Spanner

The following updates to Cloud Spanner standard SQL are now available :

Migrate for Anthos

Support for migrating Windows VM workloads has moved from the Beta stage to general availability.

This release adds full support for migrating Windows VM workloads to the Google Cloud Console, including the ability to create a Windows migration source. See Migrating a Windows VM for more.

Migrate for Anthos provides tools that you run on a Linux or Windows VM workload to determine the workload's fit for migration to a container. See Using the Linux discovery tool and Using the Windows discovery tool for more.

Custom Services Blocklist support added which lets you define a list of services to disable in a migrated container. See Custom Services Blocklist for more.

The image field value of the GenerateArtifactsFlow CRD defines the names and locations of two images created from a migrated VM. In previous releases, the names contained a predefined tag.

To ensure that the tag value is unique, the format of the tag has changed for this release to specify the timestamp of the migration.

You can also explicitly set the tag if you prefer to another value. See Setting the name of the container image for more.

When you deploy your migrated Windows containers to a cluster, you can now use a Group Managed Service Account (gMSA) to execute the container under a specific service account identity. See Configuring gMSA for more.

171123825: In some cases, migration process might fail, and Cloud Logging indicate errors such as:

"failed to load map, error 6"

or:

"failed in domap for addition of new path sdd"

Workaround: Delete the migration and restart it. In rare cases, a re-installation of the product is required.

170706786: The Linux Discovery Tool might return exit code 0 even when all information was collected successfully.

Workaround: Make sure you run the tool as a 'root' user or as a user with full sudo access.

170627229: Migrated workload of a JBoss application might fail at startup. Cloud Logging indicates such an error as:

ERROR [org.jboss.as.server] (Controller Boot Thread) ...: 
Caught exception during boot: java.lang.IllegalStateException: ...: 
Could not rename 
/opt/jboss-7.1.1/standalone/configuration/.../standalone_xml_history/current
to
/opt/jboss-7.1.1/standalone/configuration/.../standalone_xml_history/... 

Workaround: Backup and then delete the directory mentioned in the error message above. For example:

/opt/jboss-7.1.1/standalone/configuration/.../standalone_xml_history/current

167656057: Installation on a GKE cluster with ACM might fail. Indication of the error can be seen in the Migrate for Anthos upgrade job, in the v2k-system namespace.

For example:

kubectl logs -n v2k-system controllers-upgrade-fzlmz

Shows this error:

failed to validate admission controller - admission webhook "validation.gatekeeper.sh" does not support dry run

Workaround: gatekeeper is an ACM component. Manually deleting the upgrader job fixes the issue.

For example:

kubectl delete job -n v2k-system controllers-upgrade

157062328: In some cases, adding a service to the blocklist using a configmap will not actually stop that service from running on the deployed workload.

Workaround: Disable the service using in the Dockerfile (rather than a config-map), and rebuild the image.

163800225: kubectl port-forward might not work properly for a deployed workload.

Please contact support for more information.

171173082: Mistakenly creating a local VMware source on a Cloud-based cluster, normally used only in an on-prem migration, results in the source being in PROCESSING state forever.

For example, you use migctl to check the source status:

migctl source status local-vmw-src

The State displays as:

PROCESSING Message: Post "https://1.2.3.4/sdk": context deadline exceeded

Workaround: Delete the local VMware source, and create a remote/streaming VMware source.

170604382: Running migctl when not connected to a cluster results in a panic error such as the one below, followed by a stack-trace:

migctl setup install panic: Cannot create kubernetes client

Workaround: Connect a cluster, and re-run migctl.

Security Command Center

Event Threat Detection, a built-in service of Security Command Center Premium, now includes two new detectors to monitor your organization's BigQuery resources. The detectors identify data exfiltration - resources saved outside of your organization or attempts to access protected data.

Read more about available detectors in Event Threat Detection conceptual overview.

The Security Command Center API now includes a severity field for Findings. This feature is available using Security Command Center's v1p1beta1 API.

October 07, 2020

Cloud Billing

Project-level tax information in BigQuery Export tables: Starting on September 1 2020, your daily cost detail data in BigQuery shows taxes broken down by project, instead of aggregating taxes into a single line item. You don't need to make any changes to access the data.

If you have queries or visualizations that depend on tax data, you might need to update the queries to account for these changes.

Learn about viewing your tax data in BigQuery.

Cloud Composer
  • New versions of Cloud Composer images: composer-1.12.2-airflow-1.10.6, composer-1.12.2-airflow-1.10.9, and composer-1.12.2-airflow-1.10.10. The default is composer-1.12.2-airflow-1.10.9. Upgrade your Cloud SDK to use features in this release.
  • Added the log entry labels version_id and instance_id to differentiate the logs of different Airflow web server instances.
  • Airflow database upgrade logs are now published in the Composer logs under a separate log name.
  • Cloud Storage syncing logs are now published together in the Composer logs under a separate log name. They can be separated based further on pod_id.
  • Fixed upgrade rollback failures due to a mismatch in API versions error.
  • Improved handling of errors caused by a missing plugins/ directory in the Cloud Storage bucket.
  • Backported an Airflow change that fixes an issue with Airflow 1.10.9 and 1.10.10 that causes MySQL to deadlock on the rendered_task_instance_fields table when using DAG serialization.
  • Fixed a bug that undercharged customers for App Engine storage.
  • Backfilled GKE API versions to make older Composer environments GKE 1.16-proof.
  • Fixed an issue that caused bq load commands for files larger than 100MB to fail with a RedirectMissingLocation error.
Cloud Healthcare API

The Cloud Healthcare API offers single-region support in the southamerica-east1 (Osasco (São Paulo), Brazil) region.

The Cloud Healthcare API offers single-region support in the australia-southeast1 (Sydney, Australia) region.

Config Connector

Add support for the DataflowFlexTemplateJob resource

Add the transformNameMapping field to DataflowJob

Add the auditConfigs field to IAMPolicy

Add the loadBalancerType, datapathProvider, and notificationConfig fields to ContainerCluster

Add the artifacts and options fields to CloudBuildTrigger

Add support for the GRPC protocol for ComputeBackendService

Add logic to auto-trigger server-side apply metadata on resources on K8s clusters with server-side apply enabled (i.e. K8s 1.16+)

Fix issue where kubectl get gcp did not include IAMPolicy, IAMPolicyMember, and IAMAuditConfig resources (Issue #286)

October 06, 2020

BigQuery

Authorized user-defined functions (UDFs) are now generally available (GA). Authorized UDFs let you share query results without giving access to the underlying tables. For more information, see Authorized UDFs.

The Cloud Console now lets you opt in to search and autocomplete powered by Data Catalog. This feature is in beta.

Cloud NAT

The ability to enable or disable Endpoint-Independent Mapping for your gateway is now available in Preview.

Cloud Run

Cloud Run now supports request timeouts up to 60 minutes. However, timeouts greater than 15 minutes are a beta feature.

Dataproc

New sub-minor versions of Dataproc images: 1.3.71-debian10, 1.3.71-ubuntu18, 1.4.42-debian10, 1.4.42-ubuntu18, 1.5.17-debian10, 1.5.17-ubuntu18, 2.0.0-RC13-debian10, and 2.0.0-RC13-ubuntu18.

Image 1.4

Image 1.5

  • Upgraded Spark to version 2.4.7.
  • Installed google-cloud-bigquery-storage package by default in the Anaconda component.
  • Changed default value of zeppelin.notebook.storage in zeppelin-site.xml to "org.apache.zeppelin.notebook.repo.GCSNotebookRepo".

Image 2.0

  • Updated HBase to version 2.2.6.
  • Installed google-cloud-bigquery-storage in default conda environment.
  • Changed default value of zeppelin.notebook.storage in zeppelin-site.xml to "org.apache.zeppelin.notebook.repo.GCSNotebookRepo".
Istio on Google Kubernetes Engine

There is a known issue with the upgrade from GKE 1.16 to 1.17. Any custom resources you created in the istio-system namespace are deleted during an upgrade to 1.17. These resources must be manually recreated. We recommend not upgrading clusters with the Istio addon to 1.17 until the fix is rolled out. The issue only occurs during upgrades, so new clusters are not affected.

The fix was not included in release R31 as previously reported

Memorystore for Redis

Added support for Redis AUTH to Memorystore for Redis.

October 05, 2020

Cloud Monitoring

Alerting is now available for Monitoring Query Language (MQL). For more information, see Alerting policies with MQL

Compute Engine

You can use OS Login in VPC Service Controls. This feature is in Beta stage support.

Container Registry

Starting October 5, the Container Registry Service Agent is granted the Container Registry Service Agent IAM role by default when you enable the Container Registry API. Previously, this account was granted the Editor role.

To learn more about this change and how to update the role on an existing Container Registry service account with Editor permissions, see the Container Registry access control documentation

VPC Service Controls

Beta stage support for the following integration:

October 02, 2020

Anthos GKE on AWS

Anthos GKE on-AWS 1.5.0-gke.6 is now available and clusters run on 1.16.15-gke.700 and v1.17.9-gke.2800. To upgrade your clusters, perform the following steps:

  1. Upgrade your Management service to 1.5.0-gke.6.
  2. Upgrade your user clusters to 1.16.15-gke.700 or v1.17.9-gke.2800

Workload identity (preview) lets you bind Kubernetes service accounts to AWS IAM accounts with specific permissions. Workload identity blocks unwanted access to cloud resources with AWS IAM permissions. With workload identity, you can assign different IAM roles to each workload. Fine grained permissions control allows you to follow the principle of least privilege. For more details, see Creating a user cluster with workload identity

You can now route traffic from the GKE on AWS management service and Connect through an HTTP/HTTPS proxy. For more details, see Using a proxy with GKE on AWS

Improved installation experience

  • This version enables installation and upgrade by using any Google Cloud–authenticated service account. You no longer need to be on the allowlist to access GKE on AWS components..

  • Additional preflight checks enforce enablement of required Google Cloud APIs. See Google Cloud requirements for more information.

When creating multiple multiple management clusters, users may have seen name collisions with S3 bucket. Now, you can specify a custom name for your S3 bucket to avoid naming conflicts.

Compute Engine

N2D machine types are available in The Dalles, Oregon, in the us-west1-c zone. For more information, see the VM instance pricing page.

C2 machine types are now available in Sydney, Australia australia-southeast1-b. See the VM instance pricing page for details.

Config Connector

Added Cloud IAM support for ComputeImage.

Fixed an issue where an IAMPolicy cannot be deleted when the externally referenced resource does not exist.

Fixed an infinite diff condition on spec.minMasterVersion.

Istio on Google Kubernetes Engine

There is a known issue with the upgrade from GKE 1.16 to 1.17 versions lower than 1.17.9-gke.6300 (R30 or earlier). Any custom resources you created in the istio-system namespace are deleted during an upgrade to 1.17 (R30 or earlier). These resources must be manually recreated. We recommend that you upgrade only to R31 or a later version that doesn't have the issue. The issue only occurs during upgrades, so new clusters are not affected.

October 01, 2020

BigQuery

BigQuery table-level access control is now generally available. For more information, see Introduction to table access controls .

Cloud CDN

Added a new tutorial for delivering HTTP and HTTPS content over the same hostname when using Cloud CDN. While many browsers enforce the use of Transport Layer Security (TLS) and disallow non-secure content delivery, there are still use cases where non-secure delivery and secure delivery must be allowed over the same hostname.

Cloud Load Balancing

Added a new tutorial for delivering HTTP and HTTPS content over the same hostname when using Cloud CDN. While many browsers enforce the use of Transport Layer Security (TLS) and disallow non-secure content delivery, there are still use cases where non-secure delivery and secure delivery must be allowed over the same hostname.

Compute Engine

N2D machine types are now available in all three zones of us-east1-b,c,d in Moncks Corner, South Carolina. See VM instance pricing for details.

Dataproc

Launched Dataproc integration with Compute Engine sole-tenant nodes, which allows users to create a cluster in a sole-tenant node group.

Memorystore for Redis

Added Cloud Console UI support for creating Redis instances with Shared VPC using private services access.

September 30, 2020

Cloud Data Fusion

This release is in parallel with the CDAP 6.2.2 release.

Cloud Data Fusion now supports autoscaling Dataproc clusters.

Cloud Data Fusion now displays the number of pending preview runs, if any, before the current run. In the Studio, the number of pending runs is displayed under the timer.

Improved performance for skewed joins by including Distribution in the Joiner plugin settings.

Wrangler now supports BigQuery views and materialized views.

Cloud Run for Anthos

Cloud Run for Anthos on Google Cloud version 0.17.2-gke.1 is now available for the following GKE minor version:

1.17

Cloud Talent Solution Job Search

Cloud Talent Solution's consumption pricing and quota limits will change effective January 4, 2021. The new pricing and quota limits are intended to better address the variety of use cases the talent industry has for managing and searching for jobs.

Dataflow

Dataflow now supports Flex Templates in GA.

Dataproc

Creating clusters and instantiating workflow requests that succeed even when the requester did not have ActAs permission on the service account now generate a warning field in the audit log request.

New sub-minor versions of Dataproc images: 1.3.70-debian10, 1.3.70-ubuntu18, 1.4.41-debian10, 1.4.41-ubuntu18, 1.5.16-debian10, 1.5.16-ubuntu18, 2.0.0-RC12-debian10, and 2.0.0-RC12-ubuntu18.

All supported images

Upgraded Conscrypt to the 2.5.1 version.

Image 1.5

Image 1.5 and Image 2.0 Preview

Image 2.0 preview

  • YARN-9607: Auto-configuring rollover-size of IFile format for non-appendable filesystems.

  • YARN-9525: IFile format is not working against s3a remote folder.

Firestore

September 29, 2020

Anthos Service Mesh

1.6.11-asm.1, 1.5.10-asm.2, and 1.4.10-asm.18

Fixes the security issue, ISTIO-SECURITY-2020-010, with the same fixes as Istio 1.6.11. These fixes were backported to 1.6.11-asm.1, 1.5.10-asm.2 and 1.4.10-asm.18. For more information, see the Istio 1.6.11 release notes.

For details on upgrading Anthos Service Mesh, refer to the following upgrade guides:

1.6

1.5

1.4

BigQuery ML

Time series models now let you change DATA_FREQUENCY from the default value (AUTO_FREQUENCY) when forecasting multiple time series using TIME_SERIES_ID_COL.

Cloud CDN

Improvements to Cloud CDN's request collapsing behavior: If your deployment serves large objects, these improvements reduce origin load during revalidation and cache fill. Live video workloads will see the largest benefit. For more information, see Support for byte range requests.

Cloud Profiler

Cloud Profiler history view is available in beta. For more information, see Viewing historical trends.

Cloud Run

Cloud Run now supports server-side HTTP and gRPC streaming.

Dataflow

You can now use a network tags parameter to add network tags to all worker VMs that execute a particular Dataflow job.

Traffic Director

September 28, 2020

Cloud Billing

Budget alert email notifications: Cloud Billing Budgets functionality has been updated to offer more control over recipients of alert emails. In May 2020, we announced that you can use Monitoring notifications to customize your budget to send alert emails to up to five additional email recipients you specify, in addition to the default email recipients. Now, you can also opt out of the default email settings, choosing not to send budget alert emails to Billing Account Administrators and Billing Account Users on the target Cloud Billing account (that is, every user assigned a billing role of either roles/billing.admin or roles/billing.user). To opt out of the default behavior, in the Cloud Billing budget's Manage notifications settings, deselect Email alerts to billing admins and users. Read more about email notification settings in our documentation.

Committed use discounts (CUDs) are now available to purchase for Cloud VMware Engine. CUDs provide discounted prices in exchange for your commitment to use a minimum level of resources for a specified term. With spend-based committed use discounts for Cloud VMware Engine, you can earn a deep discount off your cost of use in exchange for committing to continuously use VMware Engine nodes in a particular region for a 1- or 3-year term. See the documentation for more details.

Cloud Composer
  • New versions of Cloud Composer images: composer-1.12.1-airflow-1.10.6, composer-1.12.1-airflow-1.10.9, and composer-1.12.1-airflow-1.10.10. The default is composer-1.12.1-airflow-1.10.6. Upgrade your Cloud SDK to use features in this release.
  • In-cluster image build logs will now appear in Monitoring under separate log names build-log-*, instead of in the Composer Agent logs.
  • You can now set or update machine types for the Airflow web server and Cloud SQL instance for any environment running a Composer version 1.7.2 or newer, regardless of Airflow version.
  • Airflow 1.10.6, 1.10.9, and 1.10.9: An Airflow change has been backported that lowers the network cost of DAG serialization.
  • Airflow 1.10.9: An Airflow change has been backported that improves GKE cluster resource usage, improving overall Airflow performance.
Cloud Spanner Cloud Storage

Tokyo+Osaka dual-region (asia1) launched.

Compute Engine

N2D machine types are available in The Dalles, Oregon, the us-west1-b zone. For more information, see the VM instance pricing page.

Google Cloud VMware Engine

Committed use discounts (CUDs) are available to purchase for Google Cloud VMware Engine. CUDs provide discounted prices in exchange for your commitment to use a minimum level of resources for a specified term. With spend-based committed use discounts for VMware Engine, you can earn a deep discount off your cost of use in exchange for committing to continuously use VMware Engine nodes in a particular region for a 1- or 3-year term. See the documentation for more details.