The following release notes cover the most recent changes over the last 60 days. For a comprehensive list, see the individual product release note pages .
You can see the latest product updates for all of Google Cloud on the Google Cloud release notes page.
To get the latest product updates delivered to you, add the URL of this page to your
feed
reader, or add the feed URL directly: https://cloud.google.com/feeds/gcp-release-notes.xml
March 04, 2021
AI Platform NotebooksNew Notebooks instances add labels for VM image (goog-caip-notebook) and volume (goog-caip-notebook-volume).
1.9.1-asm.1 is now available. Anthos Service Mesh 1.9 includes the features of Istio 1.9 subject to the list of Anthos Service Mesh supported features.
Google-managed control plane is now available as a public preview feature. This feature lets you move from managing istiod in your clusters to configuring the control plane as a service. Google will manage the availability, scalability and security of the control plane.
Using the managed control plane also simplifies multi-cluster mesh configuration and reduces the Kubernetes Engine privileges needed to install Anthos Service Mesh. For more information see Configuring the Google-managed control plane.
Anthos Service Mesh for Compute Engine VMs is now available as a public preview feature. With this new feature you can manage, observe, and secure services running on both Compute Engine Managed Instance Groups and Kubernetes Engine clusters in the same mesh. You can mix and choose the best environment to run your services while enjoying the benefits of Anthos Service Mesh.
This feature also improves security and usability by letting you use Compute Engine service accounts for mTLS authentication to other Compute Engine VMs and Kubernetes Engine Pods. For more information see the documentation.
Anthos Service Mesh 1.5 is no longer supported. For more information see Supported versions.
PostgreSQL 13.0 is now the default version when creating a new database.
For workload identity federation, available in beta, you can now use updated client libraries for C++, Go, Java, Node.js, and Python to automatically obtain Google credentials.
For details, see the documentation for your identity provider:
March 03, 2021
Cloud RunCloud Run reports a new Cloud Monitoring metric: Instance count, which counts the number of container instances that exist, broken down by state (active or idle).
Cloud Spanner now supports point-in-time recovery (PITR), which lets you recover data from a specific point in time in the past.
Fixed a bug where specifying a Cloud Storage URI without an object would return an internal error.
Fixed metastore.googleapis.com/service/health metric not showing up for some services.
Error Reporting has been updated to only analyze logs that are stored in global buckets in the same project where they are ingested. For more information, see Using Error Reporting with regionalized logs.
March 02, 2021
AI Platform (Unified)CMEK compliance using the client libraries
You can now use the client libraries to create resources with a customer-managed encryption key (CMEK).
For more information on creating a resource with an encryption key using REST and the command line, Node.js, and Python, see Using customer-managed encryption keys (CMEK).
Updated version of Magnitude Simba ODBC driver includes bug fixes, performance improvements, and enhancements such as support for dynamic SQL and additional DDL and DML keywords.
Updated version of Magnitude Simba JDBC driver includes bug fixes and performance improvements.
GA: Support for the Airflow Role-Based Access Control (RBAC) UI is now generally available.
GA: Support for Resource location restrictions is now generally available.
Added the --cluster-labels flag to gcloud dataproc jobs submit to allow submitting jobs to a cluster that matches specified cluster labels. Also see Submitting a Dataproc job.
Version 1.1 of the Google Cloud monitoring agent for SAP NetWeaver is now available. This new version removes automatic updates so that you can control when new versions are applied to your system. It also adds support for Bare Metal Solution environments.
For information about the new update method, see Updating the monitoring agent for SAP NetWeaver.
New SAP certifications: For SAP NetWeaver, the following Compute Engine virtual machine types that use the AMD CPU platform are certified by SAP:
- n2d-standard-128
- n2d-standard-224
- n2d custom machine type vCPU limit increased to 96
For more information, see:
March 01, 2021
Cloud RunCloud Run is now available in the following regions:
us-west2(Los Angeles)us-west3(Salt Lake city)us-west4(Las Vegas)
Cloud Run for Anthos on Google Cloud version 0.20.0-gke.6 is now available for the following GKE minor versions:
- 1.19
- 1.20
Events for Cloud Run for Anthos version 0.18.1-gke.108 is now available for the following GKE minor versions:
- 1.19
- 1.20
The maximum job size (payload) is now 1 MB total, including ~1KB request overhead.
You can now optionally receive the mutation count for a transaction in the commit response to optimize the transactions while staying within the mutation count limit. For more information, see Retrieving commit statistics for a transaction.
Dataproc 2.0 image version will become a default Dataproc image version in 2 weeks on March 15, 2021.
Google Cloud Armor Managed Protection Plus Tier is in General Availability. Managed Protection Plus Tier offers a monthly subscription that includes all of the features of Standard Tier, and bundles Google Cloud Armor WAF policy, rules, HTTP request usage, and named IP lists.
Pub/Sub message schemas are now available in the Preview launch stage.
Note: This feature release is still in progress. Some regions may not yet have access.
The preview release of version 2 of the Google Cloud monitoring agent for SAP HANA is now available. Version 2.0 represents a complete refactoring of the monitoring agent for SAP HANA.
For more information, see Monitoring agent for SAP HANA V2.0 planning guide.
Version 1.0 of the Google Cloud monitoring agent for SAP HANA is deprecated. For new installations, use the Google Cloud monitoring agent for SAP HANA V2.0.
Support for version 1 of the monitoring agent for SAP HANA ends on December 31, 2021.
For information about version 2, see Monitoring agent for SAP HANA V2.0 planning guide.
Text-to-Speech has launched Beta support of new SSML tags: <phoneme>, <mark>, <lang>, <voice>, and <say-as interpret-as="duration"> to specify durations. See the phonemes for a list of phonemes available for your language.
Support for the <prosody> SSML tag has been enhanced to produce continuous TTS when possible.
- Text-to-speech has resolved an issue that affected how volume changes are calculated, resulting in different but correct behavior.
- Text-to-speech has resolved an issue that affected how pitch changes are calculated, resulting in different but correct behavior.
Text-to-Speech has improved the continuity of mixed-media results. Now when you mix text and sounds within a <s>/<s> block, Text-to-Speech generates a much shorter pause and better transition between the synthesized speech and the sound.
Text-to-Speech has improved its handling of speech synthesis requests sent using SSML markup. These improvements might affect your applications in backward-incompatible ways. If your application is affected and you would like to temporarily opt out of the improved SSML model in order to make adjustments, please fill out this form.
Text-to-Speech has improved the verbalization and pacing of phone numbers.
February 26, 2021
AnthosAnthos 1.6.2 is now available.
Updated components:
Anthos clusters on VMware (GKE on-prem) 1.6.2-gke.0 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.6.2-gke.0 clusters run on Kubernetes 1.18.13-gke.400.
Fixed in 1.6.2-gke.0:
Fixed a
kubeletrestarting issue that was found when running workloads that rely onkubectl exec/port-forward/attach, such as Jenkins.Fixed CVE-2021-3156 in the node operating system image. CVE-2021-3156 is described in Security bulletins.
GKE on-prem 1.4.5-gke.0 is now available. To upgrade, see Upgrading GKE on-prem. GKE on-prem 1.4.5-gke.0 clusters run on Kubernetes 1.16.11-gke.11.
Fixed in 1.4.5-gke.0:
Fixed CVE-2020-1971 and CVE-2021-3156 in the node operating system image. CVE-2021-3156 is described in Security bulletins.
Fixed CVE-2020-15157 and CVE-2020-15257 in containerd.
Anthos on bare metal 1.6.2 is now available. To upgrade, see Upgrading Anthos on bare metal. Anthos on bare metal 1.6.2 runs on Kubernetes 1.18.
Fixes:
- Updated custom resource API to reject changes to Cluster and NodePool configuration fields that are not currently supported. For a list of supported mutable fields, see Configuration in Known Issues.
- Updated
bmctlto allow creating or upgrading Anthos clusters on bare metal to the currentbmctlversion (1.6.2) only. For more information about version restrictions, see Installation in Known Issues. - Fixed an issue that caused the automatic reset of bare metal machines to fail after deleting the user cluster.
- Added preflight check to verify that control group v2, or cgroup v2 for short, is not in use on the cluster machine. Anthos on bare betal 1.6.x is incompatible with cgroup v2. For more information, see Control group v2 incompatibility in Known Issues.
- Updated
csi-snapshot-validation-webhookto support certification rotation. For more information about certificate rotation, see Security in Known Issues. - Fixed an issue to prevent constant patching for
snapshot.storage.k8s.ioCRDs. - Fixed a Certificate Signing Request (CSR) issue with
kubeletto ensure fully qualified domain name(FQDN) hostnames are supported.
For information about the latest known issues, see Anthos on bare metal known issues in the Troubleshooting section.
Support for Python packages in private PyPI repositories is now in alpha. This feature is only available to alpha users. If you are interested in joining the alpha, fill in the sign up form.
- See the quickstart to get started.
- Learn more about working with Python packages in the overview.
Config Connector version 1.40.0 is now available
Added support for DataprocAutoscalingPolicy (no config-connector CLI support, expected Q2)
Added support for DataprocCluster (no config-connector CLI support, expected Q2)
Added support for DataprocWorkflowTemplate (no config-connector CLI support, expected Q2)
Added support for MemcacheInstance
New field for ComputeInstance: nicType
New fields for ComputeInstanceTemplate: nicType and resourcePolicies
New status field for BigQueryJob: status
Go client is no longer nested under generated folder.
New sub-minor versions of Dataproc images: 1.3.85-debian10, 1.3.85-ubuntu18, 1.4.56-debian10, 1.4.56-ubuntu18, 1.5.31-centos8, 1.5.31-debian10, 1.5.31-ubuntu18, 2.0.4-debian10, and 2.0.4-ubuntu18
Image 2.0: Upgraded Spark to 3.1.1 RC2 version
Allow stopping clusters that have autoscaling enabled, and allow enabling autoscaling on clusters that are STOPPED, STOPPING, or STARTING. If you stop a cluster that has autoscaling enabled, the Dataproc autoscaler will stop scaling the cluster. It will resume scaling the cluster once it has been started again. If you enable autoscaling on a stopped cluster, the autoscaling policy will only take effect once the cluster has been started (see Starting and stopping clusters).
Deactivated mysql and hive-metastore components for clusters created with a Dataproc Metastore service on an image that has the DISABLE_COMPONENT_HIVE_METASTORE and DISABLE_COMPONENT_MYSQL capabilities.
Image 1.3 - 1.5: HIVE-18871: hive on Tez execution error due to set hive.aux.jars.path to hdfs://
Project migration between organizations is now a self-serve process in public preview. For more information, see Migrating projects.
Sprite sheets now support different image compression levels with the new quality
setting.
Sprite sheets now preserve the source aspect ratio. Set the sprite width or height field, but not both (the API will automatically calculate the missing field).
The API now supports video padding with black.
Hierarchical firewall policies are now available in General Availability.
February 25, 2021
AI Platform (Unified)AI Platform (Unified) now supports Access Transparency in beta. Google Cloud organizations with certain support packages can use this feature. Learn more about using Access Transparency with AI Platform (Unified).
The client libraries for Node.js and Python now include enhancements to improve usage of training and prediction features. These client libraries include additional types and utility functions for sending training requests, sending prediction requests, and reading prediction results.
To use these enhancements, you must install the latest version of the client libraries.
The predict and explain method calls no longer require the use of a different service endpoint (for example, https://us-central1-prediction-aiplatform.googleapis.com). These methods are now available on the same endpoint as all other methods.
In addition to Docker images hosted on Container Registry, you can now use Docker images hosted on Artifact Registry and Docker Hub for custom container training on AI Platform.
The Docker images for pre-built training containers and pre-built prediction containers are now available on Artifact Registry.
You can now use a pre-built container to perform custom training with TensorFlow 2.4.
You can now use a pre-built container to serve predictions from TensorFlow 2.3 models.
You can now use a pre-built container to serve predictions from XGBoost 1.2 models.
Hierarchy Controller now includes a preview of Hierarchical Resource Quotas (HRQs). HRQs are drop-in replacements for Kubernetes Resource Quotas, but apply to resources in both a namespace as well as all of its descendants. To learn more, see Using hierarchical resource quotas.
Config Sync now takes advantage of server-side apply when possible, so will be able to handle custom resources larger than 256 kB on Kubernetes clusters with a version of 1.16 or later.
The Anthos Config Management Operator Deployment now specifies resources.limits for config-management-operator:manager.
Anthos clusters on AWS 1.6.2-gke.0 is now available.
Anthos clusters on AWS 1.6.2-gke.0 clusters run the following Kubernetes versions:
- 1.16.15-gke.5302
- 1.17.9-gke.6402
- 1.18.10-gke.902
To upgrade your clusters, perform the following steps:
- Upgrade your Management service to 1.6.2-gke.0.
- Upgrade your user clusters to a supported Kubernetes version.
This release fixes an issue where the management service fails to start when provided with a KMS alias.
Bug fixes and security improvements.
BigQuery materialized views are now generally available (GA). BigQuery materialized views are now generally available (GA). Materialized views are precomputed views that periodically cache the results of a query, enhancing performance and efficiency, and reducing costs, particularly for aggregated queries. For more information, see Introduction to materialized views.
BigQuery BI Engine now interacts with popular BI tools such as Looker, Tableau, and more, by means of an SQL interface. You must enroll to participate in the preview.
New versions of Cloud Composer images:
- composer-1.14.4-airflow-1.10.14
- composer-1.14.4-airflow-1.10.12 (default)
- composer-1.14.4-airflow-1.10.10
When an environment update operation cannot start, an error message that lists possible causes for the error is generated.
Improved the syncing of DAGs and plugins to the Airflow web server. DAG parsing is now less likely to break because of race conditions.
Added FreeTDS system package to Cloud Composer images.
Updated apache-beam package version to 2.24.0 in Airflow 1.10.10 so that Dataflow jobs now correctly create partitioned BigQuery tables. Airflow versions 1.10.12 and 1.10.14 already have apache-beam version 2.27.0 installed.
Upgraded apache-airflow-backport-providers-google package to version 2021.2.5 in Airflow 1.10.12 and 1.10.14. This is potentially a breaking change because the package contains updates of Python Google Cloud libraries. For a list of new operators and for more information about breaking changes, see the 2021.2.5 release notes.
The google-cloud-pubsublite package is installed by default in Composer images for Airflow 1.10.12 and 1.10.14.
Improved the reliability of environment upgrade operations. Added new retrying procedures and enhanced existing ones.
Fixed the cause of several update-related errors.
When an environment deletion operation fails, a correct GKE error is displayed. Before, a different GKE error was displayed in some cases.
The Airflow scheduler liveness checker is now compatible with google-cloud-logging==2.2.0.
Fixed an error when some MsSQL operators were not working with Azure instances.
Fixed an error in GKE cluster builds. In private IP environments, an additional nodepool no longer remains after an update.
Preview: You can now use the gcloud command-line tool to import images from AWS into Google Cloud. For more information, see Importing images from AWS.
General Availability release of Memorystore for Memcached.
Event notifications is now available in Preview.
Event notifications sends information about changes to your secrets and secret versions to Pub/Sub. These notifications can be used to trigger arbitrary workflows, such as restarting an application when a new secret version is added, or notifying security engineers when a secret is deleted.
Learn more at Enabling event notifications.
February 24, 2021
BigQuery Data Transfer ServiceThe BigQuery Data Transfer Service's 1-hour minimum file age requirement for transfers from Cloud Storage has been eliminated.
GA: Support for Customer Managed Encryption Keys (CMEK) is now generally available.
Cloud SQL now offers faster maintenance, with average connectivity loss lasting 90 seconds or less on average. See more about maintenance timelines.
Cloud SQL now offers faster maintenance, with average connectivity loss lasting 90 seconds or less on average. See more about maintenance timelines.
Maximum push task size is now increased to 1 MB.
You can now use Policy Simulator to simulate policy changes before you apply them. This feature is available in Preview.
Private Catalog supports Terraform. Admins can create and curate Terraform configurations as solutions for their catalogs. Learn more
An Apache Spark connector is now available for Pub/Sub Lite, allowing you to read messages from Pub/Sub Lite in your Spark clusters.
An Apache Spark connector is now available for Pub/Sub Lite, allowing you to read messages from Pub/Sub Lite in your Spark clusters.
February 23, 2021
Anthos Service Mesh1.8.3-asm.2 is now available.
This patch release contains the same bug fixes that are in Istio 1.8.3. For details on upgrading Anthos Service Mesh, refer to the following upgrade guides:
- Upgrading on GKE using the
install_asmscript - Upgrading on Anthos clusters on VMware
The BigQuery Data Transfer Service's minimum interval time between recurring transfers from Cloud Storage has been reduced from one hour to 15 minutes.
180576558: Fixed an issue where the Linux discovery tool calculated an incorrect score.
Fixed an issue where using an Envoy proxy sidecar, not as part of Istio or Anthos Service Mesh, created networking issues with the migrated workload.
The ability to connect VM interfaces other than nic0 to a Shared VPC is now available in General Availability. This feature presently only GA for individual VM instances. Support for instance templates and managed instance groups is still Preview.
February 22, 2021
Cloud BillingOptimal Recommendations for Compute Engine committed use discounts are now Generally Available. Recommendations provide you opportunities to optimize your compute costs by analyzing your VM spending trends and recommending committed use discount contracts.
Recommendations are presented in two forms:
- Optimal recommendations are based on overall usage and might cover resources that are not on all the time.
- Stable usage recommendations cover minimum stable usage over time.
For understanding and purchasing committed use discount recommendations, see the documentation.
Cloud Data Fusion Beta instances (versions 6.1.0.2 and lower that were created before November 21, 2019) will be turned down on March 1, 2021. Instead, export your pipeline, delete the old instance to avoid billing impact, create a new instance, and import your pipeline into the new instance.
The following PostgreSQL minor versions are now available. If you use maintenance windows, you might not yet have the minor version. In this case, you will see the new minor version once your maintenance update occurs. To find your maintenance window or manage maintenance updates, see Finding and setting maintenance windows.
- PostgreSQL 9.6.19 is upgraded to 9.6.20.
- PostgreSQL 10.14 is upgraded to 10.15.
- PostgreSQL 11.9 is upgraded to 11.10.
- PostgreSQL 12.4 is upgraded to 12.5.
- PostgreSQL 13 is upgraded to 13.1.
Config Connector version 1.39.0 is now available
Alpha release of Go types and clients for Config Connector resources
Added support for CloudSchedulerJob resource
Reverted webhook port to 443 to alleviate forwarding rule issue on GKE private clusters
Fixed issue with aggressive retrying of failed updates leading to exhausting quota
Dataproc 2.0 image version will become a default Dataproc image version in 3 weeks on March 15, 2021.
February 19, 2021
AI Platform Deep Learning ContainersM64 release
Upgraded TensorFlow 2.4 to 2.4.1.
Upgraded TFX and Fairness Indicators from 0.26.0 to 0.27.0.
Miscellaneous bug fixes and updates.
Swift For TensorFlow
- The Swift For TensorFlow project is entering archive mode. Containers will be deprecated and will no longer receive updates after this release.
M64 release
Upgraded TensorFlow 2.4 to 2.4.1.
Upgraded TFX and Fairness Indicators from 0.26.0 to 0.27.0.
Adding the Fast.ai book tutorials to Pytorch images.
GVNIC is enabled for all DLVM images.
Miscellaneous bug fixes and updates.
Swift For TensorFlow
- The Swift For TensorFlow project is entering archive mode. Swift images will be deprecated and will no longer receive updates after this release.
Hive configuration overrides are rejected if either the key or value contains a newline or "<" character.
Fixed a bug where services would fail to create in projects with project IDs that contain the colon "(:)" character.
Logs query builder doesn't work when selecting location and service ID.
The MetadataImport.DatabaseDump.source_database field is deprecated. It will be removed from the v1beta API channel no earlier than August 18, 2021.
Google Cloud Armor Adaptive Protection is available in Public Preview. Adaptive Protection builds machine-learning models that help you protect your Google Cloud applications, websites, and services against L7 distributed denial-of-service (DDoS) attacks.
Added upfront prepay option for 3-year and 1-year commitment contracts. VMware Engine provides an option to unlock up to 50% off the hourly rate savings on resources through the prepay upfront option. Contact Sales for more information.
February 18, 2021
Cloud LoggingCloud Logging agent for Windows version 1-14 is now available. This version changes the default Windows configuration from using gRPC to REST for sending logs to the Cloud Logging API. For more information, refer to the release information on GitHub.
Dialogflow ES now supports the europe-west1 (Belgium) region.
February 17, 2021
Compute EnginePreview: Predictive autoscaling for managed instance groups lets you improve the availability of your workloads by using Machine Learning to predict future demand and create virtual machines ahead of forecasted load.
Added password management of the CloudOwner@gve.local user for vCenter and the admin user for NSX-T Manager. VMware Engine generates a password for these users when you deploy a private cloud. You can view and reset credentials from the private cloud details page.
Added the ability to peer multiple VPCs with private clouds in a region. This improvement enables you to establish a many-to-many relationship between your VPCs and regions.
Added support for global DNS name resolution for management components of your private cloud using Cloud DNS. You can set up Cloud DNS to resolve domain names of management components of multiple private clouds (in the same or different regions) in your project.
For more information, see Configuring DNS for vCenter access.
Updated private cloud nodes so that the ESXi advanced parameter fakescsireservation and MAC learning are now enabled by default. This allows creation of a nested ESXi environment on your private cloud.
Added missing release notes for previous region launches of VMware Engine resources:
- Montréal, Québec (
northamerica-northeast1) - São Paulo, Brazil (
southamerica-east1) - Jurong West, Singapore (
asia-southeast1) - Eemshaven, Netherlands (
europe-west4) - Sydney, Australia (
australia-southeast1) - London, England (
europe-west2) - Tokyo, Japan (
asia-northeast1) - Frankfurt, Germany (
europe-west3)
February 16, 2021
AI Platform TrainingThe default boot disk type for virtual machine instances used for training jobs has changed from pd-standard to pd-ssd. Learn more about disk types for custom training and read about pricing for different disk types.
Note that for training jobs where you don't specify a DiskConfig, pricing does not change. This is because the first 100 GB of disk for each VM do not incur any charge, regardless of disk type.
BigQuery now supports exporting table data in Parquet format. This feature is in Preview. For more information, see Parquet export details.
GA: Setting and updating machine types for CloudSQL/Web Server is now generally available.
GA: Support for Domain restricted sharing is now generally available.
Cloud Composer 1.14.3 release was rolled back. If you have an environment that was created with a composer-1.14.3-airflow-* image, you can later upgrade it to a newer version.
Managing response policies and rules in Cloud DNS is available in Beta.
Zonal NEGs (with GCE_VM_IP network endpoints) can now be used as backends for internal TCP/UDP load balancers. For more information on this type of zonal NEG, see Zonal NEGs overview.
This feature is in Preview.
New sub-minor versions of Dataproc images: 1.3.84-debian10, 1.3.84-ubuntu18, 1.4.55-debian10, 1.4.55-ubuntu18, 1.5.30-centos8, 1.5.30-debian10, 1.5.30-ubuntu18, 2.0.3-debian10, and 2.0.3-ubuntu18
Fixed a bug that prevented Dataproc on GKE cluster creation.
You must now have storage.objects.get permission on the Cloud Storage object in order to import metadata from the Cloud Storage file.
You can now use IAM conditions to set limits on the roles that a member can grant and revoke. This feature is generally available.
For SAP HANA host auto-failover, version 2.0 of the gceStorageClient is now available with a new human-readable name: Google Cloud Storage Manager for SAP HANA Standby Nodes (Storage Manager for SAP HANA for short). The new version uses RPM Package Manager for installation and updates, and supports all versions of SAP HANA that are in mainstream maintenance.
For more information, see SAP HANA host auto-failover on Google Cloud.
Version 1.n releases of the gceStorageClient for SAP HANA host auto-failover are deprecated.
If you are using a version 1.n release, upgrade to version 2.0 of the gceStorageClient, the Google Cloud Storage Manager for SAP HANA Standby Nodes, at your earliest convenience, but before support is discontinued.
Version 1.n releases of the gceStorageClient will be supported until December 31, 2021.
To determine which version you are running, see Deprecation of version 1.n releases of the storage manager for SAP HANA.
Preview release of Ingress and egress rules for VPC Service Controls.
February 15, 2021
DataprocDataproc 2.0 image version will become a default Dataproc image version in 4 weeks on March 15, 2021.
February 12, 2021
Anthos Service Mesh1.6.14-asm.1 is now available.
This patch release contains a fix for CVE-2021-3156. For details on upgrading Anthos Service Mesh, refer to the following upgrade guides:
BigQuery standard SQL queries on Google Cloud Storage data are now supported by cached query results.
Google Virtual NIC (gVNIC) driver is now generally available. For more information, see Using Google Virtual NIC.
February 11, 2021
Cloud ComposerTo upgrade environments that are deployed with VPC service controls, you must add Artifact Registry to the service perimeter and configure an additional firewall rule for *.pkg.dev.
For other types of environments, if your firewall configuration does not use the default rules, you might need to add a firewall rule for *.pkg.dev as well.
It is now possible to view the details of in-process long-running operations (LRO) from within the Healthcare Browser in the Cloud Console.
Added support for Node Selector.
Added support for Task.
Added feature flags enable_dockerfile_builds, enable_custom_buildpacks and enable_custom_stacks.
Added --as and --as-group global flags to support impersonation.
Added health-check-http-endpoint flag to kf push command.
Changed the kf CLI to use kubectl kubeconfig loading logic.
Made hostname as an optional field when creating a route.
Make routes available in VCAP_APPLICATION.
Updated the Tekton version to 0.19.0.
February 10, 2021
AI Platform TrainingRuntime version 2.4 is now available. You can use runtime version 2.4 to train with TensorFlow 2.4.1, scikit-learn 0.24.0, or XGBoost 1.3.1. Runtime version 2.4 supports training with CPUs, GPUs, or TPUs.
See the full list of updated dependencies in runtime version 2.4.
New resource types now available.
The following resource types are now publicly available through the asset inventory APIs (ExportAssets and BatchGetAssetsHistory) and the Feed API.
- Compute Engine
compute.googleapis.com/Commitmentcompute.googleapis.com/Reservation
Users can now create triggers that execute builds in response to webhook events, including events from external source code management services. To learn more, see Creating webhook triggers and Building repositories hosted on Bitbucket Server.
The Cloud SQL Admin API v1beta4 is now generally available. The URL for the Admin API will continue to refer to v1beta4 for backward compatibility. To start using the Cloud SQL Admin API now, see Using the Cloud SQL Admin API.
The Cloud SQL Admin API v1beta4 is now generally available. The URL for the Admin API will continue to refer to v1beta4 for backward compatibility. To start using the Cloud SQL Admin API now, see Using the Cloud SQL Admin API.
The Cloud SQL Admin API v1beta4 is now generally available. The URL for the Admin API will continue to refer to v1beta4 for backward compatibility. To start using the Cloud SQL Admin API now, see Using the Cloud SQL Admin API.
February 09, 2021
Cloud Healthcare APIIt is now possible set the value of the writeDisposition enum when exporting FHIR resources.
Config Connector version 1.38.1 is now available
Miscellaneous bug fixes
New sub-minor versions of Dataproc images: 2.0.2-debian10, and 2.0.2-ubuntu18.
Image 2.0:
- Upgraded Spark built-in Hive to version 2.3.8.
- Upgraded Druid to version 0.20.1
- HIVE-24436: Fixed Avro NULL_DEFAULT_VALUE compatibility issue.
- SQOOP-3485: Fixed Avro NULL_DEFAULT_VALUE compatibility issue.
- SQOOP-3447: Removed usage of org.codehaus.jackson and org.json packages.
Fixed a bug for beta clusters using a Dataproc Metastore Service where using a subnetwork for the cluster resulted in an error.
Firestore now offers beta support for C++ through the Firebase C++ SDK.
You can now attach tags to resources, then use the tags to manage access to your resources. This feature is available in Preview.
If you run one of the gcloud tool's add-iam-policy-binding commands, and the IAM policy contains conditional role bindings for that role, the gcloud tool prompts you to choose one of the condition expressions that exists in the policy. If you choose a condition expression that contains a comma, the command fails.
To work around this issue, use the --condition flag to specify a condition expression on the command line.
Added support for the balanced disk type to the GcpDiskType runbook field when migrating in batches with waves. See Runbook reference for more.
Tags have released into public preview. Tags provide a way to conditionally allow or deny policies based on whether a resource has a specific tag. You can use tags and conditional enforcement of policies for fine-grained control across your resource hierarchy. For more information, see the Tags overview.
If you run one of the gcloud tool's add-iam-policy-binding commands, and the IAM policy contains conditional role bindings for that role, the gcloud tool prompts you to choose one of the condition expressions that exists in the policy. If you choose a condition expression that contains a comma, the command fails.
To work around this issue, use the --condition flag to specify a condition expression on the command line.
February 08, 2021
AI Platform Deep Learning ContainersM63 release
- SWIFT 0.12 (experimental) containers are available.
M63 release
- Nvidia driver is upgraded to 450.80.02.
- TFX version is upgraded to 0.26.1.
- Regular package refreshment and bug fixes.
The Cloud Healthcare API now supports getting HL7v2 messages in bulk. See Retrieving HL7v2 messages in bulk.
Logging truncates oversized LogEntry label keys and values. For details, see Quotas and limits.
The Cloud Spanner Console now displays database storage utilization and warns you if you are approaching the recommended limit. For more information, see storage utilization metrics.
Dataproc 2.0 image version will become a default Dataproc image version in 5 weeks on March 15, 2021.
February 05, 2021
App Engine standard environment JavaRemoved data logging in the deprecated endpoints library.
PayPal now available as a form of payment in many countries
If you have an online, auto-pay Cloud Billing account, you might be able to add PayPal as a form of payment on that account.
To learn if PayPal is available for your Cloud Billing account in your country or region, visit one of these tools:
- Cloud Billing docs: Currencies and payment methods available for Cloud Billing auto-pay accounts
- Google Payments: Payment methods & terms of service finder
To learn how to update the form of payment on your online, auto-pay Cloud Billing account, see Add, remove, or update a payment method.
Config Connector version 1.38.0 is now available
Added resourceID support to: ContainerCluster, ContainerNodePool, SourceRepoRepository and AccessContextManager resources
config-connector bulk-export now operates on LoggingLogSink resources
Increased CPU and Memory limit for ConfigConnector Operator
Security Command Center's v1 API now includes a Severity field for Findings.
The Severity field indicates the severity of a finding, as determined by the finding provider, and is included with all findings. The field is managed by finding providers and you are cautioned to not modify its values.
Uses for the field include listing findings of a certain severity level or grouping findings by severity level.
Read Using the Security Command Center dashboard to learn more about findings and finding severity.
Event Threat Detection, a built-in service of Security Command Center Premium, has launched previews for two new detectors.
IAM: Anomalous IP geolocation and IAM: Anomalous user agent detect anomalous connections to Google Cloud resources based on location and user agent, respectively.
Read more about available detectors in Event Threat Detection conceptual overview.
Documentation
- New documentation includes guides on enabling real-time email and chat notifications and investigating and responding to threats.
February 04, 2021
App Engine standard environment Java- Updated Java SDK to version 1.9.85.
- Removed deprecated File APIs.
- Updated Jetty web server to version 9.4.36.v20210114.
There is now a security level feature for HTTP functions that controls whether the function's URL supports HTTPS only, or both HTTP and HTTPS.
The Quickstart using curl or Windows PowerShell has been updated with additional information on storing and viewing DICOM, FHIR, and HL7v2 data.
Two new headers, X-CloudScheduler-JobName and X-CloudScheduler-ScheduleTime, have been added to the default headers for AppEngineHttpTarget and HttpTarget. These can be used to help with job deduplication.
February 03, 2021
Anthos GKE on AWSGKE on AWS 1.6.1-gke.2 is now available.
GKE on AWS 1.6.1-gke.2 clusters run the following Kubernetes versions:
- 1.16.15-gke.5301
- 1.17.9-gke.6401
- 1.18.10-gke.901
To upgrade your clusters, perform the following steps:
- Upgrade your Management service to 1.6.1-gke.2.
- Upgrade your user clusters to a supported Kubernetes version.
Snapshots now collect AWS EFS logs from user cluster nodes.
Bug fixes and performance improvements.
You can now configure Cloud Build to access secrets from Secret Manager via environment variables.
Timeouts for environment upgrade operations are increased.
Fix for CVE-2021-3156 vulnerability.
On a failed environment upgrade operation, the created CloudSQL database is now correctly rolled back.
Create and update operations for environments no longer fail if your account doesn't have the serviceusage.services.get permission.
Fixed SQL operation conflicts that were occurring during environment upgrade operations.
Upgrade operations that might have resulted in a semi-upgraded environment state when the operation timeout was reached are now correctly rolled back and errors are reported.
You can now enable and disable RBAC in environments with installed custom PyPI packages.
New versions of Cloud Composer images:
- composer-1.14.2-airflow-1.10.14
- composer-1.14.2-airflow-1.10.12 (default)
- composer-1.14.2-airflow-1.10.10
Preview: You can now replicate data continuously and in real time from operational data stores, such as SQL Server and MySQL, into BigQuery.
Identity-Aware Proxy (IAP) is supported with Internal HTTP(S) Load Balancing. This support is available in General Availability.
Dataflow now supports Dataflow Shuffle, Streaming Engine, FlexRS, and the following regional endpoints in GA:
asia-east2Hong Kongasia-northeast2- Japan (Osaka)asia-northeast3- Seoulasia-southeast2- Jakartaeurope-north1- Finlandus-west3- Salt Lake Cityus-west4- Las Vegas
You can now create Dataproc Metastore services in cross-product networks (shared VPC).
Dataproc Metastore now suppports the use of non-RFC 1918 private IP address ranges in metastore services.
New Cloud Monitoring service metric is now available:
metastore.googleapis.com/service/request_count
You can now update the description of metadata imports under a Dataproc Metastore service.
Fixed an issue in which a service could get stuck in the UPDATING state.
Fixed an issue where Cloud Storage buckets with single character directories would fail request validation.
February 02, 2021
Anthos Service Mesh1.8.2-asm.2 is now available.
This patch release contains the same bug fixes that are in Istio 1.8.2. For details on upgrading Anthos Service Mesh, refer to the following upgrade guides:
- Upgrading on GKE using the
install_asmscript - Upgrading on Anthos clusters on VMware
The install_asm script lets you reinstall the same version
You can use the install_asm script when you need to reinstall the same Anthos Service Mesh version to change the control plane configuration. For more information, see the following:
On-Demand Scanning is available in Preview. You can manually scan Docker container images stored locally on your computer or remotely in Artifact Registry. To get started with manual scanning, see On-Demand Scanning quickstart
Updated version of Magnitude Simba JDBC driver includes bug and security fixes and enhancements for additional DDL keywords and dynamic SQL.
For internal TCP/UDP load balancers, you can create multiple forwarding rules with the same IP address. The forwarding rules can have different protocols and ports. This feature is available in General Availability.
Generally Available: Sole-tenant nodes now support GPUs and local SSDs. For more information, see Sole-tenant nodes.
Generally Available: Specify when maintenance begins on VMs in a sole-tenant node group. For more information, see Planned maintenance.
On-Demand Scanning is available in Preview. You can manually scan container images stored locally on your computer or remotely in Container Registry. To get started with manual scanning, see On-Demand Scanning quickstart
Several new Dialogflow CX prebuilt agents have been launched. All of these prebuilt agents only support English at this time. The complete list of pre-built agents is currently:
- Financial services agent (new)
- Healthcare agent
- Order and account management agent
- Payment arrangement agent
- Small talk agent (new)
- Telecommunications agent (updated)
- Travel: baggage claim agent (new)
- Travel: car rental agent
- Travel: flight information agent
February 01, 2021
AI Platform (Unified)You can now use a pre-built container to perform custom training with PyTorch 1.6.
Notebooks Terraform Module supports Notebooks API v1
You can now use E2, N2, and C2 machine types for training. Learn about the specific machine types available for training, and learn about their pricing.
Invoices are now simpler, providing only your cost totals. View your cost details in the Cost Table and other reports in the Cloud Console.
Beginning with your January 2021 invoice or statement (available in February 2021), we removed all cost details from your invoice and statement documents, including product-level costs and costs by subaccounts (for Resellers). Invoices and statements will continue to provide header information, such as business mailing address and billing account number, the cost totals for the invoice or statement period, and remittance information.
The cost details of your invoice or statement are available in the Cloud Console, in the downloadable Cost Table report. The Cost Table report reconciles to the invoice totals and includes the product-level costs and costs by subaccounts (for Resellers), along with additional details you might need, such as costs by projects, services, SKU IDs, and labels. You can also analyze your usage costs using the Reports page or create custom reports using your exported Cloud Billing data.
For guidance on using these reports, see:
- Cost table: View and download the cost details of your invoice or statement
- Reports page: View your billing reports and cost trends
- Billing data: Export Cloud Billing data to BigQuery
- Custom reports: Visualize spend over time with Google Data Studio
- Blog post: Giving you better cost analytics capabilities—and a simpler invoice
Cloud Billing Budgets now shows your historic cost trends when you're planning your budget .
In the Cloud Billing Console, the Cloud Billing Budget creation and edit experience has been updated to include a cost trend chart showing your summarized costs by month for the previous 12 months, helping you visualize how your targeted budget amount is tracking with your spend. Your cost trend chart will show different results based on the budget filters you set (for example, filtering on specific projects or services).
You can use the chart to identify previous spending trends and help forecast future needs as you plan your budget. If you need more detailed insights, the chart is linked to the Cloud Billing Reports page; the link uses the same filters you set on your budget to configure your report view.
For more information on the cost trend chart, see Set budgets and budget alerts.
Cloud EKM adds support for Dataflow shuffle and Secret Manager. For more information, see Cloud External Key Manager.
Cloud Run for Anthos on Google Cloud version 0.19.0-gke.1 is now available for the following GKE minor versions:
- 1.18
- 1.19
- 1.20
NVIDIA® T4 GPUs are now available in the following additional regions and zones:
- Jakarta, Indonesia, APAC:
asia-southeast2-a,b
For more information about using GPUs on Compute Engine, see GPUs on Compute Engine.
Preview: You can now use schedule-based autoscaling from the Google Cloud Console.
N2D machine types are now available in London, zone europe-west2-c.
For pricing information, see VM instance pricing.
You can now create instances with up to 24 local SSD partitions for 9 TB of local SSD space using N1, N2, and N2D machine types. This is Generally available. For more information, see Local SSD 9 TB maximum capacity.
Preview: You can now create virtual machines for high performance computing (HPC) workloads using the HPC VM image.
Dataproc 2.0 image version will become a default Dataproc image version in 3 weeks on February 22, 2021.
Dialogflow CX conditions now support sentiment.
Released a fix, rolling out gradually and taking full effect 2/5/2021, for a migctl setup
installation that fails on a GKE cluster when the automatically created bucket
already exists.
Released a fix, rolling out gradually and taking full effect 2/5/2021, for a migctl crash when kubectl is not in PATH.
January 31, 2021
Cloud CDNCloud CDN now supports serving stale content and the ability to bypass the cache based on request header(s).
Serving stale content lets Google's global cache continue to serve
content to users when your origin server is unreachable or is returning errors to Cloud CDN. You can configure how long Cloud CDN will serve content beyond expiry by setting the serveWhileStale value for each backend service or bucket.
These features are available when configuring Cloud CDN enabled backend services and backend buckets in the Cloud Console, in addition to the gcloud SDK and REST API.
These features are available in Preview .
Preview: You can now configure Cloud Composer to use Artifact Registry instead of Container Registry.
January 29, 2021
Anthos clusters on bare metalAnthos on bare metal 1.6.1 is now available. To upgrade, see Upgrading Anthos on bare metal. Anthos on bare metal 1.6.1 runs on Kubernetes 1.18.6-gke.6600.
Functionality changes:
- Added upgrade support from 1.6.0. Users are able to upgrade existing Anthos bare metal cluster from 1.6.0 to 1.6.1.
- Improved upgrade preflight check. Added preflight check before cluster upgrade to validate current cluster status, machine health and other issues before proceeding to upgrade.
- Added support for deleting mounts and data from the anthos-system StorageClass during
bmctl reset. - Relaxed the requirement for an odd number of control plane node pools to allow customers to add and remove nodes for maintenance or replacement.
- Added support to force removing a broken worker node through annotation on the operator machine.
- Added etcddefrag pod to control-plane nodes, which are responsible for monitoring etcd's database size and defragmenting the database as needed. This helps reclaim etcd database size and recover etcd when its disk space is exceeded.
- Enabled kubelet server TLS certification auto-rotation. Kubelet on each node sends out CSR when nearing serving certificate expiration. A controller running inside the admin cluster validates and approves the CSR for user clusters.
- Added proxy support to connect to the OIDC provider. This allows overriding the cluster proxy configuration with a different proxy.
- Added
bmctl update clusterfor updating standalone clusters.
Fixes:
- Fixed bug causing cluster deletion stall problem because of pods refusing to evacuate, or dead nodes.
Clustered tables now support the DATETIME type for clustering columns. For more information, see Creating and using clustered tables. This feature is generally available.
New resource types now available.
The following resource types are now publicly available through the
resource search API (SearchAllResources),
policy search API (SearchAllIamPolicies),
and analyze policy APIs
(AnalyzeIamPolicy and AnalyzeIamPolicyLongrunning).
BigQuery
bigquery.googleapis.com/Table
Cloud Bigtable
bigtableadmin.googleapis.com/Clusterbigtableadmin.googleapis.com/Instancebigtableadmin.googleapis.com/Table
Pub/Sub
pubsub.googleapis.com/Topicpubsub.googleapis.com/Subscriptionpubsub.googleapis.com/Snapshot
Compute Engine (Note that Compute Engine types cover zonal, regional, and global resources.)
compute.googleapis.com/Reservationcompute.googleapis.com/Commitmentcompute.googleapis.com/ExternalVpnGatewaycompute.googleapis.com/NetworkEndpointGroupcompute.googleapis.com/NodeGroupcompute.googleapis.com/NodeTemplatecompute.googleapis.com/PacketMirroringcompute.googleapis.com/Projectcompute.googleapis.com/ResourcePolicycompute.googleapis.com/SslPolicycompute.googleapis.com/VpnGateway
Dataflow
dataflow.googleapis.com/Job
New resource types now available.
The following resource types are now publicly available through the
resource search API (SearchAllResources),
and policy search API (SearchAllIamPolicies).
Cloud SQL
sqladmin.googleapis.com/Instance
Cloud Storage
storage.googleapis.com/Bucket
Google Kubernetes Engine
container.googleapis.com/NodePool
New searchable fields now available.
The following searchable fields are now publicly available through the
resource search API (SearchAllResources).
kmsKeystatecreateTimeupdateTime
New resource types now available.
The following resource types are now publicly available through the asset inventory APIs (ExportAssets and BatchGetAssetsHistory) and the Feed API.
- Service Directory
servicedirectory.googleapis.com/Namespace
The Cloud Healthcare API offers single-region support in the us-east4 (North Virginia) region.
Cloud Logging has increased the number of entries.write API calls from 60,000 to 120,000 per minute. For more information on quotas and limits, see Quotas and limits.
Flex templates now support updating streaming jobs and Flexible Resource Scheduling (FlexRS).
Dataflow snapshots are now available in Preview.
New sub-minor versions of Dataproc images: 1.3.83-debian10, 1.3.83-ubuntu18, 1.4.54-debian10, 1.4.54-ubuntu18, 1.5.29-centos8, 1.5.29-debian10, 1.5.29-ubuntu18, 2.0.1-debian10, and 2.0.1-ubuntu18.
Image 2.0:
Upgraded Hadoop to version 3.2.2
Upgraded Scala to version 2.12.13
Upgraded Flink to version 1.12.1
Upgraded Spark to version 3.1.1 RC1
January 28, 2021
AI Platform TrainingYou can now use NVIDIA A100 GPUs and several accelerator-optimized (A2) machine types for training. You must use A100 GPUs and A2 machine types together.
A100 GPUs and A2 machine types are available in preview. Learn about their pricing.
Hierarchy Controller is upgraded to include HNC v0.7.0. This release introduces Exceptions. Exceptions let you use Kubernetes label selectors to precisely control where certain objects are propagated.
This release also removes support for the v1alpha1 API. If you were using Hierarchy Controller 1.5.1 or earlier, you must either update to Hierarchy Controller 1.5.2 or 1.6.0, and follow the HNC v0.6.0 directions to upgrade to v1alpha2.
The nomos status output has been fixed for multi-repo clusters to show git.syncBranch when git.syncRev is not specified (git.syncRev defaults to HEAD) to provide a consistent experience with mono-repo clusters.
The nomos status output has been fixed for multi-repo clusters to distinctly show status of multiple namespace repos synced to the clusters.
Pricing report access updated to allow Billing Account Users to view a version of the report
The Pricing report provides SKU prices for Google's cloud services, including Google Cloud, Google Maps Platform, and Google Workspace. Prior to this update, you could access the report only if you were a Billing Account Administrator or a Billng Account Viewer. Now, Billing Account Users can also access the report.
The data displayed in your report is dependent on your level of access to your Cloud Billing account. For more information, see View and download prices for Google's cloud services.
The default Airflow version for Composer images changes from 1.10.10 to 1.10.12.
Airflow 1.10.9 is no longer included in Cloud Composer images.
Updated google-auth package version to 1.24.0 in Cloud Composer images so that it works with VPC Service Controls. Earlier versions of google-auth caused multiple DAG execution errors when used with VPC SC.
Updated pyarrow package version to 2.0.0 in Cloud Composer images to fix an error in BigQueryHook and Pandas integration. Earlier versions of pyarrow caused the to_pandas() got an unexpected keyword argument 'timestamp_as_object' error.
New versions of Cloud Composer images:
- composer-1.14.1-airflow-1.10.14
- composer-1.14.1-airflow-1.10.12 (default)
- composer-1.14.1-airflow-1.10.10
Query Insights is now generally available. Query Insights helps you detect, diagnose, and prevent query performance problems for Cloud SQL databases. It provides self-service, intuitive monitoring, and diagnostic information that goes beyond detection to help you to identify the root cause of performance problems.
To learn more, see Overview of Query Insights. To get started now, see Troubleshooting query performance problems using Query Insights.
Manage your operating system environments by using VM Manager. VM Manager is a suite of services for reviewing, patching, and configuring your operating systems across both Linux and Windows VMs. For more information, see VM Manager.
Secret Manager Expiration is available for all customers via public preview.
Learn more at Creating and Managing Expiring Secrets.
January 27, 2021
AnthosAnthos 1.6.1 is now available.
Updated components:
Anthos clusters on VMware (GKE on-prem) 1.6.1-gke.1 is now available. To upgrade, see Upgrading Anthos clusters on VMware. Anthos clusters on VMware 1.6.1-gke.1 clusters run on Kubernetes 1.18.13-gke.400.
Fixes:
- Fixed a bug where the user cluster upgrade is blocked if the vcenter resource pool is neither directly nor indirectly specified (that is, if the vcenter resource pool is inherited and is the one used by the admin cluster) in the configs.
- Fixed CVE-2020-15157 and CVE-2020-15257 in
containerd. - Fixed an issue where upgrading the admin cluster from 1.5 to 1.6.0 breaks 1.5 user clusters that use any OIDC provider and that have no value for
authentication.oidc.capathin the user cluster configuration file.
Cloud Data Fusion Beta instances (versions 6.1.0.2 and lower that were created before November 21, 2019) will be turned down on March 1, 2021. Instead, export your pipeline, create a new instance, and import your pipeline into the new instance. This note is incorrect; see entry for February 18, 2021.
Config Connector version 1.37.0 is now available.
Added a column Status Age showing the last transition time for the value in Status, and added the column Age back to the default output of kubectl get for all Config Connector resources. Improved the value at Status and Ready columns to match against the condition name.
Added resourceID support for ArtifactRegistryRepository, Bigtable resources, DataflowJob, DNS resources, Monitoring resources, RedisInstance, ResourceManagerLien, SecretManagerSecret, Spanner resources, StorageTransferJob.
Fixed the issue with the legacy Common Name field on x509 certificate. Config Connector should be working on clusters of K8s 1.19+. (Issue #335)
Dialogflow CX beta launch of Experiments to compare the performance of flow versions to a control version while handling live traffic.
January 26, 2021
Cloud BillingThe Cloud Billing Committed Use Discounts (CUD) Analysis report has been updated to include spend-based CUDs, allowing you to easily visualize the effectiveness and financial impact of discounts you have purchased. See the documentation for more details. Learn more about spend-based committed use discounts.
Eventarc is now Generally Available (GA).
Speech-to-Text now supports regional EU and US endpoints. See the multi-region endpoints documentation for more information.
January 25, 2021
AI Platform Deep Learning ContainersGeneral Availability
AI Platform Deep Learning Containers is now generally available.
Python 2
Python 2 is no longer supported in Deep Learning Containers. Read more about Python 2 support on Google Cloud.
M62 release
Upgraded TensorFlow 2.3 to 2.3.2
Upgraded TensorFlow 2.1 to 2.1.3
Miscellaneous bug fixes and updates
Python 2
Python 2 is no longer supported in Deep Learning VM Image. Read more about Python 2 support on Google Cloud.
M62 release
- Upgraded TensorFlow 2.3 to 2.3.2
- Upgraded TensorFlow 2.1 to 2.1.3
- Miscellaneous bug fixes and updates
GPU support on Dataflow is currently available in Preview. To enroll in this Preview offering, contact Support or Sales.
Dataproc 2.0 image version will become a default Dataproc image version in 4 weeks on February 22, 2021.
Previous releases of Migrate for Anthos required that you used Google Container Registry (GCR) and Google Cloud Storage for data repositories. This release adds support for additional repositories, including ECR, S3, and Docker registries that support basic authentication. See Defining data repositories for more.
In many on-prem environments, outbound internet access is tightly controlled through the use of an HTTPS proxy server. If your environment uses a proxy server to control outbound internet access, then you can now configure Migrate for Anthos to use that proxy server. See Configuring an HTTPS proxy for more.
Migrate for Anthos now includes the deployment_spec.yaml file in artifacts.zip for Windows migrations. You can use the deployment_spec.yaml file to deploy your migrated Windows workloads. See Deploying a Windows workload to a target cluster for more.
Support added for using Anthos clusters on AWS as processing clusters to perform migrations of AWS workloads. This feature is in preview. See Prerequisites for migrating Linux VMs on AWS for more.
Removed support for the --password option to the migctl command when creating a migration source on Anthos clusters on VMware:
migctl source create local-vmware local-vmware-src --vc '1.2.3.4' --username 'admin' --password 'pass1'
You are now prompted to enter the password. See Adding a migration source for more.
172414359: Exporting multiple cloned VMs simultaneously from the same source might fail.
Workaround: Re-run the migctl migration generate-artifacts command again.
174655315: A migration might stop responding when generating artifacts and remain in the retrying state. You might also see this error in the logs or in the verbose migration status:
D 2020-12-01T18:43:53Z SHELL ERROR: '2020/12/01 18:43:53 appending [/tarlayer/layer.tar.gz]: reading tar "/tarlayer/layer.tar.gz": flate: corrupt input before offset 681999708'
Workaround: Re-run migctl migration generate-artifacts.
175000470: When adding a source when using a service account without the compute.disks.create permission, the source becomes ready but the migration will fail to create disks.
Workaround: Make sure that service account has the compute.disks.create permission.
174299021: When creating a migration source or executing a migration, you might see this error:
"Error: Internal error occurred: failed calling webhook "vmigration.kb.io": Post https://controllers-webhook-service.v2k-system.svc:443/validate-anthos-migrate-cloud-google-com-v1beta2-migration?timeout=30s: unexpected EOF"
Workaround: Recreate the source or migration.
171686793: The migctl setup upgrade --gkeop command might create a new ImageRepositiry or ArtifactRepository object that lacked Google Cloud access credentials.
Workaround: Use the following command to upgrade the cluster:
migctl setup upgrade --json-key key
Where key is the JSON key for the service account required for migctl installation. See Configuring service accounts.
If you try to mount a secret on a deployed pod you will not be able to access it in /run/secrets. This is typically an issue when giving workload identity permissions to the pod (where a secret is added by Kubernetes to hold the workload identity credentials).
The contents of the secrets directory are in /kubernetes-info/secrets.
Workaround: Run the following command on the deployed pod:
ln -s /kubernetes-info/secrets /run/secrets
If the /run mount gets deleted (by a process in the pod, or by a pod reset), you might have to run the command again.
178469863: Running migctl setup install with either the --node-selector or --tolerations flag returns an error.
Note: Running the migctl setup install command with both flags succeeds. This error only occurs when using one flag.
Workaround: Run migctl setup install without the option, and then manually add the nodeSelectors or tolerations to CSI and Controller pods. See Creating and managing cluster labels and Controlling scheduling with node taints for more.
If you delete the configuration for a Docker image file registry, create a new one with a different configuration name. You cannot recreate a configuration with the name of a previously deleted configuration.
This issue affects Docker image file registries implemented by using GCR or by using Docker registries using basic auth. It does not affect ECR. See Defining data repositories for more information.
Workaround: Use the migctl docker-registry update command to modify an existing configuration rather than deleting it and recreating it.
Network Connectivity Center is now available in Preview.
For more information, see the Network Connectivity Center overview.
Preview for the following integration:
Workflows is now Generally Available (GA).
Workflows Connectors are now available in public preview.
January 23, 2021
AI Platform NotebooksVPC-SC for Notebooks is now Generally Available
Notebooks API supports Shielded VM configuration
January 22, 2021
Cloud Composer- If you run DAGs using Pandas and BigQuery in Composer version 1.14.0 and Airflow version 1.10.14, you must update the
pyarrowPyPI package to version 1.0.0 or higher andapache-beamto version 2.27.0 or higher.
- New versions of Cloud Composer images:
composer-1.14.0-airflow-1.10.9,composer-1.14.0-airflow-1.10.10,composer-1.14.0-airflow-1.10.12, andcomposer-1.14.0-airflow-1.10.14. The default iscomposer-1.14.0-airflow-1.10.10. Upgrade your Cloud SDK to use features in this release.
- The
max-pods-per-nodeparameter configures the maximum number of pods per node in the GKE cluster. You can set this parameter when you create a new environment. This feature is available in Google Cloud SDK and Beta API.
- You can now specify maintenance windows for your environments. GKE cluster and SQL database are updated only during the specified period. This feature is available in Google Cloud SDK and Beta API.
- The maximum number of pods per node in the GKE cluster changes from 100 to 32. This change affects all newly created environments. You can use the
max-pods-per-nodeparameter when creating an environment to increase or decrease the number of pods.
The Logs Explorer now provides a higher degree of contrast that improves readability.
NVIDIA® T4 GPUs are now available in the following additional regions and zones:
- Jurong West, Singapore, APAC:
asia-southeast1-a
For more information about GPU availability on Compute Engine, see GPU regions and zones availability.
Config Connector version 1.36.0 is now available
Added a column 'Ready' showing the value of the .status.conditions0, and associated Status to the default output of kubectl get for all Config Connector resources.
Added support for referencing an organization to IAMCustomRole.
Added a new sub-command to the CLI, config-connector print-resources which shows all config connector resources and their associated level of export and bulk-export support.
Reduce the memory usage of deletiondefender and controller-manager in high-scale scenarios (1000+ resources under management).
Added resourceID support to the Compute resources.
Announcing the General Availability (GA) release of Dataproc 2.0 images. This image will become the default Dataproc image version on February 22, 2021.
2.0 image clusters:
On master nodes, initialization actions run before HDFS and YARN start. On worker nodes, initialization actions run before DataNode and NodeManager daemons start.
If you run initialization actions on a 2.0 image cluster that stages files in HDFS, poll the cluster until you determine that HDFS is available.
2.0 image clusters:
In 2.0 clusters, yarn.nm.liveness-monitor.expiry-interval-ms is set to 15000 (15 seconds). If the resource manager does not receive a heartbeat from a NodeManager during this period, it marks the NodeManager as LOST. This setting is important for clusters that use preemptible VMs. Usually, NodeManagers unregister with the resource manager when their VMs shut down, but in rare cases when they are be shut down ungracefully, it is important for the resource manager to notice this quickly.
New sub-minor versions of Dataproc images: 1.3.82-debian10, 1.3.82-ubuntu18, 1.4.53-debian10, 1.4.53-ubuntu18, 1.5.28-centos8, 1.5.28-debian10, 1.5.28-ubuntu18, 2.0.0-debian10, and 2.0.0-ubuntu18.
Fixed bug affecting cluster scale-down: If Dataproc was unable to verify whether a master node exists, for example when hitting Compute Engine read quota limits, it would erroneously put the cluster into an ERROR state.
VMware Engine nodes are now available in the following additional region:
- Montréal, Québec (
northamerica-northeast1)
New language: Text-to-Speech now supports Romanian (ro-RO). See the supported voices page for details and audio samples.
New voice: Text-to-Speech now offers 2 new Bengali (bn-IN) WaveNet voices. See the supported voices page for details and audio samples.
January 21, 2021
Anthos Anthos clusters on VMwareAnthos GKE on-prem 1.5.3-gke.0 is now available. To upgrade, see Upgrading GKE on-prem. GKE on-prem 1.5.3-gke.0 clusters run on Kubernetes 1.17.9-gke.4400.
Fixes:
Fixed CVE-2020-15157 and CVE-2020-15257 in
containerd.Cloud Run Operator is now able to successfully update custom resource definitions (CRDs).
Cloud Data Fusion 6.3.0 is now available.
In-place upgrades are now supported for minor and patch versions.
You can configure the default system compute profile in the Developer edition starting in Cloud Data Fusion version 6.3.0.
Dialogflow CX system entities can now be extended.
Service Directory is now available in GA.
Traffic Director support for xDS clients that connect and request configuration using the xDS x3 API is now in Preview.
January 20, 2021
AI Platform PredictionThe VPC Service Controls integration with AI Platform Prediction is now generally available.
Training with a custom service account is now generally available.
Support for VPC Network Peering is now generally available.
1.7.6-asm.1 is now available.
This patch release contains the same bug fixes that are in Istio 1.7.6. For details on upgrading Anthos Service Mesh, refer to the following Anthos Service Mesh upgrade guides:
- Upgrading on GKE using the
install_asmscript - Upgrading on Anthos clusters on VMware
Cloud Run now supports WebSockets, HTTP/2 and gRPC streaming.
You can now troubleshoot conditional role bindings by troubleshooting directly from audit log entries. This feature is available in Preview.
New SAP certifications: For SAP HANA, the following Bare Metal Solution bare-metal machine types are certified by SAP:
- o2-ultramem-672-metal
- o2-ultramem-896-metal
For more information, see Certified machine types for SAP HANA.
New SAP certifications: For SAP NetWeaver, the following Bare Metal Solution bare-metal machine types are certified by SAP:
- o2-standard-32-metal
- o2-standard-48-metal
- o2-standard-112-metal
- o2-highmem-224-metal
For more information, see Bare Metal Solution machine types.
Storage Transfer Service offers Preview support for specifying source and destination paths when creating a transfer. For more information, see Specifying source and destination paths.
General availability for the following integration:
January 19, 2021
AI Platform (Unified)Preview: Select AI Platform (Unified) resources can now be configured to use Customer-managed encryption keys (CMEK).
Currently you can only create resources with a CMEK key in the UI; this functionality is not currently available using the client libraries.
- New US Regions and Support platform control, enabling first-level US Person support and US data location.
- Billing integration: Assured Workload Premium Subscriptions can be purchased via offline contract by both customers and resellers
- Assured Workloads Support: Receive Premium Support from a US Person, in a US location, 24/7, to help meet compliance requirements (requires additional support services purchase).
- Existing folder support: You can now create your Assured Workloads environment inside of an existing folder.
BigQuery is now available in the Iowa (us-central1) region.
BigQuery BI Engine is now available in the Iowa (us-central1) region.
The BigQuery Data Transfer Service is now available in the Iowa (us-central1) region.
BigQuery ML is now available in the Iowa (us-central1) region.
Database auditing in Cloud SQL for PostgreSQL is generally available, through the open-source pgAudit extension. Using this extension, you can selectively record and track SQL operations performed against a given database instance.
The pgAudit extension helps you configure many of the logs often required to comply with government, financial, and ISO certifications.
The Dataproc Metastore Cloud Logging and Monitoring issue has been fixed.
Transfer service for on-premises data has updated the cost to $0.0125 per GB transferred to the destination successfully. For more information, see Transfer for on-premises pricing details.
Preview support for the following integration:
January 15, 2021
AI Platform TrainingAI Platform Training now provides pre-built PyTorch containers for PyTorch 1.6.
In addition to training with CPUs or GPUs, you can use one of the PyTorch 1.6 containers to perform PyTorch training with a TPU.
Regionalized builds from Cloud Functions are now visible in the Cloud Build History UI. To learn more, see Viewing build results.
You can now run SQL queries to retrieve lock statistics to investigate lock conflicts in your database.
Announcing the Beta release of Dataproc Service Account Based Secure Multi-tenancy, which allows you to share a cluster with multiple users. With secure multi-tenancy, users can submit interactive workloads to the cluster with isolated user identities.
New sub-minor versions of Dataproc images: 1.3.81-debian10, 1.3.81-ubuntu18, 1.4.52-debian10, 1.4.52-ubuntu18, 1.5.27-centos8, 1.5.27-debian10, 1.5.27-ubuntu18, 2.0.0-RC23-debian10, and 2.0.0-RC23-ubuntu18.
Image 2.0 preview:
The gcloud_dataproc_personal_cluster.py tool for the personal auth beta is no longer supported for new images. It will be replaced by an equivalent set of commands in an upcoming gcloud release.
The Network Topology graph now includes a checkbox, Show connections for child nodes only on focus, to display only the traffic paths between top-level entities, such as regions. When this checkbox is selected, you can still view the traffic paths between lower-level entities by selecting or holding the pointer over the lower-level entities.
Recommendations AI has migrated to the Retail API, which is now generally available.
The Recommendations Engine API (service endpoint https://recommendationengine.googleapis.com) and this documentation set remain available, but they will no longer be updated. We recommend migrating your recommendations to the Retail API (service endpoint https://retail.googleapis.com). See the new documentation:
Recommendations AI is now generally available.
This product has migrated to the Retail API from the Recommendations Engine API.
The previous API (service endpoint https://recommendationengine.googleapis.com) and its documentation set remain available, but they will no longer be updated. If you used the previous API while it was in beta, we recommend migrating your recommendations to the Retail API (service endpoint https://retail.googleapis.com).
See the new documentation:
January 14, 2021
BigQueryBigQuery's Cloud Console UI has been updated with many usability improvements for analyzing data, including multi-tab navigation, a new resource panel, and a new SQL editor. These updates are in Preview. For more information, see Using the Cloud Console.
Cloud Logging now lets you share your saved queries with other users of a project. To learn more, go to the Shared queries section on the Building queries page.
Query statistics now includes information about queries that failed, queries that timed out, and queries that were canceled by the user.
Two Queue fields that in previous versions were output only are now configurable.
taskTtlis the maximum amount of time that a task is retained in this queue.tombstoneTtlis the amount of time the task tombstone is retained after a task is deleted or executed. The tombstone is used in task de-duplication. For more information on this process, see Create Task Request.
These fields are also available in v2beta2.
New Procurement DocAI processor released in limited Preview
The following Procurement DocAI processor is now available in limited Preview:
- Procurement document splitter
For more information, see the processor documentation.
Managed Microsoft AD now supports audit logging. This feature is in the Preview stage.
January 13, 2021
Cloud Composer- Preview: You can now restart the Airflow web server using the command
gcloud beta composer environments restart-web-serveror the Beta API.
Cloud Functions has added support for a new runtime, Node 14, in Preview.
Cloud Functions has added support for a new runtime, Python 3.9, in Preview.
Cloud SQL now exposes the metric database/memory/total_usage. This metric provides visibility into the database working set (including buffer cache). You can find this metric in the Metrics explorer within the Monitoring dashboard.
For more information about database/memory/total_usage, see Cloud SQL Metrics.
Cloud SQL now exposes the metric database/memory/total_usage. This metric provides visibility into the database working set (including buffer cache). You can find this metric in the Metrics explorer within the Monitoring dashboard.
For more information about database/memory/total_usage, see Cloud SQL Metrics.
Cloud SQL now exposes the metric database/memory/total_usage. This metric provides visibility into the database working set (including buffer cache). You can find this metric in the Metrics explorer within the Monitoring dashboard.
For more information about database/memory/total_usage, see Cloud SQL Metrics.
Config Connector version 1.35.0 is now available.
Added resourceID support for: SQL resources, Pub/Sub resources, LoggingLogSink, StorageBucket, KMS resources, IAMCustomRole.
Added support for the MonitoringGroup resource.
January 12, 2021
Anthos Service Mesh1.6.14-asm.0 is now available.
This patch release contains the same bug fixes that are in Istio 1.6.14. For details on upgrading Anthos Service Mesh, refer to the following upgrade guides:
- GA: Network ACL support for the Airflow web server is now generally available.
- Preview: Composer can now be configured to use Customer-managed encryption keys (CMEK).
- New versions of Cloud Composer images:
composer-1.13.4-airflow-1.10.9,composer-1.13.4-airflow-1.10.10, andcomposer-1.13.4-airflow-1.10.12. The default iscomposer-1.13.4-airflow-1.10.10. Upgrade your Cloud SDK to use features in this release. - If environment creation succeeds when the requester did not have the
iam.serviceAccounts.actAspermission on the service account, Composer will now generate a warning in the audit log entry. - Error handling for files synchronization between buckets has been improved.
- Preview: Added support for the Airflow Role-Based Access Control (RBAC) UI for Airflow version 1.10.10 or newer and Python 3. You can enable the Airflow RBAC UI by setting the
[webserver]rbac=TrueAirflow configuration override.
- Fixed an issue where connections were inserted into the Airflow database twice.
- The Airflow UI will now always send requests to HTTPS addresses.
Added support for user configuration of Compute Engine Shielded VMs in a Dataproc Cluster.
#171638373: General stability improvements.
#171638373: Fixed Windows adaptation issue when boot partition and Windows partition were on different volumes.
Performance improvement during detach phase.
#175196444: Fixed Windows adaptation issue with network interface detection.
#174330790: Linux adaptations now archive ifcfg-* scripts to avoid Network Manager conflicts with iSCSI boot.
Security fixes applied.
January 11, 2021
AI Platform (Unified)The default boot disk type for virtual machine instances used for custom training has changed from pd-standard to pd-ssd. Learn more about disk types for custom training and read about pricing for different disk types.
If you previously used the default disk type for custom training and want to continue training with the same disk type, make sure to explicitly specify the pd-standard boot disk type when you perform custom training.
Users can now specify their own service accounts for Cloud Build to run builds. For more information, see User-specified service accounts.
The API for updating ResourceRecordSets in Cloud DNS is now available in Beta.
Cloud Functions has added support for a new runtime, Ruby, in Preview. This runtime supports Ruby 2.6 and Ruby 2.7.
Support for 1500 MTU for Cloud Interconnect is now available in General Availability.
You can now create N2D VM instances in us-east4-c Northern Virginia. See VM instance pricing for details.
Lending processors behavior update
The behavior of the following processors has been updated:
- 1003 parser
- 1040 parser
- 1099-MISC parser
- W2 parser
- W9 parser
Now, if these processors are given a multi-page input file and contains a page that is the correct document type and one of the supported versions the processor performs entity extraction for that page; subsequent applicable pages will not be processed. If the prcoessor doesn't find any applicable documents in the input file it returns an error message.
Configuring an internal load balancer in Service Directory is available in Preview.
Support for 1500 MTU for Cloud Interconnect is now available in General Availability.
January 08, 2021
Anthos Config ManagementConfig Sync unintentionally started using the absolute path in the file system with spec.git.policyDir. This has no effect on Config Sync running on the cluster, but breaks validation when running nomos vet manually against hierarchical repositories. The issue will be corrected in 1.6.1.
Added support for new persistent disk type, pd-balanced.
New sub-minor versions of Dataproc images: 1.3.80-debian10, 1.3.80-ubuntu18, 1.4.51-debian10, 1.4.51-ubuntu18, 1.5.26-centos8, 1.5.26-debian10, 1.5.26-ubuntu18, 2.0.0-RC22-debian10, and 2.0.0-RC22-ubuntu18.
Image 2.0 preview:
Image 2.0 preview:
HIVE-21646: Tez: Prevent TezTasks from escaping thread logging context
Dataproc Metastore Cloud Logging and Monitoring is unavailable. The issue will be fixed shortly.
January 07, 2021
VPC Service ControlsGeneral availability for the following integration:
January 06, 2021
AI Platform (Unified)You can now use a pre-built container to perform custom training with TensorFlow 2.3.
Hybrid Jobs are now available for inspecting external data sources.
Config Connector version 1.34.0 is now available.
Added support for IAM Member References. This allows users to create an IAMPolicyMember that references another resource as the IAM member (e.g. IAMServiceAccount, LoggingLogSink). For more information, see the memberFrom field in the IAMPolicyMember reference documentation. Support for IAM Member References is added only to IAMPolicyMember, not IAMPolicy.
Added support for the GameServicesRealm resource.
Added IAM support for ComputeDisk.
Added cacheMode, clientTtl, defaultTtl, maxTtl, negativeCaching, negativeCachingPolicy, serveWhileStale, and customResponseHeaders fields to ComputeBackendBucket.
Added customTimeBefore, daysSinceCustomTime, daysSinceNoncurrentTime, and noncurrentTimeBefore fields to StorageBucket.
Allow for IAMPolicy, IAMPolicyMember, and IAMAuditConfig to reference resources in other namespaces.
Added support for UpdateFailed, DeleteFailed, DependencyNotFound, and DependencyNotReady events to IAMPolicy, IAMPoicyMember, IAMAuditConfig.
Allow for Project and Folder resources to be migrated across folders and organizations by updating the folder-id/organization-id annotation. Only folder-to-folder or organization-to-organization migrations are allowed; folder-to-organization migrations or vice versa are not yet supported.
January 05, 2021
Channel ServicesThe Google Cloud Channel Services API is now Generally available. To get started, see Getting access to the Channel Services API.
- In an upcoming Cloud Composer version release, DAG Serialization is enabled by default when creating new Cloud Composer environments.
Traffic Director now supports TCP-based services in Preview. This brings service discovery, global load balancing, failover and many other Traffic Director capabilities to your non-HTTP services. See the setup guide to get started and the target proxies documentation for helpful background information.