Batch release notes

This page documents production updates to Batch. Check this page for announcements about new or updated features, bug fixes, known issues, and deprecated functionality.

You can see the latest product updates for all of Google Cloud on the Google Cloud page, browse and filter all release notes in the Google Cloud console, or programmatically access release notes in BigQuery.

To get the latest product updates delivered to you, add the URL of this page to your feed reader, or add the feed URL directly: https://cloud.google.com/feeds/batch-release-notes.xml

February 22, 2024

In the Google Cloud console, the Job list page has been updated to reduce latency. Although the console no longer summarizes the statuses of your jobs, you can filter based on job state when you view a list of your jobs.

Fixed the issue causing latency when listing jobs in projects that contain more than 10,000 jobs.

February 01, 2024

You can configure custom status events, which describe important events for a job's runnables. By providing additional information about a job's progress, custom status events can help make a job easier to analyze and troubleshoot.

For more information, see Configure custom status events to describe runnables and View a job's history through status events.

You can write unstructured and structured task logs:

  • An unstructured task log lets you define a log's message.
  • A structured task log lets you define multiple details for a log such as the message, the severity, custom fields, and a custom status event.

By allowing you to surface custom information in Cloud Logging, task logs can help make a job easier to analyze and troubleshoot.

For more information, see Write task logs.

You can run Batch jobs as a non-root user to meet workload or security requirements. For more information, see Create and run jobs as a non-root user.

January 25, 2024

Jobs that try to consume reserved VMs might be incorrectly delayed or prevented from running. For more information, including workarounds, see Known issues.

January 02, 2024

The limit for concurrent VMs per job now varies based on the number of zones allowed for a job's VMs:

  • The limit for concurrent VMs per single-zone job remains 2,000.
  • The limit for concurrent VMs per multi-zone job has increased to 4,000.

Learn more about Quotas and limits and Batch locations.

December 21, 2023

You can use Image streaming to enable Batch jobs to initialize without waiting for a container image to finish downloading. For more information, see Use Image streaming to reduce container startup time.

December 20, 2023

Logs from Batch jobs created after December 19, 2023 will no longer use the general-purpose generic_task monitored resource type and instead use the new batch.googleapis.com/Job monitored resource type. The batch.googleapis.com/Job monitored resource type is specific to Batch, which makes it simpler to query Cloud Logging for logs only from Batch.

When querying Cloud Logging for logs from Batch, update any filters that require the generic_task monitored resource type to specify the batch.googleapis.com/Job monitored resource type accordingly. Alternatively, you can enable the use_generic_task_monitored_resource field for your jobs to continue using the generic_task monitored resource type instead.

For more information, see the documentation for Cloud Logging monitored resources types and Batch job logs.

November 27, 2023

Documentation has been added to explain how to configure jobs that can run on reserved VMs. Using reserved VMs helps minimize a job's scheduling time and prevent resource availability errors.

For more information, see Ensure resource availability using VM reservations

November 08, 2023

You might experience latency when listing jobs in projects that contain more than 10,000 jobs. For more information, see Known issues.

Documentation has been added to explain how to configure jobs to send status notifications using Pub/Sub and how to query those notifications using BigQuery.

For more information, see the following pages:

  1. To configure your project to support status notifications, see Monitor job status using Pub/Sub notifications and BigQuery.

  2. To configure a job to send status notifications, see Create and run a job that sends Pub/Sub status notifications.

October 26, 2023

Documentation has been added to explain how to run dsub pipelines on Batch. For more information, see Orchestrate jobs by running dsub pipelines on Batch.

October 19, 2023

Documentation has been added to explain how to colocate the VMs for a job by using a compact placement policy. For example, use compact placement policies to reduce the latency between VMs for jobs with tightly coupled tasks, such as tasks that communicate using MPI libraries.

For more information, see Reduce latency by using compact placement policies.

October 18, 2023

Documentation has been added to explain how to securely reference sensitive data in a job by using Secret Manager secrets for encryption. For example, use secrets to protect sensitive data when defining custom environment variables or protect login credentials when accessing private container images from Docker Registry.

For more information, see Protect sensitive data using Secret Manager with Batch.

October 09, 2023

Job limits have increased to 100,000 tasks per task group and 5,000 parallel tasks per job. Learn more about Quotas and limits.

October 05, 2023

Batch is available in the following regions:

  • australia-southeast2 (Melbourne)
  • europe-west8 (Milan)
  • europe-west12 (Turin)
  • me-west1 (Tel Aviv)
  • northamerica-northeast2 (Toronto)
  • southamerica-east1 (São Paulo)
  • us-east5 (Columbus)

For more information, see Locations.

September 19, 2023

Documentation has been updated to reflect new default options for jobs that use GPUs:

  • Defining the machine type of the job's VMs is optional.
  • Jobs that use software from June 3, 2023 and later don't need to mount GPUs to containers.

For more information, see Create and run a job that uses GPUs.

September 05, 2023

Documentation has been added to explain how to automatically retry some or all of the failed tasks for a job. For example, automatic task retries can help prevent job failures from temporary issues like Spot VM preemption, host events, and transient networking errors.

For more information, see Automate task retries.

August 29, 2023

Documentation has been added to explain the VM OS environment for Batch. For a job's VMs, you can optionally configure the OS image and/or boot disk properties. Otherwise, a job uses the default configuration.

For more information, see the following pages:

August 25, 2023

Batch is available in the europe-west10 (Berlin) region.

For more information, see Locations.

August 22, 2023

Batch is available in the following regions:

  • asia-south2 (Delhi)
  • asia-southeast2 (Jakarta)
  • europe-southwest1 (Madrid)
  • me-central1 (Doha)

For more information, see Locations.

August 17, 2023

Batch is enforcing a 60-day retention policy for all finished (failed or succeeded) jobs:

  • Any existing jobs that have finished before August 17, 2023 are automatically deleted 60 days after, on October 16, 2023.

  • All new and existing jobs that are not yet finished on August 17, 2023 are automatically deleted 60 days after they finish running.

If you need to retain the information for a job for more than 60 days, you can export the job. For more information, see Delete and export jobs.

July 18, 2023

VPC Service Controls support for Batch is generally available (GA).

VPC Service Controls lets you create perimeters that protect the resources and data of Google Cloud services that you explicitly specify. For more information, see Overview of VPC Service Controls and Use VPC Service Controls with Batch.

July 17, 2023

Documentation has been added for Cloud Life Sciences users to explain how to migrate to Batch. For more information, see Migrate to Batch from Cloud Life Sciences.

July 08, 2023

Documentation has been added to explain how to apply labels to a Batch job and its resources. Labels are key-value pairs that can be used to group and organize Batch and Compute Engine resources. For more information, see Organize resources using labels.

July 07, 2023

Batch is available in the following regions:

  • asia-northeast2 (Osaka)
  • asia-northeast3 (Seoul)
  • australia-southeast1 (Sydney)
  • europe-west1 (Belgium)
  • europe-west9 (Paris)

For more information, see Locations.

July 03, 2023

June 20, 2023

Cloud Client Libraries for C++ are available for the Batch API. For more information, see the reference documentation.

June 05, 2023

Batch is available in the following regions:

  • asia-east2 (Hong Kong)
  • europe-central2 (Warsaw)
  • us-south1 (Dallas)
  • us-west2 (Los Angeles)
  • us-west3 (Salt Lake City)
  • us-west4 (Las Vegas)

For more information, see Locations.

May 22, 2023

Documentation has been added to explain how to use Terraform to run a Cloud Scheduler cron job that creates Batch jobs. For more information, see Create and run Batch jobs using Terraform and Cloud Scheduler.

May 17, 2023

Preview: Integrate a job into a workflow using the Batch API connector for Workflows.

May 09, 2023

Documentation has been added to explain how to run Nextflow pipelines on Batch. For more information, see Orchestrate jobs by running Nextflow pipelines on Batch.

April 27, 2023

Documentation has been added for an overview page that summarizes Batch content including pages, code samples, and videos. For more information, see Overview.

April 13, 2023

Documentation for pricing has been added to explain how you can visualize the costs associated with your Batch jobs by using Cloud Billing reports. For more information, see Pricing.

April 12, 2023

Documentation has been added to explain networking concepts and how to configure networking for Batch. For more information, see the following pages:

April 11, 2023

Batch is available in the following regions:

  • asia-northeast1 (Tokyo)
  • europe-west4 (Netherlands)

For more information, see Locations.

March 13, 2023

In the Google Cloud console, the Job details page has been updated to include an Events tab, which lists the job's status events and contains a link to the job's logs.

To view the Events tab, follow the steps to describe a job using the console.

March 06, 2023

Batch is available in the following regions:

  • asia-south1 (Mumbai)
  • asia-east1 (Taiwan)
  • europe-west3 (Frankfurt)
  • southamerica-west1 (Santiago)
  • us-east4 (Northern Virginia)

For more information, see Locations.

February 15, 2023

February 02, 2023

Documentation has been updated to include new samples. The following samples are available in Java:

For more information, see All Batch code samples.

January 17, 2023

Batch is available in the following regions:

  • northamerica-northeast1 (Montréal)
  • europe-west2 (London)

For more information, see Locations.

December 22, 2022

Documentation has been updated to include new samples. The following samples are available in Go, Node.js, and Python:

The following sample is available in Go and Python:

For more information, see All Batch code samples.

December 15, 2022

Any job can use a custom machine type. (Before, you could only use custom machine types by creating a job from a Compute Engine instance template.)

Preview: Batch supports VPC Service Controls, which lets you create perimeters that protect the resources and data of Google Cloud services that you explicitly specify. For more information about using VPC Service Controls with Batch, see Supported products and limitations.

October 21, 2022

Samples in Go are available for Batch. Documentation has been updated to include the following samples:

For more information, see All Batch code samples.

October 12, 2022

Samples in Java, Node.js, and Python are available for Batch. Documentation has been updated to include the following samples:

For more information, see All Batch code samples.

October 11, 2022

Batch is generally available (GA). Batch jobs can be created in the supported locations and resources can be created in any location supported by Compute Engine. With this release the Terms of Service in the Cloud Services Summary fully apply.

The release includes additional capabilities such as support for user-defined service accounts, VPC Service Control, and HIPAA compliance.

The documentation has been updated to include the following new content:

October 03, 2022

Starting today, to submit a job, you must also have the Service Account User (roles/iam.serviceAccountUser) IAM role on the service account used by a job. For details, see the permissions required to submit a job.

The new Batch Job Editor (roles/batch.jobsEditor) IAM role is intended to replace the existing Batch Job Administrator (roles/batch.jobsAdmin) role. Although the Batch Job Administrator role currently still has the same permissions as the Batch Job Editor role, you can now only grant users the Batch Job Editor role. Additionally, it is recommended to update any existing users with the Batch Job Administrator role to have the Batch Job Editor role instead. For more information about IAM roles, see Manage access.

September 21, 2022

The Quotas and limits are now documented for Batch.

September 14, 2022

Preview: Data Access audit logs are now available for Batch. For more information, see Batch audit logging information.

September 12, 2022

The documentation page Get started with Batch has been rewritten. Information that was previously on that page has been relocated to the following new pages:

September 07, 2022

Batch is now available in the following regions: asia-southeast1 and europe-west6. For more information, see Locations.

July 13, 2022

Batch is now available in Preview! For more information about using Batch, see the documentation.