BatchBeta

Efficiently run batch jobs using Kubernetes.

View documentation for this product.

A cloud-native batch manager

A cloud-native batch manager

Batch on Google Kubernetes Engine (GKE) is a cloud-native solution for managing your batch workloads using Kubernetes. Batch brings the functionality and familiar interface of a traditional batch job scheduler into a cloud-first world. It overcomes the rigidity of fixed clusters for batch jobs by allowing you to easily scale and dynamically allocate resources to meet your workload requirements.

Zero waste and autoscaling

Nodes are created to tailor fit submitted jobs and scale up and down with the queue. Just-in-time provisioning of resources means you only pay for what you need.

Cost management

Admins can allocate budgets to efforts rather than resource quotas (CPUs, GPUs, memory, etc.). Budgets are useful approximations to shape and prevent runaway spend in your queues.

Familiar interface

Comes with the user-friendly and familiar experience of a traditional batch scheduler, yet is optimized for the cloud—reducing the learning curve and making cloud batch job management with Kubernetes much easier.

Features

Runs on Google Kubernetes Engine (GKE)

GKE is your reliable, efficient, and secure way to run Kubernetes clusters.

Queues

Introduces a new resource, Queue, and expands on the existing notion of Jobs.

Autoscaling

Comes with its own autoscaler and automatically resizes clusters based on the demands of the workloads you want to run.

Budgets

A queue can have an optional budget that limits how many resources a queue can use in a given window. Budgets prevent runaway spend in your queues for CPU, GPUs, and memory to give all queues a chance to use the available resources.

Job dependencies

Users can submit multiple jobs, where some jobs will only be run when and if specific conditions related to previous jobs have occurred.

User account management

Differentiates between two kinds of users: admins and practitioners. Admins are those who set up, manage, and administer the batch system, including adding users. Practitioners are the end users who submit batch jobs to the system.

Data copy to/from cluster

With an optional feature, users can set up Filestore and give each user a private home directory that batch jobs can access. Batch includes tools to copy data to and from the user's directory and to manage files and directories there.

Ksub for submitting shell scripts instead of YAML

Ksub is a command-line tool to help speed up the “dev, test, iterate” process in cloud. Ksub provides an interface that is simple and familiar to users of traditional batch systems.

GPU support

Jobs which require GPUs are supported. Refer to this guide on how to use GPUs in your Google Kubernetes Engine clusters' nodes.

FAQs

No. The beta release is only supported for Google Kubernetes Engine clusters running on Google Cloud.

You are only charged for the underlying GKE resources and other Google Cloud services and products you use. The current beta release of Batch is offered at no charge.

Pricing

The current beta release of Batch is offered at no charge. Workloads deployed by Batch will be billed per GKE pricing terms. GKE is priced based on provisioned resources for the cluster. You can use the Google Cloud Pricing Calculator to estimate the cost of a Google Kubernetes Engine cluster.

Google Cloud

Get started

Learn and build

New customers get $300 in free credits to learn and build on Google Cloud for up to 12 months.

Need more help?

Our experts will help you build the right solution or find the right partner for your needs.

This product is in beta. For more information on our product launch stages, see here.