Create jobs

Preview — Using Cloud Run jobs

This feature is covered by the Pre-GA Offerings Terms of the Google Cloud Terms of Service. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

This page describes how to create and update Cloud Run jobs from an existing container image. Unlike a Cloud Run service, which listens for and serves requests, a Cloud Run job only runs its tasks and exits when finished. A job does not listen for or serve requests, and cannot accept arbitrary parameters at execution.

After you create or update a job, you can execute the job as a one-off, on a schedule or as part of a workflow. You can manage individual job executions and view the execution logs.

You can structure a job as a single task or as multiple, independent tasks (up to 10,000 tasks) that can be executed in parallel. Each task runs one container instance and can be configured to retry in case of failure. Each task is aware of its index, which is stored in the CLOUD_RUN_TASK_INDEX environment variable. The overall count of tasks is stored in the CLOUD_RUN_TASK_COUNT environment variable. If you are processing data in parallel, your code is responsible for determining which task handles which subset of the data.

You can set timeouts on tasks and specify the number of retries in case of task failure. If any task exceeds its maximum number of retries, that task is marked as failed and the job execution is marked as failed after all tasks have run.

By default each task runs for a maximum of 10 minutes: you can change this to a shorter time or a longer time up to 1 hour, by changing the task timeout setting. There is no explicit timeout for a job execution: after all tasks are complete, the job execution is complete.

Jobs use the second generation execution environment.

Permissions required to create and execute

You must have ONE of the following:

• Owner
• Editor
• Both the Cloud Run Admin and Service Account User roles
• Any custom role that includes this specific list of permissions

Supported container registries and images

You can use container images stored in Artifact Registry, Container Registry, or Docker Hub. Google recommends the use of Artifact Registry.

You can use only the following types of container images:

• Container images stored in the same project as the one you are creating the job or service in.
• Container images from other Google Cloud projects (provided that the correct IAM permissions are set).
• Public container images from Artifact Registry, Container Registry, or Docker Hub.

If you are storing container images in another type of container registry, follow the instructions under deploying images from unsupported registries.

Create a new job

You can create a new job using the Google Cloud console or gcloud command line.

Console

To create a new job:

1. Go to Cloud Run

2. Click the Jobs tab.

3. Click Create Job to display the Create job form.

   1. In the form, specify the container image containing the job code or select from a list of containers previously deployed.
   2. The job name is automatically generated from the container image. You can edit or change the job name as needed. A job name cannot be changed after the job is created.
   3. Select the region where you want your job located. The region selector highlights regions with the lowest carbon impact.
   4. Specify the number of tasks that you want to run in the job. All of the tasks must succeed for the job to succeed. By default, the tasks execute in parallel.

4. Click Container, Networking, Security to set additional job properties.

   • Under Task capacity:
   1. In the Memory dropdown menu, specify the amount of memory required. The default is the minimum required, 512MiB.
   2. In the CPU dropdown menu, specify the amount of CPU required. The default is the minimum required, 1 CPU.

   3. Under Task timeout, specify the maximum amount of time in seconds that the task can run, up to 1 hour. Each task must complete within this time. The default is 10 minutes (600 seconds).

   4. Under Number of retries per failed task, specify the number of retries in case of task failures. The default is 3 retries.

   • Under Parallelism:

     1. In most cases you can select Run as many tasks concurrently as possible,.
     2. If you need to set a lower limit due to scaling constraints on resources your job accesses, select Limit the maximum number of concurrent tasks and specify the number of concurrent tasks in the Custom parallelism limit textbox.

5. Optionally, configure other settings in the appropriate tabs:

   • Container configuration
   • CPU limits
   • Memory limits
   • Secrets
   • Environment variables
   • Labels
   • Service accounts
   • Cloud SQL connections
   • VPC connectivity

6. When you are finished configuring your job, click Create to create the job in Cloud Run.

7. To execute the job see execute jobs or execute jobs on a schedule.

Command line

1. In the Google Cloud console, activate Cloud Shell.

   Activate Cloud Shell

   At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

2. To create a new job:

   1. Run the command:

      gcloud beta run jobs create JOB_NAME --image IMAGE_URL OPTIONS

      • Replace JOB_NAME with the name of the job you want to create. You can omit this parameter, but you will be prompted for the job name if you omit it.
      • Replace IMAGE_URL with a reference to the container image, for example, us-docker.pkg.dev/cloudrun/container/job:latest.

      • Optionally, replace OPTIONS with any of the following options:

        Option	Description
        --tasks	Accepts integers greater or equal to 1. Defaults to 1; maximum is 10,000. Each task is provided the environment variables CLOUD_RUN_TASK_INDEX with a value between 0 and the number of tasks minus 1, along with CLOUD_RUN_TASK_COUNT, which is the number of tasks
        --max-retries	The number of times a failed task is retried. Once any task fails beyond this limit, the entire job is marked as failed. For example, if set to 1, a failed task will be retried once, for a total of two attempts. The default is 3. Accepts integers from 0 to 10.
        --task-timeout	Accepts a duration like "2s". Defaults to 10 minutes; maximum is 1h.
        --parallelism	The maximum number of tasks that can execute in parallel. By default, tasks will be started as quickly as possible in parallel. Refer to Parallelism for the range of values.
        --execute-now	If set, immediately after the job is created, a job execution is started. Equivalent to calling gcloud beta run jobs create followed by gcloud beta run jobs execute.

        In addition to the above options, you can set other optional configuration settings:

        • Container configuration
        • CPU limits
        • Memory limits
        • Secrets
        • Environment variables
        • Labels
        • Service accounts
        • Cloud SQL connections

        • VPC connectivity

          For a full list of available options when creating a job, refer to the gcloud beta run jobs create command line documentation.

   2. Wait for the job creation to finish. Upon successful completion, a success message is displayed

   3. To execute the job see execute jobs or execute jobs on a schedule.

Terraform

To learn how to apply or remove a Terraform configuration, see Basic Terraform commands.

To create a new Cloud Run job, use google_cloud_run_v2_job resource and modify your main.tf file as shown in the following snippet.

cloud_run_jobs_create/main.tf

View on GitHub Feedback

resource "google_cloud_run_v2_job" "default" {
  provider     = google-beta
  name         = "cloud-run-job"
  location     = "us-central1"
  launch_stage = "BETA"

  template {
    template {
      containers {
        image = "us-docker.pkg.dev/cloudrun/container/job:latest"
      }
    }
  }
}

Cloud Run locations

Cloud Run is regional, which means the infrastructure that runs your Cloud Run services is located in a specific region and is managed by Google to be redundantly available across all the zones within that region.

Meeting your latency, availability, or durability requirements are primary factors for selecting the region where your Cloud Run services are run. You can generally select the region nearest to your users but you should consider the location of the other Google Cloud products that are used by your Cloud Run service. Using Google Cloud products together across multiple locations can affect your service's latency as well as cost.

Cloud Run is available in the following regions:

Subject to Tier 1 pricing

• asia-east1 (Taiwan)
• asia-northeast1 (Tokyo)
• asia-northeast2 (Osaka)
• europe-north1 (Finland) Low CO₂
• europe-southwest1 (Madrid) Low CO₂
• europe-west1 (Belgium) Low CO₂
• europe-west4 (Netherlands)
• europe-west8 (Milan)
• europe-west9 (Paris) Low CO₂
• me-west1 (Tel Aviv)
• us-central1 (Iowa) Low CO₂
• us-east1 (South Carolina)
• us-east4 (Northern Virginia)
• us-east5 (Columbus)
• us-south1 (Dallas)
• us-west1 (Oregon) Low CO₂

Subject to Tier 2 pricing

• asia-east2 (Hong Kong)
• asia-northeast3 (Seoul, South Korea)
• asia-southeast1 (Singapore)
• asia-southeast2 (Jakarta)
• asia-south1 (Mumbai, India)
• asia-south2 (Delhi, India)
• australia-southeast1 (Sydney)
• australia-southeast2 (Melbourne)
• europe-central2 (Warsaw, Poland)
• europe-west12 (Turin)
• europe-west2 (London, UK)
• europe-west3 (Frankfurt, Germany)
• europe-west6 (Zurich, Switzerland) Low CO₂
• northamerica-northeast1 (Montreal) Low CO₂
• northamerica-northeast2 (Toronto) Low CO₂
• southamerica-east1 (Sao Paulo, Brazil) Low CO₂
• southamerica-west1 (Santiago, Chile)
• us-west2 (Los Angeles)
• us-west3 (Salt Lake City)
• us-west4 (Las Vegas)

If you already created a Cloud Run service, you can view the region in the Cloud Run dashboard in the Google Cloud console.

OK

When you create a new job, the Cloud Run service agent needs to be able to access the container, which is the case by default.

Update an existing job

Changing any configuration settings requires you to update the job, even if there is no change to the container image. Note that for any unchanged settings, the previous settings continue to be used.

You can update an existing job using the Google Cloud console, or the gcloud command line.

Console

To update an existing job:

1. Go to Cloud Run

2. Click the Jobs tab to display the list of jobs.

3. Click the desired job to display the Job details page.

4. Click Edit.

5. If you made changes to your job code, specify the new container image digest.

6. Optionally, change the number of tasks that are in the job if needed.

7. Optionally, click Container, Networking, Security to update any additional job properties as desired:

   • Under Task capacity:
   1. In the Memory dropdown menu, specify the amount of memory required. The default is the minimum required, 512MiB.
   2. In the CPU dropdown menu, specify the amount of CPU required. The default is the minimum required, 1 CPU.

   3. Under Task timeout, specify the maximum amount of time in seconds that the task can run, up to 1 hour. Each task must complete within this time. The default is 10 minutes (600 seconds).

   4. Under Number of retries per failed task, specify the number of retries in case of task failures. The default is 3 retries.

   • Under Parallelism:

     1. In most cases you can select Run as many tasks concurrently as possible,.
     2. If you need to set a lower limit due to scaling constraints on resources your job accesses, select Limit the number of concurrent tasks and specify the maximum number of concurrent tasks in the Custom parallelism limit textbox.

8. Optionally, configure other settings in the appropriate tabs:

   • Container configuration
   • CPU limits
   • Memory limits
   • Secrets
   • Environment variables
   • Labels
   • Service accounts
   • Cloud SQL connections
   • VPC connectivity

9. When you are finished configuring your job, click Save to create the job in Cloud Run and wait for the job creation to finish.

10. To execute the job see execute jobs or execute jobs on a schedule.

Command line

1. In the Google Cloud console, activate Cloud Shell.

   Activate Cloud Shell

   At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

2. Run the command:

   gcloud beta run jobs update JOB_NAME

   Replace

   • JOB_NAME with the name of the job you want to update.

   • Optionally, replace OPTIONS with the following options:

     Option	Description
     --tasks	Accepts integers equal or greater than 1. Defaults to 1; maximum is 10,000. Each task is provided the environment variables CLOUD_RUN_TASK_INDEX with a value between 0 and the number of tasks minus 1, along with CLOUD_RUN_TASK_COUNT, which is the number of tasks
     --max-retries	The number of times a failed task is retried. Once any task fails beyond this limit, the entire job is marked as failed. For example, if set to 1, a failed task will be retried once, for a total of two attempts. The default is 3. Accepts integers from 0 to 10.
     --task-timeout	Accepts a duration like "2s". Defaults to 10 minutes; maximum is 1h.
     --parallelism	The maximum number of tasks that can execute in parallel. By default, tasks will be started as quickly as possible, in parallel. Refer to Parallelism for the range of values.

   In addition to the above options, you can set other optional configuration settings:

   • Container configuration
   • CPU limits
   • Memory limits
   • Secrets
   • Environment variables
   • Labels
   • Service accounts
   • Cloud SQL connections
   • VPC connection

   For a full list of available options when creating a job, refer to the gcloud beta run jobs create command line documentation.

3. Wait for the job update to finish. Upon successful completion, a success message is displayed, similar to the following:

   Job [JOB_NAME] has been successfully updated.
   View details about this job by running `gcloud beta run jobs describe JOB_NAME`.
   See logs for this execution at: https://console.cloud.google.com/logs/viewer?project=PROJECT_ID&resource=cloud_run_revision/service_name/JOB_NAME
   

4. To execute the job see execute jobs or execute jobs on a schedule.

Terraform

Make changes to your job configuration in your main.tf file using the terraform apply command. Detailed Terraform instructions are available for:

• Parallelism
• Max retries
• Timeouts

For more information, refer to the terraform apply command line options.

What's next

After you create or update a job, you can do the following:

• Execute a job
• Execute a job on a schedule
• Manage jobs
• Manage job executions
• View job logs
• Monitor job performances
• Set memory limits
• Set environment variables