Build and create a Shell job in Cloud Run

Build and create a Shell job in Cloud Run

Learn how to create a simple Cloud Run job, then deploy from source, which automatically packages your code into a container image, uploads the container image to Artifact Registry, and then deploys to Cloud Run. You can use other languages in addition to the ones shown.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

5. Make sure that billing is enabled for your Google Cloud project.

6. Enable the Cloud Run Admin API and the Cloud Build API:

   gcloud services enable run.googleapis.com \
       cloudbuild.googleapis.com

   After the Cloud Run Admin API is enabled, the Compute Engine default service account is automatically created.

7. For Cloud Build to be able to build your sources, grant the Cloud Build Service Account role to the Compute Engine default service account by running the following:
   gcloud projects add-iam-policy-binding PROJECT_ID \
       --member=serviceAccount:[e-mail] \
       --role=roles/cloudbuild.builds.builder

   Replace PROJECT_NUMBER with your Google Cloud project number, and PROJECT_ID with your Google Cloud project ID. For detailed instructions on how to find your project ID, and project number, see Creating and managing projects.

   Granting the Cloud Build Service Account role to the Compute Engine default service account takes a couple of minutes to propagate.

   Note:

   The iam.automaticIamGrantsForDefaultServiceAccounts organization policy constraint prevents the Editor role from being automatically granted to default service accounts. If you created your organization after May 3, 2024, this constraint is enforced by default.

   We strongly recommend that you enforce this constraint to disable the automatic role grant. If you disable the automatic role grant, you must decide which roles to grant to the default service accounts, and then grant these roles yourself.

   If the default service account already has the Editor role, we recommend that you replace the Editor role with less permissive roles. To safely modify the service account's roles, use Policy Simulator to see the impact of the change, and then grant and revoke the appropriate roles.

Writing the sample job

To write a Cloud Run job that runs a Shell script:

1. Create a new directory named jobs and change directory into it:

   mkdir jobs
   cd jobs
   

2. Create a Dockerfile file with the following contents:

   
   # Use the official Ubuntu image from Docker Hub as
   # a base image
   FROM ubuntu:24.04
   
   # Execute next commands in the directory /workspace
   WORKDIR /workspace
   
   # Copy over the script to the /workspace directory
   COPY script.sh .
   
   # Just in case the script doesn't have the executable bit set
   RUN chmod +x ./script.sh
   
   # Run the script when starting the container
   CMD [ "./script.sh" ]
   

3. In the same directory, create a script.sh file for the actual job code. Copy the following sample lines into it:

   #!/bin/bash
   set -euo pipefail
   
   # In production, consider printing commands as they are executed. 
   # This helps with debugging if things go wrong and you only 
   # have the logs.
   #
   # Add -x:
   # `set -euox pipefail`
   
   CLOUD_RUN_TASK_INDEX=${CLOUD_RUN_TASK_INDEX:=0}
   CLOUD_RUN_TASK_ATTEMPT=${CLOUD_RUN_TASK_ATTEMPT:=0}
   
   echo "Starting Task #${CLOUD_RUN_TASK_INDEX}, Attempt #${CLOUD_RUN_TASK_ATTEMPT}..."
   
   # SLEEP_MS and FAIL_RATE should be a decimal
   # numbers. parse and format the input using 
   # printf. 
   #
   # printf validates the input since it 
   # quits on invalid input, as shown here:
   #
   #   $: printf '%.1f' "abc"
   #   bash: printf: abc: invalid number
   #
   SLEEP_MS=$(printf '%.1f' "${SLEEP_MS:=0}")
   FAIL_RATE=$(printf '%.1f' "${FAIL_RATE:=0}")
   
   # Wait for a specific amount of time to simulate
   # performing some work
   SLEEP_SEC=$(echo print\("${SLEEP_MS}"/1000\) | perl)
   sleep "$SLEEP_SEC" # sleep accepts seconds, not milliseconds
   
   # Fail the task with a likelihood of $FAIL_RATE
   
   # Bash does not do floating point arithmetic. Use perl 
   # to convert into integer and multiply by 100.
   FAIL_RATE_INT=$(echo print\("int(${FAIL_RATE:=0}*100"\)\) | perl)
   
   # Generate a random number between 0 and 100
   RAND=$(( RANDOM % 100))
   if (( RAND < FAIL_RATE_INT )); then 
       echo "Task #${CLOUD_RUN_TASK_INDEX}, Attempt #${CLOUD_RUN_TASK_ATTEMPT} failed."
       exit 1
   else 
       echo "Completed Task #${CLOUD_RUN_TASK_INDEX}."
   fi
   

   Cloud Run jobs allows users to specify the number of tasks the job is to execute. This sample code shows how to use the built-in CLOUD_RUN_TASK_INDEX environment variable. Each task represents one running copy of the container. Note that tasks are usually executed in parallel. Using multiple tasks is useful if each task can independently process a subset of your data.

   Each task is aware of its index, stored in the CLOUD_RUN_TASK_INDEX environment variable. The built-in CLOUD_RUN_TASK_COUNT environment variable contains the number of tasks supplied at job execution time via the --tasks parameter.

   The code shown also shows how to retry tasks, using the built-in CLOUD_RUN_TASK_ATTEMPT environment variable, which contains the number of times this task has been retried, starting at 0 for the first attempt and incrementing by 1 for every successive retry, up to --max-retries.

   The code also lets you generate failures as a way to test retries and to generate error logs so you can see what those look like.

Your code is complete and ready to be packaged in a container.

Build jobs container, send it to Artifact Registry and deploy to Cloud Run

Important: This quickstart assumes that you have owner or editor roles in the project you are using for the quickstart. Otherwise, refer to the Cloud Run Source Developer role for the required permissions for deploying a Cloud Run resource from source.

This quickstart uses deploy from source, which builds the container, uploads it to Artifact Registry, and deploys the job to Cloud Run:

gcloud run jobs deploy job-quickstart \
    --source . \
    --tasks 50 \
    --set-env-vars SLEEP_MS=10000 \
    --set-env-vars FAIL_RATE=0.1 \
    --max-retries 5 \
    --region REGION \
    --project=PROJECT_ID

where PROJECT_ID is your project ID and REGION is your region, for example, us-central1. Note that you can change the various parameters to whatever values you want to use for your testing purposes. SLEEP_MS simulates work and FAIL_RATE causes X% of tasks to fail so you can experiment with parallelism and retrying failing tasks.

Execute a job in Cloud Run

To execute the job you just created:

gcloud run jobs execute job-quickstart --region REGION

Replace REGION with the region you used when you created and deployed the job, for example us-central1.

What's next

For more information on building a container from code source and pushing to a repository, see:

• Developing Cloud Run services
• Building Containers
• Test a Cloud Run service locally
• Deploying from source code