Host GitHub runners with Cloud Run worker pools

This tutorial walks you through using self-hosted GitHub runners on worker pools to execute the workflows defined in your GitHub repository.

You will deploy a Cloud Run worker pool to handle this workload, and optionally deploy a Cloud Run function to support scaling of the worker pool.

About self-hosted GitHub runners

In a GitHub Actions workflow, runners are the machines that execute jobs. For example, a runner can clone your repository locally, install testing software, and then run commands that evaluate your code.

You can use self-hosted runners to run GitHub Actions on Cloud Run worker pool instances. This tutorial shows you how to automatically scale a pool of runners based on the number of running and unscheduled jobs, even scaling the pool to zero when there are no jobs.

Objectives

In this tutorial, you will:

• Deploy a Cloud Run worker pool to Cloud Run.
• Deploy a Cloud Run function to support scaling of the worker pool.
• Create Secret Manager secrets to securely store tokens and secrets.
• Deploy a self-hosted GitHub runner to support a GitHub repository.

Costs

In this document, you use the following billable components of Google Cloud:

• Cloud Build
• Artifact Registry
• Secret Manager
• Cloud Run

To generate a cost estimate based on your projected usage, use the pricing calculator.

New Google Cloud users might be eligible for a free trial.

Before you begin

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

1. Enable the Cloud Run, Secret Manager, Artifact Registry, and Cloud Build APIs.

   Roles required to enable APIs

   To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains the serviceusage.services.enable permission. Learn how to grant roles.

   Enable the APIs

Required roles

To get the permissions that you need to complete the tutorial, ask your administrator to grant you the following IAM roles on your project:

• Artifact Registry Repository Administrator (roles/artifactregistry.repoAdmin)
• Cloud Build Editor (roles/cloudbuild.builds.editor)
• Cloud Run Admin (roles/run.admin)
• Create Service Accounts (roles/iam.serviceAccountCreator)
• Secret Manager Admin (roles/secretmanager.admin)
• Service Account User (roles/iam.serviceAccountUser)
• Service Usage Consumer (roles/serviceusage.serviceUsageConsumer)

For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

Note: IAM basic roles might also contain permissions to complete the tutorial. You shouldn't grant basic roles in a production environment, but you can grant them in a development or test environment.

You need permission to edit the settings on a GitHub repository to configure the self-hosted runners. The repository can be user-owned, or an organisation owned repository.

Caution: GitHub recommends using self-hosted runners with private repositories only.

Retrieve the code sample

To retrieve the code sample for use:

1. Clone the sample repository to your local machine:

   git clone https://github.com/GoogleCloudPlatform/cloud-run-samples
   

2. Change to the directory that contains the Cloud Run sample code:

   cd cloud-run-samples/github-runner
   

Understand the core code

The sample is implemented as a worker pool, and autoscaler, described next.

Worker pool

The worker pool is configured with a Dockerfile that is based on the GitHub-created actions/runner image.

All the logic is self-contained in this image, apart from a small helper script.

FROM ghcr.io/actions/actions-runner:2.328.0

# Add scripts with right permissions.
USER root
# hadolint ignore=DL3045
COPY start.sh start.sh
RUN chmod +x start.sh

# Add start entrypoint with right permissions.
USER runner
ENTRYPOINT ["./start.sh"]

This helper script runs when the container is started, registering itself to the configured repository as an ephemeral instance, using a token you will create. The script also defines what actions to take when the container is scaled down.

# Configure the current runner instance with URL, token and name.
mkdir /home/docker/actions-runner && cd /home/docker/actions-runner
echo "GitHub Repo: ${GITHUB_REPO_URL} for ${RUNNER_PREFIX}-${RUNNER_SUFFIX}"
./config.sh --unattended --url ${GITHUB_REPO_URL} --pat ${GH_TOKEN} --name ${RUNNER_NAME}

# Function to cleanup and remove runner from Github.
cleanup() {
   echo "Removing runner..."
   ./config.sh remove --unattended --pat ${GH_TOKEN}
}

# Trap signals.
trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM

# Run the runner.
./run.sh & wait $!

Autoscaler

The autoscaler is a function that scales up the worker pool when there is a new job in the queue, or scales down when a job is complete. It uses the Cloud Run API to check the current number of workers in the pool, and adjusts that value as required.

try:
    current_instance_count = get_current_worker_pool_instance_count()
except ValueError as e:
    return f"Could not retrieve instance count: {e}", 500

# Scale Up: If a job is queued and we have available capacity
if action == "queued" and job_status == "queued":
    print(f"Job '{job_name}' is queued.")

    if current_instance_count < MAX_RUNNERS:
        new_instance_count = current_instance_count + 1
        try:
            update_runner_instance_count(new_instance_count)
            print(f"Successfully scaled up to {new_instance_count} instances.")
        except ValueError as e:
            return f"Error scaling up instances: {e}", 500
    else:
        print(f"Max runners ({MAX_RUNNERS}) reached.")

# Scale Down: If a job is completed, check to see if there are any more pending
# or in progress jobs and scale accordingly.
elif action == "completed" and job_status == "completed":
    print(f"Job '{job_name}' completed.")

    current_queued_actions, current_running_actions = get_current_actions()
    current_actions = current_queued_actions + current_running_actions

    if current_queued_actions >= 1:
        print(
            f"GitHub says {current_queued_actions} are still pending."
            f"Won't change scaling ({current_instance_count})."
        )
    elif current_queued_actions == 0 and current_running_actions >= 1:
        print(
            f"GitHub says no queued actions, but {current_running_actions} running actions."
            f"Won't change scaling ({current_instance_count})."
        )
    elif current_actions == 0:
        print(f"GitHub says no pending actions. Scaling to zero.")
        update_runner_instance_count(0)
        print(f"Successfully scaled down to zero.")
    else:
        print(
            f"Detected an unhandled state: {current_queued_actions=}, {current_running_actions=}"
        )
else:
    print(
        f"Workflow job event for '{job_name}' with action '{action}' and "
        f"status '{job_status}' did not trigger a scaling action."
    )

Configure IAM

This tutorial uses a custom service account with the minimum permissions required to use the provisioned resources. To set up the service account, do the following:

1. Set your project ID in gcloud:

   gcloud config set project PROJECT_ID
   

   Replace PROJECT_ID with your project ID.

2. Create a new Identity and Access Management service account:

   gcloud iam service-accounts create gh-runners
   

3. Grant the service account permissions to act as a service account on your project:

   gcloud projects add-iam-policy-binding PROJECT_ID \
     --member "serviceAccount:gh-runners@PROJECT_ID.iam.gserviceaccount.com" \
     --role=roles/iam.serviceAccountUser
   

   Replace PROJECT_ID with your project ID.

Retrieve GitHub information

The GitHub documentation for adding self-hosted runners suggests adding runners through the GitHub website, which then provides a specific token to use for authentication.

This tutorial will dynamically add and remove runners, and needs a static GitHub token to do so.

To complete this tutorial, you need to create a GitHub token with access to interact with your selected repository.

Identify the GitHub repository

In this tutorial, the GITHUB_REPO variable represents the repository name. This is the part of the GitHub repository name that after the domain name, for both personal user repositories and organization repositories.

You will be referencing the repository name that comes after the domain name for both user-owned and organization-owned repositories.

In this tutorial:

• For https://github.com/myuser/myrepo, the GITHUB_REPO is myuser/myrepo.
• For https://github.com/mycompany/ourrepo, the GITHUB_REPO is mycompany/ourrepo.

Create access token

You need to create an access token on GitHub, and securely save it in Secret Manager:

1. Ensure you are logged into your GitHub account.
2. Navigate to GitHub's Settings > Developer Settings > Personal Access Tokens page.
3. Click Generate new token, and select Generate new token (classic).
4. Create a new token with the "repo" scope.
5. Click Generate token.
6. Copy the generated token.

Create secret value

Take the secret token you just created, and store it in Secret Manager, and set access permissions.

1. Create the secret in Secret Manager:

   echo -n "GITHUB_TOKEN" | gcloud secrets create github_runner_token --data-file=-
   

   Replace the GITHUB_TOKEN with the value you copied from GitHub.

2. Grant access to your newly created secret:

   gcloud secrets add-iam-policy-binding github_runner_token \
     --member "serviceAccount:gh-runners@PROJECT_ID.iam.gserviceaccount.com" \
     --role "roles/secretmanager.secretAccessor"
   

Deploy Worker Pool

Create a Cloud Run worker pool to process GitHub actions. This pool will use an image based on the GitHub-created actions/runner image.

Set up Cloud Run worker pool

1. Navigate to the sample code for the worker pool:

   cd worker-pool-container
   

2. Deploy the worker pool:

   gcloud beta run worker-pools deploy WORKER_POOL_NAME \
     --region WORKER_POOL_LOCATION \
     --source . \
     --scaling 1 \
     --set-env-vars GITHUB_REPO=GITHUB_REPO \
     --set-secrets GITHUB_TOKEN=github_runner_token:latest \
     --service-account gh-runners@PROJECT_ID.iam.gserviceaccount.com \
     --memory 2Gi \
     --cpu 4
   

   Replace the following:

   • WORKER_POOL_NAME the name of the worker pool
   • WORKER_POOL_LOCATION the region of the worker pool
   • GITHUB_REPO the identified GitHub repo name
   • PROJECT_ID the Google Cloud project ID

   If this is the first time using Cloud Run source deploys in this project, you will be prompted to create default Artifact Registry repository.

Using worker pool

You now have a single instance in your worker pool, ready to accept jobs from GitHub actions.

To verify the you have completed the setup of your self-hosted runner, invoke a GitHub action on your repository.

For your action to use your self-hosted runners, you need to change a GitHub action's job. In the job, change the runs-on value to self-hosted.

If your repo doesn't already have any actions, you can follow the Quickstart for GitHub Actions.

Once you can configured an action to use the self-hosted runners, run the action.

Confirm the action completes successfully in the GitHub interface.

Deploy GitHub Runner Autoscaler

You deployed one worker in your original pool, which will allow processing of one action at a time. Depending on your CI usage, you may need to scale your pool to handle an influx of work to be done.

Once you deploy the worker pool with an active GitHub runner, configure the autoscaler to provision worker instances based on the job status in the actions queue.

This implementation listens for a workflow_job event. When workflow job is created, it will scale up the worker pool, and once the job is completed, scale it down again. It won't scale the pool beyond the maximum number of instances configured, and will scale to zero when all running jobs have completed.

You can adapt this autoscaler based on your workloads.

Create webhook secret value

To create a secret value for the webhook, do the following:

1. Create a Secret Manager secret containing an arbitrary string value.

   echo -n "WEBHOOK_SECRET" | gcloud secrets create github_webhook_secret --data-file=-
   

   Replace WEBHOOK_SECRET with an arbitrary string value.

2. Grant access to the secret to the autoscaler service account:

   gcloud secrets add-iam-policy-binding github_webhook_secret \
     --member "serviceAccount:gh-runners@PROJECT_ID.iam.gserviceaccount.com" \
     --role "roles/secretmanager.secretAccessor"
   

Deploy the function to receive webhook requests

To deploy the function for receiving webhook requests, do the following:

1. Navigate to the sample code for the webhook:

   cd ../autoscaler
   

2. Deploy the Cloud Run function:

   gcloud run deploy github-runner-autoscaler \
     --function github_webhook_handler \
     --region WORKER_POOL_LOCATION \
     --source . \
     --set-env-vars GITHUB_REPO=GITHUB_REPO \
     --set-env-vars WORKER_POOL_NAME=WORKER_POOL_NAME \
     --set-env-vars WORKER_POOL_LOCATION=WORKER_POOL_LOCATION \
     --set-env-vars MAX_RUNNERS=5 \
     --set-secrets GITHUB_TOKEN=github_runner_token:latest \
     --set-secrets WEBHOOK_SECRET=github_webhook_secret:latest \
     --service-account gh-runners@PROJECT_ID.iam.gserviceaccount.com \
     --allow-unauthenticated
   

   Replace the following:

   • GITHUB_REPO the part of your GitHub repository name after the domain name
   • WORKER_POOL_NAME the name of the worker pool
   • WORKER_POOL_LOCATION the region of the worker pool
   • REPOSITORY_NAME the GitHub repository name

3. Note the URL your service was deployed to. You will use this value in a later step.

4. Grant the service account permissions to update your worker pool:

   gcloud alpha run worker-pools add-iam-policy-binding WORKER_POOL_NAME \
     --member "serviceAccount:gh-runners@PROJECT_ID.iam.gserviceaccount.com" \
     --role=roles/run.developer
   

   Replace PROJECT_ID with your project ID.

Create GitHub webhook

To create the GitHub webhook, follow these steps:

1. Ensure you are logged into your GitHub account.
2. Navigate to your GitHub repository.
3. Click Settings.
4. Under "Code and automation", click Webhooks.
5. Click Add webhook.

6. Enter the following:

   1. In Payload URL, enter the URL of the Cloud Run function you deployed earlier.

      The URL will look like: https://github-runner-autoscaler-PROJECTNUM.REGION.run.app, where PROJECTNUM is the unique numeric identifier of your project, and REGION is the region you deployed the service to.

   2. For Content type, select application/json.

   3. For Secret, enter the WEBHOOK_SECRET value you created previously.

   4. For SSL verification, select Enable SSL verification.

   5. For "Which events would you like to trigger this webhook?", select Let me select individual events.

   6. In the event selection, select Workflow jobs. Unselect any other option.

   7. Click Add webhook.

Scale down your worker pool

The webhook is now in place, so you don't have to have a persistent worker in the pool. This will also ensure you have no running workers when there is no work to be done, reducing costs.

• Adjust your pool to scale to zero:

  gcloud beta run worker-pools update WORKER_POOL_NAME \
    --region WORKER_POOL_LOCATION \
    --scaling 0
  

Use your autoscaling runner

To verify your autoscaling runner is working correctly, run an action you previously configured to runs-on: self-hosted.

You can track progress of your GitHub Actions on the "Actions" tab of your repository.

You can check the execution of your webhook function and worker pool by checking the Logs tab of the Cloud Run function and Cloud Run worker pool respectively.

Clean up

To avoid additional charges to your Google Cloud account, delete all the resources you deployed with this tutorial.

Delete the project

If you created a new project for this tutorial, delete the project. If you used an existing project and need to keep it without the changes you added in this tutorial, delete resources that you created for the tutorial.

The easiest way to eliminate billing is to delete the project that you created for the tutorial.

To delete the project:

Caution: Deleting a project has the following effects:

• Everything in the project is deleted. If you used an existing project for the tasks in this document, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.

If you plan to explore multiple architectures, tutorials, or quickstarts, reusing projects can help you avoid exceeding project quota limits.

1. In the Google Cloud console, go to the Manage resources page.

   Go to Manage resources

2. In the project list, select the project that you want to delete, and then click Delete.
3. In the dialog, type the project ID, and then click Shut down to delete the project.

Delete tutorial resources

1. Delete the Cloud Run service you deployed in this tutorial. Cloud Run services don't incur costs until they receive requests.

   To delete your Cloud Run service, run the following command:

   gcloud run services delete SERVICE-NAME

   Replace SERVICE-NAME with the name of your service.

   You can also delete Cloud Run services from the Google Cloud console.

2. Remove the gcloud default region configuration you added during tutorial setup:

    gcloud config unset run/region
   

3. Remove the project configuration:

    gcloud config unset project
   

4. Delete other Google Cloud resources created in this tutorial:

   • Delete the Cloud Run function
   • Delete the Cloud Run worker pool
   • Delete the service container image from Artifact Registry.
   • Delete the service account gh-runners
   • Delete the secrets in Secret Manager

What's next

• Learn more about Cloud Run worker pools
• Explore other Cloud Run demos, tutorials, and samples