Build repositories from GitLab

Stay organized with collections Save and categorize content based on your preferences.

This page explains how you can build repositories from GitLab by using webhook triggers. If you want to build repositories from GitLab Enterprise Edition, see Building repositories from GitLab Enterprise Edition.

Before you begin

  • Enable the Cloud Build and Secret Manager APIs.

    Enable the APIs

  • To use gcloud commands on this page, install the Google Cloud CLI.

Setting up

Before creating a webhook trigger to build on GitLab, you will need to create an SSH key in order to authenticate your connection to GitLab. When you create a trigger without an associated repository and access your code on an external source code management system, such as GitLab, you'll need to retrieve your SSH key in your inline build configuration.

This section discusses how you can both create and store your SSH key prior to creating a webhook trigger.

Creating an SSH key

To access your code on GitLab, you'll need to retrieve an SSH key in your inline build config.

To create an SSH key:

  1. Open a terminal window.

  2. Create a new directory named working-dir and navigate into it:

    mkdir working-dir
cd working-dir

  3. Create a new GitLab SSH key, where gitlab.com is the URL for your GitLab repository:

    ssh-keygen -t rsa -b 4096 -N '' -C gitlab.com -f id_gitlab

    The command creates a new SSH key in working-dir/id_gitlab without a passphrase for your SSH key. Cloud Build cannot use your SSH key if it is protected with a passphrase.

Enabling SSH access on GitLab

You will need to enable SSH access on your GitLab to allow users on your server to add their own SSH keys and use those SSH keys to secure Git operations between their computer and the GitLab instance. To learn how to use SSH with GitLab, refer to GitLab and SSH keys.

Adding your public SSH access key on GitLab

To secure operations that other systems perform on your repositories managed in GitLab, you will need to add your public SSH access key to GitLab. To learn how to add your key, see Add an SSH key to your GitLab account.

Creating and storing your credentials in Secret Manager

When you create an SSH key, an id_gitlab file is created in your local environment. Since this file contains sensitive information associated with authentication, you must store the file in Secret Manager before using it to invoke a build.

In addition to the secret used when creating a webhook trigger, you will also need to create a secret in Secret Manager to validate and authorize incoming webhook events to Cloud Build.

To create and store your credentials in Secret Manager:

  1. Go to the Secret Manager page in the Google Cloud console:

    Go to the Secret Manager page

  2. On the Secret Manager page, click Create Secret.

  3. On the Create secret page, under Name, enter a name for your secret.

  4. In the Secret value field, enter a name for your secret or upload a file.

    To upload your SSH key file, click Upload to include your working-dir/id_gitlab file.

  5. Leave the Regions section unchanged.

  6. Click the Create secret button to create your secret.

After you create your secret, Google Cloud console will automatically grant the Secret Manager Secret Accessor role to your Cloud Build service account, ${PROJECT_NUMBER}@gcp-sa-cloudbuild.iam.gserviceaccount.com. If you do not see this role on your service account, complete the following steps outlined in Granting Secret Manager role to your service account.

Now that you have stored your SSH key, you can also delete the SSH key from your environment by running the following command:

rm id_gitlab*

You are now ready to create your webhook trigger.

Creating webhook triggers

Console

To create a webhook trigger that invokes builds from GitLab using the Google Cloud console:

  1. Open the Triggers page:

    Open the Build triggers page

  2. Select your project from the top of the page and click Open.

  3. Click Create trigger.

  4. Enter the following trigger settings:

    • Name: A name for your trigger.

    • Region: Select the region for your trigger.

      • If you select global as the region, Cloud Build uses the default pool to run your build.
      • If you select a non-global region and the build config file associated with the trigger specifies a private pool, Cloud Build uses the private pool to run your build. In this case, the region you specify in your trigger must match the region where you created your private pool.
      • If you select a non-global region and the build config file associated with the trigger does not specify a private pool, Cloud Build uses the default pool to run your build in the same region as your trigger.

    • Description (Optional): A description for your trigger.

    • Event: Select Webhook event to set up your trigger to start builds in response to incoming webhook events.

    • Webhook URL: Use the webhook URL to authenticate incoming webhook events.

      • Secret: You will need a secret to authenticate incoming webhook events. You can create a new secret or use an existing secret. This secret is separate from the secret associated with your SSH key.

        To create a new secret:

        1. Select Use a new secret (generated by Cloud Build).

        2. Click Create Secret.

          You will see the Create a webhook secret pop-up box.

        3. In the Secret name field, enter a name for your secret.

        4. Click Create secret to save your secret, which will automatically be created and stored for you in Secret Manager.

        To use an existing secret:

        1. Select Use an existing secret or create your own.
        2. In the Secret field, select the name of the secret you want to use from the drop-down menu or follow the instructions to add a secret by resource ID.
        3. In the Secret version field, select your secret version from the drop-down menu.

        If you use an existing secret, you may need to manually grant the Secret Manager Secret Accessor role to your Cloud Build service account, service-${PROJECT_NUMBER}@gcp-sa-cloudbuild.iam.gserviceaccount.com. To learn more, see Granting Secret Manager role to your service account.

      After you have created or selected your secret, you will see a Webhook URL preview. Your URL will contain an API key generated by Cloud Build and your secret. If Cloud Build is unable to retrieve your API key, you can manually add your API key to the URL or learn how to obtain an API key if you do not have one yet.

      You can use the URL to invoke a webhook event by making an HTTP request using the POST method.

      If you selected global as your region, use the following command to invoke a webhook event:

      curl -X POST -H "application/json" "https://cloudbuild.googleapis.com/v1/projects/${PROJECT_ID}/triggers/${TRIGGER_NAME}:webhook?key=${API_KEY}&secret=${SECRET_VALUE}" -d "{}"

      If you select a non-global region, such as us-central1, use the following command to invoke a webhook event:

      curl -X POST -H "application/json" "https://cloudbuild.googleapis.com/v1/projects/${PROJECT_ID}/locations/${REGION}/triggers/${TRIGGER_NAME}:webhook?key=${API_KEY}&secret=${SECRET_VALUE}&trigger=${TRIGGER_NAME}&projectId=${PROJECT_ID}" -d "{}"

      To learn how you can use the URL when creating a webhook in GitLab, see Creating a webhook in GitLab.

    • Source (optional): The repository to build when the webhook trigger runs. In this example, the build configuration is an inline build config so a source is not needed. The field is left blank. If you specify a source, select 1st generation. You cannot invoke a build on a webhook event when selecting 2nd generation as your source. To learn more, see Cloud Build repositories.

    • Configuration: Create an inline build config in the Google Cloud console.

      In the following example, the inline build config authenticates your connection to GitLab using your SSH key and accesses your specified repository. Following that, it checks out the commit that invoked the webhook.

      steps:
# first, setup SSH:
# 1- save the SSH key from Secret Manager to a file
# 2- add the host key to the known_hosts file
- name: gcr.io/cloud-builders/git
  args:
    - '-c'
    - |
      echo "$$SSHKEY" > /root/.ssh/id_rsa
      chmod 400 /root/.ssh/id_rsa
      ssh-keyscan gitlab.com > /root/.ssh/known_hosts
  entrypoint: bash
  secretEnv:
    - SSHKEY
  volumes:
    - name: ssh
      path: /root/.ssh
# second, clone the repository
- name: gcr.io/cloud-builders/git
  args:
    - clone
    - '-n'
    - 'git@gitlab.com/GITLAB_REPO'
    - .
  volumes:
    - name: ssh
      path: /root/.ssh
# third, checkout the specific commit that invoked this build
- name: gcr.io/cloud-builders/git
  args:
    - checkout
    - $_TO_SHA
availableSecrets:
  secretManager:
  - versionName: PATH_TO_SECRET_VERSION
    env: SSHKEY

      Where:

      • GITLAB_REPO is the path for your GitLab repo.
      • PATH_TO_SECRET_VERSION is the path to your secret version as stored in Secret Manager. This is the secret that contains your SSH key. For example, projects/project-id/secrets/secret-name/versions/1
      • SSHKEY is the name of the environment used in this case to store the path to your secret.

    • Substitutions (optional): You can choose to define trigger-specific substitution variables using this field.

      In this example, let's say you want to watch for a specific branch name associated with a commit ID and then switch to that branch name in the build definition. To obtain this data, you can create substitution variables using payload bindings to save the branch name.

      Specify the following variables and values below:

      Variable Name Variable Value
      _BRANCH $(body.ref)
      _TO_SHA $(body.after)

      To view the payload associated with GitLab events, see the GitLab documentation page on Events.

    • Filters (optional): You can create a rule within a trigger that determines whether or not your trigger will execute a build based on your substitution variables.

      Since you want the trigger to execute a build if the branch name matches main, you can use the "==" operator to check for exact matches. You can also use the "matches" keyword if you want to match by regular expression.

      Specify the following as your filters:

      • _BRANCH == refs/heads/main

      To see more example syntax for filtering you could apply to your webhook triggers, see Using CEL to filter build events.

  5. Click Create to create your build trigger.

gcloud

To create a webhook trigger that invokes builds from GitLab:

     gcloud alpha builds triggers create webhook \
       --name=TRIGGER_NAME \
       --secret=PATH_TO_SECRET \
       --substitutions='' \
       --filter='' \
       --inline-config=PATH_TO_INLINE_BUILD_CONFIG \ # or --repo=PATH_TO_REPO
       --branch=BRANCH_NAME #  or --tag=TAG_NAME

Where:

  • TRIGGER_NAME is the name of your trigger.
  • PATH_TO_SECRET is the path to your secret as stored in Secret Manager. For example, projects/my-project/secrets/my-secret/versions/2.
  • PATH_TO_INLINE_BUILD_CONFIG is the path to your inline build config. Although the build config is stored inline on the trigger you're creating, the config path here is pulled from a local file.

  • PATH_TO_REPO is the path to the repository to invoke a build on. For example, https://gitlab.com/project-namespace. Your repository URL should be a Cloud Build connected repository.

  • BRANCH_NAME is the name of your branch if you want to set your trigger to build on a branch.

  • TAG_NAME is the name of your tag if you want to set your trigger to build on a tag.

Creating a webhook in GitLab

For GitLab to make requests to Cloud Build, you will need to create a webhook in GitLab by following the instructions outlined on the GitLab documentation for Webhooks.

Now, each time updates to your repository match the trigger event you specified in your webhook, a build will automatically be invoked by Cloud Build webhook triggers.

What's next