This tutorial demonstrates how to use Secret Manager with Cloud Build to interact with private GitHub repositories. Secret Manager is a Google Cloud service that securely stores API keys, passwords, and other sensitive data.
- Set up a GitHub SSH key.
- Add the public SSH key to a private repository's deploy keys.
- Store the private SSH key in Secret Manager.
- Submit a build that accesses the key from Secret Manager and uses it to access the private repository.
CostsThis tutorial uses billable components of Google Cloud, including:
- Secret Manager
- Cloud Build
Use the Pricing Calculator to generate a cost estimate based on your projected usage.New Google Cloud users might be eligible for a free trial.
Before you begin
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.
Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.
- Enable the Cloud Build and Secret Manager APIs.
- Install and initialize the Cloud SDK.
- Optional. Complete the Secret Manager quickstart to become familiar with this product.
Create a SSH key
Open a terminal window.
Create a new directory named
workingdirand navigate into it:
mkdir workingdir cd workingdir
Create a new GitHub SSH key, where github-email is your GitHub email address:
ssh-keygen -t rsa -b 4096 -N '' -f id_github -C github-email
This command creates a new SSH key
workingdir/id_githubwithout a passphrase for your SSH key. Cloud Build cannot use your SSH key if it is protected with a passphrase.
Store the private SSH key in Secret Manager
When you create an SSH key,
id_github file is created in your environment. Because anyone can
authenticate to your account with this file, you must store the file in
Secret Manager before using it in a build.
To store your SSH key in Secret Manager:
Go to the Secret Manager page in the Cloud Console:
On the Secret Manager page, click Create Secret.
On the Create secret page, under Name, enter
In the Secret value field, click Upload and upload your
Leave the Regions section unchanged.
Click the Create secret button.
This will upload your
id_github file to Secret Manager.
Add the public SSH key to your private repository's deploy keys
Login to GitHub.
In the upper-right corner, click your profile photo, then click Your profile.
On your profile page, click Repositories, then click the name of your repository.
From your repository, click Settings.
In the sidebar, click Deploy Keys, then click Add deploy key.
Provide a title, paste your public SSH key from
Select Allow write access if you want this key to have write access to the repository. A deploy key with write access lets a deployment push to the repository.
Click Add key.
Delete the SSH key from your disk:
You need to grant the Cloud Build service account permission to access Secret Manager during the build.
Open the IAM page in the Cloud Console:
Select your project and click Open.
In the permissions table, locate the email ending with
@cloudbuild.gserviceaccount.com, and click on the pencil icon.
Secret Manager Secret Accessorrole.
Add the public SSH key to known hosts
Most machines contain a file named
known_hosts, which contains known keys for
remote hosts. The keys are often collected from the remote hosts when connecting
to them for the first time, but they can also be added manually. The keys in this
file are used to verify the identity of the remote host and protect against impersonation.
For Cloud Build to connect to GitHub, you must add the public SSH key
known_hosts file in Cloud Build's build environment. You
can do this by adding the key to a temporary
known_hosts.github file, and then
copying the contents of
known_hosts.github to the
in Cloud Build's build environment.
workingdir directory, create a file named
add the public SSH key to this file:
ssh-keyscan -t rsa github.com > known_hosts.github
In the next section when you configure the build, you'll add instructions in the
Cloud Build config file to copy the contents of
known_hosts file in Cloud Build's build environment.
Configure the build
To configure the build:
Create a build config file named
cloudbuild.yamlwith two steps: the first
gcloudstep accesses the SSH key in Secret Manager and saves it as
id_rsain a volume named
ssh, along with a copy of the
known_hosts.github. The volume is used to persist files across the build steps. The second
gitstep uses the key in
id_rsato connect to the repository at
# Access the id_github file from Secret Manager, and setup SSH steps: - name: 'gcr.io/cloud-builders/git' secretEnv: ['SSH_KEY'] entrypoint: 'bash' args: - -c - | echo "$$SSH_KEY" >> /root/.ssh/id_rsa chmod 400 /root/.ssh/id_rsa cp known_hosts.github /root/.ssh/known_hosts volumes: - name: 'ssh' path: /root/.ssh # Clone the repository - name: 'gcr.io/cloud-builders/git' args: - clone - --recurse-submodules - email@example.com:GIT_USERNAME/GIT_REPOSITORY volumes: - name: 'ssh' path: /root/.ssh availableSecrets: secretManager: - versionName: projects/PROJECT_ID/secrets/secret-name/versions/latest env: 'SSH_KEY'
Replace the placeholder values in the above commands with the following:
PROJECT_ID: The ID of the Cloud project where you've stored your secrets.
GIT_USERNAME: The GitHub username of the repository owner.
GIT_REPOSITORY: The name of the GitHub repository you want to access.
To learn about YAML multiline strings used in the snippet above, see YAML multiline.
Submit the build
To submit the build, run the following command:
gcloud builds submit --config=cloudbuild.yaml .
The output is similar to the following:
Creating temporary tarball archive of 3 file(s) totalling 4.1 KiB before compression. Uploading tarball of [.] to [gs://[PROJECT-ID]_cloudbuild/source/1504288639.02---.tgz] Created [https://cloudbuild.googleapis.com/v1/projects/[PROJECT-ID]/builds/871b68bc---]. Logs are available at [https://console.cloud.google.com/cloud-build/builds/871b68bc---?project=[PROJECT-ID]]. ----------------------------- REMOTE BUILD OUTPUT ------------------------------ starting build "871b68bc-cefc-4411-856c-2a2b7c7d2487" FETCHSOURCE Fetching storage object: gs://[PROJECT-ID]_cloudbuild/source/1504288639.02---.tgz#1504288640827178 Copying gs://[PROJECT-ID]_cloudbuild/source/1504288639.02---.tgz#1504288640827178... / [1 files][ 3.9 KiB/ 3.9 KiB] Operation completed over 1 objects/3.9 KiB. BUILD Step #0: Already have image (with digest): gcr.io/cloud-builders/gcloud Starting Step #0 Finished Step #0 Step #1: Already have image (with digest): gcr.io/cloud-builders/git Starting Step #1 Step #1: # github.com SSH-2.0-libssh_0.7.0 Finished Step #1 Step #2: Already have image (with digest): gcr.io/cloud-builders/git Starting Step #2 Step #2: Cloning into '[REPOSITORY-NAME]'... Step #2: Warning: Permanently added the RSA host key for IP address 'XXX.XXX.XXX.XXX' to the list of known hosts. Finished Step #2 PUSH DONE ----------------------------------------------------------------------------------------------------------------- ID CREATE_TIME DURATION SOURCE IMAGES STATUS 871b68bc-cefc-4411-856c-2a2b7c7d2487 XXXX-XX-XXT17:57:21+00:00 13S gs://[PROJECT-ID]_cloudbuild/source/1504288639.02---.tgz - SUCCESS
To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.
Delete the project
The easiest way to eliminate billing is to delete the project that you created for the tutorial.
To delete the project:
- In the Cloud Console, go to the Manage resources page.
- In the project list, select the project that you want to delete, and then click Delete.
- In the dialog, type the project ID, and then click Shut down to delete the project.
Delete the deploy key from your repository
On GitHub, navigate to the main page of the repository.
Under your repository name, click Settings.
In the left sidebar, click Deploy keys.
On the Deploy keys page, look for the deploy keys associated with your repository and click Delete.