Accessing private GitHub repositories

This tutorial demonstrates how to use encrypted secrets and Key Management Service (KMS) to interact with private GitHub repositories.

You require encrypted secrets to connect to private GitHub repositories only when running manual builds. When you run builds using Cloud Build triggers, you can automatically connect to any private repository you own without encrypted secrets. For information on creating triggers, see Automating builds using triggers.

Objectives

• Set up a GitHub SSH key.
• Add the SSH key to a private repository's deploy keys.
• Encrypt the SSH key using KMS.
• Submit a build that decrypts the key and uses it to access the private repository.

Costs

This tutorial uses billable components of Google Cloud, including:

• KMS
• Cloud Build

Use the Pricing Calculator to generate a cost estimate based on your projected usage.

New Google Cloud users might be eligible for a free trial.

Before you begin

1. Sign in to your Google Account.

   If you don't already have one, sign up for a new account.

2. In the Cloud Console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to the project selector page

3. Make sure that billing is enabled for your Google Cloud project. Learn how to confirm billing is enabled for your project.

4. Enable the Cloud KMS API.

   Enable the API

5. Install and initialize the Cloud SDK.
6. Create an SSH key for GitHub.
7. Add a deploy key to your private repository.

Note: Do not set a passphrase when creating SSH keys. Cloud Build cannot use your SSH key if it is protected with a passphrase.

You might also want to complete the Cloud KMS quickstart to become familiar with this product.

Create a Cloud KMS KeyRing and CryptoKey

To use your SSH key with Cloud Build, you must use a KMS CryptoKey to encrypt and decrypt the key. CryptoKeys are stored in KeyRing objects.

You can create KeyRings and CryptoKeys using the gcloud kms keyrings create and gcloud kms keys create commands.

To create a KeyRing, run the following command in your shell or terminal window. For the purposes of this tutorial, name the KeyRing my-keyring:

gcloud kms keyrings create my-keyring --location=global

Next, create a CryptoKey named github-key. This command creates the github-key CryptoKey in the my-keyring KeyRing:

gcloud kms keys create github-key \
--location=global --keyring=my-keyring \
--purpose=encryption

The --location=global flag indicates that KMS should serve read and write operations from multiple geographical locations.

Encrypt an SSH key

When you create an SSH key, an id_rsa key file is created in your environment. Because anyone can authenticate to your account with this file, you should encrypt the file before using it in a build.

You can encrypt resources using the gcloud kms encrypt command.

To encrypt your SSH key, run the following command. Be sure to specify the correct paths:

gcloud kms encrypt --plaintext-file=/path/to/sshkey/id_rsa \
--ciphertext-file=/path/to/save/encryptedkey/id_rsa.enc \
--location=global --keyring=my-keyring --key=github-key

This command creates a id_rsa.enc file, which is an encrypted copy of id_rsa.

Grant the Cloud Build service account decrypt permission

You need to grant the Cloud Build service account permission to access and decrypt the CryptoKey during the build.

To learn how to grant the service account decrypt permission, refer to Using Encrypted Resources.

Create or update known_hosts file

You need to provide a known_hosts file containing the rsa key for github.com.

To create a known_hosts file, run the following command:

ssh-keyscan -t rsa github.com > known_hosts

Note: For security, verify the host fingerprints in the known_hosts file against GitHub’s published SSH host key fingerprints.

Configure the build

To configure the build:

1. Create a build config file named cloudbuild.yaml with three steps: the first gcloud step decrypts the SSH key and stores the decrypted key in a volume named ssh. The volume is used to persist files across the build steps. The second git step configures git to use the SSH key by adding it to the known_hosts file. The third git step uses the key in known_hosts to connect to the repository at [e-mail]:[GIT-USERNAME]/[REPOSITORY].

   Note: When you create the SSH key for GitHub, the key is stored in $HOME/.ssh/id_rsa on your local machine. In the build config file below, the volume is created in /root/.ssh because in the gcloud and git builders' environment, $HOME evaluates to /root.

   # Decrypt the file containing the key
   steps:
   - name: 'gcr.io/cloud-builders/gcloud'
     args:
     - kms
     - decrypt
     - --ciphertext-file=id_rsa.enc
     - --plaintext-file=/root/.ssh/id_rsa
     - --location=global
     - --keyring=my-keyring
     - --key=github-key
     volumes:
     - name: 'ssh'
       path: /root/.ssh
   
   # Set up git with key and domain
   - name: 'gcr.io/cloud-builders/git'
     entrypoint: 'bash'
     args:
     - '-c'
     - |
       chmod 600 /root/.ssh/id_rsa
       cat <<EOF >/root/.ssh/config
       Hostname github.com
       IdentityFile /root/.ssh/id_rsa
       EOF
       mv known_hosts /root/.ssh/known_hosts
     volumes:
     - name: 'ssh'
       path: /root/.ssh
   
   # Connect to the repository
   - name: 'gcr.io/cloud-builders/git'
     args:
     - clone
     - [e-mail]:[GIT-USERNAME]/[REPOSITORY]
     volumes:
     - name: 'ssh'
       path: /root/.ssh
   

2. Create a new directory in your environment containing the following files:

   1. The id_rsa.enc file
   2. The known_hosts file
   3. The build config YAML file

Submit the build

To submit the build, run the following command from the directory containing the id_rsa.enc file, the known_hosts file, and cloudbuild.yaml file:

gcloud builds submit --config=cloudbuild.yaml .

The output is similar to the following:

Creating temporary tarball archive of 3 file(s) totalling 4.1 KiB before compression.
Uploading tarball of [.] to [gs://[PROJECT-ID]_cloudbuild/source/1504288639.02---.tgz]
Created [https://cloudbuild.googleapis.com/v1/projects/[PROJECT-ID]/builds/871b68bc---].
Logs are available at [https://console.cloud.google.com/gcr/builds/871b68bc---?project=[PROJECT-ID]].
----------------------------- REMOTE BUILD OUTPUT ------------------------------
starting build "871b68bc-cefc-4411-856c-2a2b7c7d2487"

FETCHSOURCE
Fetching storage object: gs://[PROJECT-ID]_cloudbuild/source/1504288639.02---.tgz#1504288640827178
Copying gs://[PROJECT-ID]_cloudbuild/source/1504288639.02---.tgz#1504288640827178...
/ [1 files][  3.9 KiB/  3.9 KiB]
Operation completed over 1 objects/3.9 KiB.
BUILD
Step #0: Already have image (with digest): gcr.io/cloud-builders/gcloud
Starting Step #0
Finished Step #0
Step #1: Already have image (with digest): gcr.io/cloud-builders/git
Starting Step #1
Step #1: # github.com SSH-2.0-libssh_0.7.0
Finished Step #1
Step #2: Already have image (with digest): gcr.io/cloud-builders/git
Starting Step #2
Step #2: Cloning into '[REPOSITORY-NAME]'...
Step #2: Warning: Permanently added the RSA host key for IP address 'XXX.XXX.XXX.XXX' to the list of known hosts.
Finished Step #2
PUSH
DONE
-----------------------------------------------------------------------------------------------------------------

ID                                    CREATE_TIME                DURATION  SOURCE                                                                              IMAGES  STATUS
871b68bc-cefc-4411-856c-2a2b7c7d2487  XXXX-XX-XXT17:57:21+00:00  13S       gs://[PROJECT-ID]_cloudbuild/source/1504288639.02---.tgz  -                                 SUCCESS

Cleaning up

To avoid incurring charges to your Google Cloud Platform account for the resources used in this tutorial:

Delete the project

The easiest way to eliminate billing is to delete the project that you created for the tutorial.

To delete the project:

Caution: Deleting a project has the following effects:

• Everything in the project is deleted. If you used an existing project for this tutorial, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.

If you plan to explore multiple tutorials and quickstarts, reusing projects can help you avoid exceeding project quota limits.

1. In the Cloud Console, go to the Manage resources page.

   Go to the Manage resources page

2. In the project list, select the project you want to delete and click Delete .
3. In the dialog, type the project ID, and then click Shut down to delete the project.

What's next

• Learn how to create GitHub app triggers.
• Learn more about using encrypted resources in Cloud Build.
• Learn more about KMS.