You can connect Cloud Build to a Secure Source Manager Private Service Connect instance using Cloud Build private pools.
Before you begin
- Create a Private Service Connect Secure Source Manager instance.
- If you previously configured the endpoints for your Private Service Connect instance, then you must disconnect them. To release your endpoints, follow the instructions in Release a static internal IPv4 or IPv6 address.
-
Enable the Cloud Build, Service Networking, and Compute Engine APIs.
Required roles
To get the permissions that you need to connect Cloud Build to a Private Service Connect Secure Source Manager instance,
ask your administrator to grant you the
Access Context Manager Admin role (roles/accesscontextmanager.policyAdmin
) IAM role on the organization.
Configure your service account
To follow the principle of least privilege in Cloud Build, you can configure Cloud Build to use a service account with just enough privileges to execute a build.
If you don't specify a service account, Cloud Build might automatically select a service account to execute builds on your behalf. This service account might have permissions that are unnecessarily broad for your use-case, like access to your Secure Source Manager instance and any repositories in your project.
Create a service account if you don't already have one you want to use.
Open the Cloud Build Settings page:
In the Service account permissions tab, select the service account you want to use as the Cloud Build service account.
Optionally, select Set as preferred service account to pre-populate your service account when you create new triggers.
Grant the service account required permissions
Grant the service account the required Cloud Build roles for your use-case. For information on what Cloud Build roles are required for different Cloud Build actions, see Configuring access to Cloud Build.
To grant the service account permissions to retrieve the CA certificate, run the following command.
gcloud projects add-iam-policy-binding CA_PROJECT_ID \ --member=serviceAccount:SA_EMAIL \ --role=roles/privateca.auditor \ --condition=None
Replace the following:
CA_PROJECT_ID
: with the project ID of the CA pool project.SA_EMAIL
: with the email of the service account you set as your Cloud Build service account.
To grant the service account permissions to access the Secure Source Manager instance, run the following command.
gcloud projects add-iam-policy-binding INSTANCE_PROJECT_ID \ --member=serviceAccount:SA_EMAIL \ --role=roles/securesourcemanager.instanceAccessor \ --condition=None
To grant the service account permissions to read from Secure Source Manager repositories, run the following command.
gcloud projects add-iam-policy-binding INSTANCE_PROJECT_ID \ --member=serviceAccount:SA_EMAIL \ --role=roles/securesourcemanager.repoReader \ --condition=None
Set up build logs
When you specify your own service account for builds, you must store your build logs either in Cloud Logging or in a user-created Cloud Storage bucket. You can't store your logs in the default logs bucket.
To store build logs in Cloud Logging, grant the
Logs Writer (roles/logging.logWriter
) role to your
Cloud Build service account.
For more information on how to set up your build logs, see Configure user-specified service accounts.
Create a private Cloud DNS zone
To create a managed private Cloud DNS zone, run the following command.
gcloud dns managed-zones create ZONE_NAME \
--project=PROJECT_ID \
--description=DESCRIPTION \
--dns-name="INSTANCE_LOCATION.p.sourcemanager.dev." \
--visibility="private" \
--networks="https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/networks/NETWORK"
Replace the following:
ZONE_NAME
: with the name you want to give the managed-zone.PROJECT_ID
: with the project ID of the Google Cloud project you want to use for the Cloud DNS zone.DESCRIPTION
: with a description for the zone.INSTANCE_LOCATION
: with the location where you want to create your Secure Source Manager instance.
For more information on managed private Cloud DNS zones, see Cloud DNS overview.
Configure your Virtual Private Cloud (VPC) network
Reserve an IP range that you want to use to peer the Secure Source Manager VPC with the Cloud Build private pool.
gcloud compute addresses create CB_PEER_RANGE \ --global \ --purpose=VPC_PEERING \ --prefix-length=24 \ --description="IP range for peering with Cloud Build private pool" \ --network=NETWORK \ --project=INSTANCE_PROJECT_ID
Replace the following:
CB_PEER_RANGE
: with the name of the address to create.NETWORK
: with the network resource in which the addresses should be reserved. If you're using the default network, the value isdefault
.INSTANCE_PROJECT_ID
: with the Private Service Connect instance project ID.
To create a VPC peering with
servicenetworking.googleapis.com
, run the following command.gcloud services vpc-peerings connect \ --service=servicenetworking.googleapis.com \ --ranges=CB_PEER_RANGE \ --network=NETWORK \ --project=INSTANCE_PROJECT_ID
To create a peered DNS domain to connect with Cloud Build, run the following command.
gcloud services peered-dns-domains create DNS_PEERING_DOMAIN \ --network=NETWORK \ --project=INSTANCE_PROJECT_ID \ --dns-suffix=INSTANCE_LOCATION.p.sourcemanager.dev.
Replace the following:
DNS_PEERING_DOMAIN
: with the name of the peered DNS domain you are creating.
Private DNS zones must be explicitly shared. For more information on sharing private DNS zones, see Share private DNS zones with service producers.
To export network routes to Cloud Build, run the following command.
gcloud compute networks peerings update servicenetworking-googleapis-com \ --network=NETWORK \ --export-custom-routes \ --no-export-subnet-routes-with-public-ip \ --project=INSTANCE_PROJECT_ID
To create the Cloud Build private pool in the same Google Cloud project as the Private Service Connect instance, run the following command.
gcloud builds worker-pools create PRIVATE_POOL_ID \ --project=INSTANCE_PROJECT_ID \ --region=INSTANCE_LOCATION \ --peered-network=projects/INSTANCE_PROJECT_ID/global/networks/NETWORK \ --worker-disk-size=100 \ --no-public-egress
Where
PRIVATE_POOL_ID
is the name for the private pool you are creating. The name must be between 1-63 characters, and valid characters are[a-zA-Z0-9_-]+
.To Create a Private Service Connect Network Endpoint Group (NEG) for the HTTP endpoint, run the following command:
gcloud beta compute network-endpoint-groups create HTTP_NEG_NAME \ --project=INSTANCE_PROJECT_ID \ --region=INSTANCE_LOCATION \ --network-endpoint-type=PRIVATE_SERVICE_CONNECT \ --psc-target-service=HTTP_PSC \ --network=NETWORK --subnet=SUBNET
Where:
HTTP_NEG_NAME
is the name of the NEG you want to create for the HTTP endpoint.HTTP_PSC
is the HTTP Private Service Connect endpoint value.
To create a Private Service Connect NEG for the SSH endpoint, run the following command:
gcloud beta compute network-endpoint-groups create SSH_NEG_NAME \ --project=INSTANCE_PROJECT_ID \ --region=INSTANCE_LOCATION \ --network-endpoint-type=PRIVATE_SERVICE_CONNECT \ --psc-target-service=SSH_PSC \ --network=NETWORK --subnet=SUBNET
Where:
SSH_NEG_NAME
is the name of the NEG you want to create for the SSH endpoint.SSH_PSC
is the SSH Private Service Connect endpoint value.
Set up a proxy internal load balancer (ILB)
Endpoints are not accessible from peered VPC networks. To access the Secure Source Manager endpoints, you must set up an L4 internal load balancer (ILB) for each endpoint. For more information on accessing published services through endpoints, see About accessing published services through endpoints.
To create a proxy-only subnet for ILBs, run the following command.
gcloud compute networks subnets create ILB_PROXY_SUBNET_NAME \ --purpose=REGIONAL_MANAGED_PROXY \ --role=ACTIVE \ --region=INSTANCE_LOCATION \ --network=NETWORK \ --range=CIDR_RANGE \ --project=INSTANCE_PROJECT_ID
Where:
ILB_PROXY_SUBNET_NAME
is the name of the subnet to create.CIDR_RANGE
is the primary IP address range of the subnet. You must use a subnet mask no longer than26
so that at least 64 IP addresses are available for proxies in the region. The recommended subnet mask length is/23
. For more information on proxy-only subnets, see Proxy-only subnets for Envoy-based load balancers.
To create L4 ILB for the HTTP connection NEG, run the following command.
gcloud compute backend-services create HTTP_PROXY_ILB \ --project=INSTANCE_PROJECT_ID \ --region=INSTANCE_LOCATION \ --load-balancing-scheme=INTERNAL_MANAGED
Where
HTTP_PROXY_ILB
is the name of the ILB to create for the HTTP endpoint.To create an L4 ILB for the SSH connection NEG, run the following command.
gcloud compute backend-services create SSH_PROXY_ILB \ --project=INSTANCE_PROJECT_ID \ --region=INSTANCE_LOCATION \ --load-balancing-scheme=INTERNAL_MANAGED
Where
SSH_PROXY_ILB
is the name of the ILB to create for the SSH endpoint.
Register the Private Service Connect NEG as the ILBs' backend
To register the HTTP NEG, run the following command.
gcloud compute backend-services add-backend HTTP_PROXY_ILB \ --network-endpoint-group=HTTP_NEG_NAME \ --network-endpoint-group-region=INSTANCE_LOCATION \ --project=INSTANCE_PROJECT_ID
To register the SSH NEG, run the following command.
gcloud compute backend-services add-backend SSH_PROXY_ILB \ --network-endpoint-group=SSH_NEG_NAME \ --network-endpoint-group-region=INSTANCE_LOCATION \ --project=INSTANCE_PROJECT_ID
Create target TCP proxies for the ILBs
To create a target TPC proxy for the HTTP ILB, run the following command.
gcloud compute target-tcp-proxies create ILB_HTTP_TPC_TARGET_PROXY \ --backend-service=HTTP_PROXY_ILB \ --region=INSTANCE_LOCATION \ --project=INSTANCE_PROJECT_ID
Where
ILB_HTTP_TPC_TARGET_PROXY
is the name of the target TPC proxy you want to create for the HTTP ILB.To create a target TPC proxy for the SSH ILB, run the following command.
gcloud compute target-tcp-proxies create ILB_SSH_TPC_TARGET_PROXY \ --backend-service=SSH_PROXY_ILB \ --region=INSTANCE_LOCATION \ --project=INSTANCE_PROJECT_ID
Where
ILB_SSH_TPC_TARGET_PROXY
is the name of the target TPC proxy you are creating for the SSH ILB.
Create forwarding rules to route traffic to the ILBs
To create a forwarding rule for routing traffic to the HTTP ILB, run the following command.
gcloud compute forwarding-rules create HTTP_PROXY_FORWARD \ --project=INSTANCE_PROJECT_ID \ --ports 443 \ --target-tcp-proxy=ILB_HTTP_TPC_TARGET_PROXY \ --target-tcp-proxy-region=INSTANCE_LOCATION \ --network-tier PREMIUM \ --load-balancing-scheme=INTERNAL_MANAGED \ --network=NETWORK \ --subnet=SUBNET \ --subnet-region=INSTANCE_LOCATION
Where
HTTP_PROXY_FORWARD
is the name of the forwarding rule you are creating.To create a forwarding rule for routing traffic to the SSH ILB, run the following command.
gcloud compute forwarding-rules create SSH_PROXY_FORWARD \ --project=INSTANCE_PROJECT_ID \ --ports 22 \ --target-tcp-proxy=ILB_SSH_TPC_TARGET_PROXY \ --target-tcp-proxy-region=INSTANCE_LOCATION \ --network-tier PREMIUM \ --load-balancing-scheme=INTERNAL_MANAGED \ --network=NETWORK \ --subnet=SUBNET \ --subnet-region=INSTANCE_LOCATION
Where
SSH_PROXY_FORWARD
is the name of the forwarding rule you are creating.
Create your private DNS records
Once the forwarding rules are set up, you need to register DNS records in your private Cloud DNS zone for each of the instance's hostnames.
To get the IP address for your HTTP forwarding rule, run the following command.
gcloud compute forwarding-rules describe HTTP_PROXY_FORWARD \ --project=INSTANCE_PROJECT_ID \ --region=INSTANCE_LOCATION
Where
HTTP_PROXY_FORWARD
is the name you gave the forwarding rule for HTTP.To register the API hostname, run the following command.
gcloud dns record-sets create INSTANCE_ID-INSTANCE_PROJECT_NUMBER-api.INSTANCE_LOCATION.p.sourcemanager.dev. \ --project=INSTANCE_PROJECT_ID \ --zone=ZONE_NAME \ --type=A \ --ttl=300 \ --rrdatas=HTTP_LB_IP
Replace the following:
INSTANCE_ID
: with your instance ID.INSTANCE_PROJECT_NUMBER
: with you instance's Google Cloud project number.INSTANCE_LOCATION
: with the region of your instance.INSTANCE_PROJECT_ID
: with your instance's Google Cloud project ID.ZONE_NAME
: with the name you gave the managed-zone.HTTP_LB_IP
: is the output ofgcloud compute forwarding-rules describe HTTP_PROXY_FORWARD
.
To register the Git HTTP hostname, run the following command.
gcloud dns record-sets create INSTANCE_ID-INSTANCE_PROJECT_NUMBER-git.INSTANCE_LOCATION.p.sourcemanager.dev. \ --project=INSTANCE_PROJECT_ID \ --zone=ZONE_NAME \ --type=A \ --ttl=300 \ --rrdatas=HTTP_LB_IP
To register the HTML hostname for the web interface, run the following command.
gcloud dns record-sets create INSTANCE_ID-INSTANCE_PROJECT_NUMBER.INSTANCE_LOCATION.p.sourcemanager.dev.\ --project=INSTANCE_PROJECT_ID \ --zone=ZONE_NAME \ --type=A \ --ttl=300 \ --rrdatas=HTTP_LB_IP
To get the IP address for your SSH forwarding rule, run the following command.
gcloud compute forwarding-rules describe SSH_PROXY_FORWARD \ --project=INSTANCE_PROJECT_ID \ --region=INSTANCE_LOCATION
Where
SSH_PROXY_FORWARD
is the name you gave the forwarding rule for SSH.To register the Git SSH hostname, run the following command.
gcloud dns record-sets create INSTANCE_ID-INSTANCE_PROJECT_NUMBER-ssh.INSTANCE_LOCATION.p.sourcemanager.dev. \ --project=INSTANCE_PROJECT_ID \ --zone=ZONE_NAME \ --type=A \ --ttl=300 \ --rrdatas=SSH_LB_IP
Where
SSH_LB_IP
is the output ofgcloud compute forwarding-rules describe SSH_PROXY_FORWARD
.
Create a repository in your instance
You can use a bastion host to access and create a repository in your Private Service Connect instance.
Create a bastion host VM with the
cloud-platform
scope by following the instructions in Create a bastion host VM.Create a repository by running the following command.
curl \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ https://INSTANCE_ID-PROJECT_NUMBER-api.INSTANCE_LOCATION.p.sourcemanager.dev/v1/projects/PROJECT_NUMBER/locations/INSTANCE_LOCATION/repositories?repository_id=REPOSITORY_ID \ -d '{}'
Replace the following:
INSTANCE_ID
: with the name of your Secure Source Manager instance.PROJECT_NUMBER
: with your instance's Google Cloud project number.INSTANCE_LOCATION
: with the location of your instance.REPOSITORY_ID
: with the name you want to give your repository.
A repository is created in your Private Service Connect instance.
Test access to Secure Source Manager from Cloud Build
To confirm everything is working correctly, you can use the following build config file to test connectivity and pull source code from Secure Source Manager. The build config file assumes you are using Cloud Logging to store your build logs. If you are using a user-created Cloud Storage bucket, see Execute a build using a config file for instructions on how to edit the build config file.
Before you use the following YAML file, replace the following:
CA_POOL_NAME
: with the CA pool name you used when you created your Private Service Connect instance.CA_PROJECT_ID
: with the Google Cloud project you used to create the CA pool.INSTANCE_LOCATION
: with the location of your instance.INSTANCE_ID
: with the name of your instance.INSTANCE_PROJECT_NUMBER
: with your instance's Google Cloud project number.REPOSITORY_ID
: with the name you gave your repository.SA_PROJECT_ID
: with the project ID of the service account you are using with Cloud Build.SERVICE_ACCOUNT
: with the email address of the service account you are using with Cloud Build.
steps:
- name: gcr.io/cloud-builders/gcloud
args:
- privateca
- pools
- get-ca-certs
- CA_POOL_NAME
- '--project'
- CA_PROJECT_ID
- '--location'
- INSTANCE_LOCATION
- '--output-file=cacert.pem'
- name: gcr.io/cloud-builders/git
args:
- config
- '--global'
- 'credential.https://INSTANCE_ID-INSTANCE_PROJECT_NUMBER-git.INSTANCE_LOCATION.p.sourcemanager.dev.helper'
- gcloud.sh
- name: gcr.io/cloud-builders/git
args:
- config
- '--global'
- http.sslCAInfo
- cacert.pem
- name: gcr.io/cloud-builders/git
env:
- GIT_TRACE=1
- GIT_CURL_VERBOSE=1
args:
- clone
- >-
https://INSTANCE_ID-INSTANCE_PROJECT_NUMBER-git.INSTANCE_LOCATION.p.sourcemanager.dev/INSTANCE_PROJECT_ID/REPOSITORY_ID
- https-clone
serviceAccount: 'projects/SA_PROJECT_ID/serviceAccounts/SERVICE_ACCOUNT'
options:
logging: CLOUD_LOGGING_ONLY
What's next
- To automate builds from Secure Source Manager, follow the instructions in Create a triggers file.