Using VPC Service Controls (VPC SC)

VPC Service Controls is a Google Cloud feature that allows you to set up a secure perimeter to guard against data exfiltration. This guide shows how to use VPC Service Controls with Cloud Run to add additional security to your services.

Limitations and advisories

  • For Artifact Registry or Container Registry:

    • The registry where you store your container must be in the same VPC Service Controls service perimeter as the project you're deploying to.
    • The code being built must be in the same perimeter as the registry that the container is being pushed to.
  • The continuous deployment feature is not available for projects inside a VPC Service Controls perimeter.

Setting up your organization to support VPC Service Controls

To use VPC Service Controls with Cloud Run, you must configure a service perimeter at the organization level. This setup ensures that VPC Service Controls checks are enforced when using Cloud Run and that developers can only deploy services that comply with VPC Service Controls.

Setting up a VPC Service Controls perimeter

The following steps show how to set up a VPC Service Controls perimeter.

  1. Ensure that you have the required roles for administering VPC Service Controls.

  2. Follow the VPC Service Controls Quickstart. When prompted to specify the service that you want to restrict, specify the Cloud Run API.

  3. Enable access for developers.

    Turning on VPC Service Controls for Cloud Run prevents all access from outside the perimeter, including viewing and deploying services from machines not in the perimeter, such as company laptops. In order for developers and application operators to view and deploy resources within the perimeter, you must configure the perimeter ingress policy.

    If your organization uses Access Context Manager, you can also enable access for developers by setting access levels.

As a result of setting up your service perimeter, all calls to the Cloud Run Admin API will be checked to ensure that the calls originate from within the same perimeter.

Setting up organization policies

To manage organization policies, you need the Organization Policy Administrator (roles/orgpolicy.policyAdmin) role.

To comply with VPC Service Controls and protect against data exfiltration, set up the following organization policies that control the allowable ingress and egress settings for Cloud Run in the service perimeter.

Restricting allowed ingress settings

The run.allowedIngress organization policy controls the ingress settings that developers are allowed to use for Cloud Run. Set this organization policy to mandate that developers use the value internal if you want to allow only internal requests, or internal-and-cloud-load-balancing if you want to allow requests through Cloud Load Balancing, with the access controls managed by Cloud Load Balancing.

For more information on these settings, refer to Restricting ingress for Cloud Run

Console

  1. Go to the Allowed ingress settings (Cloud Run) policy page in the Cloud Console:

    Go to organization policy

  2. Click Edit.

  3. On the Edit page, select Customize.

  4. Under Policy enforcement, select Replace.

  5. Under Policy values, select Custom.

  6. Under Policy type, select Allow.

  7. Under Custom values, enter internal. If you'd like to also allow Cloud Load Balancing traffic, click New policy value and enter internal-and-cloud-load-balancing.

  8. Click Save.

gcloud

Use the gcloud resource-manager org-policies allow command:

gcloud resource-manager org-policies allow \
  run.allowedIngress internal \
  --organization ORGANIZATION_ID

where ORGANIZATION_ID is your organization ID. If you'd like to also allow Cloud Load Balancing traffic, replace run.allowedIngress internal with run.allowedIngress internal-and-cloud-load-balancing.

After this organization policy is in place, all services must use allowed values for their ingress settings. This means that Cloud Run services can only accept traffic originating from a VPC network within the service perimeter. Attempts to set ingress to a different value will be blocked by the organization policy.

Restricting allowed VPC connector egress settings

The run.allowedVPCEgress organization policy controls the egress settings that developers are allowed to use for Cloud Run. Set this organization policy to allow only the value all-traffic:

Console

  1. Go to the Allowed VPC egress settings (Cloud Run) policy page in the Cloud Console:

    Go to organization policy

  2. Click Edit.

  3. On the Edit page, select Customize.

  4. Under Policy enforcement, select Replace.

  5. Under Policy values, select Custom.

  6. Under Policy type, select Allow.

  7. Under Custom values, enter all-traffic.

  8. Click Save.

gcloud

Use the gcloud resource-manager org-policies allow command:

gcloud resource-manager org-policies allow \
  run.allowedVPCEgress all-traffic \
  --organization ORGANIZATION_ID

where ORGANIZATION_ID is your organization ID.

Behavior after organization policy is set

After the organization policy is in place, all new revisions must use a VPC connector and must use the value all-traffic for their egress settings. This combination of settings routes all egress traffic from these revisions through a VPC network, subjecting this traffic to the VPC network's firewall rules and other settings. New revision deployments specifying a different value will be blocked by the organization policy.

Org policy and traffic migrations

For services existing before the organization policy was set, you can continue to migrate traffic to non-compliant revisions until all revisions serving traffic have become compliant.

This behavior allows you to test and gradually roll out new revisions that comply with the policy. After all revisions serving traffic are compliant with the policy, any attempt to move traffic to non-compliant revisions will be blocked by the org policy.

Setting up your project to support VPC Service Controls

For individual projects within the service perimeter, you must perform additional configuration to use VPC Service Controls.

Configuring VPC networks

This section shows how to configure your VPC network so that requests sent to the regular googleapis.com virtual IP are automatically routed to the restricted restricted virtual IP (VIP) range, 199.36.153.4/30 (restricted.googleapis.com) where your Cloud Run service is serving. Note that no code changes are needed.

For each VPC network in a project, follow these steps to block outbound traffic except for traffic to the restricted VIP range:

  1. Configure firewall rules to prevent data from leaving the VPC network:

    • Create a deny egress rule that blocks all outbound traffic.

    • Create an allow egress rule that permits traffic to 199.36.153.4/30 on TCP port 443. Ensure that it has a priority before the deny egress rule you just created—this allows egress only to the restricted VIP range.

  2. Configure DNS to resolve *.googleapis.com to restricted.googleapis.com.

  3. Configure DNS with an A record mapping *.run.app to the 199.36.153.4/30 IP range. You can do this with Cloud DNS:

    gcloud dns managed-zones create ZONE_NAME \
    --visibility=private \
    --networks=https://www.googleapis.com/compute/v1/projects/PROJECT_NAME/global/networks/default \
    --description=none \
    --dns-name=run.app
    
    gcloud dns record-sets transaction start --zone=ZONE_NAME
    
    gcloud dns record-sets transaction add --name=*.run.app. \
    --type=A 199.36.153.4 199.36.153.5 199.36.153.6 199.36.153.7 \
    --zone=ZONE_NAME \
    --ttl=300
    
    gcloud dns record-sets transaction execute --zone=ZONE_NAME
    

At this point, requests originating from within the VPC network:

  • are not able to leave the VPC network, preventing egress outside the service perimeter.
  • can only reach Google APIs and services that check VPC Service Controls, preventing exfiltration through Google APIs.

Deploying VPC Service Controls-compliant Cloud Run services

After VPC Service Controls is configured for Cloud Run, make sure all services deployed within the service perimeter enable VPC Service Controls. This means that:

  • All services must use a Serverless VPC Access connector. See Connecting to a VPC network for more information.
  • All services must allow only traffic from internal sources. See Ingress settings for more information.
  • All services must route all outgoing traffic through the VPC network. See Egress settings for more information.

Auditing existing services for VPC Service Controls compliance

After setting up VPC Service Controls, new services created in projects within the service perimeter are automatically checked for compliance. However, to avoid disruption of existing workloads, existing services continue to work and might not comply with organization policies.

Google recommends that you periodically audit your services to make sure your ingress and egress settings conform with the requirements, and update or redeploy services as necessary. For example, you could create a script that uses the Cloud Run Admin API to list your services and highlight those that do not specify the proper network settings.