Direct VPC egress with a Shared VPC network

You can enable your Cloud Run service or job to send traffic to a Shared VPC network by using Direct VPC egress with no Serverless VPC Access connector required.

This page describes how to configure Cloud Run's IAM permissions to use the subnet of the Shared VPC network, and then place your service or job on the shared subnet.

Before you begin

When using Direct VPC egress, make sure your subnet has at least a few hundred IP addresses available. If the total number of instances across all your Cloud Run services grows beyond 100, we recommend having at least 4 times (4X) the total number available.

Limitations

The following limitations apply to both Cloud Run services and jobs:

• Cloud Run supports throughput of up to 1 Gbps per individual instance. Exceeding this amount results in performance throttling.

• A Cloud Run usage quota limits the maximum number of instances you can configure to use Direct VPC egress. The maximum number is configured per Cloud Run revision or job execution. To increase the default limits, see how to increase quotas. You can check your quota using the Google Cloud console.

• Cloud Run services and jobs might experience connection breaks during networking infrastructure maintenance events. We recommend that you use client libraries that can handle occasional connection resets.

The following limitations apply to only Cloud Run jobs and not services:

• Direct VPC egress for Cloud Run jobs is available in Preview only.
• To ensure proper job executions, only use Direct VPC egress for jobs that don't require more than 8 concurrent instances and be sure to reserve a minimum of 1,024 IP addresses.

The following items are not supported by Direct VPC egress:

• VPC Flow Logs doesn't provide the name of the Cloud Run service or revision.
• VPC Flow Logs are not reported from non-VM resources like Cloud Run or on-premises machines.
• Firewall Rules Logging
• Packet Mirroring
• Network Intelligence Center
• IPv6 traffic
• Private NAT
• Using network tags in ingress firewall rules applied to the destination resource.
• Using service identity as the source service account in ingress firewall rules applied to the destination resource.
• Firewall rules cannot use Resource Manager tags attached to Cloud Run workloads.
• Cloud Run jobs that run for more than 1 hour might experience connection breaks. These can occur during maintenance events that migrate the job from one machine to another. The container receives a SIGTSTP signal 10 seconds before the event and a SIGCONT signal after the event. After the container receives the SIGCONT signal, retry the connection.

Set up IAM permissions

Before Cloud Run in a Shared VPC service project can access a Shared VPC network, you must first make sure the Cloud Run service agent has sufficient permissions to use the subnet.

1. To access the Shared VPC network, grant sufficient permissions to the Cloud Run service agent by adding one of the following roles:

   • Compute Network User (compute.networkUser) on the Shared VPC host project.

     For example, run the following command:

     gcloud projects add-iam-policy-binding HOST_PROJECT_ID \
     --member "serviceAccount:[e-mail]" \
     --role "roles/compute.networkUser"
     

     Replace the following:

     • HOST_PROJECT_ID: the ID of your Shared VPC host project.
     • SERVICE_PROJECT_NUMBER: the Shared VPC service project number where you deploy your Cloud Run service or job.

   • Compute Network Viewer (compute.networkViewer) on the Shared VPC host project and the Compute Network User (compute.networkUser) role on the Shared VPC subnet.

     For example, to grant the Compute Network Viewer role on the subnet, run the following command:

     gcloud compute networks subnets add-iam-policy-binding SUBNET_NAME \
       --region REGION \
       --member "serviceAccount:[e-mail]" \
       --role "roles/compute.networkUser" \
       --project HOST_PROJECT_ID
     

     Replace the following:

     • SUBNET_NAME: the fully qualified resource name of the subnet that you want to run your Cloud Run services.
     • REGION: the region for your Cloud Run service, which must match the region of your subnet.
     • SERVICE_PROJECT_NUMBER: the Shared VPC service project number where you deploy your Cloud Run service or job.
     • HOST_PROJECT_ID: the ID of your Shared VPC host project.

2. The Cloud Run service agent requires the Cloud Run Service Agent role on your Cloud Run project. You can verify that the role was not manually removed by running the following command:

   gcloud projects get-iam-policy SERVICE_PROJECT_ID \
     --flatten bindings \
     --filter "bindings.role:roles/run.serviceAgent"
   

   Replace SERVICE_PROJECT_ID with the project ID of your Cloud Run service or job.

For more granular control, you can ensure that the Cloud Run service agent has the following permissions:

• compute.networks.get on the Shared VPC host project
• compute.subnetworks.get on the host project or the specific subnet
• compute.subnetworks.use on the host project or the specific subnet
• compute.addresses.get on the Shared VPC service project
• compute.addresses.list on the Shared VPC service project
• compute.addresses.createInternal on the Shared VPC service project
• compute.addresses.deleteInternal on the Shared VPC service project

IP address allocation

To place your Cloud Run service or job on a VPC network, you must specify a network and subnet. Cloud Run allocates IP addresses from your subnet.

IP addresses are ephemeral, so don't create policies based on individual IPs. If you need to create a policy based on IPs, such as in firewall rules, you must use the IP address range of the entire subnet.

To change the network or subnet that your service or job uses, deploy a new service revision or execute a new job task that uses the new network and subnet values.

Scale up

To enable fast scale up in the event of a traffic surge, Cloud Run allocates IP addresses before they're needed.

At any given time, there are likely more allocated IP addresses than the number of instances that exist. To ensure that Cloud Run can obtain enough IP addresses, make sure your subnet has at least a few hundred IP addresses available. If the subnet's total number of instances across all Cloud Run services and jobs grows beyond 100, we recommend having at least 4 times (4X) the total number available. If Cloud Run cannot allocate more IP addresses, it cannot start any more service instances or job tasks until more IP addresses become available. If your IP address space is limited, see Supported IP ranges for more options. For IP allocation efficiency and ease of management, place multiple services or jobs on the same subnet.

Scale down

Even after all services or jobs scale down to zero, Cloud Run reserves some IP addresses from the subnet for up to 20 minutes in case services or jobs must quickly scale up again. Each instance requires one IP address, but Cloud Run reserves a minimum /28 subnet mask at the beginning. Once you exhaust all 16 instances, Cloud Run creates a new subnet.

To delete the subnet, you must first delete or redeploy your Cloud Run services or jobs to stop using the subnet, and then wait 1-2 hours.

Supported IP ranges

Cloud Run supports the following IPv4 ranges for your subnet:

• RFC 1918 (recommended)
  • 10.0.0.0/8
  • 172.16.0.0/12
  • 192.168.0.0/16
• RFC 6598
  • 100.64.0.0/10
• Class E (not recommended with on-premises setups)
  • 240.0.0.0/4

Note: Test non-RFC 1918 setups to identify possible compatibility issues with non-Google Cloud resources or hardware. Learn more about creating subnets with non-RFC 1918 IP ranges.

Deploy a service

Direct VPC egress allows your Cloud Run service to send traffic to a Shared VPC network without a Serverless VPC Access connector. Network costs scale to zero just like the service itself. You can also add network tags directly on Cloud Run service revisions for more granular network security, such as applying VPC firewall rules.

You can configure Direct VPC egress with a service by using the Google Cloud console, Google Cloud CLI, or YAML.

Console

1. In the Google Cloud console, go to the Cloud Run page:

   Go to Cloud Run

2. Click Deploy container and select Service if you are configure a new service. If you are configuring and deploying an existing service, click the service, then click Edit and deploy new revision.

3. If you are configuring a new service, fill out the initial service settings page as needed, then click Container(s), volumes, networking, security to expand the service configuration page.

4. Click the Networking tab.

5. Click Connect to a VPC for outbound traffic.

6. Click Send traffic directly to a VPC.

7. Select Networks shared with me.

8. In the Network field, select the Shared VPC network that you want to send traffic to.

9. In the Subnet field, select the subnet where your service receives IP addresses from.

10. Optional: Enter the names of the network tags that you want to associate with your service or services. Network tags are specified at the revision-level. Each service revision can have different network tags, such as network-tag-2.

11. For Traffic routing, select one of the following:

    • Route only requests to private IPs to the VPC to send only traffic to internal addresses through the Shared VPC network.
    • Route all traffic to the VPC to send all outbound traffic through the Shared VPC network.

12. Click Create or Deploy.

13. To verify that your service is on your Shared VPC network, click the service, then click the Networking tab. The network and subnet are listed in the VPC card.

You can now send requests from your Cloud Run service to any resource on the Shared VPC network, as allowed by your firewall rules.

gcloud

To place your service on the shared subnet, specify the fully qualified resource names for the Shared VPC network and subnet by running the following command:

gcloud run deploy SERVICE_NAME \
  --image IMAGE_URL \
  --network projects/HOST_PROJECT_ID/global/networks/VPC_NETWORK \
  --subnet projects/HOST_PROJECT_ID/regions/REGION/subnetworks/SUBNET_NAME \
  --network-tags NETWORK_TAG_NAMES \
  --vpc-egress=EGRESS_SETTING \
  --region REGION \
  --max-instances MAX
  

Replace the following:

• SERVICE_NAME: the name of your Cloud Run service.
• IMAGE_URL: the image URL of the service.
• HOST_PROJECT_ID: the ID of your Shared VPC project.
• VPC_NETWORK: the fully qualified resource name of your Shared VPC network.
• REGION: the region for your Cloud Run service, which must match the region of your subnet.
• SUBNET_NAME: the fully qualified resource name of your subnet.
• Optional: NETWORK_TAG_NAMES with the comma-separated names of the network tags you want to associate with a service. For services, network tags are specified at the revision-level. Each service revision can have different network tags, such as network-tag-2.
• EGRESS_SETTING with an egress setting value:
  • all-traffic: Sends all outbound traffic through the Shared VPC network.
  • private-ranges-only: Sends only traffic to internal addresses through the Shared VPC network.
• MAX: the maximum number of instances to use for the Shared VPC network. The maximum number of instances allowed for services is 100.

For more details and optional arguments, see the gcloud reference.

YAML

1. If you are creating a new service, skip this step. If you are updating an existing service, download its YAML configuration:

   gcloud run services describe SERVICE --format export > service.yaml

2. Update the following attributes:

   apiVersion: serving.knative.dev/v1
   kind: Service
   metadata:
     name: SERVICE_NAME
     labels:
       cloud.googleapis.com/location: REGION
   spec:
     template:
       metadata:
         annotations:
           run.googleapis.com/network-interfaces: '[{"network":"NETWORK","subnetwork":"SUBNET","tags":"NETWORK_TAG_NAMES"}]'
           run.googleapis.com/vpc-access-egress: EGRESS_SETTING
       spec:
         containers:
         - image: IMAGE

   Replace:

   • SERVICE_NAME with the name of your Cloud Run service. Service names must be 49 characters or less and must be unique per region and project.
   • REGION with the region for your Cloud Run service, which must match the region of your subnet.
   • NETWORK with the fully qualified resource name of your Shared VPC network.
   • SUBNET_NAME with the fully qualified resource name of your subnet.
   • Optional: NETWORK_TAG_NAMES with the names of the network tags you want to associate with a service. For services, network tags are specified at the revision-level. Each service revision can have different network tags, such as network-tag-2.
   • EGRESS_SETTING with an egress setting value:
     • all-traffic: Sends all outbound traffic through the Shared VPC network.
     • private-ranges-only: Sends only traffic to internal addresses through the Shared VPC network.
   • IMAGE with the URL of your service container image.

3. Create or update the service using the following command:

   gcloud run services replace service.yaml

Terraform

To learn how to apply or remove a Terraform configuration, see Basic Terraform commands.

1. Add the following to your main.tf file:

   /**
    * Copyright 2024 Google LLC
    *
    * Licensed under the Apache License, Version 2.0 (the "License");
    * you may not use this file except in compliance with the License.
    * You may obtain a copy of the License at
    *
    *      http://www.apache.org/licenses/LICENSE-2.0
    *
    * Unless required by applicable law or agreed to in writing, software
    * distributed under the License is distributed on an "AS IS" BASIS,
    * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    * See the License for the specific language governing permissions and
    * limitations under the License.
    */
   
   # Example configuration of a Cloud Run service with direct VPC
   
   resource "google_cloud_run_v2_service" "default" {
     name     = "cloudrun-service"
     location = "us-central1"
   
     deletion_protection = false # set to "true" in production
   
     template {
       containers {
         image = "us-docker.pkg.dev/cloudrun/container/hello"
       }
       vpc_access {
         network_interfaces {
           network    = "default"
           subnetwork = "default"
           tags       = ["tag1", "tag2", "tag3"]
         }
       }
     }
   }
   

Note: Administrators can restrict the egress settings that developers can select by setting the run.allowedVPCEgress organization policy.

Optionally, make your service public if you want to allow unauthenticated access to the service.

Create a job

Preview — Direct VPC egress for Jobs

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

Direct VPC egress allows your Cloud Run job to send traffic to a Shared VPC network without a Serverless VPC Access connector. You can also add network tags directly on Cloud Run jobs for more granular network security, such as applying VPC firewall rules.

You can configure Direct VPC egress with a job by using the Google Cloud console, Google Cloud CLI, or YAML.

Console

1. In the Google Cloud console, go to the Cloud Run page:

   Go to Cloud Run

2. If you are configuring a new job, click Deploy container and select Job to fill out the initial Create job form as needed. If you are configuring an existing job, click the Jobs tab, select a job, then click Edit.

3. Click Container, Variables & Secrets, Connections, Security to expand the job properties page.

4. Click the Connections tab.

5. Click Connect to a VPC for outbound traffic.

6. Click Send traffic directly to a VPC.

7. Select Networks shared with me.

8. In the Network field, select the Shared VPC network that you want to send traffic to.

9. In the Subnet field, select the subnet where your job receives IP addresses from.

10. Optional: Enter the names of the network tags that you want to associate with a job. For jobs, network tags are specified at the execution-level. Each job execution can have different network tags, such as network-tag-2.

11. For Traffic routing, select one of the following:

    • Route only requests to private IPs to the VPC to send only traffic to internal addresses through the Shared VPC network.
    • Route all traffic to the VPC to send all outbound traffic through the Shared VPC network.

12. Click Create or Update.

13. To verify that your job is on your Shared VPC network, click the job, then click the Configuration tab. The network and subnet are listed in the VPC card.

You can now execute your Cloud Run job and send requests from the job to any resource on the Shared VPC network, as allowed by your firewall rules.

gcloud

To place your job on the shared subnet, specify the fully qualified resource names for the Shared VPC network and subnet by running the following command:

gcloud run jobs create JOB_NAME \
  --image IMAGE_URL \
  --network projects/HOST_PROJECT_ID/global/networks/VPC_NETWORK \
  --subnet projects/HOST_PROJECT_ID/regions/REGION/subnetworks/SUBNET_NAME \
  --network-tags NETWORK_TAG_NAMES \
  --vpc-egress=EGRESS_SETTING \
  --region REGION \
  

Replace the following:

• JOB_NAME: the name of your Cloud Run job.
• IMAGE_URL: the image URL of the job.
• HOST_PROJECT_ID: the ID of your Shared VPC host project.
• VPC_NETWORK: the fully qualified resource name of your Shared VPC network.
• REGION: the region for your Cloud Run job, which must match the region of the subnet.
• SUBNET_NAME: the fully qualified resource name of your name of your subnet.
• Optional: NETWORK_TAG_NAMES with the comma-separated names of the network tags you want to associate with a job. Each job execution can have different network tags, such as network-tag-2.
• EGRESS_SETTING with an egress setting value:
  • all-traffic: Sends all outbound traffic through the Shared VPC network.
  • private-ranges-only: Sends only traffic to internal addresses through the Shared VPC network.

For more details and optional arguments, see the gcloud reference.

YAML

1. If you are creating a new job, skip this step. If you are updating an existing job, download its YAML configuration:

   gcloud run jobs describe JOB_NAME --format export > job.yaml

2. Update the following attributes:

   apiVersion: run.googleapis.com/v1
   kind: Job
   metadata:
     name: JOB_NAME
     annotations:
       run.googleapis.com/launch-stage: BETA
     labels:
       cloud.googleapis.com/location: REGION
   spec:
     template:
       metadata:
         annotations:
           run.googleapis.com/network-interfaces: '[{"network":"NETWORK","subnetwork":"SUBNET","tags":"NETWORK_TAG_NAMES"}]'
           run.googleapis.com/vpc-access-egress: EGRESS_SETTING
       spec:
         containers:
         - image: IMAGE

   Replace:

   • JOB_NAME with the name of your Cloud Run job. Job names must be 49 characters or less and must be unique per region and project.
   • REGION with the region for your Cloud Run job, which must match the region of your subnet.
   • NETWORK with the fully qualified resource name of your Shared VPC network.
   • SUBNET with the fully qualified resource name of your subnet.
   • Optional: NETWORK_TAG_NAMES with the names of the network tags you want to associate with a job. For jobs, network tags are specified at the execution-level. Each job execution can have different network tags, such as network-tag-2.
   • EGRESS_SETTING with an egress setting value:
     • all-traffic: Sends all outbound traffic through the Shared VPC network.
     • private-ranges-only: Sends only traffic to internal addresses through the Shared VPC network.
   • IMAGE with the URL of your job container image.

3. Create or update the job using the following command:

   gcloud run jobs replace job.yaml

Note: Administrators can restrict the egress settings that developers can select by setting the run.allowedVPCEgress organization policy.

Disconnect a service

Console

• To remove your service from the Shared VPC network:

  1. In the Google Cloud console, go to the Cloud Run page:

     Go to Cloud Run

  2. Click the service you want to remove, then click Edit and deploy new revision.

  3. Click the Networking tab.

  4. Clear Connect to a VPC for outbound traffic.

  5. Click Deploy.

  6. To verify that your service is no longer on your Shared VPC network, click the Networking tab. The network and subnet are no longer listed in the VPC card.

• To remove only the network tags while keeping the service connected to the Shared VPC network:

  1. Click the service that contains the network tags you want to remove, then click Edit and deploy new revision.

  2. Click the Networking tab.

  3. Clear the names of the network tags that you no longer want to associate with your service.

  4. Click Deploy.

gcloud

• To remove your service from the Shared VPC network, run the following command:

  gcloud run services update SERVICE_NAME --region=REGION \
  --clear-network
  

• To remove only the network tags while keeping the service connected to the Shared VPC network, run the following command:

  gcloud run services update SERVICE_NAME --region=REGION \
  --clear-network-tags
  

  Replace the following:

  • SERVICE_NAME: the name of your Cloud Run service.
  • REGION: the region for your Cloud Run service.

YAML

• To remove your service from the Shared VPC network:

  1. Download the service's YAML configuration:

     gcloud run services describe SERVICE_NAME --format export > service.yaml

  2. Remove the following content from your service.yaml file:

     run.googleapis.com/network-interfaces: '[{"network":"NETWORK","subnetwork":"SUBNET","tags":"NETWORK_TAG_NAMES"}]'

     Where

     • NETWORK: the fully qualified resource name of your Shared VPC network.
     • SUBNET: the fully qualified resource name of your subnet.
     • Optional: NETWORK_TAG_NAMES: the names of the network tags if you had associated them with a service.

  3. Update the service using the following command:

     gcloud run services replace service.yaml

• To remove only the network tags while keeping the service connected to the Shared VPC network:

  1. Download the service's YAML configuration:

     gcloud run services describe SERVICE_NAME --format export > service.yaml

  2. Remove the tags variable from the content in your service.yaml file, leaving the network and subnetwork variables in place, as shown in the following example:

     run.googleapis.com/network-interfaces: '[{"network":"NETWORK","subnetwork":"SUBNET"}]'

     Replace the following:

     • NETWORK: the fully qualified resource name of your Shared VPC network.
     • SUBNET: the fully qualified resource name of your subnet.

  3. Update the service using the following command:

     gcloud run services replace service.yaml

Disconnect a job

Preview — Direct VPC egress for Jobs

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

Console

• To remove your job from the Shared VPC network:

  1. In the Google Cloud console, go to the Cloud Run page:

     Go to Cloud Run

  2. Click the job you want to remove, then click Edit and deploy new revision.

  3. Click the Configuration tab.

  4. Clear Connect to a VPC for outbound traffic.

  5. Click Update.

  6. To verify that your job is no longer on your Shared VPC network, click the Configuration tab. The network and subnet are no longer listed in the VPC card.

• To remove only the network tags while keeping the job connected to the Shared VPC network:

  1. Click the job that contains the network tags you want to remove, then click Edit and deploy new revision.

  2. Click the Connections tab.

  3. Clear the names of the network tags that you no longer want to associate with your job.

  4. Click Update.

gcloud

• To remove your job from the Shared VPC network, run the following command:

  gcloud run jobs update JOB_NAME --region=REGION \
    --clear-network
    

• To remove only the network tags while keeping the job connected to the Shared VPC network, run the following command:

  gcloud run jobs update JOB_NAME --region=REGION \
    --clear-network-tags
    

  Replace the following:

  • JOB_NAME: the name of your Cloud Run job.
  • REGION: the region for your Cloud Run job.

YAML

• To remove your job from the Shared VPC network:

  1. Download the job's YAML configuration:

     gcloud run jobs describe JOB_NAME --format export > job.yaml

  2. Remove the following content from your job.yaml file:

     run.googleapis.com/network-interfaces: '[{"network":"NETWORK","subnetwork":"SUBNET","tags":"NETWORK_TAG_NAMES"}]'

     Where

     • NETWORK: the fully qualified resource name of your Shared VPC network.
     • SUBNET: the fully qualified resource name of your subnet.
     • Optional: NETWORK_TAG_NAMES: the names of the network tags if you had associated them with a job.

  3. Update the job using the following command:

     gcloud run jobs replace job.yaml

• To remove only the network tags while keeping the job connected to the Shared VPC network:

  1. Download the job's YAML configuration:

     gcloud run jobs describe JOB_NAME --format export > job.yaml

  2. Remove the following content from your job.yaml file:

     run.googleapis.com/network-interfaces: '[{"network":"NETWORK","subnetwork":"SUBNET","tags":"NETWORK_TAG_NAMES"}]'

     Where

     • NETWORK: the fully qualified resource name of your Shared VPC network.
     • SUBNET: the fully qualified resource name of your subnet.
     • Optional: NETWORK_TAG_NAMES: the names of the network tags if you had associated them with a job.

  3. Update the job using the following command:

     gcloud run jobs replace job.yaml

Troubleshooting

Cannot delete subnet

To delete a subnet, you must first delete all resources that use it. If Cloud Run is using a subnet, you must disconnect Cloud Run from the Shared VPC network or move it to a different subnet before deleting the subnet.

Note: After you delete or move your Cloud Run resources, wait 1-2 hours for Cloud Run to release the IPs before you delete the subnet.

Cannot detach Shared VPC network

To detach the Shared VPC network in the host project, follow the steps to Deprovision Shared VPC and be sure to disconnect any Cloud Run services or jobs from the Shared VPC network.

To see which Cloud Run resources are using the Shared VPC network, run the following command:

gcloud compute shared-vpc list-associated-resources HOST_PROJECT_ID

Replace HOST_PROJECT_ID with the ID of your Shared VPC host project.

Direct VPC subnet runs out of IP addresses

If the subnet of the Shared VPC network runs out of IP addresses, it is logged by Cloud Logging. When this occurs, Cloud Run cannot start any more service instances or job tasks until more IP addresses become available.

View allocated IP addresses

To see which IP addresses Cloud Run has allocated, go to the IP addresses page in the Google Cloud console or run the following command from the Google Cloud CLI:

gcloud compute addresses list