VPC with connectors

You can enable your Cloud Run service or job to send traffic to a VPC network by configuring a Serverless VPC Access connector.

Before you begin

  • If you don't already have a VPC network in your project, create one.

  • If you use Shared VPC, see Connecting to a Shared VPC network.

  • In the Google Cloud console, ensure that the Serverless VPC Access API is enabled for your project.

    Enable API

  • Each Serverless VPC Access connector requires its own /28 subnet to place connector instances on; this subnet must not have any other resources on it other than the connector. If you don't use Shared VPC, you can either create a subnet for the connector to use, or specify an unused custom IP range for the connector to create a subnet for its use. If you choose the custom IP range, the subnet that is created is hidden and cannot be used in firewall rules and NAT configurations.

Create a connector

To send requests to your VPC network and receive the corresponding responses without using the public internet, you can use a Serverless VPC Access connector.

You can create a connector by using the Google Cloud console, Google Cloud CLI, or Terraform.

Console

  1. Go to the Serverless VPC Access overview page.

    Go to Serverless VPC Access

  2. Click Create connector.

  3. In the Name field, enter a name for your connector. This must be in accordance with the Compute Engine naming convention, with the additional restriction that it be less than 21 characters with hyphens (-) counting as two characters.

  4. In the Region field, select a region for your connector. This must match the region of your serverless service.

    If your service or job is in the region us-central or europe-west, use us-central1 or europe-west1.

  5. In the Network field, select the VPC network to attach your connector to.

  6. Click the Subnet menu. Every connector requires its own /28 subnet to place connector instances on. A subnet cannot be used by other resources such as VMs, Private Service Connect, or load balancers.

    • If you are using Shared VPC, which requires you to use your own subnet, select an unused /28 subnet. To confirm that your subnet is not used for Private Service Connect or Cloud Load Balancing, check that the subnet purpose is PRIVATE by running the following command in the gcloud CLI:

      gcloud compute networks subnets describe SUBNET
      
      Replace:

      • SUBNET: the name of your subnet
    • If you are not using Shared VPC, create a subnet for the connector or have the connector create a subnet by selecting Custom IP range from the menu.

    • In the IP range field, enter the first address in an unreserved CIDR /28 internal IP range. This IP range must not overlap with any existing IP address reservations in your VPC network. For example, 10.8.0.0 (/28) will work in most new projects.

    • The subnet that is created is hidden and cannot be used in firewall rules and NAT configurations.

    • See which IP ranges are currently reserved in the Google Cloud console.

    • Learn more about working with subnets.

  7. (Optional) To set scaling options for additional control over the connector, click Show Scaling Settings to display the scaling form.

    1. Set the minimum and maximum number of instances for your connector, or use the defaults, which are 2 (min) and 10 (max). The connector scales out to the maximum specified if traffic usage requires it, but the connector does not scale back in when traffic decreases. You must use values between 2 and 10.
    2. In the Instance Type menu, choose the machine type to be used for the connector, or use the default e2-micro. Notice the cost sidebar on the right when you choose the instance type, which displays bandwidth and cost estimations.
  8. Click Create.

  9. A green check mark will appear next to the connector's name when it is ready to use.

gcloud

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. Update gcloud components to the latest version:

    gcloud components update
    
  3. Ensure that the Serverless VPC Access API is enabled for your project:

    gcloud services enable vpcaccess.googleapis.com
    
  4. If you are using Shared VPC, which requires you to use your own subnet, create a connector with the following command:

    gcloud compute networks vpc-access connectors create CONNECTOR_NAME \
    --region REGION \
    --subnet SUBNET \
    # If you are not using Shared VPC, omit the following line.
    --subnet-project HOST_PROJECT_ID \
    # Optional: specify minimum and maximum instance values between 2 and
    10, default is 2 min, 10 max.
    --min-instances MIN \
    --max-instances MAX \
    # Optional: specify machine type, default is e2-micro
    --machine-type MACHINE_TYPE
    

    Replace the following:

    • CONNECTOR_NAME: a name for your connector. This must be in accordance with the Compute Engine naming convention, with the additional restriction that it be less than 21 characters with hyphens (-) counting as two characters.
    • REGION: a region for your connector; this must match the region of your serverless service or job. If your service or job is in the region us-central or europe-west, use us-central1 or europe-west1.
    • SUBNET: the name of an unused /28 subnet.
      • Subnets must be used exclusively by the connector. They cannot be used by other resources such as VMs, Private Service Connect, or load balancers.
      • To confirm that your subnet is not used for Private Service Connect or Cloud Load Balancing, check that the subnet purpose is PRIVATE by running the following command in the gcloud CLI:
        gcloud compute networks subnets describe SUBNET
        
        Replace:
        • SUBNET: the name of your subnet.
      • Learn more about working with subnets.
    • HOST_PROJECT_ID: the ID of the host project; supply this only if you are using Shared VPC.
    • MIN: the minimum number of instances to use for the connector. Use an integer between 2 and 9. Default is 2. To learn about connector scaling, see Throughput and scaling.
    • MAX: the maximum number of instances to use for the connector. Use an integer between 3 and 10. Default is 10. If traffic requires it, the connector scales out to [MAX] instances, but does not scale back in. To learn about connector scaling, see Throughput and scaling.
    • MACHINE_TYPE: f1-micro, e2-micro, or e2-standard-4. To learn about connector throughput, including machine type and scaling, see Throughput and scaling.

    For more details and optional arguments, see the gcloud reference.

  5. If you are not using Shared VPC and want to supply a custom IP range instead of using an existing subnet, create a connector with the following command:

    gcloud compute networks vpc-access connectors create CONNECTOR_NAME \
    --network VPC_NETWORK \
    --region REGION \
    --range IP_RANGE
    

    Replace the following:

    • CONNECTOR_NAME: a name for your connector. This must be in accordance with the Compute Engine naming convention, with the additional restriction that it be less than 21 characters with hyphens (-) counting as two characters.
    • VPC_NETWORK: the VPC network to attach your connector to.
    • REGION: a region for your connector. This must match the region of your serverless service or job. If your service or job is in the region us-central or europe-west, use us-central1 or europe-west1.
    • IP_RANGE: an unreserved internal IP network, and a /28 of unallocated space is required. The value supplied is the network in CIDR notation (10.8.0.0/28). This IP range must not overlap with any existing IP address reservations in your VPC network. For example, 10.8.0.0/28 works in most new projects. The subnet that is created for this range is hidden and cannot be used in firewall rules and NAT configurations.

    For more details and optional arguments such as throughput controls, see the gcloud reference.

  6. Verify that your connector is in the READY state before using it:

    gcloud compute networks vpc-access connectors describe CONNECTOR_NAME \
    --region REGION
    

    Replace the following:

    • CONNECTOR_NAME: the name of your connector; this is the name that you specified in the previous step.
    • REGION: the region of your connector; this is the region that you specified in the previous step.

    The output should contain the line state: READY.

Terraform

You can use a Terraform resource to enable the vpcaccess.googleapis.com API.

resource "google_project_service" "vpcaccess-api" {
  project = var.project_id # Replace this with your project ID in quotes
  service = "vpcaccess.googleapis.com"
}

You can use Terraform modules to create a VPC network and subnet and then create the connector.

module "test-vpc-module" {
  source       = "terraform-google-modules/network/google"
  version      = "~> 9.0"
  project_id   = var.project_id # Replace this with your project ID in quotes
  network_name = "my-serverless-network"
  mtu          = 1460

  subnets = [
    {
      subnet_name   = "serverless-subnet"
      subnet_ip     = "10.10.10.0/28"
      subnet_region = "us-central1"
    }
  ]
}

module "serverless-connector" {
  source     = "terraform-google-modules/network/google//modules/vpc-serverless-connector-beta"
  version    = "~> 9.0"
  project_id = var.project_id
  vpc_connectors = [{
    name        = "central-serverless"
    region      = "us-central1"
    subnet_name = module.test-vpc-module.subnets["us-central1/serverless-subnet"].name
    # host_project_id = var.host_project_id # Specify a host_project_id for shared VPC
    machine_type  = "e2-standard-4"
    min_instances = 2
    max_instances = 7
    }
    # Uncomment to specify an ip_cidr_range
    #   , {
    #     name          = "central-serverless2"
    #     region        = "us-central1"
    #     network       = module.test-vpc-module.network_name
    #     ip_cidr_range = "10.10.11.0/28"
    #     subnet_name   = null
    #     machine_type  = "e2-standard-4"
    #     min_instances = 2
    #   max_instances = 7 }
  ]
  depends_on = [
    google_project_service.vpcaccess-api
  ]
}

Configure your service

You can configure your Cloud Run service to use a Serverless VPC Access connector for sending egress traffic. You can do this by using the Google Cloud console, Google Cloud CLI, or YAML file when you create a new service or deploy a new revision:

Console

  1. In the Google Cloud console, go to Cloud Run:

    Go to Cloud Run

  2. Click Create Service if you are configuring a new service you are deploying to. If you are configuring an existing service, click the service, then click Edit and deploy new revision.

  3. If you are configuring a new service, fill out the initial service settings page as desired, then click Container(s), volumes, networking, security to expand the service configuration page.

  4. Click the Networking tab.

    image

    • In the Network field, choose one of the following options from the menu:

      • To disconnect your service from a VPC network, select None.
      • To use an existing connector, select it from the drop-down list or select Custom to use an existing connector that's not shown in the drop-down list.
      • To create a new connector, select Add new VPC connector. See Create a Serverless VPC Access connector for details on creating a connector.
      • For Traffic routing, select one of the following:
        • Route only requests to private IPs to the VPC to send only traffic to internal addresses through the VPC network.
        • Route all traffic to the VPC to send all outbound traffic through the VPC network.
  5. Click Create or Deploy.

gcloud

To specify a connector during deployment, use the --vpc-connector flag:

gcloud run deploy SERVICE --image IMAGE_URL --vpc-connector CONNECTOR_NAME

Replace:

  • SERVICE with the name of your service.
  • IMAGE_URL with a reference to the container image, for example, us-docker.pkg.dev/cloudrun/container/hello:latest. If you use Artifact Registry, the repository REPO_NAME must already be created. The URL has the shape LOCATION-docker.pkg.dev/PROJECT_ID/REPO_NAME/PATH:TAG .
  • CONNECTOR_NAME with the name of your connector.

To attach, update, or remove a connector for an existing service, use the gcloud run services update command with either of the following flags as needed:

For example, to attach or update a connector:

gcloud run services update SERVICE --vpc-connector CONNECTOR_NAME

Replace:

  • SERVICE with the name of your service.
  • CONNECTOR_NAME with the name of your connector.

YAML

You can download and view existing service configurations using the gcloud run services describe --format export command, which yields cleaned results in YAML format. You can then modify the fields described below and upload the modified YAML using the gcloud run services replace command. Make sure you only modify fields as documented.

  1. To view and download the configuration:

    gcloud run services describe SERVICE --format export > service.yaml
  2. Add or update the run.googleapis.com/vpc-access-connector attribute under the annotations attribute under the top-level spec attribute:

    apiVersion: serving.knative.dev/v1
    kind: Service
    metadata:
      name: SERVICE
    spec:
      template:
        metadata:
          annotations:
            run.googleapis.com/vpc-access-connector: CONNECTOR_NAME
          name: REVISION

    Replace:

    • SERVICE with the name of your Cloud Run service.
    • CONNECTOR_NAME with the name of your connector.
    • REVISION with a new revision name or delete it (if present). If you supply a new revision name, it must meet the following criteria:
      • Starts with SERVICE-
      • Contains only lowercase letters, numbers and -
      • Does not end with a -
      • Does not exceed 63 characters
  3. Replace the service with its new configuration using the following command:

    gcloud run services replace service.yaml

Terraform

You can use a Terraform resource to create a service and configure it to use your connector.

# Cloud Run service
resource "google_cloud_run_v2_service" "gcr_service" {
  name     = "mygcrservice"
  provider = google-beta
  location = "us-west1"

  template {
    containers {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
      resources {
        limits = {
          cpu    = "1000m"
          memory = "512Mi"
        }
      }
      # the service uses this SA to call other Google Cloud APIs
      # service_account_name = myservice_runtime_sa
    }

    scaling {
      # Limit scale up to prevent any cost blow outs!
      max_instance_count = 5
    }

    vpc_access {
      # Use the VPC Connector
      connector = google_vpc_access_connector.connector.id
      # all egress from the service should go through the VPC Connector
      egress = "ALL_TRAFFIC"
    }
  }
}

Configure your job

After you have created a Serverless VPC Access connector, you can configure your Cloud Run job to use the connector. You can do this by using the Google Cloud console, Google Cloud CLI, or YAML when you create a new job:

Console

  1. In the Google Cloud console, go to the Cloud Run jobs page:

    Go to Cloud Run

  2. If you are configuring a new job, click the Jobs tab and fill out the initial job settings page as desired. If you are configuring an existing job, click the job, then click Edit.

  3. Click Container, variables and secrets, connections, security to expand the job properties page.

  4. Click the Connections tab.

    image

    • In the Network field, choose one of the following options from the menu:

      • To disconnect your job from a VPC network, select None.
      • To use an existing connector, select it from the drop-down list or select Custom to use an existing connector that's not shown in the drop-down list.
      • To create a new connector, select Add new VPC connector.

        See Create a Serverless VPC Access connector for details on creating a connector.
  5. Click Create or Update.

gcloud

To specify a connector during the job deployment, use the --vpc-connector flag:

gcloud run jobs create JOB --image IMAGE_URL --vpc-connector CONNECTOR_NAME

Replace:

  • JOB with the name of your job.
  • IMAGE_URL with a reference to the container image, for example, us-docker.pkg.dev/cloudrun/container/hello:latest. If you use Artifact Registry, the repository REPO_NAME must already be created. The URL has the shape LOCATION-docker.pkg.dev/PROJECT_ID/REPO_NAME/PATH:TAG .
  • CONNECTOR_NAME with the name of your connector.

To attach, update, or remove a connector for an existing service, use the gcloud run jobs update command with either of the following flags as needed:

For example, to attach or update a connector:

gcloud run jobs update JOB --vpc-connector CONNECTOR_NAME

Replace:

  • JOB with the name of your job.
  • CONNECTOR_NAME with the name of your connector.

YAML

You can download and view an existing job configuration by using the gcloud run jobs describe --format export command, which yields cleaned results in YAML format. You can then modify the fields described below and upload the modified YAML by using the gcloud run jobs replace command. Make sure you only modify fields as documented.

  1. To view and download the configuration:

    gcloud run jobs describe JOB --format export > job.yaml
  2. Add or update the run.googleapis.com/vpc-access-connector attribute under the annotations attribute under the top-level spec attribute:

    apiVersion: serving.knative.dev/v1
      kind: Job
      metadata:
        name: JOB
      spec:
        template:
          metadata:
            annotations:
              run.googleapis.com/vpc-access-connector: CONNECTOR_NAME

    Replace:

    • JOB with the name of your Cloud Run job.
    • CONNECTOR_NAME with the name of your connector. The run.googleapis.com/vpc-access-connector attribute is required when specifying an egress setting.
  3. Replace the job with its new configuration by using the following command:

    gcloud run jobs replace job.yaml

Access to VPC resources

You can restrict access from your connector by using firewall rules and policies, and you can add additional restrictions by configuring egress and ingress settings.

Firewall rules and policy settings

Allow ingress from serverless infrastructure to connector

Your connector VMs must be able to receive packets from the Google Cloud external IP address range 35.199.224.0/19. This range is used by underlying Google serverless infrastructure to ensure that services from Cloud Run, Cloud Functions, and App Engine can send packets to the connector.

Serverless VPC Access creates an ingress allow firewall rule that applies to the connector VMs by allowing packets from 35.199.224.0/19 when the connector is in the same project as the VPC network that it targets. The connector and its target VPC network are in the same project when either the connector targets a standalone VPC network or when the connector targets a Shared VPC network and the connector is located in the host project.

If you create a connector in a Shared VPC service project, a Security Admin or project owner for the Shared VPC host project must create an ingress allow firewall rule applicable to the connector VMs allowing packets from 35.199.224.0/19. The following is an example ingress allow VPC firewall rule:

gcloud compute firewall-rules create RULE_NAME \
--action=ALLOW \
--rules=TCP \
--source-ranges=35.199.224.0/19 \
--target-tags=VPC_CONNECTOR_NETWORK_TAG \
--direction=INGRESS \
--network=VPC_NETWORK \
--priority=PRIORITY \
--project=PROJECT_ID

Replace the following:

  • RULE_NAME: the name of your new firewall rule. For example, allow-vpc-connector-ingress.
  • VPC_CONNECTOR_NETWORK_TAG: the universal connector network tag vpc-connector can be used to make the rule apply to all connectors in the VPC network. Alternatively, you can use a network tag that's specific to the connector. The specific network tag takes the following form: vpc-connector-REGION-CONNECTOR_NAME where REGION is the Google Cloud region of the connector and CONNECTOR_NAME is its name.
  • VPC_NETWORK: the name of the VPC network that the connector targets.
  • PRIORITY: an integer between 0-65535. For example, 0 sets the highest priority.
  • PROJECT_ID: the project ID of the project that contains the VPC network that the connector targets.

Restrict connector VM access VPC network resources

You can restrict your connector's access to resources in its target VPC network by using VPC firewall rules or rules in firewall policies. You can accomplish these restrictions using one of the following strategies:

  • Create ingress rules whose targets represent the resources that you want to limit connector VM access to and whose sources represent the connector VMs.
  • Create egress rules whose targets represent the connector VMs and whose destinations represent the resources that you want to limit connector VM access to.

The following examples illustrate each strategy.

Restrict access using ingress rules

Choose either network tags or CIDR ranges to control the incoming traffic to your VPC network.

Network tags

The following steps show how to create ingress rules that restrict a connector's access to your VPC network based on the connector network tags.

  1. Ensure that you have the required permissions to insert firewall rules. You must have one of the following Identity and Access Management (IAM) roles:

  2. Deny connector traffic across your VPC network.

    Create an ingress firewall rule with priority lower than 1000 on your VPC network to deny ingress from the connector network tag. This overrides the implicit firewall rule that Serverless VPC Access creates on your VPC network by default.

    gcloud compute firewall-rules create RULE_NAME \
    --action=DENY \
    --rules=PROTOCOL \
    --source-tags=VPC_CONNECTOR_NETWORK_TAG \
    --direction=INGRESS \
    --network=VPC_NETWORK \
    --priority=PRIORITY
    

    Replace the following:

    • RULE_NAME: the name of your new firewall rule. For example, deny-vpc-connector.

    • PROTOCOL: one or more protocols that you want to allow from your VPC connector. Supported protocols are tcp or udp. For example, tcp:80,udp allows TCP traffic through port 80 and UDP traffic. For more information, see the documentation for the allow flag.

      For security and validation purposes, you can also configure deny rules to block traffic for the following unsupported protocols: ah, all, esp, icmp, ipip, and sctp.

    • VPC_CONNECTOR_NETWORK_TAG: the universal connector network tag if you want to restrict access for all connectors (including any connectors made in the future), or the unique network tag if you want to restrict access for a specific connector.

      • Universal network tag: vpc-connector
      • Unique network tag: vpc-connector-REGION-CONNECTOR_NAME

        Replace:

        • REGION: the region of the connector that you want to restrict
        • CONNECTOR_NAME: the name of the connector that you want to restrict

      To learn more about connector network tags, see Network tags.

    • VPC_NETWORK: the name of your VPC network

    • PRIORITY: an integer between 0-65535. For example, 0 sets the highest priority.

  3. Allow connector traffic to the resource that should receive connector traffic.

    Use the allow and target-tags flags to create an ingress firewall rule targeting the resource in your VPC network that you want the VPC connector to access. Set the priority for this rule to be a lower value than the priority of the rule you made in the previous step.

    gcloud compute firewall-rules create RULE_NAME \
    --allow=PROTOCOL \
    --source-tags=VPC_CONNECTOR_NETWORK_TAG \
    --direction=INGRESS \
    --network=VPC_NETWORK \
    --target-tags=RESOURCE_TAG \
    --priority=PRIORITY
    

    Replace the following:

    • RULE_NAME: the name of your new firewall rule. For example, allow-vpc-connector-for-select-resources.

    • PROTOCOL: one or more protocols that you want to allow from your VPC connector. Supported protocols are tcp or udp. For example, tcp:80,udp allows TCP traffic through port 80 and UDP traffic. For more information, see the documentation for the allow flag.

    • VPC_CONNECTOR_NETWORK_TAG: the universal connector network tag if you want to restrict access for all connectors (including any connectors made in the future), or the unique network tag if you want to restrict access for a specific connector. This must match the network tag that you specified in the previous step.

      • Universal network tag: vpc-connector
      • Unique network tag: vpc-connector-REGION-CONNECTOR_NAME

        Replace:

        • REGION: the region of the connector that you want to restrict
        • CONNECTOR_NAME: the name of the connector that you want to restrict

      To learn more about connector network tags, see Network tags.

    • VPC_NETWORK: the name of your VPC network

    • RESOURCE_TAG: the network tag for the VPC resource that you want your VPC connector to access

    • PRIORITY: an integer less than the priority you set in the previous step. For example, if you set the priority for the rule you created in the previous step to 990, try 980.

For more information about the required and optional flags for creating firewall rules, refer to the documentation for gcloud compute firewall-rules create.

CIDR range

The following steps show how to create ingress rules that restrict a connector's access to your VPC network based on the connector's CIDR range.

  1. Ensure that you have the required permissions to insert firewall rules. You must have one of the following Identity and Access Management (IAM) roles:

  2. Deny connector traffic across your VPC network.

    Create an ingress firewall rule with priority lower than 1000 on your VPC network to deny ingress from the connector's CIDR range. This overrides the implicit firewall rule that Serverless VPC Access creates on your VPC network by default.

    gcloud compute firewall-rules create RULE_NAME \
    --action=DENY \
    --rules=PROTOCOL \
    --source-ranges=VPC_CONNECTOR_CIDR_RANGE \
    --direction=INGRESS \
    --network=VPC_NETWORK \
    --priority=PRIORITY
    

    Replace the following:

    • RULE_NAME: the name of your new firewall rule. For example, deny-vpc-connector.

    • PROTOCOL: one or more protocols that you want to allow from your VPC connector. Supported protocols are tcp or udp. For example, tcp:80,udp allows TCP traffic through port 80 and UDP traffic. For more information, see the documentation for the allow flag.

      For security and validation purposes, you can also configure deny rules to block traffic for the following unsupported protocols: ah, all, esp, icmp, ipip, and sctp.

    • VPC_CONNECTOR_CIDR_RANGE: the CIDR range for the connector whose access you are restricting

    • VPC_NETWORK: the name of your VPC network

    • PRIORITY: an integer between 0-65535. For example, 0 sets the highest priority.

  3. Allow connector traffic to the resource that should receive connector traffic.

    Use the allow and target-tags flags to create an ingress firewall rule targeting the resource in your VPC network that you want the VPC connector to access. Set the priority for this rule to be a lower value than the priority of the rule you made in the previous step.

    gcloud compute firewall-rules create RULE_NAME \
    --allow=PROTOCOL \
    --source-ranges=VPC_CONNECTOR_CIDR_RANGE \
    --direction=INGRESS \
    --network=VPC_NETWORK \
    --target-tags=RESOURCE_TAG \
    --priority=PRIORITY
    

    Replace the following:

    • RULE_NAME: the name of your new firewall rule. For example, allow-vpc-connector-for-select-resources.

    • PROTOCOL: one or more protocols that you want to allow from your VPC connector. Supported protocols are tcp or udp. For example, tcp:80,udp allows TCP traffic through port 80 and UDP traffic. For more information, see the documentation for the allow flag.

    • VPC_CONNECTOR_CIDR_RANGE: the CIDR range for the connector you whose access you are restricting

    • VPC_NETWORK: the name of your VPC network

    • RESOURCE_TAG: the network tag for the VPC resource that you want your VPC connector to access

    • PRIORITY: an integer less than the priority you set in the previous step. For example, if you set the priority for the rule you created in the previous step to 990, try 980.

For more information about the required and optional flags for creating firewall rules, see the documentation for gcloud compute firewall-rules create.

Restrict access using egress rules

The following steps show how to create egress rules to restrict connector access.

  1. Ensure that you have the required permissions to insert firewall rules. You must have one of the following Identity and Access Management (IAM) roles:

  2. Deny egress traffic from your connector.

    Create an egress firewall rule on your Serverless VPC Access connector to prevent it from sending outgoing traffic, with the exception of established responses, to any destination.

    gcloud compute firewall-rules create RULE_NAME \
    --action=DENY \
    --rules=PROTOCOL \
    --direction=EGRESS \
    --target-tags=VPC_CONNECTOR_NETWORK_TAG \
    --network=VPC_NETWORK \
    --priority=PRIORITY
    

    Replace the following:

    • RULE_NAME: the name of your new firewall rule. For example, deny-vpc-connector.

    • PROTOCOL: one or more protocols that you want to allow from your VPC connector. Supported protocols are tcp or udp. For example, tcp:80,udp allows TCP traffic through port 80 and UDP traffic. For more information, see the documentation for the allow flag.

      For security and validation purposes, you can also configure deny rules to block traffic for the following unsupported protocols: ah, all, esp, icmp, ipip, and sctp.

    • VPC_CONNECTOR_NETWORK_TAG: the universal VPC connector network tag if you want the rule to apply to all existing VPC connectors and any VPC connectors made in the future. Or, the unique VPC connector network tag if you want to control a specific connector.

    • VPC_NETWORK: the name of your VPC network

    • PRIORITY: an integer between 0-65535. For example, 0 sets the highest priority.

  3. Allow egress traffic when the destination is in the CIDR range that you want your connector to access.

    Use the allow and destination-ranges flags to create a firewall rule allowing egress traffic from your connector for a specific destination range. Set the destination range to the CIDR range of the resource in your VPC network that you want your connector to be able to access. Set the priority for this rule to be a lower value than the priority of the rule you made in the previous step.

    gcloud compute firewall-rules create RULE_NAME \
    --allow=PROTOCOL \
    --destination-ranges=RESOURCE_CIDR_RANGE \
    --direction=EGRESS \
    --network=VPC_NETWORK \
    --target-tags=VPC_CONNECTOR_NETWORK_TAG \
    --priority=PRIORITY
    

    Replace the following:

    • RULE_NAME: the name of your new firewall rule. For example, allow-vpc-connector-for-select-resources.

    • PROTOCOL: one or more protocols that you want to allow from your VPC connector. Supported protocols are tcp or udp. For example, tcp:80,udp allows TCP traffic through port 80 and UDP traffic. For more information, see the documentation for the allow flag.

    • RESOURCE_CIDR_RANGE: the CIDR range for the connector whose access you are restricting

    • VPC_NETWORK: the name of your VPC network

    • VPC_CONNECTOR_NETWORK_TAG: the universal VPC connector network tag if you want the rule to apply to all existing VPC connectors and any VPC connectors made in the future. Or, the unique VPC connector network tag if you want to control a specific connector. If you used the unique network tag in the previous step, use the unique network tag.

    • PRIORITY: an integer less than the priority you set in the previous step. For example, if you set the priority for the rule you created in the previous step to 990, try 980.

For more information about the required and optional flags for creating firewall rules, refer to the documentation for gcloud compute firewall-rules create.

Manage your connector

By default, only requests to certain internal IPv4 addresses destinations (including internal IPv4 destinations resolved by DNS responses) are routed through a Serverless VPC Access connector. In some cases, however, you might want all outbound requests from your service or job to be routed to your VPC network.

Serverless VPC Access only supports routing IPv4 traffic. IPv6 traffic is not supported, even if you have IPv6 routes in your VPC network.

For control over routing outbound requests from your service or job, you can set the VPC egress to one of the following options:

  • Route only requests to private IPs to the VPC: Default. Traffic is routed through the VPC network only if the packets carrying the traffic have destinations that match the following:

    Packets to any other destination are routed from Cloud Run to the internet (not through any VPC network).

  • Route all traffic to the VPC: Traffic is always routed through the VPC network associated with the connector, for all packet destinations. You must use this option in these circumstances:

    • If you need to send traffic to VPC subnet ranges with privately used external IP address ranges. For more information about VPC subnet ranges, see Valid IPv4 ranges in the Subnets overview.
    • If you need to send traffic to a Private Service Connect endpoint for Google APIs whose address is a privately used external IP address. For more information about Private Service Connect endpoints for Google APIs, see Access Google APIs through endpoints.
    • If you need to send traffic to any other privately used external IP address destination that's routable within the connector's VPC network. Examples of other destinations covering privately used external IP address could include peering subnet ranges (and peering subnet ranges created from Allocated IP address ranges for services) and those destinations that are accessible using custom routes in the VPC short.

    If your VPC network includes a default route, packets can still be routed to the internet after being processed by the connector if you configure a Cloud NAT gateway to provide NAT services to the subnet used by the connector. These packets are subject to the routes in your VPC network and the firewall rules that apply to your VPC network. You can use route and firewall configuration to control internet egress for all outbound requests sent by your function through a Serverless VPC Access connector.

Control egress service traffic

You can specify a VPC egress setting by using the Google Cloud console, Google Cloud CLI, or YAML file when you create a new service or deploy a new revision:

Console

  1. In the Google Cloud console, go to Cloud Run:

    Go to Cloud Run

  2. Click Create Service if you are configuring a new service you are deploying to. If you are configuring an existing service, click the service, then click Edit and deploy new revision.

  3. If you are configuring a new service, fill out the initial service settings page as desired, then click Container(s), volumes, networking, security to expand the service configuration page.

  4. Click the Networking tab.

    image

    • After you select a VPC connector, select Route only requests to private IPs to the VPC or Route all traffic to the VPC.
  5. Click Create or Deploy.

gcloud

To specify an egress setting, use the --vpc-egress flag. You can specify an egress setting during deployment:

gcloud run deploy SERVICE \
--image IMAGE_URL \
--vpc-connector CONNECTOR_NAME \
--vpc-egress EGRESS_SETTING

Replace:

  • SERVICE with the name of your service.
  • IMAGE_URL with a reference to the container image, for example, us-docker.pkg.dev/cloudrun/container/hello:latest. If you use Artifact Registry, the repository REPO_NAME must already be created. The URL has the shape LOCATION-docker.pkg.dev/PROJECT_ID/REPO_NAME/PATH:TAG .
  • CONNECTOR_NAME with the name of your connector.
  • EGRESS_SETTING with an egress setting value:
    • all-traffic: Sends all outbound traffic through the connector.
    • private-ranges-only: Sends only traffic to internal addresses through the VPC connector.

You can also update an existing service and change the egress setting:

gcloud run services update SERVICE --vpc-egress EGRESS_SETTING

Replace:

  • SERVICE with the name of your service.
  • EGRESS_SETTING with an egress setting value as listed above.

YAML

You can download and view existing service configurations using the gcloud run services describe --format export command, which yields cleaned results in YAML format. You can then modify the fields described below and upload the modified YAML using the gcloud run services replace command. Make sure you only modify fields as documented.

  1. To view and download the configuration:

    gcloud run services describe SERVICE --format export > service.yaml
  2. Add or update the run.googleapis.com/vpc-access-egress attribute under the annotations attribute under the top-level spec attribute:

    apiVersion: serving.knative.dev/v1
    kind: Service
    metadata:
      name: SERVICE
    spec:
      template:
        metadata:
          annotations:
            run.googleapis.com/vpc-access-connector: CONNECTOR_NAME
            run.googleapis.com/vpc-access-egress: EGRESS_SETTING
          name: REVISION

    Replace:

    • SERVICE with the name of your Cloud Run service.
    • CONNECTOR_NAME with the name of your connector. The run.googleapis.com/vpc-access-connector attribute is required when specifying an egress setting.
    • EGRESS_SETTING with one of the following:
      • all-traffic: Sends all outbound traffic through the connector.
      • private-ranges-only: Sends only traffic to internal addresses through the VPC connector.
      • REVISION with a new revision name or delete it (if present). If you supply a new revision name, it must meet the following criteria:
        • Starts with SERVICE-
        • Contains only lowercase letters, numbers and -
        • Does not end with a -
        • Does not exceed 63 characters
  3. Replace the service with its new configuration using the following command:

    gcloud run services replace service.yaml

Control egress job traffic

You can specify a VPC egress setting by using the Google Cloud console, Google Cloud CLI, or YAML file when you create or update a job:

Console

  1. In the Google Cloud console, go to the Cloud Run jobs page:

    Go to Cloud Run

  2. If you are configuring a new job, click the Jobs tab and fill out the initial job settings page as desired. If you are configuring an existing job, click the job, then click Edit.

  3. Click Container, variables and secrets, connections, security to expand the job properties page.

  4. Click the Connections tab.

    image

    • After you select a connector, select Route only requests to private IPs to the VPC or Route all traffic to the VPC.
  5. Click Create or Update.

gcloud

To specify an egress setting, use the --vpc-egress flag. You can specify an egress setting during the job deployment process:

gcloud run jobs create JOB \
--image IMAGE_URL \
--vpc-connector CONNECTOR_NAME \
--vpc-egress EGRESS_SETTING

Replace:

  • JOB with the name of your job.
  • IMAGE_URL with a reference to the container image, for example, us-docker.pkg.dev/cloudrun/container/hello:latest. If you use Artifact Registry, the repository REPO_NAME must already be created. The URL has the shape LOCATION-docker.pkg.dev/PROJECT_ID/REPO_NAME/PATH:TAG .
  • CONNECTOR_NAME with the name of your connector.
  • EGRESS_SETTING with an egress setting value:
    • all-traffic: Sends all outbound traffic through the connector.
    • private-ranges-only: Sends only traffic to internal addresses through the VPC connector.

You can also update an existing job and change the egress setting:

gcloud run jobs update JOB --vpc-egress EGRESS_SETTING

Replace:

YAML

You can download and view an existing job configuration by using the gcloud run jobs describe --format export command, which yields cleaned results in YAML format. You can then modify the fields described below and upload the modified YAML by using the gcloud run jobs replace command. Make sure you only modify fields as documented.

  1. Replace the job with its new configuration by using the following command:

    gcloud run jobs replace job.yaml
  2. Add or update the run.googleapis.com/vpc-access-egress attribute under the annotations attribute under the top-level spec attribute:

    apiVersion: serving.knative.dev/v1
      kind: Job
      metadata:
        name: JOB
      spec:
        template:
          metadata:
            annotations:
              run.googleapis.com/vpc-access-connector: CONNECTOR_NAME
              run.googleapis.com/vpc-access-egress: EGRESS_SETTING

    Replace:

    • JOB with the name of your Cloud Run job.
    • CONNECTOR_NAME with the name of your connector. The run.googleapis.com/vpc-access-connector attribute is required when specifying an egress setting.
    • EGRESS_SETTING with one of the following:
      • all-traffic: Sends all outbound traffic through the connector.
      • private-ranges-only: Sends only traffic to internal addresses through the VPC connector.
  3. Replace the job with its new configuration by using the following command:

    gcloud run jobs replace job.yaml

View connector settings

You can view the current Serverless VPC Access connector settings for your service or job by using the Google Cloud console or the gcloud command-line tool:

Console

  1. Go to Cloud Run

  2. Click the service or job you are interested in to open the details page.

    • For a service, click the Revisions tab.

    • For a job, click the Configuration tab.

The Serverless VPC Access connector setting is listed under the Networking tab.

gcloud

  1. Use the following command to view settings for your service:

    gcloud run services describe SERVICE
  2. Use the following command to view settings for your job:

    gcloud run jobs describe JOB
  3. Locate the Serverless VPC Access connector setting in the returned configuration.

Disconnect from a VPC network

Disconnect a service

You can disconnect a service from your VPC network by using the Google Cloud console or the Google Cloud CLI:

Console

  1. In the Google Cloud console, go to Cloud Run:

    Go to Cloud Run

  2. Click Create Service if you are configuring a new service you are deploying to. If you are configuring an existing service, click the service, then click Edit and deploy new revision.

  3. If you are configuring a new service, fill out the initial service settings page as desired, then click Container(s), volumes, networking, security to expand the service configuration page.

  4. Click the Networking tab.

    image

    • In the Network field, select None to disconnect your service from a VPC network.
  5. Click Create or Deploy.

gcloud

To disconnect a service, use the gcloud run services update command with the following flag:

gcloud run services update SERVICE --clear-vpc-connector
  • Replace SERVICE with the name of your service.

Connectors continue to incur charges even if they have no traffic and are disconnected. For details, see pricing. If you no longer need your connector, be sure to delete it to avoid continued billing.

Disconnect a job

You can disconnect a job from your VPC network by using the Google Cloud console or the Google Cloud CLI:

Console

  1. In the Google Cloud console, go to the Cloud Run jobs page:

    Go to Cloud Run

  2. If you are configuring a new job, click the Jobs tab and fill out the initial job settings page as desired. If you are configuring an existing job, click the job, then click Edit.

  3. Click Container, variables and secrets, connections, security to expand the job properties page.

  4. Click the Connections tab.

    image

    • In the Network field, select None to disconnect your job from a VPC network.
  5. Click Create or Update.

gcloud

To disconnect a job, use the gcloud run jobs update command with the following flag:

--clear-vpc-connector

gcloud run jobs update JOB --clear-vpc-connector

Replace JOB with the name of your job.

Connectors continue to incur charges even if they have no traffic and are disconnected. For details, see pricing. If you no longer need your connector, be sure to delete it to avoid continued billing.

Update a connector

You can update and monitor the following attributes of your connector by using the Google Cloud console, Google Cloud CLI, or the API:

  • Machine (instance) type
  • Minimum and maximum number of instances
  • Recent throughput, number of instances, and CPU utilization

Update machine type

Console

  1. Go to the Serverless VPC Access overview page.

    Go to Serverless VPC Access

  2. Select the connector you want to edit and click Edit.

  3. In the Instance type list, select your preferred machine (instance) type. To learn about available machine types, see the documentation on Throughput and scaling.

gcloud

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. To update the connector machine type, run the following command in your terminal:

    gcloud beta compute networks vpc-access connectors update CONNECTOR_NAME --region=REGION --machine-type=MACHINE_TYPE
    
    Replace the following:

    • CONNECTOR_NAME: the name of your connector
    • REGION: the name of your connector's region
    • MACHINE_TYPE: your preferred machine type. To learn about available machine types, see the documentation on Throughput and scaling.

Decrease minimum and maximum number of instances

To decrease the number of minimum and maximum number of instances, you must do the following:

  1. Create a new connector with your preferred values.
  2. Update your service or function to use the new connector.
  3. Delete the old connector when you've moved its traffic.

See Create Serverless VPC Access connector for more information.

Increase minimum and maximum number of instances

Console

  1. Go to the Serverless VPC Access overview page.

    Go to Serverless VPC Access

  2. Select the connector you want to edit and click Edit.

  3. In the Minimum instances field, select your preferred minimum number of instances.

    The smallest possible value for this field is the current value. The largest possible value for this field is the current value in the Maximum instances field minus 1. For example, if the value in the Maximum instances field is 8, then the largest possible value for the Minimum instances field is 7.

  4. In the Maximum instances field, select your preferred maximum number of instances.

    The smallest possible value for this field is the current value. The largest possible value for this field is 10.

gcloud

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. To increase the minimum or maximum number of instances for the connector, run the following command in your terminal:

    gcloud beta compute networks vpc-access connectors update CONNECTOR_NAME --region=REGION --min-instances=MIN_INSTANCES --max-instances=MAX_INSTANCES
    
    Replace the following:

  • CONNECTOR_NAME: the name of your connector
  • REGION: the name of your connector's region
  • MIN_INSTANCES: your preferred minimum number of instances.
    • Smallest possible value for this field is the current value of min_instances. To find the current value, see Find the current attribute values.
    • Largest possible value for this field is the current max_instances value minus 1, because min_instances must be less than max_instances. For example, if max_instances is 8, the largest possible value for this field is 7. If your connector uses the default max-instances value of 10, the largest possible value of this field is 9. To find the value of max-instances, see Find the current attribute values.
  • MAX_INSTANCES:

    • Smallest possible value for this field is the current value of max_instances. To find the current value, see Find the current attribute values.
    • Largest possible value for this field is 10.

    If you only want to increase the minimum number of instances but not the maximum, you must still specify the maximum number of instances. Conversely, if you only want to update the maximum number of instances but not the minimum, you must still specify the minimum number of instances. To keep either the minimum or maximum number of instances at their current value, specify their current value. To find their current value, see Find the current attribute values.

Find the current attribute values

To find the current attribute values for your connector, run the following in your terminal:

gcloud compute networks vpc-access connectors describe CONNECTOR_NAME --region=REGION --project=PROJECT
Replace the following:

  • CONNECTOR_NAME: the name of your connector
  • REGION: the name of your connector's region
  • PROJECT: the name of your Google Cloud project

Monitor connector usage

Monitoring usage over time can help you determine when to adjust a connector's settings. For example, if CPU utilization spikes, you might try increasing the maximum number of instances for better results. Or if you are maxing out throughput, you might decide to switch to a larger machine type.

To display charts for the connector's throughput, number of instances, and CPU utilization metrics over time by using the Google Cloud console:

  1. Go to the Serverless VPC Access overview page.

    Go to Serverless VPC Access

  2. Click on the name of the connector you want to monitor.

  3. Select the number of days you want to display between 1 and 90 days.

  4. In the Throughput chart, hover over the chart to view the connector's recent throughput.

  5. In the Number of instances chart, hover over the chart to view the number of instances recently used by the connector.

  6. In the CPU Utilization chart, hover over the chart to view the connector's recent CPU usage. The chart displays the CPU usage distributed across instances for the 50th, 95th, and 99th percentiles.

Delete a connector

Before you delete a connector, ensure that no services or jobs are still connected to it.

For Shared VPC users who set up connectors in the Shared VPC host project, you can use the command gcloud compute networks vpc-access connectors describe to list the projects in which there are services or jobs that use a given connector.

To delete a connector, use the Google Cloud console or the Google Cloud CLI:

Console

  1. Go to the Serverless VPC Access overview page in the Google Cloud console:

    Go to Serverless VPC Access

  2. Select the connector you want to delete.

  3. Click Delete.

gcloud

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. Use the following gcloud command to delete a connector:

    gcloud compute networks vpc-access connectors delete CONNECTOR_NAME --region=REGION
    

    Replace the following:

    • CONNECTOR_NAME with the name of the connector you want to delete
    • REGION with the region where the connector is located

Troubleshooting

Service account permissions

To perform operations in your Google Cloud project, Serverless VPC Access uses the Serverless VPC Access Service Agent service account. This service account's email address has the following form:

service-PROJECT_NUMBER@gcp-sa-vpcaccess.iam.gserviceaccount.com

By default, this service account has the Serverless VPC Access Service Agent role (roles/vpcaccess.serviceAgent). Serverless VPC Access operations may fail if you change this account's permissions.

Errors

Service account needs Service Agent role error

If you use the Restrict Resource Service Usage organization policy constraint to block Cloud Deployment Manager (deploymentmanager.googleapis.com), you might see the following error message:

Serverless VPC Access service account (service-<PROJECT_NUMBER>@gcp-sa-vpcaccess.iam.gserviceaccount.com) needs Serverless VPC Access Service Agent role in the project.

Set the organization policy to either remove Deployment Manager from the denylist or add it to the allowlist.

Connector creation error

If creating a connector results in an error, try the following:

  • Specify an RFC 1918 internal IP range that does not overlap with any existing IP address reservations in the VPC network.
  • Grant your project permission to use Compute Engine VM images from the project with ID serverless-vpc-access-images. For more information about how to update your organization policy accordingly, see Set image access constraints.

Unable to access resources

If you specified a connector but still cannot access resources in your VPC network, make sure that there are no firewall rules on your VPC network with a priority lower than 1000 that deny ingress from your connector's IP address range.

If you configure a connector in a Shared VPC service project, make sure that your firewall rules allow ingress from your serverless infrastructure to the connector.

Connection refused error

If you receive connection refused errors that degrade network performance, your connections could be growing without limit across invocations of your serverless application. To limit the maximum number of connections used per instance, use a client library that supports connection pools. For detailed examples of how to use connection pools, see Manage database connections.

Resource not found error

When deleting a VPC network or a firewall rule, you might see a message that is similar to the following: The resource "aet-uscentral1-subnet--1-egrfw" was not found.

For information about this error and its solution, see Resource not found error in the VPC firewall rules documentation.

Next steps