Set up private connectivity to Google APIs and services

This document describes how to set up private connectivity from hosts in a VPC network or on-premises network to Google APIs and services that VPC Service Controls supports.

Before you read this document, we recommend that you familiarize yourself with Private Google Access concepts, specifications, and network configuration. See example topology diagrams about using Private Google Access with VPC Service Controls.

Before you begin

• Enable the APIs that you want to access through the APIs & services page in the Google Cloud console.
• Ensure that you have the required role to create or update subnets. Project owners, editors, and IAM principals with the Network Admin role can create or update subnets and assign IP addresses. For more information about roles, read the IAM roles documentation.
• Verify that a VPC network is set up for Private Google Access and Private Google Access for on-premises hosts. Both auto and custom mode VPC networks are supported. Legacy networks are not supported.
• Ensure that VM instances in a VPC network have a private IP address (no public IP address) and are in a subnet with Private Google Access enabled.
• For on-premises hosts, ensure that you have an existing Cloud VPN tunnel or an Cloud Interconnect connection to your VPC network.

Overview of procedure

To set up private connectivity, complete the following tasks:

• Configure routes for the destination 199.36.153.4/30. For more information, see Configure routes.
• Configure firewall rules to let the appropriate traffic to the restricted Google APIs IP address range. For more information, see Configure firewall rules.
• Configure DNS so that traffic to Google APIs resolves to the Restricted Google APIs IP address range. For more information, see Configure DNS.

Configure routes to restricted.googleapis.com

Use restricted.googleapis.com to provide access to Cloud and Developer APIs that support VPC Service Controls. The restricted.googleapis.com domain resolves to a VIP (virtual IP address) range 199.36.153.4/30. This IP address range is not accessible from the internet.

Although VPC Service Controls are enforced for compatible and configured services, regardless of the domain you use, restricted.googleapis.com provides additional risk mitigation for data exfiltration. restricted.googleapis.com denies access to Google APIs and services that are not supported by VPC Service Controls.

For both Private Google Access and Private Google Access for on-premises hosts, your VPC network must include a route for the destination 199.36.153.4/30 whose next hop is the default internet gateway. Even though the next hop is a default internet gateway, the traffic sent to 199.36.153.4/30 remains within Google's network.

If your VPC network does not have a default route whose next hop is the default internet gateway, you can create a custom static route whose destination is 199.36.153.4/30 and whose next hop is the default internet gateway. When you have a custom static route with the destination 199.36.153.4/30, to prevent access to the internet, you must remove other routes whose next hop is the default internet gateway.

For more information about working with VPC routes, see Using Routes in the VPC documentation.

Configure a custom static route in a VPC network

Add a custom static route to enable access to Google-managed services that VPC Service Controls supports.

• Add a custom route that lets access only to Google-managed services secured by VPC Service Controls.

  gcloud compute routes create ROUTE_NAME \
    --network=NETWORK_NAME \
    --destination-range=199.36.153.4/30 \
    --next-hop-gateway=default-internet-gateway
  

  Replace the following:

  • ROUTE_NAME: a name for the custom route.
  • NETWORK_NAME: the name of your VPC network.

Announce the restricted route to hosts in an on-premises network

If you're using Private Google Access for on-premises hosts, configure routes so that Google API traffic is forwarded through your Cloud VPN or Cloud Interconnect connection. To announce the restricted VIP (199.36.153.4/30) to your on-premises network, use Cloud Router Custom Route Advertisement. This IP address range is only accessible to on-premises hosts that can reach your VPC network through private IP addresses.

Note: If you have multiple tunnels or interconnects, you can't create cross-region asymmetric routes back to your on-premises network. Google Cloud doesn't support them.

You can add this custom route advertisement to a Cloud Router (for all BGP sessions on the router) or a select BGP session (for a single Cloud VPN tunnel or VLAN attachment).

To create a custom route advertisement for the restricted range for all BGP sessions on an existing Cloud Router, follow these steps:

Console

1. In the Google Cloud console, go to the Cloud Router page.
   Go to Cloud Router
2. Select the Cloud Router to update.
3. In the Cloud Router's detail page, select Edit.
4. Expand the Advertised routes section.
5. For the Routes, select Create custom routes.
6. To continue advertising the subnets available to the Cloud Router, select Advertise all subnets visible to the Cloud Router. Enabling this option mimics the Cloud Router's default behavior.
7. To add an advertised route, select Add custom route .
8. Configure the route advertisement.
   • Source — Select Custom IP range .
   • IP address range — Specify 199.36.153.4/30.
   • Description — Add a description of Restricted Google APIs IPs.
9. After you add routes, select Save.

gcloud

Run the update command, using either the --set-advertisement-ranges or --add-advertisement-ranges flag to specify the custom IP ranges:

• To set custom IP ranges, use the --set-advertisement-ranges flag. Any existing custom advertisements are replaced. The following example updates the my-router Cloud Router to advertise all subnets and the Restricted Google APIs IPs range 199.36.153.4/30:

  gcloud compute routers update my-router \
      --advertisement-mode CUSTOM \
      --set-advertisement-groups ALL_SUBNETS \
      --set-advertisement-ranges 199.36.153.4/30
  

• To append custom IP ranges to an existing advertisement, use the --add-advertisement-ranges flag. Note that this flag requires the Cloud Router's advertisement mode to already be set to custom. The following example adds the Restricted Google APIs IPs custom IP to the Cloud Router's advertisements:

  gcloud compute routers update my-router \
      --add-advertisement-ranges 199.36.153.4/30
  

To create a custom route advertisement for the restricted range on a specific BGP session of an existing Cloud Router, follow these steps:

Console

1. In the Google Cloud console, go to the Cloud Router page.
   Go to Cloud Router
2. Select the Cloud Router that contains the BGP session to update.
3. In the Cloud Router's detail page, select the BGP session to update.
4. In the BGP session details page, select Edit.
5. For the Routes, select Create custom routes.
6. Select Advertise all subnets visible to the Cloud Router to continue advertising the subnets available to the Cloud Router. Enabling this option mimics the Cloud Router's default behavior.
7. Select Add custom route to add an advertised route.
8. Configure the route advertisement.
   • Source — Select Custom IP range to specify a custom IP range.
   • IP address range — Specify 199.36.153.4/30.
   • Description — Add a description of Restricted Google APIs IPs.
9. After you're done adding routes, select Save.

gcloud

Run the update-bgp-peer command, using either the --set-advertisement-ranges or --add-advertisement-ranges flag to specify the custom IP ranges.

• To set custom IP ranges, use the --set-advertisement-ranges flag. Any existing custom advertisements are replaced. The following example updates the my-bgp-session BGP session on the my-router Cloud Router to advertise all subnets and the custom IP range 199.36.153.4/30:

  gcloud compute routers update-bgp-peer my-router \
      --peer-name my-bgp-session \
      --advertisement-mode CUSTOM \
      --set-advertisement-groups ALL_SUBNETS \
      --set-advertisement-ranges 199.36.153.4/30
  

• To append custom IP ranges to existing ones, use the --add-advertisement-ranges flag. Note that this flag requires the Cloud Router's advertisement mode to already be set to custom. The following example adds the 199.36.153.4/30 Restricted Google APIs IPs to the Cloud Router's advertisements:

  gcloud compute routers update-bgp-peer my-router \
      --peer-name my-bgp-session \
      --add-advertisement-ranges 199.36.153.4/30
  

  For more information about custom advertisements, see Custom Router Advertisements.

Configure firewall rules

For Private Google Access, VM instances use internal IP addresses and don't require external IP addresses to reach protected Google API resources. However, it's possible for VM instances to possess external IP addresses or otherwise meet the requirements for Internet access. In addition to custom routes, you can restrict egress traffic from VM instances in your VPC network by creating firewall rules to deny egress traffic.

By default, the implied allow egress firewall rule permits VM instances to send traffic to any destination if an applicable route exists. You can first create an egress deny rule to block all outbound traffic. You can then create higher priority egress rules that permit traffic to selected destinations in your VPC network and to the 199.36.153.4/30 (restricted.googleapis.com) IP address range. All communication to restricted.googleapis.com is on TCP port 443.

Note: Before blocking internet access, carefully consider the impact on your VM instances. Blocking internet access can reduce your risk of data exfiltration, but it can also block legitimate traffic, including essential traffic for software updates. Without internet access, you are only able to access your VM instances through an on-premises network connected through a Cloud VPN tunnel or Cloud Interconnect connection. After you block internet access, VM instances cannot call third-party APIs and services.

For more information about working with VPC firewall rules, see Using Firewall Rules in the VPC documentation.

Configure your on-premises firewall rules to let traffic from your on-premises hosts to reach 199.36.153.4/30.

Configure DNS

For general use of VPC Service Controls, we recommend that you use Cloud DNS response policies to configure DNS for your VPC networks. When you use Cloud DNS, you don't need to create a managed private zone to configure DNS. A response policy uses passthru behavior to let names, such as www.googleapis.com, be allowed past the wildcard name in the example *.googleapis.com. For more information, see Managing response policies and rules.

You can also use managed private zones for your VPC networks. Cloud DNS private DNS zones enable you to host a DNS zone accessible from authorized VPC networks. To configure forwarding from certain on-premises name servers, you can make use of the Restricted Google APIs IP addresses. You can then create a private zone for googleapis.com with a DNS A record that maps restricted.googleapis.com and appropriate CNAME records for each *.googleapis.com name. For more information, see Managing zones.

For on-premises access, you can configure a Cloud DNS inbound forwarding policy to enable on-premises name servers to query a Cloud DNS managed private zone. You can also configure an on-premises name server, such as one using BIND:

Note: For general use of VPC Service Controls, we strongly recommend that you use response policies for your VPC networks. We suggest that you use BIND only for on-premises implementations.

Configure DNS with Cloud DNS

To enable DNS resolution for VM instances in your VPC network, hosts in an on-premises network, or both, use Cloud DNS. If you're using Shared VPC, see Private zones and Shared VPC in the Cloud DNS documentation. Also, if you're using Shared VPC, ensure that the Shared VPC network host project is included in the same service perimeter as projects that connect to the network.

You can configure DNS with Cloud DNS by using response policies or private zones.

Configure Cloud DNS using response policies

Beta — Managing response policies

This feature is covered by the Pre-GA Offerings Terms of the Google Cloud Terms of Service. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

Direct specific names to restricted VIP addresses

You can configure a response policy that has local CNAME data for each zone to translate Google API requests to restricted Google APIs. Names that aren't specified continue to be resolved by using regular DNS.

For example, you can create a response policy for pubsub.googleapis.com, which has local CNAME data for a zone, to translate Google API requests to restricted.googleapis.com. In the meantime, www.googleapis.com, which is unspecified, continues to be resolved by using regular DNS.

In the following example configuration, you create a policy and apply it to a specific VPC network.

gcloud

1. To create a response policy, run the gcloud dns response-policies create command:

   gcloud dns response-policies create RESPONSE_POLICY_NAME \
       --networks=NETWORK \
       --description=DESCRIPTION
   

   Replace the following:

   • RESPONSE_POLICY_NAME: the name or ID of the response policy that you want to create, such as myresponsepolicy
   • NETWORK: a comma-separated list of network names to associate with the response policy, such as network1,network2
   • DESCRIPTION: a description of the response policy, such as My new response policy

2. To add a rule to the policy, run the gcloud dns response-policies rules create command:

   gcloud dns response-policies rules create RESPONSE_POLICY_RULE_NAME \
       --response-policy=RESPONSE_POLICY_NAME \
       --dns-name=pubsub.googleapis.com. \
       --local-data=name="pubsub.googleapis.com.",type="A",ttl=300,rrdatas="199.36.153.4|199.36.153.5|199.36.153.6|199.36.153.7"
   

   Replace the following:

   • RESPONSE_POLICY_RULE_NAME: a name for the response policy rule that you want to create, such as myresponsepolicyrule
   • RESPONSE_POLICY_NAME: the name of the response policy, such as myresponsepolicy

API

1. Create a response policy by URL:

   {
     kind: "dns#responsePolicy",
     response_policy_name: RESPONSE_POLICY_NAME,
     description: RESPONSE_POLICY_DESCRIPTION,
     networks: [
       {
         network_url: URL_TO_NETWORK;
       }
     ]
   }
   

   Replace the following:

   • RESPONSE_POLICY_NAME: a name for the response policy
   • RESPONSE_POLICY_DESCRIPTION: a description of the response policy

   • URL_TO_NETWORK: the URL for which you're creating the response policy

     Note: It is not necessary to refer to the parent response policy in the JSON. The parent response policy is inferred from the URL that you use.

2. Add a rule to the policy:

   {
     kind: "dns#responsePolicyRules",
     rule_name: RULE_NAME,
     dns_name: DNS_NAME,
     local_data: [
       {
         name: "pubsub.googleapis.com.",
         type: "A",
         ttl: 300,
         rrdata: ["199.36.153.4", "199.36.153.5", "199.36.153.6", "199.36.153.7"]
       }
     ]
   }
   

   Replace the following:

   • RULE_NAME: a name for the rule that you are creating, such as pubsub
   • DNS_NAME: the DNS name for which you are creating the rule, such as pubsub.googleapis.com.; note the trailing dot

Direct all names except some to restricted VIP addresses

You can set rules to exempt some DNS responses from a policy rule that covers an entire domain or a large IP address block. This concept is called passthru behavior. By using passthru behavior, you can let names that lack support for service controls be allowed past the wildcard name.

For example, you can let www.googleapis.com be allowed past the wildcard name in the example *.googleapis.com. The exact match for www takes precedence over the wildcard *.

In the following example configuration, you create a policy with a specified name and apply it to a specific VPC network. The rule allows www.googleapis.com to bypass the wildcard *.googleapis.com.

gcloud

1. To create a response policy, run the gcloud dns response-policies create command:

   gcloud dns response-policies create RESPONSE_POLICY_NAME \
       --networks=NETWORK \
       --description=DESCRIPTION
   

   Replace the following:

   • RESPONSE_POLICY_NAME: the name or ID of the response policy that you want to create, such as myresponsepolicy
   • NETWORK: a comma-separated list of network names to associate with the response policy, such as network1,network2
   • DESCRIPTION: a description of the response policy, such as My new response policy

2. To add a bypass rule to the policy, run the gcloud dns response-policies rules create command, and set the --behavior flag to bypassResponsePolicy:

   gcloud dns response-policies rules create RESPONSE_POLICY_RULE_NAME \
       --response-policy=RESPONSE_POLICY_NAME \
       --dns-name=DNS_NAME \
       --behavior=bypassResponsePolicy
   

   Replace the following:

   • RESPONSE_POLICY_RULE_NAME: a name for the response policy rule that you want to create, such as myresponsepolicyrule
   • RESPONSE_POLICY_NAME: the name of the response policy, such as myresponsepolicy
   • DNS_NAME: the DNS or domain name, such as www.googleapis.com

API

1. Create a response policy:

   {
     kind: "dns#responsePolicy",
     response_policy_name: RESPONSE_POLICY_NAME,
     description: RESPONSE_POLICY_DESCRIPTION,
     networks: [
       {
         network_url: URL_TO_NETWORK;
       }
     ]
   }
   

   Replace the following:

   • RESPONSE_POLICY_NAME: a name for the response policy, such as my-response-policy
   • RESPONSE_POLICY_DESCRIPTION: a description of the response policy, such as my response policy
   • URL_TO_NETWORK: the URL for which you're creating the response policy

2. Add a rule to the policy:

   {
     kind: "dns#responsePolicyRules",
     rule_name: RULE_NAME,
     dns_name: DNS_NAME,
     local_data: [
       {
         name: "*.googleapis.com.",
         type: "A",
         ttl: 300,
         rrdata: ["restricted.googleapis.com."]
       }
     ]
   }
   {
     kind: "dns#responsePolicyRules",
     rule_name: "www-passthru",
     dns_name: "www.googleapis.com.",
     behavior: BYPASS_RESPONSE_POLICY
   }
   

   Replace the following:

   • RULE_NAME: a name for the rule that you are creating, such as googleapis
   • DNS_NAME: the wildcard DNS name for which you are creating the rule, such as *.googleapis.com.; note the trailing dot

Configure Cloud DNS by using private zones

To configure Cloud DNS when you cannot use response policies, use private zones.

1. Create a managed private zone for your VPC network.

   gcloud beta dns managed-zones create ZONE_NAME \
    --visibility=private \
    --networks=https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/networks/NETWORK_NAME \
    --description=DESCRIPTION \
    --dns-name=googleapis.com
   

   Replace the following:

   • ZONE_NAME: a name for the zone that you are creating. For example, vpc. This name is used in each of the following steps.
   • PROJECT_ID: the ID of the project that hosts your VPC network.
   • NETWORK_NAME: the name of your VPC network.
   • DESCRIPTION: an optional, human-readable description of the managed zone.

2. Start a transaction.

   gcloud dns record-sets transaction start --zone=ZONE_NAME
   

   Replace ZONE_NAME with the name of the zone that you created in the first step.

3. Add DNS records.

   gcloud dns record-sets transaction add --name=*.googleapis.com. \
       --type=CNAME restricted.googleapis.com. \
       --zone=ZONE_NAME \
       --ttl=300
   

   Replace ZONE_NAME with the name of the zone that you created in the first step.

   gcloud dns record-sets transaction add --name=restricted.googleapis.com. \
       --type=A 199.36.153.4 199.36.153.5 199.36.153.6 199.36.153.7 \
       --zone=ZONE_NAME \
       --ttl=300
   

   Replace ZONE_NAME with the name of the zone that you created in the first step.

4. Execute the transaction.

   gcloud dns record-sets transaction execute --zone=ZONE_NAME
   

   Replace ZONE_NAME with the name of the zone that you created in the first step.

5. Optional. To enable on-premises hosts to reach the restricted VIP, complete the following steps:

   1. Create a DNS policy and enable inbound DNS forwarding to make VPC network's name resolution services externally available to systems in on-premises networks,

      gcloud beta dns policies create POLICY_NAME \
       --networks=https://www.googleapis.com/compute/v1/projects/PROJECT_ID/global/networks/NETWORK_NAME \
       --enable-inbound-forwarding \
       --description=DESCRIPTION
      

      Replace the following:

      • POLICY_NAME: a name for the policy that you are creating. For example, apipolicy.
      • PROJECT_ID: the ID of the project that hosts your VPC network.
      • NETWORK_NAME: the name of your VPC network.
      • DESCRIPTION: an optional, human-readable description of the managed zone.

   2. In your on-premises network, point your on-premises DNS to the Cloud DNS forwarder IP address. To find the forwarder IP address, use the compute addresses list command:

      gcloud compute addresses list --filter='name ~ ^dns-forwarding.*' \
       --format='csv[no-heading](address, subnetwork)'
      

      Note: Your VPC network only has a forwarder IP address if you have configured an inbound DNS forwarding policy for it. For more information about DNS policies, see the Cloud DNS server policies overview and Creating an inbound server policy.

Configure DNS with BIND

If you use BIND for DNS resolution, you can configure it to resolve Google API requests to the restricted Google APIs. Use the following example BIND configuration, which makes use of response policy zones (RPZ) to achieve this behavior:

1. Add the following lines to /etc/bind/named.conf:

   include "/etc/bind/named.conf.options";
   include "/etc/bind/named.conf.local";
   

2. Add the following lines to /etc/bind/named.conf.options:

   options {
     directory "/var/cache/bind";
   
     dnssec-validation no;
   
     auth-nxdomain no;    # conform to RFC 1035
     listen-on-v6 { any; };
     listen-on { any; };
     response-policy { zone "googleapis.zone"; };
     allow-query { any;};
   };
   

3. Add the following lines to /etc/bind/named.conf.local:

   include "/etc/bind/named.conf.default-zones";
   
   
   zone "googleapis.zone" {
     type master;
     file "/etc/bind/db.googleapis.zone";
     allow-query {none;};
   };
   

4. Add the following lines to /etc/bind/db.googleapis.zone:

   $TTL 1H
   @                       SOA LOCALHOST. noreply.localhost(1 1h 15m 30d 2h)
                           NS  LOCALHOST.
   
   
   *.googleapis.com CNAME restricted.googleapis.com.
   restricted.googleapis.com CNAME rpz-passthru.
   

Configure DNS in special cases

When you need to configure DNS in special cases, keep the following in mind:

• Custom BIND is not supported when using Dataflow. To customize DNS resolution when using Dataflow with VPC Service Controls, use Cloud DNS private zones instead of using custom BIND servers. To use your own on-premises DNS resolution, consider using a Google Cloud DNS forwarding method.

• You might need to also configure DNS for gcr.io if, for example, you're using Google Kubernetes Engine (GKE). For more information, see Setting up Container Registry for GKE private clusters.