Access Google APIs using Private Service Connect with consumer HTTP(S) service controls

Private Service Connect lets you connect to services outside of your Virtual Private Cloud (VPC) network using internal IP addresses that you define.

You can create a Private Service Connect endpoint with consumer HTTP(S) service controls using an internal HTTP(S) load balancer, which lets you maintain consistent policies across multiple service producers.

See Using Private Service Connect to access Google APIs with consumer HTTP(S) service controls for more information.

An internal HTTP(S) load balancer used for Private Service Connect can be reached from Shared VPC networks and from connected networks.

Roles

The Compute Load Balancer Admin role (roles/compute.loadBalancerAdmin) contains the permission required to perform the tasks described in this guide.

Supported services

See the list of supported regional service endpoints.

Before you begin

  • Create a proxy-only subnet if you don't already have one. There must be a single proxy-only subnet in the VPC network and region where you intend to create the internal HTTP(S) load balancer. This subnet is used by all internal HTTP(S) load balancers within that network and region.

  • If you don't already have them, create a private key and a certificate for your domain. See step 1 of using self-managed SSL certificates. You need a private key and certificate to create the target HTTPS proxy when you configure the load balancer.

Create a network endpoint group

For each service that you want to make available using the load balancer, create a Private Service Connect NEG.

gcloud beta compute network-endpoint-groups create NEG_NAME \
  --network-endpoint-type=private-service-connect \
  --psc-target-service=TARGET_SERVICE \
  --region=REGION

Replace the following:

  • NEG_NAME: a name for the network endpoint group.

  • TARGET_SERVICE: the target service that you want to connect to. See the list of supported regional service endpoints.

  • REGION: the region to create the network endpoint group in. The region must be the same region as the service you want to connect to.

Configure the load balancer

All load balancer components must be created in the same region as the Private Service Connect NEG.

  1. For each NEG that you have created, create a backend service.

    gcloud compute backend-services create BACKEND_SERVICE_NAME \
      --load-balancing-scheme=INTERNAL_MANAGED \
      --protocol=HTTPS \
      --region=REGION
    

    Replace the following:

    • BACKEND_SERVICE_NAME: the name of the backend service.
    • REGION: the region to create the backend service in.
  2. For each backend service that you have created, add the corresponding NEG to the backend service.

    gcloud beta compute backend-services add-backend BACKEND_SERVICE_NAME \
      --network-endpoint-group=NEG_NAME \
      --region=REGION
    

    Replace the following:

    • BACKEND_SERVICE_NAME: the name of the backend service.
    • NEG_NAME: the name of the network endpoint group.
    • REGION: the region for the backend.
  3. Create a URL map for the load balancer.

    A URL map must reference a default backend service. If you're configuring your load balancer with one backend service, set that backend service as the default. If you're configuring your load balancer to use multiple backend services, referenced by host rules and path matchers that you create in the next step, pick one of the backend services to be the URL map default.

    gcloud compute url-maps create URL_MAP_NAME \
      --default-service=DEFAULT_BACKEND_SERVICE_NAME \
      --region=REGION
    

    Replace the following:

    • URL_MAP_NAME: a name for the URL map.

    • DEFAULT_BACKEND_SERVICE_NAME: the name of the load balancer's default. The default is used when no host rule matches the requested hostname.

    • REGION: the region to create the URL map in.

  4. Add additional backend services to the URL map.

    If your URL map needs to reference two or more backend services, complete the following steps. If your URL map only references one backend service, skip this step.

    1. For each backend service, add a path matcher. You must create one path matcher for each backend service.

      gcloud compute url-maps add-path-matcher URL_MAP_NAME \
      --path-matcher-name=PATH_MATCHER \
      --default-service=BACKEND_SERVICE_NAME \
      --region=REGION
      

      Replace the following:

      • URL_MAP_NAME: the name of the URL map.
      • PATH_MATCHER: a name for the path matcher.
      • BACKEND_SERVICE_NAME: the name of the backend service.
      • REGION: the region of the URL map.
    2. For each hostname, add a host rule.

      Each host rule can reference only one path matcher, but two or more host rules can reference the same path matcher.

      gcloud compute url-maps add-host-rule URL_MAP_NAME \
      --hosts=HOST \
      --path-matcher-name=PATH_MATCHER \
      --region=REGION
      

      Replace the following:

      • URL_MAP_NAME: the name of the URL map.
      • HOST: the hostname to send requests to for this service. For example, spanner.example.com.
      • PATH_MATCHER: the name of the path matcher.
      • REGION: the region of the URL map.
  5. Create the target HTTPS proxy.

    Create a regional SSL certificate resource using the gcloud compute ssl-certificates create command. Google-managed certificates aren't supported with internal HTTP(S) load balancers.

    gcloud compute ssl-certificates create CERTIFICATE \
      --certificate=LB_CERT \
      --private-key=LB_PRIVATE_KEY \
      --region=REGION
    

    Replace the following:

    • CERTIFICATE: a name for the certificate.

    • LB_CERT: the path to the PEM-formatted certificate file for your self-managed certificate.

    • LB_PRIVATE_KEY: the path to the PEM-formatted private key file for your self-managed certificate.

    • REGION: the region for the certificate.

    Use the regional SSL certificate resource to create a target HTTPS proxy with the gcloud compute target-https-proxies create command.

    gcloud compute target-https-proxies create PROXY_NAME \
        --url-map=URL_MAP_NAME \
        --region=REGION \
        --ssl-certificates=CERTIFICATE
    

    Replace the following:

    • PROXY_NAME: a name for the target HTTPS proxy.
    • URL_MAP_NAME: the name of the URL map.
    • REGION: the region for the certificate resource.
    • CERTIFICATE: the name of the certificate resource.
  6. Create the forwarding rule.

    Create the forwarding rule using the gcloud beta compute forwarding-rules create command.

    gcloud beta compute forwarding-rules create FWD_RULE \
      --load-balancing-scheme=INTERNAL_MANAGED \
      --network=NETWORK \
      --address=FWD_RULE_IP_ADDRESS \
      --ports=443 \
      --region=REGION \
      --target-https-proxy=PROXY_NAME \
      --target-https-proxy-region=PROXY_REGION \
      [--service-directory-registration=SD_SERVICE_NAME]
    

    Replace the following:

    • FWD_RULE: a name for the forwarding rule.

    • NETWORK: the network to create the forwarding rule in.

    • FWD_RULE_IP_ADDRESS: the IP address that you want to use for the forwarding rule.

    • REGION: the region for the forwarding rule.

    • PROXY_NAME: the name of the target HTTPS proxy.

    • PROXY_REGION: the region of the target HTTPS proxy.

    • SD_SERVICE_NAME: the URI of the Service Directory service that you want to register the load balancer with, in this format: projects/PROJECT/locations/REGION/namespaces/NAMESPACE_NAME/services/SERVICE_NAME. Specifying --service-directory-registration is optional.

Configure DNS records

Configure DNS records for each host that you added to the URL map, pointing to the IP address of the forwarding rule.

For example, say you have configured Private Service Connect with HTTP(S) consumer service controls with the following configurations:

  • A Private Service Connect NEG that uses target service asia-east1-spanner.googleapis.com.

  • An internal HTTP(S) load balancer that uses that Private Service Connect NEG as a backend.

  • A URL map that defines a host rule for spanner.example.com.

For this configuration to work correctly, you create a DNS record that points spanner.example.com to the IP address of the forwarding rule.

Any requests sent to spanner.example.com are sent to the load balancer, which forwards the request to asia-east1-spanner.googleapis.com.

If you're using Cloud DNS to manage DNS, see adding a DNS record.

Verifying the configuration

  1. Create the VM instance.

    gcloud compute instances create VM_NAME \
      --network=NETWORK \
      --image-project=debian-cloud --image-family=debian-9 \
      --zone=ZONE
    

    Replace the following:

    • VM_NAME: a name for the virtual machine.

    • NETWORK: the network for the VM.

    • ZONE: the zone for the VM.

  2. Connect to the VM.

    gcloud compute ssh VM_NAME --zone=ZONE
    
  3. Use curl to verify the configuration. This command sets the Host header and bypasses DNS resolution by specifying a user-defined IP address. You can omit the port if you are using the default port for the protocol, for example using port 443 for HTTPS.

    If needed, you can skip certificate validation using the -k flag. You might need to skip validation if you used a self-signed certificate to configure the target HTTPS proxy, or if the VM does not have the certificate of the certificate authority that signed your certificate.

    curl [-k] -s 'https://HOST:443/RESOURCE_URI' \
        -H 'Host: HOST' \
        --connect-to HOST:443:FWD_RULE_IP_ADDRESS:443
    

    Replace the following:

    • HOST: the hostname that is configured in the URL map.

    • RESOURCE_URI: the rest of the URI of the resource that you want to use for verification. For example, if the load balancer is forwarding requests to a Cloud Spanner regional endpoint, you could use v1/projects/<var>PROJECT_ID</var>/instanceConfigs.

    • FWD_RULE_IP_ADDRESS: the IP address assigned to the forwarding rule.

Using Private Service Connect from on-premises hosts

If your on-premises network is connected to a VPC network, you can connect to the internal HTTP(S) load balancer which provides Private Service Connect using consumer HTTP(S) service controls.

  • Your on-premises network must be connected to the VPC network containing your internal HTTP(S) load balancer using Cloud VPN tunnels or Cloud Interconnect attachments (VLANs) in the same region as the internal HTTP(S) load balancer.

  • The on-premises network must have appropriate routes to your internal HTTP(S) load balancer. Ensure that each Cloud Router managing the BGP session for the Cloud VPN tunnel or Cloud Interconnect attachment (VLAN) has been configured to advertise the primary IP address range of the subnet used by your internal HTTP(S) load balancer forwarding rule. Cloud Routers advertise subnet routes by default.

  • You must configure on-premises systems so that the hostnames in the URL map of your internal HTTP(S) load balancer resolve to the internal HTTP(S) load balancer forwarding rule IP address. You can create the DNS records in on-premises name servers or you can use Cloud DNS.

    If you've created the DNS records using Cloud DNS managed private zones, complete the following steps:

    • Create an inbound server policy in the VPC network to which your on-premises network connects.

    • Identify the inbound forwarder entry points, in the regions where your Cloud VPN tunnels and Cloud Interconnect attachments (VLANs) are located, in the VPC network to which your on-premises network connects.

    • Configure on-premises DNS name servers to forward DNS requests for the hostnames in the URL map to a Cloud DNS inbound forwarder entry point.

Figure 1. By configuring Private Service Connect, Cloud Router, and on-premises hosts, you can connect to Google APIs and services using regional service endpoints.