Setting up internet NEGs

This guide uses an example to teach the fundamentals of using a custom origin in an internet network endpoint group (NEG). A custom origin is an internet endpoint that is external to Google Cloud. You can use an internet NEG as a backend for an external HTTP(S) load balancer and improve performance by using Cloud CDN caching.

The guide steps through how to configure a global external HTTP(S) load balancer with a Cloud CDN-enabled backend service that proxies to a custom origin server at backend.example.com.

In the example, the load balancer accepts HTTPS requests from clients, and proxies these requests as HTTP/2 to the custom origin. This example assumes that the origin supports HTTP/2.

Other options would be to configure a load balancer to accept HTTP or HTTP/2 requests, and use HTTPS when proxying requests to the custom origin.

Setting up an internet NEG involves:

  • Defining the internet endpoint in an internet NEG.
  • Adding an internet NEG as the backend to a backend service.
  • Defining which user traffic to map to this backend service by configuring your external HTTP(S) load balancer's URL map.

Before you begin

Before following this guide, familiarize yourself with the following:

Optional: Installing gcloud

If you prefer to work from the command line, install the gcloud command-line tool. See gcloud command-line tool overview for conceptual and installation information about the tool.

If you haven't run the gcloud command-line tool previously, first run gcloud init to initialize your gcloud directory.

Optional: Creating a new project

These instructions require a project. If you do not already have a project, set one up now.

We recommend that users with the resourcemanager.projects.create permission create a new project before following the rest of this how-to. This simplifies cleanup at the end of the guide.

Permissions

To follow this guide, you need to create an internet NEG and create or modify an external HTTP(S) load balancer in a project. You should be either a project owner or editor, or you should have both of the following Compute Engine Cloud IAM roles:

Task Required Role
Create and modify load balancer components Network Admin
Create and modify NEGs Compute Instance Admin

Configuring a load balancer with a custom origin

This guide shows you how to configure and test an internet NEG.

Setup overview

The steps in this section describe how to configure the following:

  • Creating an internet NEG and attaching a network endpoint defining your INTERNET_IP_PORT or INTERNET_FQDN_PORT endpoint.
  • Associating this NEG with the external HTTP(S) load balancer's backend service.
  • Adding the forwarding rule for this external HTTP(S) load balancer.

A sample architecture looks like this:

Typical use case for custom origins
Typical use case for custom origins

In the diagram, www.example.com has a load balancer frontend with the IP address 120.1.1.1. When there is a cache miss, user requests for /cart/id/1223515 are fetched from the custom origin by way of HTTP/2. All other incoming traffic is directed to either the Google Cloud backend service with Compute Engine VMs or to the backend bucket, based on the URL map.

You must create the following resources to set up this example:

  • A forwarding rule with the 120.1.1.1 IP address directs incoming requests to a target proxy.
    • The networkTier of the forwarding rule must be PREMIUM.
  • The target proxy checks each request against the URL map to determine the appropriate backend service for the request.
    • For custom origins, the target proxy must be TargetHttpProxy or TargetHttpsProxy. This example uses TargetHttpsProxy.
  • Cloud CDN enabled (optional) on the backend service allows caching and serving responses from Cloud CDN caches.
  • The backend service configuration directs traffic to one internet network endpoint group. This internet NEG contains the network endpoint where the external HTTP(S) load balancer sends traffic when there is a Cloud CDN cache miss.
  • This example includes a user-defined request header, which is required when the custom origin expects a particular value for the HTTP request's Host header.

Creating the NEG and internet endpoint

Console

  1. Go to the Network Endpoint Groups page in the Google Cloud Console.
    Go to the Network Endpoint Groups page
  2. Click CREATE NETWORK ENDPOINT GROUP.
  3. Enter the Name of the network endpoint group: example-fqdn-neg.
  4. For the Network endpoint group type, select Network endpoint group (Internet).
  5. Set the Default port.
  6. For the Network endpoint, select Fully qualified domain name and port.
  7. Enter backend.example.com as the FQDN.
  8. For the Default port number, enter 443.
  9. Click Create

gcloud

  1. Create an internet NEG, and set the --network-endpoint-type to internet-fqdn-port (the hostname and port where your origin can be reached):

    gcloud compute network-endpoint-groups create example-fqdn-neg \
        --network-endpoint-type="internet-fqdn-port" --global
    
  2. Add your endpoint to the NEG. If a port isn't specified, the port selection defaults to port 80 (HTTP) or 443 (HTTPS; HTTP/2) depending on the protocol configured in the backend service. Make sure to include the --global flag.

    gcloud compute network-endpoint-groups update example-fqdn-neg \
        --add-endpoint="fqdn=backend.example.com,port=443" \
        --global
    
  3. List the created internet NEG:

    gcloud compute network-endpoint-groups list --global
    
    NAME                   LOCATION  ENDPOINT_TYPE       SIZE
    example-fqdn-neg       global    INTERNET_FQDN_PORT  1
    
  4. List the endpoint within that NEG:

    gcloud compute network-endpoint-groups list-network-endpoints example-fqdn-neg \
        --global
    
    INSTANCE  IP_ADDRESS  PORT  FQDN
                                backend.example.com
    

Using an existing load balancer

The following example updates an existing load balancer.

If you don't already have an external HTTP(S) load balancer, skip this section and go to Using a new load balancer.

In the existing load balancer, the default service is a Google Cloud service. The example modifies the existing URL map by adding a path matcher that sends all requests for cart/id/1223515 to the images backend service, which is associated with the internet NEG.

Console

Create the backend service and add the internet NEG

  1. Go to the Load balancing page in the Google Cloud Console.
    Go to the Load balancing page
  2. To add the backend service to an existing load balancer, select your external HTTP(S) load balancer and click Edit.
  3. Click Backend configuration.
  4. In the Create or select a backend service pull-down menu, select Create a backend service.
  5. Set the Name of the backend service to images.
  6. For the Backend type, select Internet Network Endpoint Group.
  7. Select the protocol that you intend to use from the load balancer to the internet NEG. For this example, HTTP/2.
  8. Under New backend, select example-fqdn-neg.
  9. Click Done.
  10. Select Enable Cloud CDN.
  11. In Advanced configurations, under Custom request headers, click Add header.
    1. For the Header name, enter Host.
    2. For the Header value, enter backend.example.com.
  12. Click Create.
  13. Keep the window open to continue.

Attach the backend service to an existing URL map

  1. Click Host and path rules.
  2. The first row or rows have Google Cloud services in the right-hand column and one of them is already populated with the default rule Any unmatched (default) for Hosts and Paths.
  3. Ensure that there is row with images selected in the right-hand column. If it doesn't exist, click Add host and path rule, and select images. Populate the other fields as follows:
    1. Set Hosts to *.
    2. In the Paths field, enter /cart/id/1223515.

Review and finalize

  1. Click Review and finalize.
  2. Compare your settings to what you intended to create.
  3. If everything looks correct, click Create to create your external HTTP(S) load balancer.

gcloud

  1. Create a new backend service for the NEG.

    gcloud compute backend-services create images \
       --global \
       --enable-cdn \
       --protocol=HTTP2
    
  2. Configure the backend service to add the custom request header Host: backend.example.com to the request.

    gcloud compute backend-services update images \
       --custom-request-header "Host: backend.example.com" --global
    
  3. Use the backend-services add-backend command to add the internet NEG to the backend service.

    gcloud compute backend-services add-backend images \
      --network-endpoint-group "example-fqdn-neg" \
      --global-network-endpoint-group \
      --global
    
  4. Attach the new backend service to the load balancer's URL map by creating a new matching rule to direct requests to that backend.

    gcloud compute url-maps add-path-matcher example-url-map \
      --default-service=gcp-service-example \
      --path-matcher-name=custom-origin-path-matcher-example \
      --backend-service-path-rules='/cart/id/1223515=images'
    

    where:

    • example-url-map is the name of your existing URL map.
    • gcp-service-example is the name of an existing default backend service.
    • custom-origin-path-matcher-example is the name of this new path rule.
    • /cart/id/1223515 is the path.
    • images is the name of the new backend service with the attached internet NEG.

Using a new load balancer

The following example creates an external HTTP(S) load balancer that uses your internet NEG as the default (any host, any path) service.

If you already have an existing external HTTP(S) load balancer that you want to modify, skip this section and go to Using an existing load balancer.

Console

Create the backend service and add the internet NEG

  1. Go to the Load balancing page in the Google Cloud Console.
    Go to the Load balancing page
  2. Click Create load balancer.
  3. In HTTP(S) Load Balancing, click Start configuration.
  4. Select From Internet to my VMs.
  5. Enter a name for the new load balancer. This name becomes the name for your new URL map. For this example, call it example-url-map.
  6. Click Backend configuration.
  7. In the Create or select a backend service pull-down menu, select Create a backend service.
  8. Set the Name of the backend service to images.
  9. For the Backend type, select Internet Network Endpoint Group.
  10. Select the protocol that you intend to use from the load balancer to the internet NEG. For this example, HTTP/2.
  11. Under New backend, select example-fqdn-neg.
  12. Click Done.
  13. Select Enable Cloud CDN.
  14. In Advanced configurations, under Custom request headers, click Add header.
    1. For the Header name, enter Host.
    2. For the Header value, enter backend.example.com.
  15. Click Create.
  16. Keep the window open to continue.

Attach the backend service to a new URL map

  1. Click Host and path rules.
  2. Ensure that there is a row with images selected in the right-hand column. The first two columns are already populated with the default rule Any unmatched (default).
  3. Keep the window open to continue.

Adding the frontend

  1. Click Frontend configuration.
  2. In the Name field, enter example-forwarding-rule.
  3. In the Protocol field, select HTTPS.
  4. Click the Certificate drop-down list.
    1. If you already have a self-managed SSL certificate resource you want to use as the primary SSL certificate, select it from the drop-down menu.
    2. Otherwise, select Create a new certificate.
    3. Select Upload my certificate or Create Google managed certificate.
    4. If you selected Upload my certificate, complete these steps.
      1. Fill in a Name of example-ssl.
      2. In the appropriate fields upload your Public key certificate (.crt file), Certificate chain (.csr file), and Private key (.key file).
      3. Click Create.
    5. If you choose Create Google managed certificate, enter a Domain.
  5. To add certificate resources in addition to the primary SSL certificate resource:
    1. Click Add certificate.
    2. Select a certificate from the Certificates list or click Create a new certificate and follow the instructions above.
  6. Click Done.
  7. Keep the window open to continue.

Review and finalize

  1. Click Review and finalize.
  2. Compare your settings to what you intended to create.
  3. If everything looks correct, click Create to create your external HTTP(S) load balancer.

gcloud

  1. Create a new backend service for the NEG.

    gcloud compute backend-services create images \
       --global \
       --enable-cdn \
       --protocol=HTTP2
    
  2. Configure the backend service to add the custom request header Host: backend.example.com to the request.

    gcloud compute backend-services update images \
       --custom-request-header "Host: backend.example.com" --global
    
  3. Use the backend-services add-backend command to add the internet NEG to the backend service.

    gcloud compute backend-services add-backend images \
      --network-endpoint-group "example-fqdn-neg" \
      --global-network-endpoint-group \
      --global
    
  4. Create a URL map that specifies the backend service as the value of --default-service.

    gcloud compute url-maps create example-url-map \
     --default-service images \
     --global
    
  5. To create a self-managed SSL certificate resource:

    gcloud compute ssl-certificates create example-ssl \
        --certificate [CRT_FILE_PATH] \
        --private-key [KEY_FILE_PATH]
    
    1. To create a Google-managed SSL certificate resource:
    gcloud beta compute ssl-certificates create example-ssl \
      --domains [DOMAIN]
    
  6. Create a new target HTTPS proxy, attaching the URL map and SSL certificate.

    gcloud compute target-https-proxies create example-target-https-proxy \
      --url-map=example-url-map \
      --ssl-certificates=example-ssl \
      --global
    
  7. Create a global forwarding rule, which configures a global anycast IP address. The load balancer listens on the anycast IP address on the configured ports. Clients can connect to the backend through the Google Cloud network.

    gcloud compute forwarding-rules create example-forwarding-rule \
       --ip-protocol=TCP \
       --ports=443 \
       --global \
       --target-https-proxy=example-target-https-proxy
    

Testing

Retrieve the IP address of your forwarding rule, and use your browser or a command-line tool (for example, curl) to connect to it.

gcloud compute forwarding-rules list

For help with troubleshooting, see Troubleshooting custom origin and internet NEG issues.

What's next