Using protocol forwarding

Compute Engine supports protocol forwarding, which lets you create forwarding rule objects that can send packets to target instances that do not have a NAT policy applied to them. Each target instance contains a single virtual machine (VM) instance that receives and handles traffic from the corresponding forwarding rules.

Protocol forwarding can be used in a number of scenarios, including:

  • External protocol forwarding

    You can set up multiple forwarding rules to point to a single target instance, allowing you to use multiple external IP addresses with one VM instance. You can use this in scenarios where you may want to serve data from just one VM instance, but through different external IP addresses, or different protocols and ports. This is especially useful for setting up SSL virtual hosting. You can also switch regional forwarding rules from using target instances to backend services and vice versa.

  • Virtual private networks (VPN)

    Refer to:

  • Internal protocol forwarding

    You can use protocol forwarding for internal regional forwarding rules with VPC subnet ranges. Use this feature to configure internal forwarding rules that send TCP or UDP traffic to a target instance in the same region. You can also switch internal regional forwarding rules from using target instances to backend services and vice versa.

For external protocol forwarding and virtual private networks (VPN), Compute Engine supports protocol forwarding for the following protocols:

For internal protocol forwarding, only TCP and UDP are supported:

Protocol forwarding is charged at the same rates as the load balancing service. For more information, read the pricing page.

Permissions

Make sure you have an Identity and Access Management (IAM) role that allows you to create forwarding rules. For more information, see the permissions section in the forwarding rules document.

Quickstart

This quickstart assumes you are familiar with bash.

To create a forwarding rule that sends traffic to a single instance, you must:

  1. Create a target instance.

    Your target instance is a single VM instance, but this instance can exist at the time you create the target instance, or it can be created afterward.

  2. Create a forwarding rule.

    Your target instance must exist before you create a forwarding rule. If incoming packets match the IP, protocol, and (if applicable) the port range that is being served by your forwarding rule, the forwarding rule directs that traffic to your target instance.

The rest of this quickstart demonstrates the above steps end-to-end by:

  1. Setting up an Apache server on a VM instance.
  2. Creating a target instance and corresponding forwarding rules.
  3. Send traffic to a single target instance.

At the end of this quickstart, you should know how to set up protocol forwarding from multiple forwarding rules to a single target instance.

Set up a VM instance and install Apache

To create a single VM instance with Apache installed:

  1. Create some startup scripts for your new instance.

    Depending on your operating system, your startup script contents might differ.

    • If you plan to use a Debian image on your instance, run the following command: 

      me@local:~$ echo "sudo apt update && sudo apt -y install apache2 && mkdir -p /var/www1 &&
mkdir -p /var/www2 && mkdir -p /var/www3 && hostname > /var/www/html/index.html &&
echo w1 > /var/www1/index.html && echo w2 > /var/www2/index.html && echo w3 > /var/www3/index.html" > 
      
$HOME/pf_startup.sh

    • If you plan to use a CentOS image on your instance, run the following command: 

      me@local:~$ echo "sudo yum -y install httpd && sudo service httpd restart && mkdir -p /var/www1 &&
mkdir -p /var/www2 && mkdir /var/www3 && hostname > /var/www/html/index.html &&
echo w1 > /var/www1/index.html && echo w2 > /var/www2/index.html && echo w3 > /var/www3/index.html" > 
      
$HOME/pf_startup.sh

  2. Create a tag for your future VM, to help apply a firewall to it later:

    me@local:~$ TAG="www-tag"

  3. Create a VM instance to handle traffic for your forwarding rules:

    gcloud compute instances create pf-instance \
   --image-project=debian-cloud \
   --image-family=debian-10 \
   --tags=$TAG \
   --metadata-from-file=startup-script=$HOME/pf_startup.sh

  4. Create a firewall rule to allow external traffic to this VM instance:

    gcloud compute firewall-rules create www-firewall \
   --target-tags=$TAG \
   --allow tcp

You have successfully set up a VM instance. Now you can start setting up your protocol forwarding configuration.

Create a target instance and corresponding forwarding rules

  1. Create a target instance.

    Target instances contain a single VM instance that receives and handles traffic from a forwarding rule. Target instances do not have a NAT policy, so you can use them to set up your own VPN connections using IPsec protocols directly.

    You must create a target instance before you can create a forwarding rule object because forwarding rules must reference an existing target resource. You cannot create a forwarding rule that directs traffic to a non-existing target resource. For this example, create a target instance as follows:

    gcloud compute target-instances create pf-target-instance \
   --instance=pf-instance

  2. Create forwarding rule objects.

    A forwarding rule object directs traffic by IP protocol and port to a specified target instance. For more information, see forwarding rules.

    For this example, the following commands create three forwarding rules, each with an ephemeral IP address that forwards TCP traffic to your target instance. Optionally, if you have some static external IP addresses, you can use them with these forwarding rules by specifying the --address IP_ADDRESS flag.

    gcloud compute forwarding-rules create pf-rule1 \
   --ip-protocol=TCP \
   --load-balancing-scheme=EXTERNAL \
   --network-tier=PREMIUM \
   --ports=80 \
   --target-instance=pf-target-instance

    gcloud compute forwarding-rules create pf-rule2 \
   --ip-protocol=TCP \
   --load-balancing-scheme=EXTERNAL \
   --network-tier=PREMIUM \
   --ports=80 \
   --target-instance=pf-target-instance

    gcloud compute forwarding-rules create pf-rule3 \
   --ip-protocol=TCP \
   --load-balancing-scheme=EXTERNAL \
   --network-tier=PREMIUM \
   --ports=80 \
   --target-instance=pf-target-instance

You can start sending traffic to your target instance.

Send traffic to your instance

  1. Get the external IP addresses of your new forwarding rules.

    Run gcloud compute forwarding-rules list to get the external IP addresses of your forwarding rules. For example, the following table lists the ephemeral IP addresses that are allocated for the forwarding rules created earlier.

    If you opted to use reserved IP addresses, they are listed here in place of the ephemeral IP addresses.

    gcloud compute forwarding-rules list

    NAME     REGION      IP_ADDRESS     IP_PROTOCOL TARGET
pf-rule1 us-central1 [ADDRESS_1]    TCP         us-central1-a/targetInstances/pf-target-instance
pf-rule2 us-central1 [ADDRESS_2]    TCP         us-central1-a/targetInstances/pf-target-instance
pf-rule3 us-central1 [ADDRESS_3]    TCP         us-central1-a/targetInstances/pf-target-instance

    Make a note of the IP addresses for the next step.

  2. Configure the VM instance's Apache virtual hosts to serve different information based on the destination URL.

    First, connect to your instance:

    gcloud compute ssh pf-instance

    Then, edit the /etc/apache2/sites-enabled/000-default.conf file and add the following lines. For VirtualHostcd .. lines, use the IP addresses you saw when you ran the previous step.

    <VirtualHost [ADDRESS_1]>
 DocumentRoot /var/www1
 <Directory /var/www1>
  Require all granted
 </Directory>
</VirtualHost>
<VirtualHost [ADDRESS_2]>
 DocumentRoot /var/www2
 <Directory /var/www2>
  Require all granted
 </Directory>
</VirtualHost>
<VirtualHost [ADDRESS_3]>
 DocumentRoot /var/www3
 <Directory /var/www3>
  Require all granted
 </Directory>
</VirtualHost>

    Lastly, restart Apache:

    user@myinst:~$ sudo /etc/init.d/apache2 restart

  3. Try sending some traffic to your instance.

    On your local machine, make a request to the external IP addresses served by the forwarding rules you created.

    Use curl to send traffic to the IP addresses. The response returns w1, w2, or w3 depending on the IP address.

    me@local:~$curl ADDRESS_1
w1

    me@local:~$ curl ADDRESS_2
w2

    me@local:~$ curl ADDRESS_3
w3

    Your protocol forwarding configuration is ready.

Forwarding rules

Forwarding rules work in conjunction with target instances to support protocol forwarding features. While forwarding rules are also used with other products, such as Cloud Load Balancing, this section only covers forwarding rules as part of protocol forwarding to an individual VM.

Forwarding rule resources live in the Forwarding Rules collection. Each forwarding rule matches a particular IP address, protocol, and optionally, port range to a single target instance. When traffic is sent to an external IP address that is served by a forwarding rule, the forwarding rule directs that traffic to the target instances.

Some points to keep in mind when working with forwarding rules:

  • The name of a forwarding rule must be unique in this project, from 1-63 characters long and match the regular expression: [a-z]([-a-z0-9]*[a-z0-9])? which means the first character must be a lowercase letter, and all following characters must be a dash, lowercase letter, or digit, except the last character, which cannot be a dash.

  • If you do not specify a forwarding rule protocol, the default TCP is used. Other supported protocols are UDP ESP, AH, SCTP, ICMP, and L3_DEFAULT.

    The L3_DEFAULT forwarding rule protocol is to forward all TCP, UDP, ESP and ICMP traffic. L3_DEFAULT is only supported when the load balancing scheme is EXTERNAL.

  • Port ranges can only be specified for TCP, UDP, and SCTP protocols.

  • All ports must be configured if:

    • You are using L3_DEFAULT forwarding rules.

    • You expect fragmented UDP packets. Use only one UDP or L3_DEFAULT forwarding rule per IP address, and configure the forwarding rule to accept traffic on all ports. This ensures that all fragments arrive at the same forwarding rule even if they don't have the same destination port.

    To configure all ports, either set --ports=ALL using gcloud, or set allPorts to True using the API.

Add a forwarding rule

To add a new forwarding rule, you can use the gcloud compute forwarding-rules create command or create a HTTP POST request to the Forwarding Rules collection.

gcloud

To add a new forwarding rule using gcloud:

gcloud compute forwarding-rules create FORWARDING_RULE_NAME \
    --load-balancing-scheme=SCHEME \
    --region=REGION \
    --target-instance-zone=ZONE \
    --ip-protocol=PROTOCOL \
    --ports=PORTS \
    --target-instance=TARGET_INSTANCE_NAME

If you omit the --target-instance-zone flag, the Google Cloud CLI prompts you to choose a zone if you did not set the compute/zone property with gcloud config set compute/zone. For more information, see Set default properties.

If you omit the --region flag, the Google Cloud CLI prompts you to choose a region if you did not set the compute/region property with gcloud config set compute/region. For more information, see Set default properties.

For a complete description of the flags you can use, see the forwarding rules create command, or type gcloud compute forwarding-rules create --help.

API

To add a forwarding rule using the API, perform a HTTP POST request to the following URI:

https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/us-central1/forwardingRules

Your request body should contain the following fields:

{
  "name": "example-forwarding-rule",
  "IPAddress": "`10.1.1.1",
  "IPProtocol": "TCP",
  "portRange": "80",
  "target": "zones/us-central1-f/targetInstances/example-target-instances"
}

For information about listing forwarding rules, getting information about a specific forwarding rule, and deleting forwarding rules, see the gcloud CLI and API reference pages:

Target instances

A Target Instance resource contains one VM instance that handles traffic from one or more forwarding rules and is ideal for forwarding certain types of protocol traffic that should be managed by a single source (e.g., ESP and AH), but you can also use a target instance for TCP and UDP protocols. Target instances do not have a NAT policy applied to them, so they can be used for traffic that requires non-NAT'ed IPsec traffic for virtual private networks (VPN).

Target instances must be in the same region as the forwarding rule. Target instances must also be in the same zone as the VM instance. For example, if your forwarding rule is in us-central1 and the instance you want to use is in us-central1-a, the target instance must be in us-central1-a. If the instance is in us-central1-b, the target instance must also be in us-central1-b.

Add a target instance

To add a new target instance, you can use the gcloud compute target-instances command or create an HTTP POST request to the targetInstances collection. You cannot create target instance resources by using the console. The following example shows creating a target instance by using the Google Cloud CLI:

gcloud compute target-instances create TARGET_INSTANCE \
    --instance=INSTANCE \
    --network=NETWORK

If you omit the --zone flag, the Google Cloud CLI prompts you to choose a zone if you did not set the compute/zone property with gcloud config set compute/zone. For more information, see Set default properties.

You can use the --network flag to specify the network that this target instance should use to forward traffic. This is useful for target instances with multiple NICs. If the --network flag is not specified, traffic will be forwarded to the network that the default network interface (nic0) belongs to.

For a complete description of the flags you can use, see the create command, or type gcloud compute target-instances create --help.

In the API, make an HTTP POST request to the following URI:

https://compute.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/targetInstances

With the following request body:

body = {
  "name": "example-target-instance",
  "instance": "zones/us-central1-f/instances/INSTANCE",
  "network": "projects/PROJECT_ID/global/networks/NETWORK
}

For information about listing target instances, getting information about a specific target instance, and deleting target instances, see the gcloud CLI and API reference pages:

Protocol forwarding

Protocol forwarding can be used for regional forwarding rules. Use this feature to configure forwarding rules that send traffic to a target instance in the same region. You can also easily switch regional forwarding rules from using target instances to backend services and vice versa.

A target instance contains one VM instance that handles traffic from one or more forwarding rules. A forwarding rule cannot point to multiple target instances or backend services.

Protocol forwarding can be used for both internal and external regional forwarding rules. The following diagram shows an example architecture with an internal forwarding rule.

Internal forwarding rule with a target instance (click to enlarge).
Internal forwarding rule with a target instance (click to enlarge)

Use protocol forwarding with forwarding rules in the following circumstances.

  • You want to deploy a single backend instance for your service and you want to manage health checks and other aspects yourself.
  • You want to preserve a forwarding rule IP address and you want to change the target instance to which the forwarding rule points.
  • You want to keep your deployment stable by having the ability to move easily to multiple backend instances (backend service) from a single instance (target instance) without changing your forwarding rule IP address.

Transition between a target instance and a backend service

You can update a forwarding rule from directing traffic to a target instance with a single VM instance to directing traffic to a backend service or the reverse. When you update a forwarding rule in this way, the forwarding rule IP address is preserved. Existing connections might be interrupted during this transition because the change enables or disables load balancing for backend service instances.

Transition from a backend service to a target instance (click to enlarge).
Transition from a backend service to a target instance (click to enlarge)

Updating the target of a forwarding rule

To change from pointing to a target instance to pointing to a regional backend service:

gcloud compute forwarding-rules set-target my-forwarding-rule \
    --backend-service=my-backend-service

You can then transition back to a target instance:

gcloud compute forwarding-rules set-target my-forwarding-rule \
    --target-instance=my-target-instance

Enforcing protocol forwarding settings across a project, folder, or organization

Use an organization policy to restrict the protocol forwarding types that can be created in your organization. Set the following organization policy constraint:

constraints/compute.restrictProtocolForwardingCreationForTypes

When you set the compute.restrictProtocolForwardingCreationForTypes constraint, you specify the protocol forwarding types that are disallowed. For a list of available types, see Restrict protocol forwarding creation.

Setting up an organization policy

Console

To set a protocol forwarding organization policy from the console, complete the following steps:

  1. In the Google Cloud console, go to the Organization policies page.

    Go to the Organization policies page

  2. In the Filter field, type protocol and select constraints/compute.restrictProtocolForwardingCreationForTypes.
  3. Click Restrict Protocol Forwarding Based on type of IP Address.
  4. Click Edit to edit your existing protocol forwarding policy constraints.
  5. To create a custom policy, select Customize.
  6. After making any changes, click Save to apply the constraint settings.

gcloud

To set a protocol forwarding organization policy, use the gcloud resource-manager org-policies enable-enforce command.

  1. Find your organization ID.

    gcloud organizations list

  2. Create the policy file, as shown in the following examples.

    List the values to be denied

    {
  "constraint": "constraints/compute.restrictProtocolForwardingCreationForTypes",
   "listPolicy": {
     "deniedValues": [
       "INTERNAL",
       "EXTERNAL"
     ]
   }
 }

    Allow internal forwarding rules

    {
  "constraint": "constraints/compute.restrictProtocolForwardingCreationForTypes",
   "listPolicy": {
     "deniedValues": [
       "EXTERNAL"
     ]
   }
 }

    Deny all forwarding rules

    {
  "constraint": "constraints/compute.restrictProtocolForwardingCreationForTypes",
  "listPolicy": {
    "allValues": "DENY"
  }
}

  3. Set the constraint in your organization. Replace ORGANIZATION_ID with your organization ID.

    gcloud resource-manager org-policies set-policy POLICY_FILE \
    --organization=ORGANIZATION_ID

    You can also apply the a protocol forwarding organization policy to a folder or a project with the --folder or the --project flags, and the folder ID and project ID, respectively.

    For folders, run the following command:

    gcloud resource-manager org-policies set-policy POLICY_FILE \
    --folder=FOLDER_ID

    For projects, run the following command:

    gcloud resource-manager org-policies set-policy POLICY_FILE \
    --project=PROJECT_ID

    Replace the following:

After you set the policy, the policy is enforced when adding the respective Google Cloud forwarding rules.

The constraint is not enforced on existing protocol forwarding configurations.

If you attempt to create protocol forwarding of a type that violates the constraint, the attempt fails and an error message is generated. The error message has the following format:

Constraint constraints/compute.restrictProtocolForwardingCreationForTypes
violated for projects/PROJECT_NAME. Forwarding Rule projects/PROJECT_NAME/region/REGION/forwardingRules/FORWARDING_RULE_NAME
of type SCHEME is not allowed.

If you set multiple restrictProtocolForwardingCreationForTypes constraints at different resource levels, and if you set the inheritFromParent field to true, then the constraints are enforced hierarchically.

To learn more about setting organization policies, including descriptions of the available options, see Creating and managing organization policies and Using constraints.

Limitations

  • Health checks are not supported with target instances. You must use ensure that the necessary software is running and operational on the VM referenced by the target instance.
  • When you use target instances with internal forwarding rules, the only protocols supported are TCP and UDP.

Quotas and limits

  • You can configure 100 internal forwarding rules pointing to target instances per network.

  • You can have 5 ports per internal forwarding rule.

What's next?