Protocol forwarding overview

Protocol forwarding uses a regional forwarding rule to deliver packets of a specific protocol to a single virtual machine (VM) instance. The forwarding rule can have an internal or an external IP address. Protocol forwarding delivers packets while preserving the destination IP address of the forwarding rule. The forwarding rule references an object called a target instance, which, in turn, references a single VM instance.

You can use protocol forwarding to do the following:

Provide an IP address which can be moved from one instance to another by either changing the VM referenced by the target instance object or by changing the target instance referenced by the forwarding rule.
Forward packets to different VMs based on protocol and port. Two forwarding rules can share the same IP address as long as their port and protocol information is unique.
(External protocol forwarding only) Define additional external IP addresses for a given network interface. Unlike a network interface with a 1:1 NAT configuration for its external IPv4 address, protocol forwarding preserves the destination IP address of the forwarding rule.
Send packets whose source IP addresses match the forwarding rule's IP address.

Protocol forwarding is different from a pass-through load balancer in the following ways:

No load balancing. A target instance only distributes packets to a single VM.
No health check. Unlike a backend service, a target instance does not support a health check. You must use other means to ensure that the necessary software is running and operational on the VM referenced by the target instance.

Architecture

Protocol forwarding uses regional external or regional internal forwarding rules and a zonal target instance object. The target instance and the VM it references must be located in a zone in the forwarding rule's region.

External protocol forwarding. You can set up multiple forwarding rules to point to a single target instance, which lets you use multiple external IP addresses with one VM instance. You can use this in scenarios where you may want to serve data from just one VM instance, but through different external IP addresses or different protocols and ports. This is especially useful for setting up SSL virtual hosting. External protocol forwarding can handle connections from IPv6 clients.

External protocol forwarding supports the following protocols: AH, ESP, GRE, ICMP, ICMPv6, SCTP, TCP, and UDP

The following diagram shows an example of external protocol forwarding architecture. To learn how to set this up, see Set up external protocol forwarding.

External protocol forwarding architecture
Internal protocol forwarding. Internal protocol forwarding uses either a regional internal IPv4 address (from the primary IPv4 address range of a subnet) or a regional internal IPv6 address (from the IPv6 address range of a subnet).

Internal protocol forwarding supports the TCP and UDP protocols.

The following diagram shows an example of internal protocol forwarding architecture. To learn how to set this up, see Set up internal protocol forwarding.

Internal protocol forwarding architecture

With internal protocol forwarding, you can change the target of a forwarding rule to switch between a target instance and a backend service of a pass-through load balancer. For details, see Switch between a target instance and a backend service.

Forwarding rules

Each forwarding rule matches an IP address, protocol, and optionally, port information (if specified and if the protocol supports ports). When a forwarding rule references a target instance, Google Cloud routes packets that match the forwarding rule's address, protocol, and port specification to the VM referenced by the target instance.

Internal protocol forwarding:
- IPv4 address support: A regional internal IPv4 address (reserved static or ephemeral) from the primary IPv4 range of a subnet.
- IPv6 address support: The forwarding rule references a /96 range of IP addresses from the subnet's /64 internal IPv6 address range. The subnet must be a dual-stack subnet with the ipv6-access-type set to INTERNAL. Internal IPv6 addresses are available only in Premium Tier. Reserving a regional internal IPv6 address is supported only for instances, so you must use an ephemeral IPv6 address for the forwarding rule.
- Protocol options: TCP(default) and UDP.
- Port specification options: a list of up to five contiguous or non-contiguous ports or all ports.
External protocol forwarding:
- IPv4 address support: The forwarding rule references a single regional external IPv4 address. Regional external IPv4 addresses come from a pool unique to each Google Cloud region. The IP address can be a reserved static address or an ephemeral address.
- IPv6 address support: The forwarding rule references a /96 range of IP addresses from the subnet's /64 external IPv6 address range. The subnet must be a dual-stack subnet with the ipv6-access-type set to EXTERNAL. External IPv6 addresses are available only in Premium Tier. The IPv6 address range can be a reserved static address or an ephemeral address.
- Protocol options: AH, ESP, ICMP, SCTP, TCP (default), UDP, and L3_DEFAULT :
  - The L3_DEFAULT forwarding rule protocol setting can be used to serve all IP protocol traffic.
  - IPv6 forwarding rules don't support the ICMP protocol setting because the ICMP protocol only supports IPv4 addresses. To serve ICMPv6 and GRE traffic, set the forwarding rule protocol to L3_DEFAULT.
- Port specification options: a contiguous port range or all ports.

Keep the following points in mind when working with forwarding rules:

For protocol forwarding, a forwarding rule can only reference a single target instance.
For internal passthrough Network Load Balancers and backend service-based external passthrough Network Load Balancers, a forwarding rule can only reference a single backend service.
You can switch between internal protocol forwarding and an internal passthrough Network Load Balancer without deleting and re-creating the forwarding rule. To switch between external protocol forwarding and a backend service-based external passthrough Network Load Balancer, you must delete and re-create the forwarding rule. For details, see Switch between a target instance and a backend service.
Port information can only be specified for protocols that have a concept of port: TCP, UDP, or SCTP.
The L3_DEFAULT protocol option forwards all AH, ESP, GRE, ICMP, ICMPv6, SCTP, TCP, and UDP protocols. For the TCP, UDP, and SCTP protocols, L3_DEFAULT forwards all ports.
If you expect fragmented UDP packets, do one of the following to ensure that all fragments (including those without port information) are delivered to the instance:
- Use a single L3_DEFAULT forwarding rule, or
- Use a single UDP forwarding rule configured to forward all ports.

Target instances

A target instance is a zonal resource that references one VM instance in the same zone. The forwarding rule that references the target instance must be in the region containing the target instance's zone. Because a target instance doesn't have a Cloud NAT policy applied to it, it can be used for IPsec traffic that can't traverse NAT.

Multi-NIC support

A target instance supports specifying the network interface (NIC) of the VM instance it references. Use the --network flag to specify the name of a VPC network where the referenced VM has a NIC:

If you omit the --network flag, the target instance delivers packets to the nic0 interface of the VM it references.
If you use the --network flag, the VM that the target instance references must have a NIC in that VPC network.
For internal protocol forwarding, the subnet used by the forwarding rule must be located in the VPC network used by the target instance's network interface.

IPv6 support

If you want the external protocol forwarding deployment to support IPv6 traffic, the VM instance must be configured in a dual-stack subnet that is in the same region as the IPv6 forwarding rule. You can use a subnet with the ipv6-access-type set to either EXTERNAL or INTERNAL for the VM instance. Using a subnet with ipv6-access-type set to INTERNAL requires you to use a separate dual-stack subnet with ipv6-access-type set to EXTERNAL for the external forwarding rule. For instructions, see Add a dual-stack subnet.

Additionally, the VM instance itself must be a dual-stack instance. Set the VM's stack-type to IPv4_IPv6. The VM inherits the ipv6-access-type setting (either EXTERNAL or INTERNAL) from the subnet. For instructions, see Create a VM and enable IPv6. If you want to use an existing VM, update the VM to be dual-stack by using the gcloud compute instances network-interfaces update command.

IP addresses for request and return packets

When a target instance receives a packet from a client, the request packet's source and destination IP addresses are as shown in this table.

**Table 1.** Source and destination IP addresses for request packets
Protocol forwarding type	Source IP address	Destination IP address
External protocol forwarding	The external IP address associated with a Google Cloud VM or an external IP address of a client on the internet.	The IP address of the forwarding rule.
Internal protocol forwarding	A client's internal IP address; for Google Cloud clients, the primary internal IPv4 address or IPv6 address or an IPv4 address from an alias IP range of a VM's network interface.	The IP address of the forwarding rule.

Software running on the target instance VMs should be configured to do the following:

Listen on (bind to) the forwarding rule IP address or any IP address (0.0.0.0 or ::).
If the forwarding rule's protocol supports ports, then listen on (bind to) a port that's included in the forwarding rule.

Return packets are sent directly from the target instance to the client. The response packet's source and destination IP addresses depend on the protocol:

TCP is connection-oriented. Target instances must reply with packets that have source IP addresses that match the forwarding rule's IP address. This ensures that the client can associate the response packets with the appropriate TCP connection.
AH, ESP, GRE, ICMP, ICMPv6, and UDP are connectionless. Target instances can send response packets which have source IP addresses that either match the forwarding rule's IP address, or match any IP address assigned to the VM's NIC in the same VPC network as the forwarding rule. Practically speaking, most clients expect the response to come from the same IP address to which they sent packets.

The following table summarizes sources and destinations for return packets:

**Table 2.** Source and destination IP addresses for return packets
Traffic type	Source IP address	Destination IP address
TCP	The IP address of the forwarding rule.	The request packet's source IP address.
AH, ESP, GRE, ICMP, ICMPv6, and UDP^*	For most use cases, the IP address of the forwarding rule.^†	The request packet's source IP address.

^* AH, ESP, GRE, ICMP, and ICMPv6 are only supported with external protocol forwarding.

^† With internal protocol forwarding, it is possible to set the response packet's source to the VM NIC's primary internal IPv4 address or IPv6 address or an alias IP address range. If the VM has IP forwarding enabled, arbitrary IP address sources can also be used. Not using the forwarding rule's IP address as a source is an advanced scenario because the client receives a response packet from an internal IP address that does not match the IP address to which it sent a request packet.

Outbound internet connectivity from target instances

VM instances referenced by target instances can initiate connections to the internet by using the IP address of the associated forwarding rule as the source IP address of the outbound connection.

Generally, a VM instance always uses its own external IP address or Cloud NAT to initiate connections. You use the forwarding rule IP address to initiate connections from target instances only in special scenarios such as when you need VM instances to originate and receive connections at the same external IP address.

Outbound packets sent from target instance VMs directly to the internet have no restrictions on traffic protocols and ports. Even if an outbound packet is using the forwarding rule's IP address as the source, the packet's protocol and source port don't have to match the forwarding rule's protocol and port specification. However, inbound response packets must match the forwarding rule IP address, protocol, and destination port of the forwarding rule. For more information, see Paths for external passthrough Network Load Balancers and external protocol forwarding.

This path to internet connectivity from a target instance VM is the default intended behavior according to Google Cloud's implied firewall rules. However, if you have security concerns about leaving this path open, you can use targeted egress firewall rules to block unsolicited outbound traffic to the internet.

Limitations

A forwarding rule cannot point to more than one target instance.
Health checks are not supported with target instances. You must ensure that the necessary software is running and operational on the VM referenced by the target instance.
Internal protocol forwarding for IPv6 traffic has the following limitations:
- The L3_DEFAULT protocol is not supported. Use either TCP or UDP.
- Multi-NIC is not supported.

API and gcloud reference

For forwarding rules, see the following:

For target instances, see the following:

Pricing

Protocol forwarding is charged at the same rate as load balancing. There is a charge for the forwarding rule and a charge for the inbound data processed by the target instance.

For all pricing information, see Pricing.

Quotas and limits

For the quotas on forwarding rules for protocol forwarding, see Quotas and limits: Forwarding rules.

What's next

Troubleshoot protocol forwarding