Cloud NAT address and port concepts

This page describes how Cloud NAT gateways use external IP addresses and how they allocate source ports to Compute Engine VM instances and GKE nodes that use the gateways. Before reviewing this information, familiarize yourself with the Cloud NAT overview.

NAT IP addresses

A NAT IP address is a regional external IP address, routable on the internet. A VM without an external IP address, in a subnet served by a Cloud NAT gateway, uses a NAT IP address when it sends packets to a destination on the internet.

You assign NAT IP addresses to a Cloud NAT gateway using one of these methods:

  • Automatic NAT IP address allocation: When you select this option, or choose Google Cloud defaults, Cloud NAT automatically adds regional external IP addresses to your gateway based on the number of VMs using the gateway and the number of ports reserved for each VM. It also automatically removes a NAT IP address when it no longer needs any source ports on that NAT IP address.

    • When a Cloud NAT gateway adds a NAT IP address, it creates a static (reserved) regional external IP address. These addresses count towards per-project quotas.
    • With automatic allocation, you cannot predict the next IP address that will be allocated. If you depend on knowing the set of possible NAT IP addresses ahead of time (for example, to create an allow whitelist), you should use manual NAT IP address assignment instead.
    • If you switch to manual NAT IP address assignment later, the automatically reserved regional external IP addresses are deleted. See switching assignment method for details.
  • Manual NAT IP address assignment: When you select this option, you create and manually assign static (reserved) regional external IP addresses to your Cloud NAT gateway. You can increase or decrease the number of manually assigned NAT IP addresses by editing the Cloud NAT gateway.

    • When using manual NAT IP address assignment, you must calculate the number of regional external IP addresses you need for the Cloud NAT gateway. If your gateway runs out of NAT IP addresses, Cloud NAT drops packets. Dropped packets are logged when you turn on error logging using Cloud NAT logging.
    • See the port reservation example for example calculations.

Refer to Cloud NAT limits for the maximum number of automatically allocated or manually assigned NAT IP addresses.

Switching assignment method

You can switch a Cloud NAT gateway from automatic NAT IP address allocation to manual NAT IP address assignment; however, the NAT IP addresses cannot be preserved. Even though automatically allocated NAT IP addresses are static, they cannot be moved to a manual NAT IP address assignment. For example, you cannot start using a Cloud NAT gateway with automatically allocated NAT IP addresses and later use those same addresses when you switch the NAT gateway to manually assigned NAT IP addresses.

The set of regional external IP addresses that Cloud NAT uses for automatic NAT IP address allocation are different from the set of regional external IP addresses that you can manually choose.

Draining NAT IP addresses

When you configure a Cloud NAT gateway with manual NAT IP address assignment, you can choose what happens when you need to reduce the number of NAT IP addresses the gateway uses:

  • If you remove a manually assigned NAT IP address, established NAT connections are broken immediately.

  • You can choose to drain a manually assigned NAT IP address instead. Draining instructs the Cloud NAT gateway to stop using the NAT IP address for new connections, but continue using it for established connections. Established connections are permitted to close normally instead of being abruptly terminated. For instructions, see Drain external IP addresses associated with NAT.

Ports

Each NAT IP address on a Cloud NAT gateway offers 64,512 TCP source ports and 64,512 UDP source ports. TCP and UDP each support 65,536 ports per IP address, and Cloud NAT doesn't use the first 1,024 well-known (privileged) ports.

When you create a Cloud NAT gateway, you specify a minimum number of ports per VM instance. When a Cloud NAT gateway performs source network address (SNAT) on a packet sent by a VM, it changes the packet's source IP address and source port.

Port reservation procedure

Cloud NAT uses this procedure to provision source address and source port tuples for each VM that the Cloud NAT gateway serves. When you use a Cloud NAT gateway to provide NAT services for a private Google Kubernetes Engine cluster, the number of source address and port tuples are assigned to the whole node VM, shared by all Pods on the node.

  1. Determine the VM internal IP addresses for which NAT should be performed. The VM internal IP addresses for which a Cloud NAT gateway should perform NAT is determined by the subnet IP address ranges that the gateway has been configured to serve.

    • If the Cloud NAT gateway is configured to perform NAT for the primary IP address range of the subnet used by the VM's network interface, the gateway performs NAT for both the VM's primary internal IP address and any of the VM's alias IP ranges from the subnet's primary IP address range.

    • If the Cloud NAT gateway is configured to perform NAT for a secondary IP address range of the subnet used by the VM's network interface, the gateway performs NAT for any alias IP ranges from that subnet's secondary IP address range.

  2. Calculate the number of source ports to assign to the VM.

    1. If the Cloud NAT gateway should perform NAT for one or more of the VM's alias IP ranges whose netmasks are shorter than /32 — that is, if the gateway should perform NAT for at least one of the VM's alias IP ranges that consists of more than a single IP address, Cloud NAT takes the maximum of the following two numbers and proceeds to the next step using that maximum as input:

      • the minimum ports per VM instance you specified
      • the number 1,024

      Otherwise, the Cloud NAT gateway proceeds to the next step using just the minimum ports per VM instance as input.

    2. The Cloud NAT gateway uses this input to determine how many source ports it should assign to the VM.

  3. Reserve source address and source port tuples for the VM. The number of source address and source port tuples matches how many source ports the Cloud NAT gateway determined should be assigned to the VM.

    It is possible for the source address and source port tuples to span more than one NAT IP address if the Cloud NAT gateway uses two or more NAT IP addresses. A single NAT IP address might not have enough unreserved source ports to accommodate the number of source address and source port tuples that the VM needs.

Ports and connections

The number of source address and source port tuples that a Cloud NAT gateway reserves for a VM limits the number of connections the VM can make to a unique destination:

  • A unique destination means a unique 3-tuple consisting of a destination IP address, a destination port, and an IP protocol (like TCP or UDP).

  • A connection means a unique 5-tuple consisting of the source address and source port tuple combined with a unique destination 3-tuple. Because the UDP protocol is connectionless, the concept of connection is reduced to a 5-tuple associated with a unique UDP datagram.

Suppose that a Cloud NAT gateway calculates 1,024 for the fixed number of ports for a VM by following the port reservation procedure. The Cloud NAT gateway reserves 1,024 unique combinations of source address and source port tuples for the VM. The Cloud NAT gateway can process 1,024 simultaneous connections to each unique destination 3-tuple. As examples:

  • The gateway supports 1,024 simultaneous connections to destination IP address 203.0.113.99 on port 80 using the TCP protocol, and

  • The gateway supports another 1,024 simultaneous connections that same destination IP address on port 443, also using the TCP protocol, and

  • The gateway supports another 1,024 simultaneous connections to a different destination IP address on port 80, using the TCP protocol.

As long as at least one piece of information in the destination 3-tuple changes — the destination IP address, the destination port, the protocol — the same source address and source port tuple can be simultaneously re-used for other connections.

Source port reuse for TCP connections

After a Cloud NAT gateway closes a TCP connection, Google Cloud enforces a two-minute delay before the gateway can re-use the same source address and source port tuple with the same destination (destination IP address, destination port, and protocol).

You cannot reduce this delay; however, you can do one of the following:

  • Increase the minimum number of ports per VM instance so that the port allocation procedure assigns the VM more source address and source port tuples.

  • If a VM needs to rapidly open and close TCP connections to the same destination IP address and destination port, using the same protocol, you should assign an external IP address to the VM and use firewall rules to limit unsolicited ingress connections instead of using Cloud NAT.

Source ports and security

If you depend on source port randomization as a security measure, you need to consider the following:

Examples

The following examples demonstrate how Cloud NAT reserves source addresses and source ports for a VM and how it performs NAT for packets sent to the internet.

Port reservation

The following examples demonstrate applications of the port reservation procedure:

Suppose you're configuring a Cloud NAT gateway to provide NAT for the primary IP address range of a subnet and the VMs using that subnet do not have any alias IP ranges from the subnet's primary IP address range. Round down the result of any division operation to the closest integer. ⌊⌋ is the floor (greatest integer) function, meaning discard any fractional result of division.

  • If you configure the Cloud NAT gateway with a single NAT IP address using manual assignment, and you set the minimum number of ports per VM instance to 64, the gateway can provide NAT services for up to 1,008 VMs:

    ⌊(1 NAT IP address) × (64,512 ports per address) / (64 ports per VM)⌋ = 1,008 VMs

  • If need to support more than 1,008 VMs, you can assign a second NAT IP address to the Cloud NAT gateway. With two NAT IP addresses, keeping the minimum number of ports per VM at 64, you can support 2,016 VMs:

    ⌊(2 NAT IP addresses) × (64,512 ports per address) / (64 ports per VM)⌋ = 2,016 VMs

  • If set the minimum number of ports per VM to 4,096, each NAT IP address can support 15 VMs. This calculation is rounded down to the closest integer:

    ⌊(1 NAT IP addresses) × (64,512 ports per address) / (4,096 ports per VM)⌋ = 15 VMs

NAT flow

In this example, a VM with primary internal IP address 10.240.0.4, without an external IP address, needs to download an update from the external IP address 203.0.113.1. You've configured the nat-gw-us-east gateway as follows:

  • minimum ports per instance: 64
  • manually assigned two NAT IP addresses: 192.0.2.50 and 192.0.2.60
  • provide NAT for the primary IP address range of subnet-1
Cloud NAT translation example (click to enlarge)
Cloud NAT translation example (click to enlarge)

Cloud NAT follows the port reservation procedure, to reserve the following source address and source port tuples for each of the VMs in the network. For example, the Cloud NAT gateway reserves 64 source ports for the VM with internal IP address 10.240.0.4. The NAT IP address 192.0.2.50 has 64 unreserved ports, so the gateway reserves the following set of 64 source address and source port tuples for that VM:

  • 192.0.2.50:34000 through 192.0.2.50:34063

When the VM sends a packet to the update server, 203.0.113.1, on destination port 80, using the TCP protocol...

  • The VM sends a request packet with these attributes:

    • Source address: 10.240.0.4, the primary internal IP address of the VM
    • Source port: 24000, the ephemeral source port chosen by the VM's operating system
    • Destination address: 203.0.113.1, the update server's external IP address
    • Destination port: 80, the destination port for HTTP traffic to the update server
    • Protocol: TCP
  • The nat-gw-us-east gateway performs SNAT on egress, re-writing the request packet's source address and source port. The modified packet is sent to the internet if the VPC network has a route for the 203.0.113.1 destination whose next hop is the default internet gateway. A default route commonly meets this requirement.

    • Source address: 192.0.2.50, from one of the VM's reserved source address and source port tuples
    • Source port: 34022, an unused source port from one of the VM's reserved source port tuples
    • Destination address: 203.0.113.1, unchanged
    • Destination port: 80, unchanged
    • Protocol: TCP, unchanged
  • When the update server sends a response packet, that packet arrives on the nat-gw-us-east gateway with these attributes:

    • Source address: 203.0.113.1, the update server's external IP address
    • Source port: 80, the HTTP response from the update server
    • Destination address: 192.0.2.50, matching the original source address of the request packet
    • Destination Port: 34022, matching the source port of the request packet
    • Protocol: TCP, unchanged
  • The nat-gw-us-east gateway performs DNAT on the response packet, re-writing the response packet's destination address and destination port so the packet is delivered to the VM:

    • Source address: 203.0.113.1, unchanged
    • Source port: 80, unchanged
    • Destination address: 10.240.0.4, the primary internal IP address of the VM
    • Destination Port: 24000, matching the original ephemeral source port of the request packet
    • Protocol: TCP, unchanged

What's next