Troubleshooting

This guide can help you solve common issues with Cloud NAT.

Common issues

VMs can reach the internet unexpectedly, without Cloud NAT

If your virtual machine (VM) instances or container instances can reach the internet without Cloud NAT, but you don't want them to, check for the following issues:

  • Determine if the VM's network interface has an external IP address. If the network interface has an external IP address assigned to it, Google Cloud automatically performs one-to-one NAT for packets whose sources match the interface's primary internal IP address. For more information, see Cloud NAT specifications.

    To determine if a VM has an external IP address, see changing or assigning an external IP address to an existing instance.

  • Ensure that your Google Kubernetes Engine (GKE) cluster is a private cluster. Each node VM in a non-private cluster has an external IP address, so each node can use routes in your Virtual Private Cloud (VPC) network whose next hop is the default internet gateway without relying on Cloud NAT. For more information, including how non-private clusters interact with Cloud NAT gateways, see GKE interaction.

  • List routes in your VPC network, looking for ones that could provide internet connectivity through a next hop different than the default internet gateway. As examples:

    • Custom static routes whose next hops are VMs, internal TCP/UDP load balancers, or Cloud VPN tunnels might indirectly provide internet connectivity. For example, the next hop VMs or backend VMs for an internal TCP/UDP load balancer might have external IP addresses themselves, or a Cloud VPN tunnel might connect to a network that offers internet access.

    • Custom dynamic routes learned from on-premises networks by Cloud Routers in your VPC network might connect to a network that offers internet access.

  • Keep in mind that other custom routes in your VPC network might have higher priorities than routes whose next hops are default internet gateways. For information about how Google Cloud evaluates routes, see routing applicability and order.

No logs are generated

  • Verify that NAT logging is enabled.
  • Double-check that your view of the logs isn't filtering out the logs that you are looking for. For instructions, see Viewing logs.

  • Make sure that a firewall rule isn't blocking traffic. Firewall rules that block egress (outbound) traffic are applied before the traffic would have been sent to the NAT gateway. You can use Firewall Rules Logging to see if your custom egress rules are blocking outbound traffic.

  • Review cases where NAT is not performed on traffic. The destination for your traffic might not be handled by NAT.

Certain logs are excluded

  • Verify that NAT logging is enabled and that your log filter is not excluding logs that you want to keep. You can clear a logs filter so that nothing is excluded.

  • Cloud NAT does not log every single event. During periods of heavy egress traffic, NAT logging is throttled, proportional to the machine type of the VM. Translation or error logs might be dropped, and it is not possible to determine what is omitted during throttling.

Need to allocate more IP addresses

If an instance is unable to reach the internet, and if you are manually allocating IP addresses, you might need to add more IP addresses.

In the Cloud Console, if you see the message You need to allocate at least 'X' more IP addresses to allow all instances to access the internet, then you need to allocate more IP addresses. For more information, see NAT IP addresses.

You can monitor this issue by defining alerting policies on the message nat_allocation_failed.

Frequently asked questions

Regional restriction for Cloud NAT

Can I use the same Cloud NAT gateway in more than one region?

No. Cloud NAT gateways are regional resources, associated with a single region, VPC network, and Cloud Router.

You can create additional Cloud NAT gateways in other regions or other VPC networks. To determine if you can create more than one gateway in a given region and VPC network, see subnet IP address range applicability.

Are the external NAT IP addresses used by Cloud NAT gateways global or regional?

Cloud NAT gateways use regional external IP addresses as NAT IP addresses. Even though they are regional, they are publicly routable. For information about different ways that NAT IP addresses can be allocated or assigned, see NAT IP addresses.

When Cloud NAT can and cannot be used

Does Cloud NAT apply to instances, including GKE node VMs, that have external IP addresses?

Generally, no. If the network interface of a VM has an external IP address, Google Cloud always performs 1-to-1 NAT for packets sent from the primary internal IP address of the network interface without using Cloud NAT. However, Cloud NAT could still provide NAT services to packets sent from alias IP address ranges of that same network interface. For additional details, see Cloud NAT specifications and GKE interaction.

Can I use Cloud NAT for communication between VMs in a VPC network?

No, Cloud NAT is designed to provide connectivity to the internet only.

Can I use Cloud NAT to connect a VPC network to another network to work around overlapping IP addresses?

No, Cloud NAT cannot apply to any custom route whose next hop is not the default internet gateway. For example, Cloud NAT cannot apply to traffic sent to a next hop Cloud VPN tunnel, even if the destination is a publicly routable IP address.

Does Cloud NAT let a source VM whose network interface lacks an external IP address send traffic to a destination VM or load balancer that has an external IP address, even when the source and destination are in the same VPC network?

Yes. The network path involves sending traffic out of the VPC network through a default internet gateway, and then receiving it in the same network.

When the source VM sends a packet to the destination, Cloud NAT performs source NAT (SNAT) before delivering the packet to the second instance. Cloud NAT performs destination NAT (DNAT) for responses from the second instance to the first. For a step-by-step example, see NAT flow.

Unsolicited incoming connections not supported

Does Cloud NAT allow for inbound connections (for example, SSH) to instances without external IP addresses?

No, Cloud NAT does not support unsolicited incoming connections. For more information, see Cloud NAT specifications.

If you need to connect to a VM that doesn't have an external IP address, see connecting to instances that do not have external IP addresses. For example, as part of the Cloud NAT example Compute Engine setup, you connect to a VM without an external IP address by using Identity-Aware Proxy.

Cloud NAT and ports

Why does a VM have a fixed number of ports (64 by default)?

When a Cloud NAT gateway provides NAT for a VM, it reserves source address and source port tuples according to the port reservation procedure.

For more information, see port reservation examples.

Can I change the minimum number of ports reserved for a VM?

Yes. You can increase or decrease the minimum number of ports per VM when you create a new Cloud NAT gateway or by editing it later. Each Cloud NAT gateway reserves source address and source port tuples according to the port reservation procedure.

For additional information about decreasing the minimum number of ports, see the next question.

Can I decrease the minimum number of ports per VM after creating the Cloud NAT gateway?

Yes; however, decreasing the minimum number of ports could result in the port reservation procedure reserving a smaller number of ports per VM. When this happens, existing TCP connections might be reset and, if so, must be re-established.

Cloud NAT and other Google services

Does Cloud NAT enable access to Google APIs and services?

When you enable Cloud NAT for a subnet's primary IP range, Google Cloud automatically enables Private Google Access. For more information, see Private Google Access interaction.

What's next