Maximum transmission unit

The maximum transmission unit (MTU) is the size, in bytes, of the largest possible IP packet, including IP headers, layer 4 protocol headers, and layer 4 data, that can fit inside an Ethernet frame.

Valid VPC network MTU sizes

Virtual Private Cloud (VPC) networks use a default MTU of 1,460 bytes. You can set a VPC network's MTU to any value between 1,300 bytes and 8,896 bytes (inclusive). Common custom MTU sizes are 1,500 bytes (standard Ethernet) or 8,896 bytes (the maximum possible). Google recommends you configure the MTU for each virtual machine (VM) instance network interface (NIC) to match the MTU of the VPC network to which it is connected. For more information, see VMs and MTU settings.

Communication between Google Cloud VMs within VPC networks

When sending and receiving VMs use the same VPC network or peered VPC networks that have identical MTUs, IP packets up to the MTU size can be sent between the two VMs, if the interfaces for both VMs are configured to use the VPC network MTU.

To avoid MTU mismatch issues, Google recommends that you use the same MTU for all of your connected VPC networks. While that is the recommended practice, you are not forced to have identical MTUs among connected VPC networks. For details about how protocols handle situations where there is a MTU mismatch between VPC networks, see Mismatched MTUs, MSS clamping, path MTU discovery.

From the perspective of a sending VM, paths to the following destinations represent VM-to-VM traffic routed within a VPC network:

  • A regional internal IPv4 address in a subnet primary IPv4 or subnet secondary IPv4 address range, including private IPv4 address ranges and privately used public IPv4 address ranges, used by these destination resources:
    • The primary internal IPv4 address of a receiving VM's network interface (NIC).
    • An internal IPv4 address in an alias IP range of a receiving VM's NIC.
    • An internal IPv4 address of an internal forwarding rule for either protocol forwarding or for an internal passthrough Network Load Balancer.
  • Internal IPv6 subnet address ranges used by these destination resources:
    • An IPv6 address from the /96 IPv6 address range assigned to a dual-stack receiving VM's NIC.
    • An IPv6 address from the /96 IPv6 address range of an internal forwarding rule for either protocol forwarding or for an internal passthrough Network Load Balancer.
  • External IPv6 subnet address ranges used by these destination resources when packets are routed using subnet routes or peering subnet routes within the VPC network:
    • An IPv6 address from the /96 IPv6 address range assigned to a dual-stack receiving VM's NIC.
    • An IPv6 address from the /96 IPv6 address range of an external forwarding rule for either protocol forwarding or for an external passthrough Network Load Balancer.

The following VM-to-VM paths are treated in the same way as Communication to destinations outside of a VPC network:

  • If the packet destination is an external IPv4 address of a receiving Google Cloud VM's NIC.
  • If the packet destination is an external IPv4 address of a external passthrough Network Load Balancer.
  • If the packet destination is an external IPv4 address of a forwarding rule for protocol forwarding
  • If the packet destination is an external IPv6 address of a Google Cloud VM's NIC, external passthrough Network Load Balancer, or forwarding rule for external protocol forwarding and the applicable route in the VPC network uses a default internet gateway next hop. In this scenario, receiving VMs are neither in the same VPC network as the sending VM nor in a VPC network connected to the sending VM's VPC network using VPC Network Peering.

Communication to destinations outside of a VPC network

Google Cloud processes packets sent to destinations outside of the sending VM's VPC network as shown in the following table. Destinations outside of a sending VM's VPC network include publicly routable IP addresses for resources outside of Google Cloud and customer-usable external IP addresses within Google Cloud.

Because the internet generally uses an MTU of 1,500 bytes, keeping IP packet size at 1,500 bytes or less usually avoids MTU-related packet loss.

Situation Behavior
TCP SYN and SYN-ACK packets Google Cloud performs MSS clamping if necessary, changing the MSS to ensure packets fits within the MTU.
IP packet MTU between 1,300 bytes and 1,600 bytes (inclusive) Google Cloud makes no changes to the packet, except for SYN and SYN-ACK packets as discussed in the first row.
IP packet larger than 1,600 bytes Google Cloud drops the packet and sends an ICMP Fragmentation Needed message both when the DF bit is on and also when the DF bit is off.

Communication to Google APIs and services

VMs using any valid VPC network MTU size can send packets to Google APIs and services, including using Private Google Access and Private Service Connect for Google APIs. The details in this section also apply to on-premises resources that send packets to Google APIs and services using Private Google Access for on-premises hosts.

The traffic path to Google APIs and services described in this section is implemented by Google Front Ends (GFEs). These GFEs use non-configurable, fixed MTUs. Traffic from Google Cloud to Google APIs and services always uses the TCP protocol: If a VM connects to Google APIs and services from a VPC network whose MTU does not match the MTU of the GFE, the segment size is negotiated by using TCP MSS advertisement as described in Mismatched MTUs, MSS clamping, path MTU discovery.

Packet source Packet destination

Any internal IPv4 address: primary internal IPv4 address or internal IPv4 address from an alias IP range of the VM NIC

An external IPv4 address assigned to the VM NIC using a 1-1 NAT access config: In this situation, Google Cloud performs 1-1 NAT on egress, converting an original source primary internal IPv4 address to a source external IPv4 address specified in the access config.
  • Google APIs and services IPv4 addresses for the default domains
  • 199.36.153.4/30 (restricted.googleapis.com)
  • 199.36.153.8/30 (private.googleapis.com)
  • Private Service Connect endpoint for Google APIs and services
External or internal IPv6 address, for dual stack VMs
  • Google APIs and services IPv6 addresses for the default domains
  • 2600:2d00:0002:1000::/64 (restricted.googleapis.com)
  • 2600:2d00:0002:2000::/64 (private.googleapis.com)

Communication through Cloud VPN tunnels

Cloud VPN has both a gateway MTU for encapsulated packets and a payload MTU for packets before and after encapsulation.

For precise payload MTU values and other Cloud VPN MTU information, see MTU considerations in the Cloud VPN documentation.

Communication through Cloud Interconnect (VLAN) attachments

Google recommends that you use the same MTU for all VLAN attachments that are connected to the same VPC network, and that you set the MTU of the VPC network to the same value. For details about Cloud Interconnect VLAN attachment MTUs, see Cloud Interconnect MTU.

Jumbo frame support

The following table summarizes jumbo frame support among Google Cloud products and features:

Product or feature Jumbo frame support
Compute Engine Yes
Cloud Interconnect Yes
Cloud VPN No
Google APIssupported services No

VMs and MTU settings

As a best practice, match a VM's NIC MTU to the MTU of the VPC network to which the NIC is connected:

  • Each NIC MTU for a Linux VM based on a Google-provided OS image is automatically set to the respective VPC network MTU by using DHCP Option 26.

  • Each NIC MTU for a Windows VM based on a Google-provided OS image is configured with a fixed MTU of 1,460 bytes. If you change the MTU of a VPC network that contains Windows VMs based on Google-provided OS images, you must change the MTU for the Windows VM.

  • If you use custom guest OS images, you must configure NIC MTUs or verify that the guest OS accepts the VPC network MTU by using DHCP Option 26.

  • If a VM has multiple network interfaces, set each NIC MTU to the respective VPC network MTU.

  • If a NIC MTU must differ from the VPC network MTU, set the NIC MTU to less than the VPC network MTU. Forcibly decreasing the NIC MTU is advantageous for some advanced networking scenarios.

Changing the MTU of a VPC network

If you change the MTU of a VPC network with running VMs, keep the following considerations in mind:

  • If you reduce the VPC network MTU, you must stop and start each VM. Rebooting a VM from within its guest operating system does not update its MTU.

  • If you increase the VPC network MTU, running VMs using the VPC network won't take advantage of the increased VPC network MTU until the VMs have been stopped and started. Until each VM has been stopped and restarted, the VM continues to use the previous (lower) MTU value.

For instructions, see Change the MTU setting of a VPC network.

GKE and MTU settings

The MTU selected for a Pod interface is dependent on the Container Network Interface (CNI) used by the cluster Nodes and the underlying VPC MTU setting. For more information, see Pods.

The Pod interface MTU value is either 1460 or inherited from the primary interface of the Node.

CNI MTU GKE Standard
kubenet 1460 Default
kubenet
(GKE version 1.26.1 and later) 		Inherited Default
Calico 1460

Enabled by using --enable-network-policy.

For details, see Control communication between Pods and Services using network policies.
netd Inherited Enabled by using any of the following:
GKE Dataplane V2 Inherited

Enabled by using --enable-dataplane-v2.

For details, see Using GKE Dataplane V2.

Mismatched MTUs, MSS clamping, path MTU discovery

This section describes how TCP and non-TCP protocols handle mismatched MTUs.

TCP protocol

The TCP protocol handles MTU mismatches automatically. Both the client and server individually calculate their own effective TCP maximum segment size (MSS) values each time a TCP connection is opened. The client and server don't have to agree on an identical effective MSS value.

  • Effective TCP maximum segment size (MSS) of the client: The largest amount of transmissible data in a TCP segment sent from a client to a server is the minimum of the following two values:

    • The value of the MSS field in the SYN-ACK packet received by the client from the server during TCP connection establishment.

    • The MTU of the client's network interface, minus 40 bytes. The subtracted 40 bytes includes 20 bytes for the IP header and 20 bytes for the base TCP header.

  • Effective TCP maximum segment size (MSS) of the server: The largest amount of transmissible data in a TCP segment sent from a server to a client is the minimum of the following two values:

    • The value of the MSS field in the SYN packet received by the server from the client during TCP connection establishment.

    • The MTU of the server's network interface, minus 40 bytes. The subtracted 40 bytes includes 20 bytes for the IP header and 20 bytes for the base TCP header.

TCP MSS clamping

TCP MSS clamping is a process where a network device between a client and server changes the MSS values in SYN and SYN-ACK packets as it routes the packets between the client and server. Google Cloud uses MSS clamping whenever it sends packets to destinations outside of a VPC network.

Cloud VPN tunnels and Cloud Interconnect VLAN attachments also use MSS clamping. For more information, see Packet encapsulation and processing in the Cloud VPN documentation and Cloud Interconnect MTU.

VPC networks don't perform MSS clamping for packets routed by the next hops within a VPC network because the TCP protocol itself is sufficient.

Non-TCP protocols

Other protocols, such as UDP, require special care when two different VPC network MTUs are involved. It is the responsibility of a sending system to emit packets that fit within its network interface MTU, the MTU of the receiving system's network interface, and the MTU of all networks in between. Google Cloud does not perform IP fragmentation for packets routed by the next hops within a VPC network.

When an IP packet is too large to be delivered—for example, when the packet exceeds the MTU of the VPC network where the receiving VM's NIC is located— Google Cloud drops the packet. If the packet has the DF bit set, Google Cloud also sends a Fragmentation Needed (ICMP over IPv4) or Packet Too Big (ICMPv6) message back to the sender.

Google Cloud sends a Fragmentation Needed or Packet Too Big message in the following circumstances, even when a DF bit is not set:

  • If the VPC network MTU is less than 1,600 bytes, and the packet being sent exceeds the VPC network MTU.
  • If the VPC network MTU is 1,600 bytes or more, and the packet being sent exceeds 1,600 bytes.

ICMP Fragmentation Needed or Packet Too Big messages are necessary for a VM that is sending packets to use Path MTU discovery (PMTUD). To illustrate how PMTUD works, consider the following example with two VMs in different VPC networks that are connected by using VPC Network Peering:

  • The sending VM has a NIC in a VPC network whose MTU is 8,896 bytes.
  • The receiving VM has a NIC in a VPC network whose MTU is 1,460 bytes.
  • The sending VM emits an 8,000-byte IP packet whose Don't Fragment (DF) bit is set. Because the packet is too large to be delivered to the receiving VM, Google Cloud sends a Fragmentation Required or Packet Too Big message to the sending VM. This message indicates the largest possible IP packet size that the sender can use when it attempts to re-transmit packets for the connection.
  • The sending VM's operating system uses this information to lower the IP packet size when sending subsequent packets to the receiving VM.

PMTUD has the following additional requirements because PMTUD-generated Fragmentation Needed or Packet Too Big packets use the ICMP protocol and have sources that match an original packet's destination:

  • You must configure ingress allow VPC firewall rules or rules in firewall policies such that ICMP (for IPv4) or ICMPv6 (for IPv6) are allowed from sources that match the original packet destinations. To simplify firewall configuration, consider allowing ICMP and ICMPv6 from all sources.
  • Forwarding rules for internal passthrough Network Load Balancer and internal protocol forwarding must use the L3_DEFAULT protocol so that they process both ICMP for PMTUD and the protocol used by the original packet.

What's next

Try it for yourself

If you're new to Google Cloud, create an account to evaluate how VPC performs in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

Try VPC free