Maximum transmission unit

The maximum transmission unit (MTU) is the size, in bytes, of the largest possible IP packet, including IP headers, layer 4 protocol headers, and layer 4 data, that can fit inside an Ethernet frame.

Valid VPC network MTU sizes

Virtual Private Cloud (VPC) networks use a default MTU of 1,460 bytes. You can set a VPC network's MTU to any value between 1,300 bytes and 8,896 bytes (inclusive). Common custom MTU sizes are 1,500 bytes (standard Ethernet) or 8,896 bytes (the maximum possible). Google recommends you configure the MTU for each virtual machine (VM) instance network interface (NIC) to match the MTU of the VPC network to which it is connected. For more information, see VMs and MTU settings.

Communication between Google Cloud VMs within VPC networks

When sending and receiving VMs use the same VPC network or peered VPC networks that have identical MTUs, IP packets up to the MTU size can be sent between the two VMs, if the interfaces for both VMs are configured to use the VPC network MTU.

To avoid MTU mismatch issues, Google recommends that you use the same MTU for all of your connected VPC networks. While that is the recommended practice, you are not forced to have identical MTUs among connected VPC networks. For details about how protocols handle situations where there is a MTU mismatch between VPC networks, see MSS clamping and path MTU discovery.

From the perspective of a sending VM, paths to the following destinations represent VM-to-VM traffic routed within a VPC network:

  • A regional internal IPv4 address in a subnet primary IPv4 or subnet secondary IPv4 address range, including private IPv4 address ranges and privately used public IPv4 address ranges, used by these destination resources:
    • The primary internal IPv4 address of a receiving VM's network interface (NIC).
    • An internal IPv4 address in an alias IP range of a receiving VM's NIC.
    • An internal IPv4 address of an internal forwarding rule for either protocol forwarding or for an internal passthrough Network Load Balancer.
  • Internal IPv6 subnet address ranges used by these destination resources:
    • An IPv6 address from the /96 IPv6 address range assigned to a dual-stack receiving VM's NIC.
    • An IPv6 address from the /96 IPv6 address range of an internal forwarding rule for either protocol forwarding or for an internal passthrough Network Load Balancer.
  • External IPv6 subnet address ranges used by these destination resources when packets are routed using subnet routes or peering subnet routes within the VPC network:
    • An IPv6 address from the /96 IPv6 address range assigned to a dual-stack receiving VM's NIC.
    • An IPv6 address from the /96 IPv6 address range of an external forwarding rule for either protocol forwarding or for an external passthrough Network Load Balancer.

The following VM-to-VM paths are treated in the same way as Communication to destinations outside of a VPC network:

  • If the packet destination is an external IPv4 address of a receiving Google Cloud VM's NIC.
  • If the packet destination is an external IPv4 address of a external passthrough Network Load Balancer.
  • If the packet destination is an external IPv4 address of a forwarding rule for protocol forwarding
  • If the packet destination is an external IPv6 address of a Google Cloud VM's NIC, external passthrough Network Load Balancer, or forwarding rule for external protocol forwarding and the applicable route in the VPC network uses a default internet gateway next hop. In this scenario, receiving VMs are neither in the same VPC network as the sending VM nor in a VPC network connected to the sending VM's VPC network using VPC Network Peering.

Communication to destinations outside of a VPC network

Google Cloud processes packets sent to destinations outside of the sending VM's VPC network as shown in the following table. Destinations outside of a sending VM's VPC network include publicly routable IP addresses for resources outside of Google Cloud and customer-usable external IP addresses within Google Cloud.

Because the internet generally uses an MTU of 1,500 bytes, keeping IP packet size at 1,500 bytes or less usually avoids MTU-related packet loss.

Situation Behavior
TCP SYN and SYN-ACK packets Google Cloud performs MSS clamping if necessary, changing the MSS to ensure packets fits within the MTU.
IP packet MTU between 1,300 bytes and 1,600 bytes (inclusive) Google Cloud makes no changes to the packet, except for SYN and SYN-ACK packets as discussed in the first row.
IP packet larger than 1,600 bytes Google Cloud drops the packet and sends an ICMP Fragmentation Needed message both when the DF bit is on and also when the DF bit is off.

Communication to Google APIs and services

VMs using any valid VPC network MTU size can send packets to Google APIs and services, including using Private Google Access and Private Service Connect for Google APIs. The details in this section also apply to on-premises resources that send packets to Google APIs and services using Private Google Access for on-premises hosts.

The traffic path to Google APIs and services described in this section is implemented by Google Front Ends (GFEs). These GFEs use 1,460-byte MTUs. Traffic from Google Cloud to Google APIs and services always uses the TCP protocol. If a VM connects to Google APIs and services from a VPC network whose MTU is not 1,460 bytes, the segment size is negotiated by using TCP MSS advertisement.

Packet source Packet destination

Any internal IPv4 address: primary internal IPv4 address or internal IPv4 address from an alias IP range of the VM NIC

An external IPv4 address assigned to the VM NIC using a 1-1 NAT access config: In this situation, Google Cloud performs 1-1 NAT on egress, converting an original source primary internal IPv4 address to a source external IPv4 address specified in the access config.

  • Google APIs and services IPv4 addresses for the default domains
  • 199.36.153.4/30 (restricted.googleapis.com)
  • 199.36.153.8/30 (private.googleapis.com)
  • Private Service Connect endpoint for Google APIs and services
External or internal IPv6 address, for dual stack VMs
  • Google APIs and services IPv6 addresses for the default domains
  • 2600:2d00:0002:1000::/64 (restricted.googleapis.com)
  • 2600:2d00:0002:2000::/64 (private.googleapis.com)

Communication through Cloud VPN tunnels

Cloud VPN has both a gateway MTU for encapsulated packets and a payload MTU for packets before and after encapsulation.

For precise payload MTU values and other Cloud VPN MTU information, see MTU considerations in the Cloud VPN documentation.

Communication through Cloud Interconnect (VLAN) attachments

Google recommends that you use the same MTU for all VLAN attachments that are connected to the same VPC network, and that you set the MTU of the VPC network to the same value. For details about Cloud Interconnect VLAN attachment MTUs, see Cloud Interconnect MTU.

Jumbo frame support

The following table summarizes jumbo frame support among Google Cloud products and features:

Product or feature Jumbo frame support
Compute Engine Yes
Cloud Interconnect Yes
Cloud VPN No
Google APIssupported services No

VMs and MTU settings

As a best practice, match a VM's NIC MTU to the MTU of the VPC network to which the NIC is connected:

  • Each NIC MTU for a Linux VM based on a Google-provided OS image is automatically set to the respective VPC network MTU by using DHCP Option 26.

  • Each NIC MTU for a Windows VM based on a Google-provided OS image is configured with a fixed MTU of 1,460 bytes. If you change the MTU of a VPC network that contains Windows VMs based on Google-provided OS images, you must change the MTU for the Windows VM.

  • If you use custom guest OS images, you must configure NIC MTUs or verify that the guest OS accepts the VPC network MTU by using DHCP Option 26.

  • If a VM has multiple network interfaces, set each NIC MTU to the respective VPC network MTU.

  • If a NIC MTU must differ from the VPC network MTU, set the NIC MTU to less than the VPC network MTU. Forcibly decreasing the NIC MTU is advantageous for some advanced networking scenarios.

Changing the MTU of a VPC network

If you change the MTU of a VPC network with running VMs, keep the following considerations in mind:

  • If you reduce the VPC network MTU, you must stop and start each VM. Rebooting a VM from within its guest operating system does not update its MTU.

  • If you increase the VPC network MTU, running VMs using the VPC network won't take advantage of the increased VPC network MTU until the VMs have been stopped and started. Until each VM has been stopped and restarted, the VM continues to use the previous (lower) MTU value.

For instructions, see Change the MTU setting of a VPC network.

GKE and MTU settings

The MTU selected for a Pod interface is dependent on the Container Network Interface (CNI) used by the cluster Nodes and the underlying VPC MTU setting. For more information, see Pods.

The Pod interface MTU value is either 1460 or inherited from the primary interface of the Node.

CNI MTU GKE Standard
kubenet 1460 Default
kubenet
(GKE version 1.26.1 and later)
Inherited Default
Calico 1460

Enabled by using --enable-network-policy.

For details, see Control communication between Pods and Services using network policies.

netd Inherited Enabled by using any of the following:
GKE Dataplane V2 Inherited

Enabled by using --enable-dataplane-v2.

For details, see Using GKE Dataplane V2.

MSS clamping and path MTU discovery

This section describes how different protocols deal with mismatched MTUs by using MSS clamping and path MTU discovery technologies:

  • TCP traffic. The TCP protocol handles mismatched MTUs automatically. As part of connection establishment, both the sender VM and receiving VM specify a maximum TCP segment size in their SYN and SYN-ACK packets. The greatest amount of TCP data that can be transmitted by a sender VM is the minimum of the receiving VM's advertised MSS (from its SYN-ACK) and the sender VM's MTU minus 40 bytes (20 for the IP header and 20 for the base TCP header). Google Cloud VPC networks don't perform MSS clamping for packets sent between VMs in VPC networks.

  • Other protocols. Other protocols (such as UDP) require special care when two different VPC network MTUs are involved. In general, it is the responsibility of a sending VM to emit packets that fit within the receiving VM's NIC MTU. Google Cloud does not perform IP fragmentation when sending packets between VMs in the same VPC network or in VPC networks that are connected by using VPC Network Peering.

    When an IP packet is too large to be delivered—for example, when the capacity of the receiving VM's NIC MTU is too small to receive the incoming packet— Google Cloud drops the packet. If the packet has the DF bit set, Google Cloud also sends a Fragmentation Needed (ICMP over IPv4) or Packet Too Big (ICMPv6) message back to the sender.

    Google Cloud sends a Fragmentation Needed or Packet Too Big message in the following circumstances, even when a DF bit is not set:

    • If the VPC network MTU is less than 1,600 bytes, and the packet being sent exceeds the VPC network MTU.
    • If the VPC network MTU is 1,600 bytes or more, and the packet being sent exceeds 1,600 bytes.

    ICMP Fragmentation Needed or Packet Too Big messages are necessary for a VM that is sending packets to use Path MTU discovery (PMTUD). To illustrate how PMTUD works, consider the following example with two VMs in different VPC networks that are connected by using VPC Network Peering:

    • The sending VM has a NIC in a VPC network whose MTU is 8,896 bytes.
    • The receiving VM has a NIC in a VPC network whose MTU is 1,460 bytes.
    • The sending VM emits an 8,000-byte IP packet whose Don't Fragment (DF) bit is set. Because the packet is too large to be delivered to the receiving VM, Google Cloud sends a Fragmentation Required or Packet Too Big message to the sending VM. This message indicates the largest possible IP packet size that the sender can use when it attempts to re-transmit packets for the connection.
    • The sending VM's operating system uses this information to lower the IP packet size when sending subsequent packets to the receiving VM.

PMTUD has the following additional requirements because PMTUD-generated Fragmentation Needed or Packet Too Big packets use the ICMP protocol and have sources that match an original packet's destination:

  • You must configure ingress allow VPC firewall rules or rules in firewall policies such that ICMP (for IPv4) or ICMPv6 (for IPv6) are allowed from sources that match the original packet destinations. To simplify firewall configuration, consider allowing ICMP and ICMPv6 from all sources.
  • Forwarding rules for internal passthrough Network Load Balancer and internal protocol forwarding must use the L3_DEFAULT protocol so that they process both ICMP for PMTUD and the protocol used by the original packet.

What's next

Try it for yourself

If you're new to Google Cloud, create an account to evaluate how VPC performs in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

Try VPC free