Version 1.7. This version is supported as outlined in the Anthos version support policy, offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on bare metal. For more details, see the release notes 1.7. This is the most recent version. For a complete list of each minor and patch release in chronological order, see the combined release notes.

Available versions: 1.7  |   1.6

Network requirements

Network requirements

External network requirements

Anthos clusters on bare metal requires an internet connection for operational purposes. Anthos clusters on bare metal retrieves cluster components from Container Registry and the cluster is registered with Connect.

You can connect to Google using the public internet (with HTTPS), through a Virtual Private Network (VPN), or through a Dedicated Interconnect.

Internal network requirements

Anthos clusters on bare metal can work with L2 or L3 connectivity between cluster nodes and requires load balancer nodes be in the same L2 domain. The load balancer nodes can be the control plane nodes or a dedicated set of nodes. See Choosing and configuring load balancers for configuration information.

The L2 network requirement applies whether you run the load balancer on the control plane node pool or in a dedicated set of nodes.

The requirements for load balancer machines are:

  • All load balancers for a given cluster are in the same L2 domain.
  • All VIPs must be in the load balancer machine subnet and routable to the gateway of the subnet.
  • Users are responsible to allow ingress load balancer traffic.

Pod networking

Anthos clusters on bare metal 1.7.0 and later versions allows you to configure up to 250 pods per node. Kubernetes assigns a CIDR block to each node so that each pod can have a unique IP address. The size of the CIDR block corresponds to the maximum number of pods per node. The following table lists the size of the CIDR block that Kubernetes assigns to each node based on the configured maximum pods per node:

Maximum pods per node CIDR block per node Number of IP addresses
32 /26 64
33 – 64 /25 128
65 – 128 /24 256
129 - 250 /23 512

Running 250 pods per node requires Kubernetes to reserve a /23 CIDR block for each node. Assuming that the cluster's clusterNetwork.pods.cidrBlocks is configured to the default value of /16, this allows the cluster to have a limit of (2(23-16))=128 nodes. If you intend to grow the cluster beyond this limit, you can either increase the value of clusterNetwork.pods.cidrBlocks or decrease the value of nodeConfig.podDensity.maxPodsPerNode.

Single user cluster deployment with high availability

The following diagram illustrates a number of key networking concepts for Anthos clusters on bare metal in one possible network configuration.

Anthos clusters on bare metal typical network configuration

  • The control plane nodes run load balancers, and they are all on the same L2 network, while other connections, including worker nodes, only require L3 connectivity.
  • Configuration files define IP addresses for worker node pools, as well as virtual IP addresses for Services, for ingress and for control plane (Kubernetes API) access.
  • A connection to Google Cloud is also required.

Port usage

This section shows how UDP and TCP ports are used on cluster and load balancer nodes.

Master nodes

ProtocolDirectionPort rangePurposeUsed by
UDPInbound6081GENEVE EncapsulationSelf
TCPInbound22Provisioning and updates of admin cluster nodesAdmin workstation
TCPInbound6444Kubernetes API serverAll
TCPInbound2379 - 2380etcd server client APIkube-apiserver, etcd
TCPInbound10250kubelet APISelf, Control plane
TCPInbound10251kube-schedulerSelf
TCPInbound10252kube-controller-managerSelf
TCPBoth4240CNI health checkAll

Worker nodes

ProtocolDirectionPort rangePurposeUsed by
TCPInbound22Provisioning and updates of user cluster nodesAdmin cluster nodes
UDPInbound6081GENEVE EncapsulationSelf
TCPInbound10250kubelet APISelf, Control plane
TCPInbound30000 - 32767NodePort ServicesSelf
TCPBoth4240CNI health checkAll

Load balancer nodes

ProtocolDirectionPort rangePurposeUsed by
TCPInbound22Provisioning and updates of user cluster nodesAdmin cluster nodes
UDPInbound6081GENEVE EncapsulationSelf
TCPInbound443*Cluster managementAll
TCPBoth4240CNI health checkAll
TCPInbound7946Metal LB health checkLB nodes
UDPInbound7946Metal LB health checkLB nodes

* This port can be configured in the cluster config, using the controlPlaneLBPort field.

Multi-cluster port requirements

In a multi-cluster configuration, added clusters must include the following ports to communicate with the admin cluster.

ProtocolDirectionPort rangePurposeUsed by
TCPInbound22Provisioning and updates of cluster nodesAll nodes
TCPInbound443*Kubernetes API server for added clusterControl plane, LB nodes

* This port can be configured in the cluster config, using the controlPlaneLBPort field.

Configuring firewalld ports

Starting with Anthos clusters on bare metal 1.7.0, you are not required to disable firewalld to run Anthos clusters on bare metal on Red Hat Enterprise Linux (RHEL) or CentOS. To use firewalld, you must open the UDP and TCP ports used by master, worker, and load balancer nodes as described in Port usage on this page. The following example configurations show how you can open ports with firewall-cmd, the firewalld command line client.

Master node example configuration

The following example block of commands shows how you can open the needed ports on servers running master nodes:

firewall-cmd --permanent --zone=public --add-port=22/tcp
firewall-cmd --permanent --zone=public --add-port=4240/tcp
firewall-cmd --permanent --zone=public --add-port=6444/tcp
firewall-cmd --permanent --zone=public --add-port=6081/udp
firewall-cmd --permanent --zone=public --add-port=10250-10252/tcp
firewall-cmd --permanent --zone=public --add-port=2379-2380/tcp
firewall-cmd --permanent --zone=public --add-port=443/tcp
firewall-cmd --permanent --zone=public --add-port=30000-32767/tcp
firewall-cmd --reload

Worker node example configuration

The following example block of commands shows how you can open the needed ports on servers running worker nodes:

firewall-cmd --permanent --zone=public --add-port=22/tcp
firewall-cmd --permanent --zone=public --add-port=4240/tcp
firewall-cmd --permanent --zone=public --add-port=6444/tcp
firewall-cmd --permanent --zone=public --add-port=6081/udp
firewall-cmd --permanent --zone=public --add-port=10250/tcp
firewall-cmd --permanent --zone=public --add-port=443/tcp
firewall-cmd --permanent --zone=public --add-port=30000-32767/tcp
firewall-cmd --reload

Load balancer node example configuration

The following example block of commands shows how you can open the needed ports on servers running load balancer nodes:

firewall-cmd --permanent --zone=public --add-port=22/tcp
firewall-cmd --permanent --zone=public --add-port=4240/tcp
firewall-cmd --permanent --zone=public --add-port=6444/tcp
firewall-cmd --permanent --zone=public --add-port=7946/tcp
firewall-cmd --permanent --zone=public --add-port=7946/udp
firewall-cmd --permanent --zone=public --add-port=6081/udp
firewall-cmd --permanent --zone=public --add-port=10250/tcp
firewall-cmd --permanent --zone=public --add-port=443/tcp
firewall-cmd --permanent --zone=public --add-port=30000-32767/tcp
firewall-cmd --reload