Migrate from the Seesaw load balancer to MetalLB

Version 1.16 

• Latest version
• Version 1.16
• Earlier versions (unsupported)

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

This document shows how to migrate from the Seesaw load balancer to the MetalLB load balancer.

Using MetalLB has several benefits compared to other load balancing options.

Warning: Don't migrate to MetalLB if you are relying on Seesaw for client IP address preservation. More specifically, don't migrate to MetalLB if there are Services in the cluster with spec.externalTrafficPolicy set to Local. You can replace USER_CLUSTER_KUBECONFIG with the path of the user cluster kubeconfig file and run the following command to check:

kubectl --kubeconfig USER_CLUSTER_KUBECONFIG \
  get svc -A -o yaml | grep "externalTrafficPolicy: Local"

User cluster migration

In your user cluster configuration file, choose a node pool, and set enableLoadBalancer to true:

nodePools:
- name: pool-1
  replicas: 3
  enableLoadBalancer: true

Update the cluster:

gkectl update cluster --kubeconfig ADMIN_CLUSTER_KUBECONFIG --config USER_CLUSTER_CONFIG

Replace the following:

• ADMIN_CLUSTER_KUBECONFIG: the path of the admin cluster kubeconfig file

• USER_CLUSTER_CONFIG: the path of the user cluster configuration file

Next, remove the Seesaw sections from the file, and add a MetalLB section.

Then update the cluster again:

gkectl update cluster --kubeconfig ADMIN_CLUSTER_KUBECONFIG --config USER_CLUSTER_CONFIG

Verify that the metallb components are running successfully:

kubectl --kubeconfig USER_CLUSTER_KUBECONFIG get pods \
    --namespace kube-system --selector app=metallb

The output shows Pods for the MetalLB controller and speaker. For example:

metallb-controller-744884bf7b-rznr9   1/1     Running
metallb-speaker-6n8ws                 1/1     Running
metallb-speaker-nb52z                 1/1     Running
metallb-speaker-rq4pp                 1/1     Running

Important: The migration process will stop network connections to all VIPs in the user cluster for Kubernetes Services of type LoadBalancer for two to ten minutes. After the migration is complete, the connections will work again.

After a successful migration, manually delete the Seesaw VMs, which are already powered off, for the user cluster. You can find the Seesaw VM names in the vmnames section of the seesaw-for-[USERCLUSTERNAME].yaml file in your configuration directory.

Example: User cluster, static IP addresses

Suppose you have a user cluster that uses static IP addresses for its cluster nodes. Also suppose that the cluster has two Services of type LoadBalancer, and the external addresses for those Services are 172.16.21.41 and 172.16.21.45.

Adjust the user cluster configuration file as follows:

• Keep the network.hostConfig section.
• Set loadBalancer.kind to MetalLB.
• Remove the loadBalancer.seesaw section.
• Add a loadBalancer.metalLB section.

Example:

network:
  hostConfig:
    dnsServers:
    - "172.16.255.1"
    - "172.16.255.2"
    ntpServers:
    - "216.239.35.0"
loadBalancer:
  vips:
    controlPlaneVIP: "172.16.20.30"
    ingressVIP: "172.16.20.31"
  kind: MetalLB Seesaw
  seesaw:
    ipBlockFilePath: "user-cluster-1-ipblock.yaml"
    vrid: 1
    masterIP: ""
    cpus: 4
    memoryMB: 3072
  metalLB:
    addressPools:
    - name: "address-pool-1"
      addresses:
      - "172.16.20.31/32"
      - "172.16.20.40 - 172.16.21.49"
  

Key points from the preceding example:

• Even though the cluster will no longer use the Seesaw load balancer, the network.hostConfig section is needed, because the cluster nodes use static IP addresses.

• The value of ingressVIP appears in the MetalLB address pool.

• The external IP addresses, 172.16.21.41 and 172.16.21.45, for the existing Services of type LoadBalancer are included in the MetalLB address pool.

Example: User cluster, DHCP

Suppose you have a user cluster that uses DHCP for its cluster nodes. Also suppose that the cluster has two Services of type LoadBalancer, and the external addresses for those Services are 172.16.21.61 and 172.16.21.65.

Adjust the user cluster configuration file as follows:

• Remove the network.hostConfig section.
• Set loadBalancer.kind to MetalLB.
• Remove the loadBalancer.seesaw section.
• Add a loadBalancer.metalLB section.

Example:

network:
  hostConfig:
    dnsServers:
    - "172.16.255.1"
    - "172.16.255.2"
    ntpServers:
    - "216.239.35.0"
loadBalancer:
  vips:
    controlPlaneVIP: "172.16.20.50"
    ingressVIP: "172.16.20.51"
  kind: MetalLB Seesaw
  seesaw:
    ipBlockFilePath: "user-cluster-2-ipblock.yaml"
    vrid: 1
    masterIP: ""
    cpus: 4
    memoryMB: 3072
  metalLB:
    addressPools:
    - name: "address-pool-1"
      addresses:
      - "172.16.20.51/32"
      - "172.16.20.60 - 172.16.21.69"
  

Key points from the preceding example:

• The cluster will no longer use the Seesaw load balancer, and the cluster does not use static IP addresses for its cluster nodes. So the network.hostConfig section is not needed.

• The value of ingressVIP appears in the MetalLB address pool.

• The external IP addresses, 172.16.21.61 and 172.16.21.65, for the existing Services of type LoadBalancer are included in the MetalLB address pool.

Admin cluster migration

In your admin cluster configuration file, set loadBalancer.kind to MetalLB, and remove the loadBalancer.seesaw section.

Update the cluster:

gkectl update admin --kubeconfig  ADMIN_CLUSTER_KUBECONFIG --config ADMIN_CLUSTER_CONFIG

Replace the following:

• ADMIN_CLUSTER_KUBECONFIG: the path of the admin cluster kubeconfig file

• ADMIN_CLUSTER_CONFIG: the path of the admin cluster configuration file

Verify that the metallb components are running successfully:

kubectl --kubeconfig ADMIN_CLUSTER_KUBECONFIG get pods \
    --namespace kube-system --selector app=metallb

The output shows Pods for the MetalLB controller and speaker. For example:

metallb-controller-744884bf7b-rznr9   1/1     Running
metallb-speaker-6n8ws                 1/1     Running
metallb-speaker-nb52z                 1/1     Running
metallb-speaker-rq4pp                 1/1     Running

Important: The migration process will stop network connections to all VIPs in the admin cluster for Kubernetes Services of type LoadBalancer for two to ten minutes. After the migration is complete, the connections will work again.

After a successful migration, manually delete the Seesaw VMs, which are already powered off, for the admin cluster. You can find the Seesaw VM names in the vmnames section of the seesaw-for-gke-admin.yaml file in your configuration directory.

Example: Admin cluster, static IP addresses

Suppose you have an admin cluster that uses static IP addresses for its cluster nodes.

Adjust the admin cluster configuration file as follows:

• Keep the network.hostConfig section.
• Set loadBalancer.kind to MetalLB.
• Remove the loadBalancer.seesaw section.

Example:

network:
  hostConfig:
    dnsServers:
    - "172.16.255.1"
    - "172.16.255.2"
    ntpServers:
    - "216.239.35.0"
loadBalancer:
  vips:
    controlPlaneVIP: "172.16.20.30"
  kind: MetalLB Seesaw
  seesaw:
    ipBlockFilePath: "user-cluster-1-ipblock.yaml"
    vrid: 1
    masterIP: ""
    cpus: 4
    memoryMB: 3072
  

Key point from the preceding example:

• Even though the cluster will no longer use the Seesaw load balancer, the network.hostConfig section is needed, because the cluster nodes use static IP addresses.

Example: Admin cluster, DHCP

Suppose you have an admin cluster that uses DHCP for its cluster nodes.

Adjust the admin cluster configuration file as follows:

• Remove the network.hostConfig section.
• Set loadBalancer.kind to MetalLB.
• Remove the loadBalancer.seesaw section.

Example:

network:
  hostConfig:
    dnsServers:
    - "172.16.255.1"
    - "172.16.255.2"
    ntpServers:
    - "216.239.35.0"
loadBalancer:
  vips:
    controlPlaneVIP: "172.16.20.30"
  kind: MetalLB Seesaw
  seesaw:
    ipBlockFilePath: "user-cluster-1-ipblock.yaml"
    vrid: 1
    masterIP: ""
    cpus: 4
    memoryMB: 3072
  

Key point from the preceding example:

• The cluster will no longer use the Seesaw load balancer, and the cluster does not use static IP addresses for its cluster nodes. So the network.hostConfig section is not needed.

Troubleshooting

If gkectl update fails during user cluster migration, and the metallb Pods are not running in the user cluster, manually power on the user cluster Seesaw VMs. This will re-establish traffic to currently used VIPs. But newly created VIPs might not be served by the Seesaw VMs if the load-balancer-seesaw Pod is not running. If that is the case, create a support ticket.

If gkectl update fails during admin cluster migration, and the metallb Pods are not running in the admin cluster, manually power on the admin cluster Seesaw VMs. This might allow traffic to currently used control plane VIPs for user clusters to work again. But the VIP for the control plane of the admin cluster itself might not work. In that case, edit the kubeconfig file of the admin cluster to directly use the IP address of the admin cluster control-plane node.

Also, in the kube-system namespace, change the kube-apiserver Service type from ClusterIP to LoadBalancer. If necessary, create a support ticket.