Configure external IP addresses for GKE on VMware with F5 BIG-IP load balancers

The default Anthos Service Mesh installation on GKE on VMware assumes that external IP addresses are automatically allocated for LoadBalancer services. This is not true for GKE on VMware with F5 BIG-IP load balancers. Because of this, you need to allocate one or more external IP addresses, depending on your service mesh configuration:

  • One external IP address for the Anthos Service Mesh ingress Gateway resource, for example the gateway that your customers use to access your workloads from across the internet.
  • Another external IP address for your clusters to communicate with each other if they exist on different networks within your service mesh. This is referred to as the east-west gateway.

Configuring the ingress gateway IP address

To configure an external IP address for the ingress gateway, use the instructions in one of the following sections, depending on your GKE on VMware load balancing mode:

    Integrated mode

  1. Patch the istio-ingressgateway Service's configuration with the external IP address for the ingress gateway: 
    kubectl patch svc istio-ingressgateway -n istio-system --type='json' -p '[{"op": "add", "path": "/spec/loadBalancerIP", "value": "INGRESS_GATEWAY_IP"}]'

    2. Manual mode

  2. View the istio-ingressgateway service's configuration in your shell: 
    kubectl get svc -n istio-system istio-ingressgateway -o yaml
    Each of the ports for Anthos Service Mesh's gateways are displayed. The command output is like the following: 
      ...
  ports:
  - name: status-port
    nodePort: 30391
    port: 15020
    protocol: TCP
    targetPort: 15020
  - name: http2
    nodePort: 31380
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    nodePort: 31390
    port: 443
    protocol: TCP
    targetPort: 443
  - name: tcp
    nodePort: 31400
    port: 31400
    protocol: TCP
    targetPort: 31400
  - name: https-kiali
    nodePort: 31073
    port: 15029
    protocol: TCP
    targetPort: 15029
  - name: https-prometheus
    nodePort: 30253
    port: 15030
    protocol: TCP
    targetPort: 15030
  - name: https-grafana
    nodePort: 30050
    port: 15031
    protocol: TCP
    targetPort: 15031
  - name: https-tracing
    nodePort: 31204
    port: 15032
    protocol: TCP
    targetPort: 15032
  - name: tls
    nodePort: 30158
    port: 15443
    protocol: TCP
    targetPort: 15443
  ...
  3. Expose these ports through your load balancer.

    For example, the service port named http2 has port 80 and nodePort 31380. Suppose the node addresses for your user cluster are 192.168.0.10, 192.168.0.11, and 192.168.0.12, and your load balancer's VIP is 203.0.113.1.

    Configure your load balancer so that traffic sent to 203.0.113.1:80 is forwarded to 192.168.0.10:31380, 192.168.0.11:31380, or 192.168.0.12:31380. You can select the service ports that you want to expose on this given VIP.

Confirm that the ingress gateway was assigned an external IP address. There might be a slight delay that requires you to repeat this command until you see the expected result:

kubectl --context="${CTX_CLUSTER1}" get svc istio-ingressgateway -n istio-system

The expected output is:

NAME                    TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)   AGE
istio-ingressgateway   LoadBalancer   10.80.6.124   34.75.71.237   ...       51s

Configuring the east-west gateway IP address

To configure an external IP address for the east-west gateway, use the instructions in one of the following sections, depending on your GKE on VMware load balancing mode:

    Integrated mode

  1. Patch the istio-eastwestgateway Service's configuration with the external IP address for the east-west gateway: 
    kubectl patch svc istio-eastwestgateway -n istio-system --type='json' -p '[{"op": "add", "path": "/spec/loadBalancerIP", "value": "EAST_WEST_GATEWAY_IP"}]'

    2. Manual mode

  2. View the istio-eastwestgateway service's configuration in your shell: 
    kubectl get svc -n istio-system istio-eastwestgateway -o yaml
    Each of the ports for Anthos Service Mesh's gateways are displayed. The command output is like the following: 
      ports:
  - name: status-port
    nodePort: 31781
    port: 15021
    protocol: TCP
    targetPort: 15021
  - name: tls
    nodePort: 30498
    port: 15443
    protocol: TCP
    targetPort: 15443
  - name: tls-istiod
    nodePort: 30879
    port: 15012
    protocol: TCP
    targetPort: 15012
  - name: tls-webhook
    nodePort: 30336
    port: 15017
    protocol: TCP
    targetPort: 15017
  ...
  3. Expose these ports through your load balancer.

    For example, the service port named http2 has port 80 and nodePort 31380. Suppose the node addresses for your user cluster are 192.168.0.10, 192.168.0.11, and 192.168.0.12, and your load balancer's VIP is 203.0.113.1.

    Configure your load balancer so that traffic sent to 203.0.113.1:80 is forwarded to 192.168.0.10:31380, 192.168.0.11:31380, or 192.168.0.12:31380. You can select the service ports that you want to expose on this given VIP.

Confirm that the east-west gateway was assigned an external IP address. There might be a slight delay that requires you to run repeat this command until you see the expected result:

kubectl --context="${CTX_CLUSTER1}" get svc istio-eastwestgateway -n istio-system

The expected output is:

NAME                    TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)   AGE
istio-eastwestgateway   LoadBalancer   10.80.6.124   34.75.71.237   ...       51s

What's next?