Configure external load balancers

External load balancers (ELB) expose services to access from outside the organization from a pool's IP addresses assigned to the organization from the larger instance-external IP pool.

ELB Virtual IP (VIP) addresses don't conflict between organizations and are unique across all organizations. For this reason, you must use ELB services only for services that clients outside the organization necessarily have to access.

Workloads running inside the organization can access ELB services as long as you enable the workloads to exit the organization. This traffic pattern effectively requires outbound traffic from the organization before returning to the internal service.

Before you begin

To configure ELB services, you must have the following:

Own the project you are configuring the load balancer for. For more information, see Create a project.
A customized ProjectNetworkPolicy (PNP) ingress policy to allow traffic to this ELB service. For more information, see Configure PNP to allow traffic to ELB.
The necessary identity and access roles:
- Project NetworkPolicy Admin: has access to manage project network policies in the project namespace Ask your Organization IAM Admin to grant you the Project NetworkPolicy Admin (project-networkpolicy-admin) role.
- Load Balancer Admin: Ask your Organization IAM Admin to grant you the Load Balancer Admin (load-balancer-admin) role.
- Global Load Balancer Admin: Ask your Organization IAM Admin to grant you the Global Load Balancer Admin (global-load-balancer-admin) role. For more information, see Predefined role descriptions.

Configure PNP to allow traffic to ELB

For ELB services to function, you must configure and apply your own customized ProjectNetworkPolicy ingress policy to allow traffic to the workloads of this ELB service. Network policies control access to your workloads, not the load balancer itself. ELBs expose workloads to your customer network, requiring explicit network policies to allow external traffic to the workload port, such as 8080.

Specify the external CIDR address to allow traffic to the workloads of this ELB:

kubectl --kubeconfig MANAGEMENT_API_SERVER apply -f - <<EOF
apiVersion: networking.gdc.goog/v1
kind: ProjectNetworkPolicy
metadata:
  namespace: PROJECT
  name: allow-inbound-traffic-from-external
spec:
  policyType: Ingress
  subject:
    subjectType: UserWorkload
  ingress:
  - from:
    - ipBlock:
        cidr: CIDR
    ports:
    - protocol: TCP
      port: PORT
EOF

Replace the following:

MANAGEMENT_API_SERVER: the kubeconfig path of the Management API server's kubeconfig path. If you have not yet generated a kubeconfig file for the API server in your targeted zone, see Sign in for details.
PROJECT: the name of your GDC project.
CIDR: the external CIDR that the ELB needs to be accessed from. This policy is required as the external load balancer uses Direct Server Return (DSR), which preserves the source external IP address and bypasses the load balancer on the return path. For more information, see Create a global ingress firewall rule for cross-organization traffic.
PORT: the backend port on the pods behind the load balancer. This value is found in the .spec.ports[].targetPortfield of the manifest for the Service resource.

Create an external load balancer

You can create global or zonal ELBs. The scope of global ELBs span across a GDC universe. The scope of zonal ELBs is limited the zones specified at the time of creation. For more information, see Global and zonal load balancers.

Create ELBs using three different methods in GDC:

Use the gdcloud CLI to create global or zonal ELBs.
Use the Networking Kubernetes Resource Model (KRM) API to create global or zonal ELBs.
Use the Kubernetes Service directly in the Kubernetes cluster. This method is only available for zonal ELBs.

You can target pod or VM workloads using the KRM API and gdcloud CLI. You can only target workloads in the cluster where the Service object is created when you use the Kubernetes Service directly in Kubernetes cluster.

Create a zonal ELB

Create a zonal ELB using the gdcloud CLI, the KRM API, or the Kubernetes Service in the Kubernetes cluster:

gdcloud

Create an ELB that targets pod or VM workloads using the gdcloud CLI.

This ELB targets all of the workloads in the project matching the label defined in the Backend object.

To create an ELB using the gdcloud CLI, follow these steps:

Create a Backend resource to define the endpoint for the ELB:
```
gdcloud compute backends create BACKEND_NAME \
  --labels=LABELS \
  --project=PROJECT_NAME \
  --zone=ZONE \
  --cluster=CLUSTER_NAME
```
Replace the following:
- BACKEND_NAME: your chosen name for the backend resource, such as my-backend.
- LABELS: A selector defining which endpoints between pods and VMs to use for this backend resource. For example, app=web.
- PROJECT_NAME: the name of your project.
- ZONE: the zone to create the backend resource in.
- CLUSTER_NAME: the cluster to which the scope of the defined selectors is limited to. If this field is not specified, all of the endpoints with the given label are selected. This is an optional field.
Skip this step if this ELB is for pod workloads. If you are configuring an ELB for VM workloads, define a health check for the ELB:
```
gdcloud compute health-checks create tcp HEALTH_CHECK_NAME \
  --check-interval=CHECK_INTERVAL \
  --healthy-threshold=HEALTHY_THRESHOLD \
  --timeout=TIMEOUT \
  --unhealthy-threshold=UNHEALTHY_THRESHOLD \
  --port=PORT \
  --zone=ZONE
```
Replace the following:
- HEALTH_CHECK_NAME: your chosen name for the health check resource, such as my-health-check.
- CHECK_INTERVAL: the amount of time in seconds from the start of one probe to the start of the next one. The default value is to 5. This field is optional.
- HEALTHY_THRESHOLD: the time to wait before claiming failure. The default value is to 5. This field is optional.
- TIMEOUT: the amount of time in seconds to wait before claiming failure. The default value is to 5. This field is optional.
- UNHEALTHY_THRESHOLD: the number of sequential probes that must fail for the endpoint to be considered unhealthy. The default value is to 2. This field is optional.
- PORT: the port on which the health check is performed. The default value is to 80. This field is optional.
- ZONE: the zone you are creating this ELB in.
Create a BackendService resource and add to it the previously created Backend resource:
```
gdcloud compute backend-services create BACKEND_SERVICE_NAME \
  --project=PROJECT_NAME \
  --target-ports=TARGET_PORTS \
  --zone=ZONE \
  --health-check=HEALTH_CHECK_NAME
```
Replace the following:
- BACKEND_SERVICE_NAME: the chosen name for this backend service.
- TARGET_PORTS: a comma-separated list of target ports that this backend service translates, where each target port specifies the protocol, the port on the forwarding rule, and the port on the backend instance. You can specify multiple target ports. This field must be in the format protocol:port:targetport, such as TCP:80:8080. This field is optional.
- HEALTH_CHECK_NAME: the name of the health check resource. This field is optional. Only include this field if you are configuring an ELB for VM workloads.

Add the BackendService resource to the previously created Backend resource:

gdcloud compute backend-services add-backend BACKEND_SERVICE_NAME \
  --backend=BACKEND_NAME \
  --project=PROJECT_NAME \
  --zone=ZONE

Create an external ForwardingRule resource that defines the VIP the service is available at:
```
gdcloud compute forwarding-rules create FORWARDING_RULE_EXTERNAL_NAME \
  --backend-service=BACKEND_SERVICE_NAME \
  --cidr=CIDR \
  --ip-protocol-port=PROTOCOL_PORT \
  --load-balancing-scheme=EXTERNAL \
  --zone=ZONE \
  --project=PROJECT_NAME
```
Replace the following:
- FORWARDING_RULE_EXTERNAL_NAME with your chosen name for the forwarding rule.
- CIDR: This field is optional. If not specified, an IPv4/32 CIDR is automatically reserved from the zonal IP pool. Specify the name of a Subnet resource in the same namespace as this forwarding rule. A Subnet resource represents the request and allocation information of a zonal subnet. For more information on Subnet resources, see Example custom resources.
- PROTOCOL_PORT: the protocol and port to expose on the forwarding rule. This field must be in the format ip-protocol=TCP:80. The exposed port must be the same as what the actual application is exposing inside of the container.
To verify the configured ELB, confirm the Ready condition on each of the created objects. To get the assigned IP address of the load balancer, describe the forwarding rule:
```
gdcloud compute forwarding-rules describe FORWARDING_RULE_EXTERNAL_NAME
```
Test the traffic with a curl request to the VIP at port 444.

API

Create an ELB that targets pod or VM workloads using the KRM API. This ELB targets all of the workloads in the project matching the label defined in the Backend object.

To create a zonal ELB using the KRM API, follow these steps:

Create a Backend resource to define the endpoints for the ELB. Create Backend resources for each zone the workloads are placed in:
```
kubectl --kubeconfig MANAGEMENT_API_SERVER apply -f - <<EOF
apiVersion: networking.gdc.goog/v1
kind: Backend
metadata:
  namespace: PROJECT_NAME
  name: BACKEND_NAME
spec:
  clusterName: CLUSTER_NAME
  endpointsLabels:
    matchLabels:
      app: server
EOF
```
Replace the following:
- MANAGEMENT_API_SERVER: the kubeconfig path of the Management API server's kubeconfig path. If you have not yet generated a kubeconfig file for the API server in your targeted zone, see Sign in for details.
- PROJECT_NAME: the name of your project.
- BACKEND_NAME: the name of the Backend resource.
- CLUSTER_NAME: This is an optional field. This field specifies the cluster to which the scope of the defined selectors are limited to. This field does not apply to VM workloads. If a Backend resource doesn't have the clusterName field included, the specified labels apply to all of the workloads in the project.
You can use the same Backend resource for each zone, or create Backend resources with different label sets for each zone.
Skip this step if this ELB is for pod workloads. If you are configuring an ELB for VM workloads, define a health check for the ELB:
```
kubectl --kubeconfig MANAGEMENT_API_SERVER apply -f - <<EOF
apiVersion: networking.gdc.goog/v1
kind: HealthCheck
metadata:
  namespace: PROJECT_NAME
  name: HEALTH_CHECK_NAME
spec:
  tcpHealthCheck:
    port: PORT
  timeoutSec: TIMEOUT
  checkIntervalSec: CHECK_INTERVAL
  healthyThreshold: HEALTHY_THRESHOLD
  unhealthyThreshold: UNHEALTHY_THRESHOLD
EOF
```
Replace the following:
- HEALTH_CHECK_NAME: your chosen name for the health check resource, such as my-health-check.
- PORT: the port on which the health check is performed. The default value is to 80.
- TIMEOUT: the amount of time in seconds to wait before claiming failure. The default value is to 5.
- CHECK_INTERVAL: the amount of time in seconds from the start of one probe to the start of the next one. The default value is to 5.
- HEALTHY_THRESHOLD: the number of sequential probes that must pass for the endpoint to be considered healthy. The default value is to 2.
- UNHEALTHY_THRESHOLD: the number of sequential probes that must fail for the endpoint to be considered unhealthy. The default value is to 2.
Create a BackendService object using the previously created Backend resource. If you are configuring an ELB for VM workloads, include the HealthCheck resource.
```
kubectl --kubeconfig MANAGEMENT_API_SERVER apply -f - <<EOF
apiVersion: networking.gdc.goog/v1
kind: BackendService
metadata:
  namespace: PROJECT_NAME
  name: BACKEND_SERVICE_NAME
spec:
  backendRefs:
  - name: BACKEND_NAME
  healthCheckName: HEALTH_CHECK_NAME
EOF
```
Replace the following:
- BACKEND_SERVICE_NAME: the chosen name for your BackendService resource.
- HEALTH_CHECK_NAME: the name of your previously created HealthCheck resource. Don't include this field if you are configuring an ELB for pod workloads.
Create an external ForwardingRule resource defining the VIP the service is available at.
```
kubectl --kubeconfig MANAGEMENT_API_SERVER apply -f - <<EOF
apiVersion: networking.gdc.goog/v1
kind: ForwardingRuleExternal
metadata:
  namespace: PROJECT_NAME
  Name: FORWARDING_RULE_EXTERNAL_NAME
spec:
  cidrRef: CIDR
  ports:
  - port: PORT
    Protocol: PROTOCOL
  backendServiceRef:
    name: BACKEND_SERVICE_NAME
EOF
```
Replace the following:
- FORWARDING_RULE_EXTERNAL_NAME: the chosen name for your ForwardingRuleExternal resource.
- CIDR: This field is optional. If not specified, an IPv4/32 CIDR is automatically reserved from the zonal IP pool. Specify the name of a Subnet resource in the same namespace as this forwarding rule. A Subnet resource represents the request and allocation information of a zonal subnet. For more information on Subnet resources, see Example custom resources.
- PORT: Use the ports field to specify an array of L4 ports for which packets are forwarded to the backends configured with this forwarding rule. At least one port has to be specified. the protocol and port to expose on the forwarding rule. Use the port field to specify a port number. The exposed port must be the same as what the actual application is exposing inside of the container.
- PROTOCOL: the protocol to use for the forwarding rule, such as TCP. An entry in the ports array must look like the following:
```
ports:
- port: 80
  protocol: TCP
```
To validate the configured ELB, confirm the Ready condition on each of the created objects. Try and test the traffic with a curl request to the VIP.

To get the VIP, use kubectl get:

kubectl --kubeconfig MANAGEMENT_API_SERVER get forwardingruleexternal -n PROJECT_NAME

The output looks like the following:

NAME           BACKENDSERVICE                               CIDR              READY
elb-name       BACKEND_SERVICE_NAME        10.200.32.59/32   True

Kubernetes Service

You can create ELBs in GDC by creating a Kubernetes Service of type LoadBalancer in a Kubernetes cluster.

To create an ELB service, do the following:

Create a YAML file for the Service definition of type LoadBalancer.

Note: Contrary to the process for internal load balancers, don't add annotations on the file. If you don't specify the internal annotation, then the service is external-facing by default.

The following Service object is an example of an ELB service:
```
apiVersion: v1
kind: Service
metadata:
  name: ELB_SERVICE_NAME
  namespace: PROJECT_NAME
spec:
  ports:
  - port: 1235
    protocol: TCP
    targetPort: 1235
  selector:
    k8s-app: my-app
  type: LoadBalancer
```
Replace the following:
- ELB_SERVICE_NAME: the name of the ELB service.
- PROJECT_NAME: the namespace of your project that contains the backend workloads.
The port field configures the frontend port you expose on the VIP address. The targetPort field configures the backend port to which you want to forward the traffic on the backend workloads. The load balancer supports Network Address Translation (NAT). The frontend and backend ports can be different.
On the selector field of the Service definition, specify pods or virtual machines as the backend workloads.

The selector defines which workloads to take as backend workloads for this service, based on matching the labels you specify with labels on the workloads. The Service can only select backend workloads in the same project and same cluster where you define the Service.

For more information about service selection, see https://kubernetes.io/docs/concepts/services-networking/service/.
Save the Service definition file in the same project as the backend workloads.
Apply the Service definition file to the cluster:
```
kubectl apply -f ELB_FILE
```
Replace ELB_FILE with the name of the Service definition file for the ELB service.

When you create an ELB, the service gets two IP addresses. One is an internal IP address accessible only from within the same cluster. The other is the external IP address, accessible from inside and outside the organization. You can obtain the IP addresses of the ELB service by viewing the service status:
```
kubectl -n PROJECT_NAME get svc ELB_SERVICE_NAME
```
Replace the following:
- PROJECT_NAME: the namespace of your project that contains the backend workloads.
- ELB_SERVICE_NAME: the name of the ELB service.
You must obtain an output similar to the following example:
```
NAME                    TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)          AGE
elb-service             LoadBalancer   10.0.0.1      20.12.1.11      1235:31931/TCP   22h
```
The EXTERNAL-IP is the IP address of the service that is accessible from outside the organization.

If you don't obtain an output, ensure that you created the ELB service successfully.

Create a global ELB

Create a global ELB using the gdcloud CLI or the KRM API.