This guide uses an example to teach the fundamentals of Google Cloud Internal TCP/UDP Load Balancing. Before following this guide, familiarize yourself with the following:
Permissions
To follow this guide, you need to create instances and modify a network in a project. You should be either a project owner or editor, or you should have all of the following Compute Engine IAM roles:
Task | Required Role |
---|---|
Create networks, subnets, and load balancer components | Network Admin |
Add and remove firewall rules | Security Admin |
Create instances | Compute Instance Admin |
For more information, see the following guides:
Setup
This guide shows you how to configure and test an internal TCP/UDP load balancer. The steps in this section describe how to configure the following:
- A sample VPC network with custom subnets
- Firewall rules that allow incoming connections to backend VMs
- Four backend VMs:
- Two VMs in an unmanaged instance group in zone
us-west1-a
- Two VMs in an unmanaged instance group in zone
us-west1-c
- Two VMs in an unmanaged instance group in zone
- One client VM to test connections
- The following internal TCP/UDP load balancer components:
- A health check for the backend service
- An internal backend service in the
us-west1
region to manage connection distribution to the two zonal instance groups - An internal forwarding rule and internal IP address for the frontend of the load balancer
The architecture for this example looks like this:
Configuring a network, region, and subnet
The example internal TCP/UDP load balancer described on this page is created in a
custom mode VPC network named
lb-network
.
This example's backend VMs and load balancer's components are located in this region and subnet:
- Region:
us-west1
- Subnet:
lb-subnet
, with primary IP address range10.1.2.0/24
To demonstrate global access, this example creates a second test client VM in a different region and subnet:
- Region:
europe-west1
- Subnet:
europe-subnet
, with primary IP address range10.3.4.0/24
To create the example network and subnet, follow these steps.
Console
- In the Google Cloud Console, go to the VPC networks page.
Go to VPC networks - Click Create VPC network.
- Enter a Name of
lb-network
. - In the Subnets section:
- Set the Subnet creation mode to Custom.
- In the New subnet section, enter the following information:
- Name:
lb-subnet
- Region:
us-west1
- IP address range:
10.1.2.0/24
- Click Done.
- Name:
- Click Add subnet and enter the following information:
- Name:
europe-subnet
- Region:
europe-west1
- IP address range:
10.3.4.0/24
- Click Done.
- Name:
- Click Create.
gcloud
Create the custom VPC network:
gcloud compute networks create lb-network --subnet-mode=custom
- Within the
lb-network
network, create a subnet for backends in theus-west1
region and another subnet for testing global access in theeurope-west1
region:
gcloud compute networks subnets create lb-subnet \ --network=lb-network \ --range=10.1.2.0/24 \ --region=us-west1
gcloud compute networks subnets create europe-subnet \ --network=lb-network \ --range=10.3.4.0/24 \ --region=europe-west1
- Within the
api
Make a POST
request to the
networks.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/global/networks
{
"routingConfig": {
"routingMode": "REGIONAL"
},
"name": "lb-network",
"autoCreateSubnetworks": false
}
Make two POST
requests to the
subnetworks.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/regions/us-west1/subnetworks
{
"name": "lb-subnet",
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"ipCidrRange": "10.1.2.0/24",
"privateIpGoogleAccess": false
}
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/regions/europe-west1/subnetworks
{
"name": "europe-subnet",
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"ipCidrRange": "10.3.4.0/24",
"privateIpGoogleAccess": false
}
Configuring firewall rules
This example uses the following firewall rules:
fw-allow-lb-access
: An ingress rule, applicable to all targets in the VPC network, allowing traffic from sources in the10.1.2.0/24
and10.3.4.0/24
ranges. This rule allows incoming traffic from any client located in either of the two subnets. It later allows you to configure and test global access.fw-allow-ssh
: An ingress rule, applicable to the instances being load balanced, that allows incoming SSH connectivity on TCP port 22 from any address. You can choose a more restrictive source IP range for this rule; for example, you can specify just the IP ranges of the system from which you will initiating SSH sessions. This example uses the target tagallow-ssh
to identify the VMs to which it should apply.fw-allow-health-check
: An ingress rule, applicable to the instances being load balanced, that allows traffic from the Google Cloud health checking systems (130.211.0.0/22
and35.191.0.0/16
). This example uses the target tagallow-health-check
to identify the instances to which it should apply.
Without these firewall rules, the default deny ingress rule blocks incoming traffic to the backend instances.
Console
- In the Google Cloud Console, go to the Firewall page.
Go to Firewall - Click Create firewall rule and enter the following
information to create the rule to allow subnet traffic:
- Name:
fw-allow-lb-access
- Network:
lb-network
- Priority:
1000
- Direction of traffic: ingress
- Action on match: allow
- Targets: All instances in the network
- Source filter:
IP ranges
- Source IP ranges:
10.1.2.0/24
and10.3.4.0/24
- Protocols and ports: Allow all
- Name:
- Click Create.
- Click Create firewall rule again to create the rule to allow incoming
SSH connections:
- Name:
fw-allow-ssh
- Network:
lb-network
- Priority:
1000
- Direction of traffic: ingress
- Action on match: allow
- Targets: Specified target tags
- Target tags:
allow-ssh
- Source filter:
IP ranges
- Source IP ranges:
0.0.0.0/0
- Protocols and ports: Choose Specified protocols and ports then
type:
tcp:22
- Name:
- Click Create.
- Click Create firewall rule a third time to create the rule to allow
Google Cloud health checks:
- Name:
fw-allow-health-check
- Network:
lb-network
- Priority:
1000
- Direction of traffic: ingress
- Action on match: allow
- Targets: Specified target tags
- Target tags:
allow-health-check
- Source filter:
IP ranges
- Source IP ranges:
130.211.0.0/22
and35.191.0.0/16
- Protocols and ports: Allow all
- Name:
- Click Create.
gcloud
Create the
fw-allow-lb-access
firewall rule to allow communication from with the subnet:gcloud compute firewall-rules create fw-allow-lb-access \ --network=lb-network \ --action=allow \ --direction=ingress \ --source-ranges=10.1.2.0/24,10.3.4.0/24 \ --rules=tcp,udp,icmp
Create the
fw-allow-ssh
firewall rule to allow SSH connectivity to VMs with the network tagallow-ssh
. When you omitsource-ranges
, Google Cloud interprets the rule to mean any source.gcloud compute firewall-rules create fw-allow-ssh \ --network=lb-network \ --action=allow \ --direction=ingress \ --target-tags=allow-ssh \ --rules=tcp:22
Create the
fw-allow-health-check
rule to allow Google Cloud health checks.gcloud compute firewall-rules create fw-allow-health-check \ --network=lb-network \ --action=allow \ --direction=ingress \ --target-tags=allow-health-check \ --source-ranges=130.211.0.0/22,35.191.0.0/16 \ --rules=tcp,udp,icmp
api
Create the fw-allow-lb-access
firewall rule by making a POST
request to
the firewalls.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/global/firewalls
{
"name": "fw-allow-lb-access",
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"priority": 1000,
"sourceRanges": [
"10.1.2.0/24", "10.3.4.0/24"
],
"allowed": [
{
"IPProtocol": "tcp"
},
{
"IPProtocol": "udp"
},
{
"IPProtocol": "icmp"
}
],
"direction": "INGRESS",
"logConfig": {
"enable": false
},
"disabled": false
}
Create the fw-allow-ssh
firewall rule by making a POST
request to
the firewalls.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/global/firewalls
{
"name": "fw-allow-ssh",
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"priority": 1000,
"sourceRanges": [
"0.0.0.0/0"
],
"targetTags": [
"allow-ssh"
],
"allowed": [
{
"IPProtocol": "tcp",
"ports": [
"22"
]
}
],
"direction": "INGRESS",
"logConfig": {
"enable": false
},
"disabled": false
}
Create the fw-allow-health-check
firewall rule by making a POST
request to
the firewalls.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/global/firewalls
{
"name": "fw-allow-health-check",
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"priority": 1000,
"sourceRanges": [
"130.211.0.0/22",
"35.191.0.0/16"
],
"targetTags": [
"allow-health-check"
],
"allowed": [
{
"IPProtocol": "tcp"
},
{
"IPProtocol": "udp"
},
{
"IPProtocol": "icmp"
}
],
"direction": "INGRESS",
"logConfig": {
"enable": false
},
"disabled": false
}
Creating backend VMs and instance groups
This example uses two unmanaged instance groups each having two backend
(server) VMs. To demonstrate the regional nature of Internal TCP/UDP Load Balancing, the two instance groups are placed in separate zones, us-west1-a
and us-west1-c
.
- Instance group
ig-a
contains these two VMs:vm-a1
vm-a2
- Instance group
ig-c
contains these two VMs:vm-c1
vm-c2
Traffic to all four of the backend VMs is load balanced.
To support this example and the additional configuration options, each of the four VMs runs an Apache web server that listens on the following TCP ports: 80, 8008, 8080, 8088, 443, and 8443.
Each VM is assigned an internal IP address in the lb-subnet
and an ephemeral
external (public) IP address. You can remove the external IP addresses
later.
External IP address for the backend VMs are not required; however, they are useful for this example because they permit the backend VMs to download Apache from the internet, and they make it easy to connect via SSH.
By default, Apache is configured to bind to any IP address. internal TCP/UDP load balancers deliver packets by preserving the destination IP. Ensure that server software running on your backend VMs is listening on the IP address of the load balancer's internal forwarding rule. If you configure multiple internal forwarding rules, ensure that your software listens to the internal IP address associated with each one. The destination IP address of a packet delivered to a backend VM by an internal TCP/UDP load balancer is the internal IP address of the forwarding rule.
For instructional simplicity, these backend VMs run Debian Debian GNU/Linux 10.
Console
Create backend VMs
- In the Google Cloud Console, go to the VM instances page.
Go to VM instances - Repeat the following steps to create four VMs, using the following name
and zone combinations.
- Name:
vm-a1
, zone:us-west1-a
- Name:
vm-a2
, zone:us-west1-a
- Name:
vm-c1
, zone:us-west1-c
- Name:
vm-c2
, zone:us-west1-c
- Name:
- Click Create instance.
- Set the Name as indicated in step 2.
- For the Region, choose
us-west1
, and choose a Zone as indicated in step 2. - In the Boot disk section, ensure that the Debian operating system and the 10 (buster) version are selected for the boot disk options. Click Choose to change the image if necessary.
Click Management, security, disks, networking, sole tenancy and make the following changes:
- Click Networking and add the following Network tags:
allow-ssh
andallow-health-check
- Click Edit
- Network:
lb-network
- Subnet:
lb-subnet
- Primary internal IP: Ephemeral (automatic)
- External IP: Ephemeral
under
Network interfaces and make the
following changes then click Done:
- Network:
Click Management. In the Startup script field, copy and paste the following script contents. The script contents are identical for all four VMs:
#! /bin/bash if [ -f /etc/startup_script_completed ]; then exit 0 fi apt-get update apt-get install apache2 -y a2ensite default-ssl a2enmod ssl file_ports="/etc/apache2/ports.conf" file_http_site="/etc/apache2/sites-available/000-default.conf" file_https_site="/etc/apache2/sites-available/default-ssl.conf" http_listen_prts="Listen 80\nListen 8008\nListen 8080\nListen 8088" http_vh_prts="*:80 *:8008 *:8080 *:8088" https_listen_prts="Listen 443\nListen 8443" https_vh_prts="*:443 *:8443" vm_hostname="$(curl -H "Metadata-Flavor:Google" \ http://169.254.169.254/computeMetadata/v1/instance/name)" echo "Page served from: $vm_hostname" | \ tee /var/www/html/index.html echo "Page served from: $vm_hostname" | \ tee /var/www/html/index.html prt_conf="$(cat "$file_ports")" prt_conf_2="$(echo "$prt_conf" | sed "s|Listen 80|${http_listen_prts}|")" prt_conf="$(echo "$prt_conf_2" | sed "s|Listen 443|${https_listen_prts}|")" echo "$prt_conf" | tee "$file_ports" http_site_conf="$(cat "$file_http_site")" http_site_conf_2="$(echo "$http_site_conf" | sed "s|*:80|${http_vh_prts}|")" echo "$http_site_conf_2" | tee "$file_http_site" https_site_conf="$(cat "$file_https_site")" https_site_conf_2="$(echo "$https_site_conf" | sed "s|_default_:443|${https_vh_prts}|")" echo "$https_site_conf_2" | tee "$file_https_site" systemctl restart apache2 touch /etc/startup_script_completed
- Click Networking and add the following Network tags:
Click Create.
Create instance groups
- In the Google Cloud Console, go to the Instance groups page.
Go to Instance groups - Repeat the following steps to create two unmanaged instance groups each
with two VMs in them, using these combinations.
- Instance group:
ig-a
, zone:us-west1-a
, VMs:vm-a1
andvm-a2
- Instance group:
ig-c
, zone:us-west1-c
, VMs:vm-c1
andvm-c2
- Instance group:
- Click Create instance group.
- Click New unmanaged instance group.
- Set Name as indicated in step 2.
- In the Location section, choose
us-west1
for the Region, and then choose a Zone as indicated in step 2. - For Network, enter
lb-network
. - For Subnetwork, enter
lb-subnet
. - In the VM instances section, add the VMs as indicated in step 2.
- Click Create.
gcloud
Create the four VMs by running the following command four times, using these four combinations for
[VM-NAME]
and[ZONE]
. The script contents are identical for all four VMs.[VM-NAME]
ofvm-a1
and[ZONE]
ofus-west1-a
[VM-NAME]
ofvm-a2
and[ZONE]
ofus-west1-a
[VM-NAME]
ofvm-c1
and[ZONE]
ofus-west1-c
[VM-NAME]
ofvm-c2
and[ZONE]
ofus-west1-c
gcloud compute instances create [VM-NAME] \ --zone=[ZONE] \ --image-family=debian-10 \ --image-project=debian-cloud \ --tags=allow-ssh,allow-health-check \ --subnet=lb-subnet \ --metadata=startup-script='#! /bin/bash if [ -f /etc/startup_script_completed ]; then exit 0 fi apt-get update apt-get install apache2 -y a2ensite default-ssl a2enmod ssl file_ports="/etc/apache2/ports.conf" file_http_site="/etc/apache2/sites-available/000-default.conf" file_https_site="/etc/apache2/sites-available/default-ssl.conf" http_listen_prts="Listen 80\nListen 8008\nListen 8080\nListen 8088" http_vh_prts="*:80 *:8008 *:8080 *:8088" https_listen_prts="Listen 443\nListen 8443" https_vh_prts="*:443 *:8443" vm_hostname="$(curl -H "Metadata-Flavor:Google" \ http://169.254.169.254/computeMetadata/v1/instance/name)" echo "Page served from: $vm_hostname" | \ tee /var/www/html/index.html prt_conf="$(cat "$file_ports")" prt_conf_2="$(echo "$prt_conf" | sed "s|Listen 80|${http_listen_prts}|")" prt_conf="$(echo "$prt_conf_2" | sed "s|Listen 443|${https_listen_prts}|")" echo "$prt_conf" | tee "$file_ports" http_site_conf="$(cat "$file_http_site")" http_site_conf_2="$(echo "$http_site_conf" | sed "s|*:80|${http_vh_prts}|")" echo "$http_site_conf_2" | tee "$file_http_site" https_site_conf="$(cat "$file_https_site")" https_site_conf_2="$(echo "$https_site_conf" | sed "s|_default_:443|${https_vh_prts}|")" echo "$https_site_conf_2" | tee "$file_https_site" systemctl restart apache2 touch /etc/startup_script_completed'
Create the two unmanaged instance groups in each zone:
gcloud compute instance-groups unmanaged create ig-a \ --zone=us-west1-a gcloud compute instance-groups unmanaged create ig-c \ --zone=us-west1-c
Add the VMs to the appropriate instance groups:
gcloud compute instance-groups unmanaged add-instances ig-a \ --zone=us-west1-a \ --instances=vm-a1,vm-a2 gcloud compute instance-groups unmanaged add-instances ig-c \ --zone=us-west1-c \ --instances=vm-c1,vm-c2
api
For the four VMs, use the following VM names and zones:
[VM-NAME]
ofvm-a1
and[ZONE]
ofus-west1-a
[VM-NAME]
ofvm-a2
and[ZONE]
ofus-west1-a
[VM-NAME]
ofvm-c1
and[ZONE]
ofus-west1-c
[VM-NAME]
ofvm-c2
and[ZONE]
ofus-west1-c
You can get the current DEBIAN_IMAGE_NAME
by running the following
gcloud
command:
gcloud compute images list \ --filter="family=debian-10"
Create four backend VMs by making four POST
requests to the
instances.insert
method:
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/zones/[ZONE]/instances
{
"name": "[VM-NAME]",
"tags": {
"items": [
"allow-health-check",
"allow-ssh"
]
},
"machineType": "https://www.googleapis.com/compute/v1/projects/project-id/zones/[ZONE]/machineTypes/e2-standard-2",
"canIpForward": false,
"networkInterfaces": [
{
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/subnetworks/lb-subnet",
"accessConfigs": [
{
"type": "ONE_TO_ONE_NAT",
"name": "external-nat",
"networkTier": "PREMIUM"
}
]
}
],
"disks": [
{
"type": "PERSISTENT",
"boot": true,
"mode": "READ_WRITE",
"autoDelete": true,
"deviceName": "[VM-NAME]",
"initializeParams": {
"sourceImage": "projects/debian-cloud/global/images/debian-image-name",
"diskType": "projects/project-id/zones/zone/diskTypes/pd-standard",
"diskSizeGb": "10"
}
}
],
"metadata": {
"items": [
{
"key": "startup-script",
"value": "#! /bin/bash\napt-get update\napt-get install apache2 -y\na2ensite default-ssl\na2enmod ssl\nfile_ports=\"/etc/apache2/ports.conf\"\nfile_http_site=\"/etc/apache2/sites-available/000-default.conf\"\nfile_https_site=\"/etc/apache2/sites-available/default-ssl.conf\"\nhttp_listen_prts=\"Listen 80\\nListen 8008\\nListen 8080\\nListen 8088\"\nhttp_vh_prts=\"*:80 *:8008 *:8080 *:8088\"\nhttps_listen_prts=\"Listen 443\\nListen 8443\"\nhttps_vh_prts=\"*:443 *:8443\"\nvm_hostname=\"$(curl -H \"Metadata-Flavor:Google\" \\\nhttp://169.254.169.254/computeMetadata/v1/instance/name)\"\necho \"Page served from: $vm_hostname\" | \\\ntee /var/www/html/index.html\nprt_conf=\"$(cat \"$file_ports\")\"\nprt_conf_2=\"$(echo \"$prt_conf\" | sed \"s|Listen 80|${http_listen_prts}|\")\"\nprt_conf=\"$(echo \"$prt_conf_2\" | sed \"s|Listen 443|${https_listen_prts}|\")\"\necho \"$prt_conf\" | tee \"$file_ports\"\nhttp_site_conf=\"$(cat \"$file_http_site\")\"\nhttp_site_conf_2=\"$(echo \"$http_site_conf\" | sed \"s|*:80|${http_vh_prts}|\")\"\necho \"$http_site_conf_2\" | tee \"$file_http_site\"\nhttps_site_conf=\"$(cat \"$file_https_site\")\"\nhttps_site_conf_2=\"$(echo \"$https_site_conf\" | sed \"s|_default_:443|${https_vh_prts}|\")\"\necho \"$https_site_conf_2\" | tee \"$file_https_site\"\nsystemctl restart apache2"
}
]
},
"scheduling": {
"preemptible": false
},
"deletionProtection": false
}
Create two instance groups by making a POST
request to the
instanceGroups.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/zones/us-west1-a/instanceGroups
{
"name": "ig-a",
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/subnetworks/lb-subnet"
}
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/zones/us-west1-c/instanceGroups
{
"name": "ig-c",
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/subnetworks/lb-subnet"
}
Add instances to each instance group by making a POST
request to the
instanceGroups.addInstances
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/zones/us-west1-a/instanceGroups/ig-a/addInstances
{
"instances": [
{
"instance": "https://www.googleapis.com/compute/v1/projects/project-id/zones/us-west1-a/instances/vm-a1",
"instance": "https://www.googleapis.com/compute/v1/projects/project-id/zones/us-west1-a/instances/vm-a2"
}
]
}
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/zones/us-west1-c/instanceGroups/ig-c/addInstances
{
"instances": [
{
"instance": "https://www.googleapis.com/compute/v1/projects/project-id/zones/us-west1-c/instances/vm-c1",
"instance": "https://www.googleapis.com/compute/v1/projects/project-id/zones/us-west1-c/instances/vm-c2"
}
]
}
Creating a client VM
This example creates a client VM (vm-client
) in the same region as the backend
(server) VMs. The client is used to validate the load balancer's configuration
and demonstrate expected behavior as described in the
testing section.
Console
- In the Google Cloud Console, go to the VM instances page.
Go to VM instances - Click Create instance.
- Set the Name to
vm-client
. - Set the Zone to
us-west1-a
. - Click Management, security, disks, networking, sole tenancy and make
the following changes:
- Click Networking and add the
allow-ssh
to Network tags. - Click the edit button under Network interfaces and make the
following changes then click Done:
- Network:
lb-network
- Subnet:
lb-subnet
- Primary internal IP: Ephemeral (automatic)
- External IP: Ephemeral
- Network:
- Click Networking and add the
- Click Create.
gcloud
The client VM can be in any zone in the same region as the
load balancer, and it can use any subnet in that region. In this example,
the client is in the us-west1-a
zone, and it uses the same
subnet as the backend VMs.
gcloud compute instances create vm-client \ --zone=us-west1-a \ --image-family=debian-10 \ --image-project=debian-cloud \ --tags=allow-ssh \ --subnet=lb-subnet
api
Make a POST
request to the
instances.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/zones/us-west1-a/instances
{
"name": "vm-client",
"tags": {
"items": [
"allow-ssh"
]
},
"machineType": "https://www.googleapis.com/compute/v1/projects/project-id/zones/us-west1-a/machineTypes/e2-standard-2",
"canIpForward": false,
"networkInterfaces": [
{
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/subnetworks/lb-subnet",
"accessConfigs": [
{
"type": "ONE_TO_ONE_NAT",
"name": "external-nat",
"networkTier": "PREMIUM"
}
]
}
],
"disks": [
{
"type": "PERSISTENT",
"boot": true,
"mode": "READ_WRITE",
"autoDelete": true,
"deviceName": "vm-client",
"initializeParams": {
"sourceImage": "projects/debian-cloud/global/images/debian-image-name",
"diskType": "projects/project-id/zones/us-west1-a/diskTypes/pd-standard",
"diskSizeGb": "10"
}
}
],
"scheduling": {
"preemptible": false
},
"deletionProtection": false
}
Configuring load balancer components
These steps configure all of the internal TCP/UDP load balancer components starting with the health check and backend service, and then the frontend components:
Health check: In this example, we use an HTTP health check that simply checks for an HTTP
200
(OK) response. For more information, see the health checks section of the Internal TCP/UDP Load Balancing overview.Backend service: Because we need to pass HTTP traffic through the internal load balancer, we need to use TCP, not UDP.
Forwarding rule: This example creates a single internal forwarding rule.
Internal IP address: In this example, we specify an internal IP address,
10.1.2.99
, when we create the forwarding rule.
Console
Create the load balancer and configure a backend service
- In the Google Cloud Console, go to the Load balancing page.
Go to Load balancing - Click Create load balancer.
- Under TCP load balancing, click Start configuration.
- Under Internet facing or internal only select Only between my VMs.
- Click Continue.
- Set the Name to
be-ilb
. - Click Backend configuration and make the following changes:
- Region:
us-west1
- Network:
lb-network
- Under Backends, in the New item section, select the
ig-a
instance group and click Done. - Click Add backend. In the New item section that appears,
select the
ig-c
instance group and click Done again. - Under Health check, choose Create another health check,
enter the following information, and click Save and continue:
- Name:
hc-http-80
- Protocol:
HTTP
- Port:
80
- Proxy protocol:
NONE
- Request path:
/
Note that when you use the Cloud Console to create your load balancer, the health check is global. If you want to create a regional health check, usegcloud
or the API.
- Name:
- Verify that there is a blue check mark next to Backend configuration before continuing. Review this step if not.
- Region:
- Click Frontend configuration. In the New Frontend IP and port
section, make the following changes:
- Name:
fr-ilb
- Subnetwork:
lb-subnet
- From Internal IP, choose Reserve a static internal IP address,
enter the following information, and click Reserve:
- Name:
ip-ilb
- Static IP address: Let me choose
- Custom IP address:
10.1.2.99
- Name:
- Ports: Choose Multiple, and enter
80,8008,8080,8088
for the Port number. - Verify that there is a blue check mark next to Frontend configuration before continuing. Review this step if not.
- Name:
- Click Review and finalize. Double-check your settings.
- Click Create.
gcloud
Create a new regional HTTP health check to test HTTP connectivity to the VMs on 80.
gcloud compute health-checks create http hc-http-80 \ --region=us-west1 \ --port=80
Create the backend service for HTTP traffic:
gcloud compute backend-services create be-ilb \ --load-balancing-scheme=internal \ --protocol=tcp \ --region=us-west1 \ --health-checks=hc-http-80 \ --health-checks-region=us-west1
Add the two instance groups to the backend service:
gcloud compute backend-services add-backend be-ilb \ --region=us-west1 \ --instance-group=ig-a \ --instance-group-zone=us-west1-a gcloud compute backend-services add-backend be-ilb \ --region=us-west1 \ --instance-group=ig-c \ --instance-group-zone=us-west1-c
Create a forwarding rule for the backend service. When you create the forwarding rule, specify
10.1.2.99
for the internal IP address in the subnet.gcloud compute forwarding-rules create fr-ilb \ --region=us-west1 \ --load-balancing-scheme=internal \ --network=lb-network \ --subnet=lb-subnet \ --address=10.1.2.99 \ --ip-protocol=TCP \ --ports=80,8008,8080,8088 \ --backend-service=be-ilb \ --backend-service-region=us-west1
api
Create the health check by making a POST
request to the
regionHealthChecks.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/regions/us-west1/regionHealthChecks
{
"name": "hc-http-80",
"type": "HTTP",
"httpHealthCheck": {
"port": 80
}
}
Create the regional backend service by making a POST
request to the
regionBackendServices.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/regions/us-west1/backendServices
{
"name": "be-ilb",
"backends": [
{
"group": "https://www.googleapis.com/compute/v1/projects/project-id/zones/us-west1-a/instanceGroups/ig-a",
"balancingMode": "CONNECTION"
},
{
"group": "https://www.googleapis.com/compute/v1/projects/project-id/zones/us-west1-c/instanceGroups/ig-c",
"balancingMode": "CONNECTION"
}
],
"healthChecks": [
"https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/healthChecks/hc-http-80"
],
"loadBalancingScheme": "INTERNAL",
"connectionDraining": {
"drainingTimeoutSec": 0
}
}
Create the forwarding rule by making a POST
request to the
forwardingRules.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/regions/us-west1/forwardingRules
{
"name": "fr-ilb",
"IPAddress": "10.1.2.99",
"IPProtocol": "TCP",
"ports": [
"80", "8008", "8080", "8088"
],
"loadBalancingScheme": "INTERNAL",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/subnetworks/lb-subnet",
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"backendService": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/backendServices/be-ilb",
"networkTier": "PREMIUM"
}
Testing
These tests show how to validate your load balancer configuration and learn about its expected behavior.
Testing load balancing
This test contacts the load balancer from a separate client VM; that is, not from a backend VM of the load balancer. The expected behavior is for traffic to be distributed among the four backend VMs because no session affinity has been configured.
Connect to the client VM instance.
gcloud compute ssh vm-client --zone=us-west1-a
Make a web request to the load balancer using
curl
to contact its IP address. Repeat the request so you can see that responses come from different backend VMs. The name of the VM generating the response is displayed in the text in the HTML response, by virtue of the contents of/var/www/html/index.html
on each backend VM. Expected responses look like:Page served from: vm-a1
,Page served from: vm-a2
, and so on.curl http://10.1.2.99
The forwarding rule is configured to serve ports
80
,8008
,8080
, and8088
. To send traffic to those other ports, append a colon (:
) and the port number after the IP address, like this:curl http://10.1.2.99:8008
If you add a service label to the internal forwarding rule, you can use internal DNS to contact the load balancer using its service name.
curl http://web-test.fr-ilb.il4.us-west1.lb.project-id.internal
Pinging the load balancer's IP address
This test demonstrates an expected behavior: You cannot ping the IP address of the load balancer. This is because internal TCP/UDP load balancers are implemented in virtual network programming — they are not separate devices.
Connect to the client VM instance.
gcloud compute ssh vm-client --zone=us-west1-a
Attempt to ping the IP address of the load balancer. Notice that you do not get a response and that the
ping
command times out after 10 seconds in this example.timeout 10 ping 10.1.2.99
Sending requests from load balanced VMs
This test demonstrates that requests to the load balancer that originate from any of the backend VMs — the server VMs being load balanced — are always answered by the same VM making the request.
Internal TCP/UDP Load Balancing is implemented using virtual network programming and VM configuration in the guest OS. On Linux VMs, the Linux Guest Environment performs the local configuration by installing a route in the guest OS routing table. Because of this local route, traffic to the IP address of the load balancer stays on the load balanced VM itself. (This local route is different from the routes in the VPC network.)
Connect to a backend VM, such as
vm-a1
:gcloud compute ssh vm-a1 --zone=us-west1-a
Make a web request to the load balancer (by IP address or service name) using
curl
. Repeat the request, and note that the response is sent from the backend VM that makes the request. The expected response when testing fromvm-a1
is always:Page served from: vm-a1
.curl http://10.1.2.99
Inspect the local routing table, looking for a destination that matches the IP address of the load balancer itself,
10.1.2.99
. This route is a necessary part of Internal TCP/UDP Load Balancing, but it also demonstrates why a request from a VM behind the load balancer is always responded to by the same VM.ip route show table local | grep 10.1.2.99
Additional configuration options
This section expands on the configuration example to provide alternative and additional configuration options. All of the tasks are optional. You can perform them in any order.
Enabling global access
You can enable global access for
your example internal TCP/UDP load balancer to make it accessible to clients in all regions.
The backends of your example load balancer must still be located in one region
(us-west1
).
To configure global access, make the following configuration changes.
Console
Edit the load balancer's forwarding rule
- In the Google Cloud Console, go to the Load balancing page.
Go to Load balancing In the Name column, click your internal TCP/UDP load balancer. The example load balancer is named
be-ilb
.Click Frontend configuration.
Click Edit
.Under Global access, select Enable.
Click Done.
Click Update.
On the Load balancer details page, verify that the frontend configuration
says Regional (REGION
) with global access.
gcloud
Update the example load balancer's forwarding rule,
fr-ilb
to include the--allow-global-access
flag.gcloud compute forwarding-rules update fr-ilb \ --region=us-west1 \ --allow-global-access
Make sure that the forwarding rule allows global access.
gcloud compute forwarding-rules describe fr-ilb \ --region=us-west1 \ --format="get(name,region,allowGlobalAccess)"
The word
True
appears in the output, after the name and region of the forwarding rule, when global access is enabled.
api
Make a PATCH
request to the
forwardingRules/patch
method.
PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/regions/us-west1/forwardingRules/fr-ilb
{
"allowGlobalAccess": true
}
Create a VM client to test global access
Console
- In the Google Cloud Console, go to the VM instances page.
Go to VM instances - Click Create instance.
- Set the Name to
vm-client2
. - Set the Zone to
europe-west1-b
. - Click Management, security, disks, networking, sole tenancy and make
the following changes:
- Click Networking and add the
allow-ssh
to Network tags. - Click the edit button under Network interfaces and make the
following changes then click Done:
- Network:
lb-network
- Subnet:
europe-subnet
- Primary internal IP: Ephemeral (automatic)
- External IP: Ephemeral
- Network:
- Click Networking and add the
- Click Create.
gcloud
The client VM can be in any zone in the same region as the
load balancer, and it can use any subnet in that region. In this example,
the client is in the europe-west1-b
zone, and it uses the same
subnet as the backend VMs.
gcloud compute instances create vm-client2 \ --zone=europe-west1-b \ --image-family=debian-10 \ --image-project=debian-cloud \ --tags=allow-ssh \ --subnet=europe-subnet
api
Make a POST
request to the
instances.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/zones/europe-west1-b/instances
{
"name": "vm-client2",
"tags": {
"items": [
"allow-ssh"
]
},
"machineType": "https://www.googleapis.com/compute/v1/projects/project-id/zones/europe-west1-b/machineTypes/e2-standard-2",
"canIpForward": false,
"networkInterfaces": [
{
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/project-id/regions/europe-west1/subnetworks/europe-subnet",
"accessConfigs": [
{
"type": "ONE_TO_ONE_NAT",
"name": "external-nat",
"networkTier": "PREMIUM"
}
]
}
],
"disks": [
{
"type": "PERSISTENT",
"boot": true,
"mode": "READ_WRITE",
"autoDelete": true,
"deviceName": "vm-client2",
"initializeParams": {
"sourceImage": "projects/debian-cloud/global/images/debian-image-name",
"diskType": "projects/project-id/zones/europe-west1-b/diskTypes/pd-standard",
"diskSizeGb": "10"
}
}
],
"scheduling": {
"preemptible": false
},
"deletionProtection": false
}
Connect to the VM client and test connectivity
gcloud compute ssh vm-client2 --zone=europe-west1-b
Test connecting to the load balancer on all configured ports, as you did
from the vm-client
in the us-west1
region. Test
HTTP connectivity on the four ports configured on the forwarding rule:
curl http://10.1.2.99 curl http://10.1.2.99:8008 curl http://10.1.2.99:8080 curl http://10.1.2.99:8088
Configuring managed instance groups
The example configuration created two unmanaged instance groups. You can instead use managed instance groups, including zonal and regional managed instance groups, as backends for Internal TCP/UDP Load Balancing.
Managed instance groups require that you create an instance template. This procedure demonstrates how to replace the two unmanaged zonal instance groups from the example with a single, regional managed instance group. A regional managed instance group automatically creates VMs in multiple zones of the region, making it simpler to distribute production traffic among zones.
Managed instance groups also support autoscaling and autohealing. If you use autoscaling with Internal TCP/UDP Load Balancing, you cannot scale based on load balancing.
This procedure shows you how to modify the backend service for the example internal TCP/UDP load balancer so that it uses a regional managed instance group.
Console
Instance template
- In the Google Cloud Console, go to the VM instance templates page.
Go to VM instance templates - Click Create instance template.
- Set the Name to
template-vm-ilb
. - Choose a machine type.
- For Boot disk, click Change, choose the Debian operating system and the 10 (buster) version.
- Click Save to confirm the options for this boot disk.
Click Management, security, disks, networking, sole tenancy.
- Click Networking and make the following changes:
- Network:
lb-network
- Subnet:
lb-subnet
- Network tags:
allow-ssh
andallow-health-check
- Network:
- Click Management. In the Startup script field, copy and paste the following script contents:
#! /bin/bash if [ -f /etc/startup_script_completed ]; then exit 0 fi apt-get update apt-get install apache2 -y a2ensite default-ssl a2enmod ssl file_ports="/etc/apache2/ports.conf" file_http_site="/etc/apache2/sites-available/000-default.conf" file_https_site="/etc/apache2/sites-available/default-ssl.conf" http_listen_prts="Listen 80\nListen 8008\nListen 8080\nListen 8088" http_vh_prts="*:80 *:8008 *:8080 *:8088" https_listen_prts="Listen 443\nListen 8443" https_vh_prts="*:443 *:8443" vm_hostname="$(curl -H "Metadata-Flavor:Google" \ http://169.254.169.254/computeMetadata/v1/instance/name)" echo "Page served from: $vm_hostname" | \ tee /var/www/html/index.html echo "Page served from: $vm_hostname" | \ tee /var/www/html/index.html prt_conf="$(cat "$file_ports")" prt_conf_2="$(echo "$prt_conf" | sed "s|Listen 80|${http_listen_prts}|")" prt_conf="$(echo "$prt_conf_2" | sed "s|Listen 443|${https_listen_prts}|")" echo "$prt_conf" | tee "$file_ports" http_site_conf="$(cat "$file_http_site")" http_site_conf_2="$(echo "$http_site_conf" | sed "s|*:80|${http_vh_prts}|")" echo "$http_site_conf_2" | tee "$file_http_site" https_site_conf="$(cat "$file_https_site")" https_site_conf_2="$(echo "$https_site_conf" | sed "s|_default_:443|${https_vh_prts}|")" echo "$https_site_conf_2" | tee "$file_https_site" systemctl restart apache2 touch /etc/startup_script_completed
- Click Networking and make the following changes:
Click Create.
Managed instance group
- In the Google Cloud Console, go to the Instance groups page.
Go to VM instance groups - Click Create instance group.
- Set the Name to
ig-ilb
. - For Location, choose Multi-zone, and set the Region to
us-west1
. - Set the Instance template to
template-vm-ilb
. - (Optional) Configure autoscaling. You cannot autoscale the instance group based on HTTP load balancing usage because the instance group is a backend for Internal TCP/UDP Load Balancing.
- Set the Minimum number of instances to
1
and the Maximum number of instances to6
. - (Optional) Configure
autohealing.
If you configure autohealing, use the same health check used by the backend
service for the internal TCP/UDP load balancer. In this example, use
hc-http-80
. - Click Create.
gcloud
Create the instance template. Optionally, you can set other parameters, such as machine type, for the image template to use.
gcloud compute instance-templates create template-vm-ilb \ --image-family=debian-10 \ --image-project=debian-cloud \ --tags=allow-ssh,allow-health-check \ --subnet=lb-subnet \ --region=us-west1 \ --network=lb-network \ --metadata=startup-script='#! /bin/bash if [ -f /etc/startup_script_completed ]; then exit 0 fi apt-get update apt-get install apache2 -y a2ensite default-ssl a2enmod ssl file_ports="/etc/apache2/ports.conf" file_http_site="/etc/apache2/sites-available/000-default.conf" file_https_site="/etc/apache2/sites-available/default-ssl.conf" http_listen_prts="Listen 80\nListen 8008\nListen 8080\nListen 8088" http_vh_prts="*:80 *:8008 *:8080 *:8088" https_listen_prts="Listen 443\nListen 8443" https_vh_prts="*:443 *:8443" vm_hostname="$(curl -H "Metadata-Flavor:Google" \ http://169.254.169.254/computeMetadata/v1/instance/name)" echo "Page served from: $vm_hostname" | \ tee /var/www/html/index.html prt_conf="$(cat "$file_ports")" prt_conf_2="$(echo "$prt_conf" | sed "s|Listen 80|${http_listen_prts}|")" prt_conf="$(echo "$prt_conf_2" | sed "s|Listen 443|${https_listen_prts}|")" echo "$prt_conf" | tee "$file_ports" http_site_conf="$(cat "$file_http_site")" http_site_conf_2="$(echo "$http_site_conf" | sed "s|*:80|${http_vh_prts}|")" echo "$http_site_conf_2" | tee "$file_http_site" https_site_conf="$(cat "$file_https_site")" https_site_conf_2="$(echo "$https_site_conf" | sed "s|_default_:443|${https_vh_prts}|")" echo "$https_site_conf_2" | tee "$file_https_site" systemctl restart apache2 touch /etc/startup_script_completed'
Create one regional managed instance group using the template:
gcloud compute instance-groups managed create ig-ilb \ --template=template-vm-ilb \ --region=us-west1 \ --size=6
Add the regional managed instance group as a backend to the backend service that you already created:
gcloud compute backend-services add-backend be-ilb \ --region=us-west1 \ --instance-group=ig-ilb \ --instance-group-region=us-west1
Disconnect the two unmanaged (zonal) instance groups from the backend service:
gcloud compute backend-services remove-backend be-ilb \ --region=us-west1 \ --instance-group=ig-a \ --instance-group-zone=us-west1-a gcloud compute backend-services remove-backend be-ilb \ --region=us-west1 \ --instance-group=ig-c \ --instance-group-zone=us-west1-c
Removing external IP addresses from backend VMs
When you created the backend VMs, each was assigned an ephemeral external IP address so it could download Apache via a startup script. Because the backend VMs are only used by an internal TCP/UDP load balancer, you can remove their external IP addresses. Removing external IP addresses prevents the backend VMs from accessing the internet directly.
Console
- In the Google Cloud Console, go to the VM instances page.
Go to VM instances - Repeat the following steps for each backend VM.
- Click the name of the backend VM (for example,
vm-a1
) to view VM instance details. - Click Edit .
- In the Network interfaces section, click the Edit button.
- From the External IP pop-up, choose None, and click Done.
- Click Save.
gcloud
To look up the zone for an instance – for example, if you're using a regional managed instance group – run the following command for each instance to determine its zone. Replace
[SERVER-VM]
with the name of the VM to look up.gcloud compute instances list --filter="name=[SERVER-VM]"
Repeat the following step for each backend VM. Replace
[SERVER-VM]
with the name of the VM, and replace and[ZONE]
with the VM's zone.gcloud compute instances delete-access-config [SERVER-VM] \ --zone=[ZONE] \ --access-config-name=external-nat
api
Make a POST
request to the
instances.deleteAccessConfig
method for each backend VM, replacingvm-a1
with the name of the VM, and replacing and us-west1-a
with the VM's zone.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/zones/us-west1-a/instances/vm-a1/deleteAccessConfig?accessConfig=external-nat&networkInterface=None
Accepting traffic on all ports
The load balancer's forwarding rule, not its backend service, determines the port or ports on which the load balancer accepts traffic. For information about the purpose of each component, see Components.
When you created this example load balancer's forwarding
rule, you configured ports 80
, 8008
, 8080
,
and 8088
. The startup script that installs Apache also configures it to
accept HTTPS connections on ports 443
and 8443
.
To support these six ports, you can configure the forwarding rule to accept traffic on all ports. With this strategy, you can also configure the firewall rule or rules that allow incoming connections to backend VMs so that they only permit certain ports.
This procedure shows you how to replace the example load balancer's forwarding rule with one that accepts traffic on all ports.
For more information about when to use this setup, see Internal TCP/UDP Load Balancing and forwarding rules with a common IP address.
Console
Delete your forwarding rule and create a new one
- In the Google Cloud Console, go to the Load balancing page.
Go to Load balancing - Click your
be-ilb
load balancer and click Edit. - Click Frontend configuration.
- Hover over your 10.1.2.9 forwarding rule and click the trashcan icon to delete it.
- Click Add frontend IP and port.
- In the New Frontend IP and port
section, make the following changes:
- Name:
fr-ilb
- Subnetwork:
lb-subnet
- From Internal IP, select
ip-ilb
- Ports: All.
- Click Done.
- Verify that there is a blue check mark next to Frontend configuration before continuing. Review this step if not.
- Name:
- Click Review and finalize. Double-check your settings.
- Click Create.
gcloud
Delete your existing forwarding rule,
fr-ilb
.gcloud compute forwarding-rules delete fr-ilb \ --region=us-west1
Create a replacement forwarding rule, with the same name, whose port configuration uses the keyword
ALL
. The other parameters for the forwarding rule remain the same.gcloud compute forwarding-rules create fr-ilb \ --region=us-west1 \ --load-balancing-scheme=internal \ --network=lb-network \ --subnet=lb-subnet \ --address=10.1.2.99 \ --ip-protocol=TCP \ --ports=ALL \ --backend-service=be-ilb \ --backend-service-region=us-west1
api
Delete the forwarding rule by making a DELETE
request to the
forwardingRules.delete
method.
DELETE https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/regions/us-west1/forwardingRules/fr-ilb
Create the forwarding rule by making a POST
request to the
forwardingRules.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/regions/us-west1/forwardingRules
{
"name": "fr-ilb",
"IPAddress": "10.1.2.99",
"IPProtocol": "TCP",
"allPorts": true,
"loadBalancingScheme": "INTERNAL",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/subnetworks/lb-subnet",
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"backendService": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/backendServices/be-ilb",
"networkTier": "PREMIUM"
}
Testing the traffic on all ports setup
Connect to the client VM instance and test HTTP and HTTPS connections.
Connect to the client VM:
gcloud compute ssh vm-client --zone=us-west1-a
Test HTTP connectivity on all four ports:
curl http://10.1.2.99 curl http://10.1.2.99:8008 curl http://10.1.2.99:8080 curl http://10.1.2.99:8088
Test HTTPS connectivity on ports
443
and8443
. The--insecure
flag is required because each Apache server in the example setup uses a self-signed certificate.curl https://10.1.2.99 --insecure curl https://10.1.2.99:8443 --insecure
Observe that HTTP requests (on all four ports) and HTTPS requests (on both ports) are distributed among all of the backend VMs.
Accepting traffic on multiple ports using two forwarding rules
When you created this example load balancer's forwarding
rule, you configured ports 80
, 8008
, 8080
,
and 8088
. The startup script that installs Apache also configures it to
accept HTTPS connections on ports 443
and 8443
.
An alternative strategy to configuring a single forwarding rule to accept traffic on all ports is to create multiple forwarding rules, each supporting five or fewer ports.
This procedure shows you how to replace the example load balancer's forwarding
rule with two forwarding rules, one handling traffic on ports 80
, 8008
,
8080
, and 8088
, and the other handling traffic on ports 443
and 8443
.
For more information about when to use this setup, see Internal TCP/UDP Load Balancing and forwarding rules with a common IP address.
Console
- In the Google Cloud Console, go to the Forwarding Rules page.
Go to Forwarding Rules - Select
fr-ilb
. - Click Delete.
- Go to the Load balancing page.
Go to the Load balancing page - Select your internal TCP/UDP load balancer. In this example,
be-ilb
. - Click Edit.
- Click Frontend configuration.
- Click Add frontend IP and port.
- In the New Frontend IP and port
section, make the following changes:
- Name:
fr-ilb-http
- Subnetwork:
lb-subnet
- Under Purpose, select Shared.
- From Internal IP, choose Reserve a static internal IP address,
enter the following information, and click Reserve:
- Name:
internal-10-1-2-99
- Static IP address: Let me choose
- Custom IP address:
10.1.2.99
- Click Reserve.
- Name:
- Ports: Choose Multiple, and enter
80,8008,8080,8088
for the Port number. - Click Done.
- Name:
- Click Add frontend IP and port.
- In the New Frontend IP and port
section, make the following changes:
- Name:
fr-ilb-https
- Subnetwork:
lb-subnet
- Under Purpose, select Shared.
- Under IP address, select
internal-10-1-2-99
. - Ports: Choose Multiple, and enter
443,8443
for the Port number. - Click Done.
- Verify that there is a blue check mark next to Frontend configuration before continuing. Review this step if not.
- Name:
- Click Review and finalize. Double-check your settings.
- Click Update.
gcloud
Delete your existing forwarding rule,
fr-ilb
.gcloud compute forwarding-rules delete fr-ilb \ --region=us-west1
Create a static (reserved) internal IP address for
10.1.2.99
and set its--purpose
flag toSHARED_LOADBALANCER_VIP
. The--purpose
flag is required so that two internal forwarding rules can use the same internal IP address.gcloud compute addresses create internal-10-1-2-99 \ --region=us-west1 \ --subnet=lb-subnet \ --addresses=10.1.2.99 \ --purpose=SHARED_LOADBALANCER_VIP
- Create two replacement forwarding rules with the following parameters:
gcloud compute forwarding-rules create fr-ilb-http \ --region=us-west1 \ --load-balancing-scheme=internal \ --network=lb-network \ --subnet=lb-subnet \ --address=10.1.2.99 \ --ip-protocol=TCP \ --ports=80,8008,8080,8088 \ --backend-service=be-ilb \ --backend-service-region=us-west1
gcloud compute forwarding-rules create fr-ilb-https \ --region=us-west1 \ --load-balancing-scheme=internal \ --network=lb-network \ --subnet=lb-subnet \ --address=10.1.2.99 \ --ip-protocol=TCP \ --ports=443,8443 \ --backend-service=be-ilb \ --backend-service-region=us-west1
api
Delete the forwarding rule by making a DELETE
request to the
forwardingRules.delete
method.
DELETE https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/regions/us-west1/forwardingRules/fr-ilb
Create a static (reserved) internal IP address for 10.1.2.99
and set its purpose to SHARED_LOADBALANCER_VIP
by making a POST
request to the
addresses.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/regions/us-west1/addresses
{
"name": "internal-10-1-2-99",
"address": "10.1.2.99",
"prefixLength": 32,
"addressType": INTERNAL,
"purpose": SHARED_LOADBALANCER_VIP,
"subnetwork": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/subnetworks/lb-subnet"
}
Create two forwarding rules by making two POST
requests to the
forwardingRules.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/regions/us-west1/forwardingRules
{
"name": "fr-ilb-http",
"IPAddress": "10.1.2.99",
"IPProtocol": "TCP",
"ports": [
"80", "8008", "8080", "8088"
],
"loadBalancingScheme": "INTERNAL",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/subnetworks/lb-subnet",
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"backendService": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/backendServices/be-ilb",
"networkTier": "PREMIUM"
}
{
"name": "fr-ilb-https",
"IPAddress": "10.1.2.99",
"IPProtocol": "TCP",
"ports": [
"443", "8443"
],
"loadBalancingScheme": "INTERNAL",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/subnetworks/lb-subnet",
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"backendService": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/backendServices/be-ilb",
"networkTier": "PREMIUM"
}
Testing the traffic on multiple ports setup
Connect to the client VM instance and test HTTP and HTTPS connections.
Connect to the client VM:
gcloud compute ssh vm-client --zone=us-west1-a
Test HTTP connectivity on all four ports:
curl http://10.1.2.99 curl http://10.1.2.99:8008 curl http://10.1.2.99:8080 curl http://10.1.2.99:8088
Test HTTPS connectivity on ports
443
and8443
. The--insecure
flag is required because each Apache server in the example setup uses a self-signed certificate.curl https://10.1.2.99 --insecure curl https://10.1.2.99:8443 --insecure
Observe that HTTP requests (on all four ports) and HTTPS requests (on both ports) are distributed among all of the backend VMs.
Using session affinity
The example configuration creates a backend service without session affinity.
This procedure shows you how to update the backend service for the example internal TCP/UDP load balancer so that it uses session affinity based on a hash created from the client's IP addresses and the IP address of the load balancer's internal forwarding rule. Note that, currently, setting session affinity for an internal UDP load balancer is not supported.
Console
- In the Google Cloud Console, go to the Load balancing page.
Go to Load balancing - Click be-ilb (the name of the backend service that you created for this example) and click Edit.
- On the Edit Internal load balancer page, click Backend configuration.
- Select Client IP from the Session affinity pop-up menu.
- Click Update.
gcloud
Use the following gcloud
command to update the be-ilb
backend service,
specifying client IP session affinity:
gcloud compute backend-services update be-ilb \ --region=us-west1 \ --session-affinity CLIENT_IP
api
Make a PATCH
request to the
regionBackendServices/patch
method.
PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/regions/us-west1/backendServices/be-ilb
{
"sessionAffinity": "CLIENT_IP"
}
For more information about using session affinity to influence traffic distribution and a description of each option, see Traffic distribution.
Creating a forwarding rule in another subnet
This procedure creates a second IP address and forwarding rule in a different subnet to demonstrate that you can create multiple forwarding rules for one internal TCP/UDP load balancer. The region for the forwarding rule must match the region of the backend service.
Subject to firewall rules, clients in any subnet in the region can contact either internal TCP/UDP load balancer IP address.
Console
Add the second subnet
- In the Google Cloud Console, go to the VPC networks page.
Go to VPC networks - Click Create VPC network.
- Click
lb-network
. - In the Subnets section:
- Click Add subnet.
- In the New subnet section, enter the following information:
- Name:
second-subnet
- Region:
us-west1
- IP address range:
10.5.6.0/24
- Click Add.
- Name:
Add the second forwarding rule
- In the Google Cloud Console, go to the Load balancing page.
Go to Load balancing - Click your
be-ilb
load balancer and click Edit. - Click Frontend configuration.
- Click Add frontend IP and port.
- In the New Frontend IP and port
section, make the following changes:
- Name:
fr-ilb-2
- Subnetwork:
second-subnet
- From Internal IP, select
ip-ilb
- Ports: 80, 443.
- Click Done.
- Verify that there is a blue check mark next to Frontend configuration before continuing. Review this step if not.
- Name:
- Click Review and finalize. Double-check your settings.
- Click Create.
gcloud
Create a second subnet in the
lb-network
network in theus-west1
region:gcloud compute networks subnets create second-subnet \ --network=lb-network \ --range=10.5.6.0/24 \ --region=us-west1
Create a second forwarding rule for ports 80 and 443. The other parameters for this rule, including IP address and backend service, are the same as for the primary forwarding rule,
fr-ilb
.gcloud compute forwarding-rules create fr-ilb-2 \ --region=us-west1 \ --load-balancing-scheme=internal \ --network=lb-network \ --subnet=second-subnet \ --address=10.5.6.99 \ --ip-protocol=TCP \ --ports=80,443 \ --backend-service=be-ilb \ --backend-service-region=us-west1
api
Make a POST
requests to the
subnetworks.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/regions/us-west1/subnetworks
{
"name": "second-subnet",
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"ipCidrRange": "10.5.6.0/24",
"privateIpGoogleAccess": false
}
Create the forwarding rule by making a POST
request to the
forwardingRules.insert
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID
/regions/us-west1/forwardingRules
{
"name": "fr-ilb-2",
"IPAddress": "10.5.6.99",
"IPProtocol": "TCP",
"ports": [
"80", "443"
],
"loadBalancingScheme": "INTERNAL",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/subnetworks/lb-subnet",
"network": "https://www.googleapis.com/compute/v1/projects/project-id/global/networks/lb-network",
"backendService": "https://www.googleapis.com/compute/v1/projects/project-id/regions/us-west1/backendServices/be-ilb",
"networkTier": "PREMIUM"
}
Testing the new forwarding rule
Connect to the client VM instance and test HTTP and HTTPS connections to the IP addresses.
- Connect to the client VM:
gcloud compute ssh vm-client --zone=us-west1-a
- Test HTTP connectivity to the IP addresses:
curl http://10.1.2.99 curl http://10.5.6.99
- Test HTTPS connectivity. Use of
--insecure
is required because the Apache server configuration in the example setup uses self-signed certificates.
curl https://10.1.2.99 --insecure curl https://10.5.6.99 --insecure
- Observe that requests are handled by all of the backend VMs, regardless of the protocol (HTTP or HTTPS) or IP address used.
Using subsetting
The example configuration creates a backend service without subsetting.
This procedure shows you how to enable subsetting on the backend service for the example internal TCP/UDP load balancer so that the deployment can scale to a larger number of backend instances.
Subsetting is an advanced feature that is supported for instance groups only during the Preview release.
You should only enable subsetting if you need to support more than 250 backend VMs on a single load balancer.
For more information about this use case, see Subsetting.
gcloud
Use the following gcloud
command to update the be-ilb
backend service,
specifying subsetting:
gcloud beta compute backend-services update be-ilb \ --subsetting-policy=CONSISTENT_HASH_SUBSETTING
api
Make a PATCH
request to the
regionBackendServices/patch
method.
PATCH https://compute.googleapis.com/compute/beta/projects/PROJECT_ID
/regions/us-west1/backendServices/be-ilb
{
"subsetting":
{
"policy": CONSISTENT_HASH_SUBSETTING
}
}
What's next
- See Internal TCP/UDP Load Balancing Concepts for important fundamentals.
- See Failover concepts for Internal TCP/UDP Load Balancing for important information about failover.
- See Internal load balancing and DNS names for available DNS name options your load balancer can use.
- See Configuring failover for Internal TCP/UDP Load Balancing for configuration steps and an example internal TCP/UDP load balancer failover configuration.
- See Internal TCP/UDP Load Balancing Logging and Monitoring for information on configuring Logging and Monitoring for Internal TCP/UDP Load Balancing.
- See Internal TCP/UDP Load Balancing and Connected Networks for information about accessing internal TCP/UDP load balancers from peer networks connected to your VPC network.
- See Troubleshooting Internal TCP/UDP Load Balancing for information on how to troubleshoot issues with your internal TCP/UDP load balancer.
- Clean up the load balancer setup.