This guide uses an example to teach the fundamentals of Google Cloud Internal TCP/UDP Load Balancing. Before following this guide, familiarize yourself with the following:
Permissions
To follow this guide, you need to create instances and modify a network in a project. You should be either a project owner or editor, or you should have all of the following Compute Engine IAM roles:
Task | Required Role |
---|---|
Create networks, subnets, and load balancer components | Network Admin |
Add and remove firewall rules | Security Admin |
Create instances | Instance Admin |
Setup
This guide shows you how to configure and test an internal TCP/UDP load balancer. The steps in this section describe how to configure the following:
- A sample VPC network with custom subnets
- Firewall rules that allow incoming connections to backend VMs
- Four backend VMs:
- Two VMs in an unmanaged instance group in zone
us-west1-a
- Two VMs in an unmanaged instance group in zone
us-west1-c
- Two VMs in an unmanaged instance group in zone
- One client VM to test connections
- The following internal TCP/UDP load balancer components:
- A health check for the backend service
- An internal backend service in the
us-west1
region to manage connection distribution to the two zonal instance groups - An internal forwarding rule and internal IP address for the frontend of the load balancer
The architecture for this example looks like this:
Configuring a network, region, and subnet
You need a VPC network with at least one subnet. An internal TCP/UDP load balancer is regional. Traffic within the VPC network is routed to the load balancer if its source is from a subnet in the same region as the load balancer.
This example uses the following VPC network, region, and subnet:
Network: The network is a custom mode VPC network named
lb-network
.Region: The region is
us-west1
.Subnet: The subnet,
lb-subnet
, uses the10.1.2.0/24
IP range.
To create the example network and subnet, follow these steps.
Console
- Go to the VPC networks page in the Google Cloud Console.
Go to the VPC network page - Click Create VPC network.
- Enter a Name of
lb-network
. - In the Subnets section:
- Set the Subnet creation mode to Custom.
- In the New subnet section, enter the following information:
- Name:
lb-subnet
- Region:
us-west1
- IP address range:
10.1.2.0/24
- Click Done.
- Name:
- Click Create.
gcloud
Create the custom VPC network:
gcloud compute networks create lb-network --subnet-mode=custom
Create a subnet in the
lb-network
network in theus-west1
region:gcloud compute networks subnets create lb-subnet \ --network=lb-network \ --range=10.1.2.0/24 \ --region=us-west1
api
Make a POST
request to the
networks.insert
method.
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks
{
"routingConfig": {
"routingMode": "REGIONAL"
},
"name": "lb-network",
"autoCreateSubnetworks": false
}
Make a POST
request to the
subnetworks.insert
method.
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/subnetworks
{
"name": "lb-subnet",
"network": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/lb-network",
"ipCidrRange": "10.1.2.0/24",
"privateIpGoogleAccess": false
}
Configuring firewall rules
This example uses the following firewall rules:
fw-allow-lb-subnet
: An ingress rule, applicable to all targets in the VPC network, allowing traffic from sources in the10.1.2.0/24
range. This rules allows incoming traffic from any source within thelb-subnet
to the instances (VMs) being load balanced.fw-allow-ssh
: An ingress rule, applicable to the instances being load balanced, that allows incoming SSH connectivity on TCP port 22 from any address. You can choose a more restrictive source IP range for this rule; for example, you can specify just the IP ranges of the system from which you will initiating SSH sessions. This example uses the target tagallow-ssh
to identify the VMs to which it should apply.fw-allow-health-check
: An ingress rule, applicable to the instances being load balanced, that allows traffic from the Google Cloud health checking systems (130.211.0.0/22
and35.191.0.0/16
). This example uses the target tagallow-health-check
to identify the instances to which it should apply.
Without these firewall rules, the default deny ingress rule blocks incoming traffic to the backend instances.
Console
- Go to the Firewall rules page in the Google Cloud Console.
Go to the Firewall rules page - Click Create firewall rule and enter the following
information to create the rule to allow subnet traffic:
- Name:
fw-allow-lb-subnet
- Network:
lb-network
- Priority:
1000
- Direction of traffic: ingress
- Action on match: allow
- Targets: All instances in the network
- Source filter:
IP ranges
- Source IP ranges:
10.1.2.0/24
- Protocols and ports: Allow all
- Name:
- Click Create.
- Click Create firewall rule again to create the rule to allow incoming
SSH connections:
- Name:
fw-allow-ssh
- Network:
lb-network
- Priority:
1000
- Direction of traffic: ingress
- Action on match: allow
- Targets: Specified target tags
- Target tags:
allow-ssh
- Source filter:
IP ranges
- Source IP ranges:
0.0.0.0/0
- Protocols and ports: Choose Specified protocols and ports then
type:
tcp:22
- Name:
- Click Create.
- Click Create firewall rule a third time to create the rule to allow
Google Cloud health checks:
- Name:
fw-allow-health-check
- Network:
lb-network
- Priority:
1000
- Direction of traffic: ingress
- Action on match: allow
- Targets: Specified target tags
- Target tags:
allow-health-check
- Source filter:
IP ranges
- Source IP ranges:
130.211.0.0/22
and35.191.0.0/16
- Protocols and ports: Allow all
- Name:
- Click Create.
gcloud
Create the
fw-allow-lb-subnet
firewall rule to allow communication from with the subnet:gcloud compute firewall-rules create fw-allow-lb-subnet \ --network=lb-network \ --action=allow \ --direction=ingress \ --source-ranges=10.1.2.0/24 \ --rules=tcp,udp,icmp
Create the
fw-allow-ssh
firewall rule to allow SSH connectivity to VMs with the network tagallow-ssh
. When you omitsource-ranges
, Google Cloud interprets the rule to mean any source.gcloud compute firewall-rules create fw-allow-ssh \ --network=lb-network \ --action=allow \ --direction=ingress \ --target-tags=allow-ssh \ --rules=tcp:22
Create the
fw-allow-health-check
rule to allow Google Cloud health checks.gcloud compute firewall-rules create fw-allow-health-check \ --network=lb-network \ --action=allow \ --direction=ingress \ --target-tags=allow-health-check \ --source-ranges=130.211.0.0/22,35.191.0.0/16 \ --rules=tcp,udp,icmp
api
Create the fw-allow-lb-subnet
firewall rule by making a POST
request to
the firewalls.insert
method.
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/firewalls
{
"name": "fw-allow-lb-subnet",
"network": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/lb-network",
"priority": 1000,
"sourceRanges": [
"10.1.2.0/24"
],
"allowed": [
{
"IPProtocol": "tcp"
},
{
"IPProtocol": "udp"
},
{
"IPProtocol": "icmp"
}
],
"direction": "INGRESS",
"logConfig": {
"enable": false
},
"disabled": false
}
Create the fw-allow-ssh
firewall rule by making a POST
request to
the firewalls.insert
method.
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/firewalls
{
"name": "fw-allow-ssh",
"network": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/lb-network",
"priority": 1000,
"sourceRanges": [
"0.0.0.0/0"
],
"targetTags": [
"allow-ssh"
],
"allowed": [
{
"IPProtocol": "tcp",
"ports": [
"22"
]
}
],
"direction": "INGRESS",
"logConfig": {
"enable": false
},
"disabled": false
}
Create the fw-allow-health-check
firewall rule by making a POST
request to
the firewalls.insert
method.
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/firewalls
{
"name": "fw-allow-health-check",
"network": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/lb-network",
"priority": 1000,
"sourceRanges": [
"130.211.0.0/22",
"35.191.0.0/16"
],
"targetTags": [
"allow-health-check"
],
"allowed": [
{
"IPProtocol": "tcp"
},
{
"IPProtocol": "udp"
},
{
"IPProtocol": "icmp"
}
],
"direction": "INGRESS",
"logConfig": {
"enable": false
},
"disabled": false
}
Creating backend VMs and instance groups
This example uses two unmanaged instance groups each having two backend
(server) VMs. To demonstrate the regional nature of Internal TCP/UDP Load
Balancing, the two instance groups are placed in separate zones, us-west1-a
and us-west1-c
.
- Instance group
ig-a
contains these two VMs:vm-a1
vm-a2
- Instance group
ig-c
contains these two VMs:vm-c1
vm-c2
Traffic to all four of the backend VMs is load balanced. Each of the four
run an Apache web server on TCP ports 80 and 443. Each is assigned an
internal IP address in the lb-subnet
and an ephemeral external (public) IP
address. You can remove the external IP addresses
later.
External IP address for the backend VMs are not required; however, they are useful for this example because they permit the backend VMs to download Apache from the internet, and they make it easy to connect via SSH.
By default, Apache is configured to bind to any IP address. Internal TCP/UDP load balancers deliver packets by preserving the destination IP. Ensure that server software running on your backend VMs is listening on the IP address of the load balancer's internal forwarding rule. If you configure multiple internal forwarding rules, ensure that your software listens to the internal IP address associated with each one. The destination IP address of a packet delivered to a backend VM by an internal TCP/UDP load balancer is the internal IP address of the forwarding rule.
For instructional simplicity, these backend VMs run Debian GNU/Linux 9.
Console
Create backend VMs
- Go to the VM instances page in the Google Cloud Console.
Go to the VM instances page - Repeat the following steps to create four VMs, using the following name
and zone combinations.
- Name:
vm-a1
, zone:us-west1-a
- Name:
vm-a2
, zone:us-west1-a
- Name:
vm-c1
, zone:us-west1-c
- Name:
vm-c2
, zone:us-west1-c
- Name:
- Click Create instance.
- Set the Name as indicated in step 2.
- For the Region, choose
us-west1
, and choose a Zone as indicated in step 2. - In the Boot disk section, ensure that the selected image is Debian GNU/Linux 9 Stretch. Click Choose to change the image if necessary.
Click Management, security, disks, networking, sole tenancy and make the following changes:
- Click Networking and add the following Network tags:
allow-ssh
andallow-health-check
- Click the edit button under Network interfaces and make the
following changes then click Done:
- Network:
lb-network
- Subnet:
lb-subnet
- Primary internal IP: Ephemeral (automatic)
- External IP: Ephemeral
- Network:
Click Management. In the Startup script field, copy and paste the following script contents. The script contents are identical for all four VMs:
#! /bin/bash apt-get update apt-get install apache2 -y a2ensite default-ssl a2enmod ssl vm_hostname="$(curl -H "Metadata-Flavor:Google" \ http://169.254.169.254/computeMetadata/v1/instance/name)" echo "Page served from: $vm_hostname" | \ tee /var/www/html/index.html systemctl restart apache2
- Click Networking and add the following Network tags:
Click Create.
Create instance groups
- Go to the Instance groups page in the Google Cloud Console.
Go to the Instance groups page - Repeat the following steps to create two unmanaged instance groups each
with two VMs in them, using these combinations.
- Instance group:
ig-a
, zone:us-west1-a
, VMs:vm-a1
andvm-a2
- Instance group:
ig-c
, zone:us-west1-c
, VMs:vm-c1
andvm-c2
- Instance group:
- Click Create instance group.
- Set Name as indicated in step 2.
- In the Location section, select Single-zone, choose
us-west1
for the Region, and then choose a Zone as indicated in step 2. - In the Group type section, select Unmanaged instance group.
- For Network, enter
lb-network
. - For Subnetwork, enter
lb-subnet
. - In the VM instances section, add the VMs as indicated in step 2.
- Click Create.
gcloud
Create the four VMs by running the following command four times, using these four combinations for
[VM-NAME]
and[ZONE]
. The script contents are identical for all four VMs.[VM-NAME]
ofvm-a1
and[ZONE]
ofus-west1-a
[VM-NAME]
ofvm-a2
and[ZONE]
ofus-west1-a
[VM-NAME]
ofvm-c1
and[ZONE]
ofus-west1-c
[VM-NAME]
ofvm-c2
and[ZONE]
ofus-west1-c
gcloud compute instances create [VM-NAME] \ --zone=[ZONE] \ --image-family=debian-9 \ --image-project=debian-cloud \ --tags=allow-ssh,allow-health-check \ --subnet=lb-subnet \ --metadata=startup-script='#! /bin/bash apt-get update apt-get install apache2 -y a2ensite default-ssl a2enmod ssl vm_hostname="$(curl -H "Metadata-Flavor:Google" \ http://169.254.169.254/computeMetadata/v1/instance/name)" echo "Page served from: $vm_hostname" | \ tee /var/www/html/index.html systemctl restart apache2'
Create the two unmanaged instance groups in each zone:
gcloud compute instance-groups unmanaged create ig-a \ --zone=us-west1-a gcloud compute instance-groups unmanaged create ig-c \ --zone=us-west1-c
Add the VMs to the appropriate instance groups:
gcloud compute instance-groups unmanaged add-instances ig-a \ --zone=us-west1-a \ --instances=vm-a1,vm-a2 gcloud compute instance-groups unmanaged add-instances ig-c \ --zone=us-west1-c \ --instances=vm-c1,vm-c2
api
For the four VMs, use the following VM names and zones:
[VM-NAME]
ofvm-a1
and[ZONE]
ofus-west1-a
[VM-NAME]
ofvm-a2
and[ZONE]
ofus-west1-a
[VM-NAME]
ofvm-c1
and[ZONE]
ofus-west1-c
[VM-NAME]
ofvm-c2
and[ZONE]
ofus-west1-c
Create four backend VMs by making four POST
requests to the
instances.insert
method:
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instances
{
"name": "[VM-NAME]",
"tags": {
"items": [
"allow-health-check",
"allow-ssh"
]
},
"machineType": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/machineTypes/n1-standard-1",
"canIpForward": false,
"networkInterfaces": [
{
"network": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/lb-network",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/subnetworks/lb-subnet",
"accessConfigs": [
{
"type": "ONE_TO_ONE_NAT",
"name": "external-nat",
"networkTier": "PREMIUM"
}
]
}
],
"disks": [
{
"type": "PERSISTENT",
"boot": true,
"mode": "READ_WRITE",
"autoDelete": true,
"deviceName": "[VM-NAME]",
"initializeParams": {
"sourceImage": "projects/debian-cloud/global/images/debian-9-stretch-v20190618",
"diskType": "projects/[PROJECT_ID]/zones/us-west1-a/diskTypes/pd-standard",
"diskSizeGb": "10"
}
}
]
"metadata": {
"items": [
{
"key": "startup-script",
"value": "#! /bin/bash\napt-get update\napt-get install apache2 -y\na2ensite default-ssl\na2enmod ssl\nvm_hostname=\"$(curl -H \"Metadata-Flavor:Google\" \\\nhttp://169.254.169.254/computeMetadata/v1/instance/name)\"\necho \"Page served from: $vm_hostname\" | \\\ntee /var/www/html/index.html\nsystemctl restart apache2"
}
]
},
"scheduling": {
"preemptible": false
},
"deletionProtection": false
}
Create two instance groups by making a POST
request to the
instanceGroups.insert
method.
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/us-west1-a/instanceGroups
{
"name": "ig-a",
"network": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/lb-network",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/subnetworks/lb-subnet"
}
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/us-west1-c/instanceGroups
{
"name": "ig-c",
"network": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/lb-network",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/subnetworks/lb-subnet"
}
Add instances to each instance group by making a POST
request to the
instanceGroups.addInstances
method.
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/us-west1-a/instanceGroups/ig-a/addInstances
{
"instances": [
{
"instance": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/us-west1-a/instances/vm-a1",
"instance": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/us-west1-a/instances/vm-a2"
}
]
}
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/us-west1-c/instanceGroups/ig-c/addInstances
{
"instances": [
{
"instance": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/us-west1-c/instances/vm-c1",
"instance": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/us-west1-c/instances/vm-c2"
}
]
}
Creating a client VM
This example creates a client VM (vm-client
) in the same region as the backend
(server) VMs. The client is used to validate the load balancer's configuration
and demonstrate expected behavior as described in the
testing section.
Console
- Go to the VM instances page in the Google Cloud Console.
Go to the VM instances page - Click Create instance.
- Set the Name to
vm-client
. - Set the Zone to
us-west1-a
. - Click Management, security, disks, networking, sole tenancy and make
the following changes:
- Click Networking and add the
allow-ssh
to Network tags. - Click the edit button under Network interfaces and make the
following changes then click Done:
- Network:
lb-network
- Subnet:
lb-subnet
- Primary internal IP: Ephemeral (automatic)
- External IP: Ephemeral
- Network:
- Click Networking and add the
- Click Create.
gcloud
The client VM can be in any zone in the same region as the
load balancer, and it can use any subnet in that region. In this example,
the client is in the us-west1-a
zone, and it uses the same
subnet as the backend VMs.
gcloud compute instances create vm-client \ --zone=us-west1-a \ --image-family=debian-9 \ --image-project=debian-cloud \ --tags=allow-ssh \ --subnet=lb-subnet
api
Make a POST
request to the
instances.insert
method.
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/us-west1-a/instances
{
"name": "vm-client",
"tags": {
"items": [
"allow-ssh"
]
},
"machineType": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/us-west1-a/machineTypes/n1-standard-1",
"canIpForward": false,
"networkInterfaces": [
{
"network": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/lb-network",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/subnetworks/lb-subnet",
"accessConfigs": [
{
"type": "ONE_TO_ONE_NAT",
"name": "external-nat",
"networkTier": "PREMIUM"
}
]
}
],
"disks": [
{
"type": "PERSISTENT",
"boot": true,
"mode": "READ_WRITE",
"autoDelete": true,
"deviceName": "[VM-NAME]",
"initializeParams": {
"sourceImage": "projects/debian-cloud/global/images/debian-9-stretch-v20190618",
"diskType": "projects/[PROJECT_ID]/zones/us-west1-a/diskTypes/pd-standard",
"diskSizeGb": "10"
}
}
]
"scheduling": {
"preemptible": false
},
"deletionProtection": false
}
Configuring load balancer components
These steps configure all of the internal TCP/UDP load balancer components starting with the health check and backend service, and then the frontend components:
Health check: In this example, we use an HTTP health check that simply checks for an HTTP
200
(OK) response. For more information, see the health checks section of the Internal TCP/UDP Load Balancing overview.Backend service: Because we need to pass HTTP traffic through the internal load balancer, we need to use TCP, not UDP.
Forwarding rule: This example creates a single internal forwarding rule.
Internal IP address: In this example, we specify an internal IP address,
10.1.2.99
, when we create the forwarding rule.
Console
Create the load balancer and configure a backend service
- Go to the Load balancing page in the Google Cloud Console.
Go to the Load balancing page - Click Create load balancer.
- Under TCP load balancing, click Start configuration.
- Under Internet facing or internal only select Only between my VMs.
- Click Continue.
- Set the Name to
be-ilb
. - Click Backend configuration and make the following changes:
- Region:
us-west1
- Network:
lb-network
- Under Backends, in the New item section, select the
ig-a
instance group and click Done. - Click Add backend. In the New item section that appears,
select the
ig-c
instance group and click Done again. - Under Health check, choose Create another health check,
enter the following information, and click Save and continue:
- Name:
hc-http-80
- Protocol:
HTTP
- Port:
80
- Proxy protocol:
NONE
- Request path:
/
- Name:
- Verify that there is a blue check mark next to Backend configuration before continuing. Review this step if not.
- Region:
- Click Frontend configuration. In the New Frontend IP and port
section, make the following changes:
- Name:
fr-ilb
- Subnetwork:
ilb-subnet
- From Internal IP, choose Reserve a static internal IP address,
enter the following information, and click Reserve:
- Name:
ip-ilb
- Static IP address: Let me choose
- Custom IP address:
10.1.2.99
- Name:
- Ports: Choose Single, and enter
80
for the Port number. - Verify that there is a blue check mark next to Frontend configuration before continuing. Review this step if not.
- Name:
- Click Review and finalize. Double-check your settings.
- Click Create.
gcloud
Create a new HTTP health check to test TCP connectivity to the VMs on 80.
gcloud compute health-checks create http hc-http-80 \ --port=80
Create the backend service for HTTP traffic:
gcloud compute backend-services create be-ilb \ --load-balancing-scheme=internal \ --protocol=tcp \ --region=us-west1 \ --health-checks=hc-http-80
Add the two instance groups to the backend service:
gcloud compute backend-services add-backend be-ilb \ --region=us-west1 \ --instance-group=ig-a \ --instance-group-zone=us-west1-a gcloud compute backend-services add-backend be-ilb \ --region=us-west1 \ --instance-group=ig-c \ --instance-group-zone=us-west1-c
Create a forwarding rule for the backend service. When you create the forwarding rule, specify
10.1.2.99
for the internal IP in the subnet.gcloud compute forwarding-rules create fr-ilb \ --region=us-west1 \ --load-balancing-scheme=internal \ --network=lb-network \ --subnet=lb-subnet \ --address=10.1.2.99 \ --ip-protocol=TCP \ --ports=80 \ --backend-service=be-ilb \ --backend-service-region=us-west1
api
Create the health check by making a POST
request to the
healthChecks.insert
method.
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/healthChecks
{
"name": "hc-http-80",
"type": "HTTP",
"httpHealthCheck": {
"port": 80
}
}
Create the regional backend service by making a POST
request to the
regionBackendServices.insert
method.
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/backendServices
{
"name": "be-ilb",
"backends": [
{
"group": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/us-west1-a/instanceGroups/ig-a",
"balancingMode": "CONNECTION"
}
{
"group": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/us-west1-c/instanceGroups/ig-c",
"balancingMode": "CONNECTION"
}
],
"healthChecks": [
"https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/healthChecks/hc-http-80"
],
"loadBalancingScheme": "INTERNAL",
"connectionDraining": {
"drainingTimeoutSec": 0
}
}
Create the forwarding rule by making a POST
request to the
forwardingRules.insert
method.
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/forwardingRules
{
"name": "fr-ilb",
"IPAddress": "10.1.2.99",
"IPProtocol": "TCP",
"ports": [
"80"
],
"loadBalancingScheme": "INTERNAL",
"subnetwork": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/subnetworks/lb-subnet",
"network": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/networks/lb-network",
"backendService": "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/backendServices/be-ilb",
"networkTier": "PREMIUM"
}
Testing
These tests show how to validate your load balancer configuration and learn about its expected behavior.
Testing load balancing
This test contacts the load balancer from a separate client VM; that is, not from a backend VM of the load balancer. The expected behavior is for traffic to be distributed among the four backend VMs because no session affinity has been configured.
Connect to the client VM instance.
gcloud compute ssh vm-client --zone=us-west1-a
Make a web request to the load balancer using
curl
to contact its IP address. Repeat the request so you can see that responses come from different backend VMs. The name of the VM generating the response is displayed in the text in the HTML response, by virtue of the contents of/var/www/html/index.html
on each backend VM. Expected responses look like:Page served from: vm-a1
,Page served from: vm-a2
, and so on.curl http://10.1.2.99
If you add a service label to the internal forwarding rule, you can use internal DNS to contact the load balancer using its service name.
curl http://web-test.fr-ilb.il4.us-west1.lb.[PROJECT_ID].internal
Pinging the load balancer's IP address
This test demonstrates an expected behavior: You cannot ping the IP address of the load balancer. This is because internal TCP/UDP load balancers are implemented in virtual network programming — they are not separate devices.
Connect to the client VM instance.
gcloud compute ssh vm-client --zone=us-west1-a
Attempt to ping the IP address of the load balancer. Notice that you do not get a response and that the
ping
command times out after 10 seconds in this example.timeout 10 ping 10.1.2.99
Sending requests from load balanced VMs
This test demonstrates that requests to the load balancer that originate from any of the backend VMs — the server VMs being load balanced — are always answered by the same VM making the request.
Internal TCP/UDP Load Balancing is implemented using virtual network programming and VM configuration in the guest OS. On Linux VMs, the Linux Guest Environment performs the local configuration by installing a route in the guest OS routing table. Because of this local route, traffic to the IP address of the load balancer stays on the load balanced VM itself. (This local route is different from the routes in the VPC network.)
Connect to a backend VM, such as
vm-a1
:gcloud compute ssh vm-a1 --zone=us-west1-a
Make a web request to the load balancer (by IP address or service name) using
curl
. Repeat the request, and note that the response is sent from the backend VM that makes the request. The expected response when testing fromvm-a1
is always:Page served from: vm-a1
.curl http://10.1.2.99
Inspect the local routing table, looking for a destination that matches the IP address of the load balancer itself,
10.1.2.99
. This route is a necessary part of Internal TCP/UDP Load Balancing, but it also demonstrates why a request from a VM behind the load balancer is always responded to by the same VM.ip route show table local | grep 10.1.2.99
Additional configuration options
This section expands on the configuration example providing alternative and additional configuration options. All of the tasks are optional. You can perform them in any order.
Configuring managed instance groups
The example configuration created two unmanaged instance groups. You can instead use managed instance groups, including zonal and regional managed instance groups, as backends for Internal TCP/UDP Load Balancing.
Managed instance groups require that you create an instance template. This procedure demonstrates how to replace the two unmanaged zonal instance groups from the example with a single, regional managed instance group. A regional managed instance group automatically creates VMs in multiple zones of the region, making it simpler to distribute production traffic among zones.
Managed instance groups also support autoscaling and autohealing. If you use autoscaling with Internal TCP/UDP Load Balancing, you cannot scale based on load balancing.
This procedure shows you how to modify the backend service for the example internal TCP/UDP load balancer so that it uses a regional managed instance group.
Console
Instance template
- Go to the VM instance templates page in the Google Cloud Console.
Go to the VM instance templates page - Click Create instance template.
- Set the Name to
template-vm-ilb
. - Choose a machine type.
- For Boot disk, click Change, choose Debian GNU/Linux 9 Stretch, then click Select.
- Click Management, security, disks, networking, sole tenancy.
- Click Networking and make the following changes:
- Network:
lb-network
- Subnet:
lb-subnet
- Network tags:
allow-ssh
andallow-health-check
- Network:
- Click Management. In the Startup script field, copy and paste
the following script contents:
#! /bin/bash apt-get update apt-get install apache2 -y a2ensite default-ssl a2enmod ssl vm_hostname="$(curl -H "Metadata-Flavor:Google"
http://169.254.169.254/computeMetadata/v1/instance/name)" echo "Page served from: $vm_hostname" |
tee /var/www/html/index.html systemctl restart apache2
- Click Networking and make the following changes:
- Click Create.
Managed instance group
- Go to the VM instance groups page in the Google Cloud Console.
Go to the VM instance groups page - Click Create instance group.
- Set the Name to
ig-ilb
. - For Location, choose Multi-zone, and set the Region to
us-west1
. - Set the Instance template to
template-vm-ilb
. - (Optional) Configure autoscaling. You cannot autoscale the instance group based on HTTP load balancing usage because the instance group is a backend for Internal TCP/UDP Load Balancing.
- Set the Minimum number of instances to
1
and the Maximum number of instances to6
. - (Optional) Configure
autohealing.
If you configure autohealing, use the same health check used by the backend
service for the Internal TCP/UDP Load Balancer. In this example, use
hc-http-80
. - Click Create.
gcloud
Create the instance template. Optionally, you can set other parameters, such as machine type, for the image template to use.
gcloud compute instance-templates create template-vm-ilb \ --image-family=debian-9 \ --image-project=debian-cloud \ --tags=allow-ssh,allow-health-check \ --subnet=lb-subnet \ --region=us-west1 \ --network=lb-network \ --metadata=startup-script='#! /bin/bash apt-get update apt-get install apache2 -y a2ensite default-ssl a2enmod ssl vm_hostname="$(curl -H "Metadata-Flavor:Google" \ http://169.254.169.254/computeMetadata/v1/instance/name)" echo "Page served from: $vm_hostname" | \ tee /var/www/html/index.html systemctl restart apache2'
Create one regional managed instance group using the template:
gcloud compute instance-groups managed create ig-ilb \ --template=template-vm-ilb \ --region=us-west1 \ --size=6
Add the regional managed instance group as a backend to the backend service you already created:
gcloud compute backend-services add-backend be-ilb \ --region=us-west1 \ --instance-group=ig-ilb \ --instance-group-region=us-west1
Disconnect the two unmanaged (zonal) instance groups from the backend service:
gcloud compute backend-services remove-backend be-ilb \ --region=us-west1 \ --instance-group=ig-a \ --instance-group-zone=us-west1-a gcloud compute backend-services remove-backend be-ilb \ --region=us-west1 \ --instance-group=ig-c \ --instance-group-zone=us-west1-c
Removing external IP addresses from backend VMs
When you created the backend VMs, each was assigned an ephemeral external IP address so it could download Apache via a startup script. Because the backend VMs are only used by an internal load balancer, you can remove their external IP addresses. Removing external IP addresses prevents the backend VMs from accessing the internet directly.
Console
- Go to the VM instances page in the Google Cloud Console.
Go to the VM instances page - Repeat the following steps for each backend VM.
- Click the name of the backend VM (for example,
vm-a1
) to view the VM instance details page. - Click Edit.
- In the Network interfaces section, click the Edit button.
- From the External IP pop-up, choose None, and click Done.
- Click Save.
gcloud
To look up the zone for an instance – for example, if you're using a regional managed instance group – run the following command for each instance to determine its zone. Replace
[SERVER-VM]
with the name of the VM to look up.gcloud compute instances list --filter="name=[SERVER-VM]"
Repeat the following step for each backend VM. Replace
[SERVER-VM]
with the name of the VM, and replace and[ZONE]
with the VM's zone.gcloud compute instances delete-access-config [SERVER-VM] \ --zone=[ZONE] \ --access-config-name=external-nat
api
Make a POST
request to the
instances.deleteAccessConfig
method for each backend VM, replacingvm-a1
with the name of the VM, and replacing and us-west1-a
with the VM's zone.
POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/us-west1-a/instances/v1-a1/deleteAccessConfig?accessConfig=external-nat&networkInterface=None
Accepting traffic on multiple ports
The component that controls the port for incoming traffic is the internal forwarding rule. This procedure shows you how to replace the forwarding rule for port 80 with one that accepts traffic on ports 80 and 443. When you create the backend VMs, the startup script that installs and configures Apache also configures it to serve HTTPS traffic on port 443. You must replace a forwarding rule because Google Cloud does not support updating forwarding rules.
gcloud
Delete your existing forwarding rule,
fr-ilb
.gcloud compute forwarding-rules delete fr-ilb \ --region=us-west1
Create a replacement forwarding rule with the same name for ports 80 and 443. The other parameters for this rule, including IP address and backend service, are the same.
gcloud compute forwarding-rules create fr-ilb \ --region=us-west1 \ --load-balancing-scheme=internal \ --network=lb-network \ --subnet=lb-subnet \ --address=10.1.2.99 \ --ip-protocol=TCP \ --ports=80,443 \ --backend-service=be-ilb \ --backend-service-region=us-west1
Connect to the client VM instance and test HTTP and HTTPS connections.
Connect to the client VM:
gcloud compute ssh vm-client --zone=us-west1-a
Test HTTP connectivity:
curl http://10.1.2.99
Test HTTPS connectivity. The
--insecure
flag is required because the Apache server configuration in the example setup uses self-signed certificates.curl https://10.1.2.99 --insecure
Observe that HTTP requests (on port 80) and HTTPS requests (on port 443) are handled by all of the backend VMs.
Using session affinity
The example configuration creates a backend service without session affinity.
This procedure shows you how to update the backend service for the example internal TCP/UDP load balancer so that it uses session affinity based on client IP addresses.
The load balancer directs a particular client's requests to the same backend VM based on a hash created from the client's IP address and the load balancer's IP address (the internal IP address of an internal forwarding rule)
Console
- Go to the Load balancing page in the Google Cloud Console.
Go to the Load balancing page - Click be-ilb (the name of the backend service you created for this example) and click Edit.
- On the Edit Internal load balancer page, click Backend configuration.
- Select Client IP from the Session affinity pop-up menu.
- Click Update.
gcloud
Use the following gcloud
command to update the be-ilb
backend service,
specifying client IP session affinity:
gcloud compute backend-services update be-ilb \ --session-affinity CLIENT_IP
api
Make a PATCH
request to the
regionBackendServices/patch
method.
PATCH https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/regions/us-west1/backendServices/be-ilb
{
"sessionAffinity": "CLIENT_IP"
}
For more information about using session affinity to influence traffic distribution and a description of each option, see Traffic distribution.
Creating a forwarding rule in another subnet
This procedure creates a second IP address and forwarding rule in a different subnet to demonstrate that you can create multiple forwarding rules for one internal TCP/UDP load balancer. The region for the forwarding rule must match the region of the backend service.
Subject to firewall rules, clients in any subnet in the region can contact either internal TCP/UDP load balancer IP address.
Create a second subnet in the
lb-network
network in theus-west1
region:gcloud compute networks subnets create second-subnet \ --network=lb-network \ --range=10.5.6.0/24 \ --region=us-west1
Create a second forwarding rule for ports 80 and 443. The other parameters for this rule, including IP address and backend service, are the same as for the primary forwarding rule,
fr-ilb
.gcloud compute forwarding-rules create fr-ilb-2 \ --region=us-west1 \ --load-balancing-scheme=internal \ --network=lb-network \ --subnet=second-subnet \ --address=10.5.6.99 \ --ip-protocol=TCP \ --ports=80,443 \ --backend-service=be-ilb \ --backend-service-region=us-west1
Connect to the client VM instance and test HTTP and HTTPS connections to the IP addresses.
- Connect to the client VM:
gcloud compute ssh vm-client --zone=us-west1-a
- Test HTTP connectivity to the IP addresses:
curl http://10.1.2.99 curl http://10.5.6.99
- Test HTTPS connectivity. Use of
--insecure
is required because the Apache server configuration in the example setup uses self-signed certificates.
curl https://10.1.2.99 --insecure curl https://10.5.6.99 --insecure
- Observe that requests are handled by all of the backend VMs, regardless of the protocol (HTTP or HTTPS) or IP address used.
What's next
- See Internal TCP/UDP Load Balancing Concepts for important fundamentals.
- See Failover concepts for Internal TCP/UDP Load Balancing for important information about failover.
- See Internal Load Balancing and DNS Names for available DNS name options your load balancer can use.
- See Configuring failover for Internal TCP/UDP Load Balancing for configuration steps and an example internal TCP/UDP load balancer failover configuration.
- See Internal TCP/UDP Load Balancing Logging and Monitoring for information on configuring Stackdriver logging and monitoring for Internal TCP/UDP Load Balancing.
- See Internal TCP/UDP Load Balancing and Connected Networks for information about accessing internal TCP/UDP load balancers from peer networks connected to your VPC network.
- See Troubleshooting Internal TCP/UDP Load Balancing for information on how to troubleshoot issues with your internal TCP/UDP load balancer.