Adding Health Checks

Google Cloud Platform (GCP) health checks determine whether instances are "healthy" and available to do work. This document describes using health checks with load balancing.

For information on the newer style health checks (HealthChecks) that can be configured for TCP, SSL, HTTP, or HTTPS probing, proceed directly to the Overview section.

For information on the legacy HttpHealthChecks and HttpsHealthChecks resources, see the Legacy health checks section. Currently, only Network load balancing requires legacy HttpHealthChecks. All other load balancers can use the newer style.

For information on using health checks with managed instance group autohealing, see Setting up health checking for managed instance groups.

Overview

A health checker polls instances at specified intervals. Instances that do not respond successfully to a specified number of consecutive probes are marked as UNHEALTHY. No new connections are sent to such instances, though existing connections are allowed to continue. The health checker continues to poll unhealthy instances. If an instance later responds successfully to a specified number of consecutive probes, it is marked HEALTHY again and can receive new connections.

Supported protocols

Newer-style health checks support the following protocols:

HTTP and HTTPS health checks

If traffic from the load balancer to your instances uses the HTTP or HTTPS protocol, then HTTP or HTTPS health checks verify that the instance is healthy and the web server is up and serving traffic.

For an HTTP(S) health check probe to be deemed successful, the instance must return a valid HTTP response with code 200 and close the connection normally within the configured period. If it does this a specified number of times in a row, the health check returns a status of HEALTHY for that instance. If an instance fails a specified number of health check probes in a row, it is marked UNHEALTHY without any notification being sent. UNHEALTHY instances do not receive new connections, but existing connections are allowed to continue. UNHEALTHY instances continue to receive health check probes. If an instance later passes a health check by successfully responding to a specified number of consecutive health check probes, it is marked HEALTHY and starts receiving new connections, again without any notification.

TCP and SSL health checks

TCP and SSL health checks can be used when the service expects a TCP or SSL connection that is not HTTP(S).When you configure the health check to be of type SSL, an SSL connection is used to connect to the instances. When you configure the health check to be of type TCP, a TCP connection is used. An instance passes the health check if a specified number of probes in a row are successful, and it fails the health check if a specified number of probes in a row are unsuccessful. Existing connections are allowed to continue on instances that have failed their health check. A TCP or SSL health check probe can use one of the following checks:

  • Simple handshake (default): The health checker attempts only a simple TCP or SSL handshake. If it is successful, the instance passes that round of the probe.
  • Request/response: You provide a request string for the health checker to send after completing the TCP or SSL handshake. If the instance returns the response string you've configured, the instance passes that round of the probe. Both the request and response strings can be up to 1024 bytes.

Health check frequency

To ensure high availability, GCP creates redundant copies of each health checker. These redundant health checkers also probe your instances. If any health checker fails, a redundant one can take over with no delay.

If you examine the logs on your instance, you might see health check polling happening more frequently than than you have configured. This is because the redundant health checkers are also following your settings. These redundant health checkers are created automatically and are not separately user configurable.

Creating and working with health checks

Create a health check

Console


Use the gcloud command-line tool to create health checks.

gcloud


gcloud compute health-checks create [tcp | ssl | http | https] my-health-check \
    [--check-interval=CHECK_INTERVAL; default="5s"] \
    [--description=DESCRIPTION] \
    [--healthy-threshold=HEALTHY_THRESHOLD; default="2"] \
    [--host=HOST] \
    [--port=PORT; default="80"] \
    [--port-name=PORT_NAME] \
    [--proxy-header=PROXY_HEADER; default="NONE"] \
    [--request-path=REQUEST_PATH; default="/"] \
    [--timeout=TIMEOUT; default="5s"] \
    [--unhealthy-threshold=UNHEALTHY_THRESHOLD; default="2"]

For a full description of all parameters, see the SDK documentation.

API


To make a request to the API, send a POST request to the following URI:

POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/healthChecks

{
  "name" : [NAME]
}

The body of your request must contain, at a minimum, the name field.

For a full description of all parameters, see the API documentation.

List health checks

Lists the health checks in the current project.

Console


Go to the Health checks page in the Google Cloud Platform Console.
Go to the Health checks page

gcloud


gcloud compute health-checks list

NAME            PROTOCOL
my-health-check SSL

API


https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/healthChecks

Describe a health check

Provides detailed information about a specific health check.

Console


  1. Go to the Health checks page in the Google Cloud Platform Console.
    Go to the Health checks page
  2. Click on a health check.

gcloud


gcloud compute health-checks describe my-health-check

checkIntervalSec: 5
creationTimestamp: '2016-02-20T20:47:26.034-08:00'
description: ''
healthyThreshold: 2
id: '1423984233044836273'
kind: compute#healthCheck
name: my-health-check
selfLink: https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/healthChecks/my-health-check
sslHealthCheck:
  port: 443
timeoutSec: 5
type: SSL
unhealthyThreshold: 2

API


https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/healthChecks/[HEALTH_CHECK]

Update a health check

To modify a parameter in a health check, run the following command and pass in any of the create parameters. Any specified parameters will be changed. All unspecified parameters will be left the same.

Console


  1. Go to the Health checks page in the Google Cloud Platform Console.
    Go to the Health checks page
  2. Click the name of the health check you wish to update.
  3. Click Edit.
  4. Update any necessary fields.
  5. Click Save.

gcloud


gcloud compute health-checks [tcp|ssl|http|https] update
    [--options]

Example:

gcloud compute health-checks update ssl my-health-check \
    --description "SSL health check"

Updated [https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/healthChecks/my-health-check].

API


In the API, you can choose to update your health check using the standard PUT request, or use PATCH to partially update your health. PATCH will only update the fields that you specify.

https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/healthChecks/[HEALTH_CHECK]

Health check source IPs and firewall rules

Network load balancing

When a health check is used with Network load balancing, the health check probes come from addresses in the ranges 209.85.152.0/22, 209.85.204.0/22, and 35.191.0.0/16. You need to create firewall rules that allow these connections to all your load balanced instances.

HTTP(S), SSL proxy, TCP proxy, and Internal load balancing

When a health check is used with HTTP(S), SSL proxy, TCP proxy, or Internal load balancing, the health check probes come from addresses in the ranges 130.211.0.0/22 and 35.191.0.0/16. You need to create firewall rules that allow these connections to all your load balanced instances.

Configure a firewall rule to allow health checking

Configure a firewall rule to allow health check probes from the health checker.

Console


  1. Go to the Firewall rules page in the Google Cloud Platform Console.
    Go to the Firewall rules page
  2. Click Create firewall rule.
  3. Set Name to allow-health-check.
  4. Set Network to my-custom-network.
  5. Set Source filter to IP ranges.
  6. Set Source IP ranges to 130.211.0.0/22 and 35.191.0.0/16.
  7. Set Allowed protocols and ports to tcp.
  8. Set Target tags to health-check-tag.
  9. Click Create.

gcloud


gcloud compute firewall-rules create allow-health-check \
    --network my-custom-network \
    --source-ranges 130.211.0.0/22,35.191.0.0/16 \
    --target-tags health-check-tag \
    --allow tcp

NAME                NETWORK            SRC_RANGES                    RULES  SRC_TAGS  TARGET_TAGS
allow-health-check  my-custom-network  130.211.0.0/22,35.191.0.0/16  tcp              health-check-tag

API


POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/firewalls
{
  "kind": "compute#firewall",
  "name": "allow-health-check",
  "sourceRanges": [
    "130.211.0.0/22",
    "35.191.0.0/16"
  ],
  "sourceTags": [],
  "targetTags": [
    "health-check-tag"
    ],
      "allowed": [
    {
      "IPProtocol": "tcp"
    }
  ],
  "network": "projects/[PROJECT_ID]/global/networks/my-custom-network"
}

Legacy health checks

Legacy health checks work similarly to normal health checks, but each one is limited to a specific protocol (HTTP or HTTPS):

  • HTTP health checks: required for network load balancing. Existing HTTP load balancing configurations may continue to use legacy health checks, but we recommend that you replace legacy health checks with newer-style health checks.
  • HTTPS health checks: Existing HTTPS load balancing configurations may continue to use legacy health checks, but we recommend that you replace legacy health checks with newer-style health checks.

For an HTTP or HTTPS health check to be deemed successful, the instance must return a valid HTTP response with code 200 and close the connection normally within the timeoutSec period.

Create a legacy health check

To create a health check object with the gcloud command-line tool, use the http-health-checks create or https-health-checks create sub-command.

Console


  1. Go to the Health checks page in the Google Cloud Platform Console.
    Go to the Health checks page
  2. Add a Name.
  3. Populate other fields as needed.
  4. Click Create.

gcloud


gcloud compute http-health-checks create NAME
    [--check-interval CHECK_INTERVAL; default="5s"]
    [--description DESCRIPTION]
    [--healthy-threshold HEALTHY_THRESHOLD; default="2"]
    [--host HOST]
    [--port PORT; default="80"]
    [--request-path REQUEST_PATH; default="/"]
    [--timeout TIMEOUT; default="5s"]
    [--unhealthy-threshold UNHEALTHY_THRESHOLD; default="2"]

or

gcloud compute https-health-checks create NAME
    [--check-interval CHECK_INTERVAL; default="5s"]
    [--description DESCRIPTION]
    [--healthy-threshold HEALTHY_THRESHOLD; default="2"]
    [--host HOST]
    [--port PORT; default="443"]
    [--request-path REQUEST_PATH; default="/"]
    [--timeout TIMEOUT; default="5s"]
    [--unhealthy-threshold UNHEALTHY_THRESHOLD; default="2"]

API


To make a request to the API, send a POST request to the following URI:

POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/httpHealthChecks

{
  "name" : [NAME]
}

or

POST https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/httpsHealthChecks

{
  "name" : [NAME]
}

The body of your request must contain, at a minimum, the name field.

Update legacy health checks

To update the properties of an existing health check with the gcloud command-line tool, use the http-health-checks update sub-command. The values of your request will update the existing values of the health check.

Console


  1. Go to the Health checks page in the Google Cloud Platform Console.
    Go to the Health checks page
  2. Click the name of the health check you wish to update.
  3. Click Edit.
  4. Update any necessary fields.
  5. Click Save.

gcloud


gcloud compute http-health-checks update [HEALTH_CHECK]
    [--check-interval CHECK_INTERVAL]
    [--description DESCRIPTION]
    [--healthy-threshold HEALTHY_THRESHOLD]
    [--host HOST]
    [--port PORT]
    [--request-path REQUEST_PATH]
    [--timeout TIMEOUT]
    [--unhealthy-threshold UNHEALTHY_THRESHOLD]
    

or

gcloud compute https-health-checks update [HEALTH_CHECK]
    [--check-interval CHECK_INTERVAL]
    [--description DESCRIPTION]
    [--healthy-threshold HEALTHY_THRESHOLD]
    [--host HOST]
    [--port PORT]
    [--request-path REQUEST_PATH]
    [--timeout TIMEOUT]
    [--unhealthy-threshold UNHEALTHY_THRESHOLD]
    

API


In the API, you can choose to update your health check using the standard PUT request, or use PATCH to partially update your health. PATCH will only update the fields that you specify.

https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/httpHealthChecks/[HEALTH_CHECK]

or

https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/httpsHealthChecks/[HEALTH_CHECK]

Your request body should contain your desired fields and values to update for this health check.

List legacy health checks

To list health checks with the gcloud command-line tool, use the http-health-checks list sub-command for HTTP health checks or the https-health-checks list sub-command for HTTPS health checks:

Console


  1. Go to the Health checks page in the Google Cloud Platform Console.
    Go to the Health checks page
  2. Click on a health check.

gcloud


gcloud compute http-health-checks list

or

gcloud compute https-health-checks list

API


https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/httpHealthChecks

or

https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/httpsHealthChecks

Describe a legacy health check

To get information about a specific legacy health check with the gcloud command-line tool, use the http-health-checks describe sub-command for HTTP health checks or the https-health-checks describe sub-command for HTTPS health checks:

Console


  1. Go to the Health checks page in the Google Cloud Platform Console.
    Go to the Health checks page
  2. Click on a health check.

gcloud


gcloud compute http-health-checks describe [HEALTH_CHECK]

or

gcloud compute https-health-checks describe [HEALTH_CHECK]

API


https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/httpHealthChecks/[HEALTH_CHECK]

or

https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/global/httpsHealthChecks/[HEALTH_CHECK]

Associate the health check with your configuration

Creating a health check resource does not automatically apply the health check resource to your load balancer. After you add a health check, you must associate it with a target pool or backend service before it can perform health checking.

Before a health check can be used with a backend service, you must configure firewall rules that allow the health check probes to reach the instances.

A target pool or backend service will likely contain more than one instance. It is not possible to define different health check parameters for each instance. You can only define health check parameters that apply to all instances in that target pool or backend service.

Network load balancing

You can add a health check to an existing target pool, or to a new target pool. This example adds the health check object to an existing target pool:

gcloud compute target-pools add-health-checks [TARGET_POOL] \
    --http-health-check [HEALTH_CHECK]

HTTP(S), TCP proxy, SSL proxy, and Internal load balancing

When you create your backend service object, you must specify a health check. See the backend service documentation for details.

Handling unhealthy instances

When an instance becomes unhealthy, it may not become unhealthy in a way that registers with your existing health checks. During this time, the load balancing service will continue sending new requests to the instance.

You can force an instance to fail health checks by blocking the IP addresses of the health checkers. This allows you to troubleshoot the instance without shutting it down.

The following commands work for instances that have iptables installed.

For a Network load balancer instance, run the following command. This will only block health check traffic. All other traffic will continue to reach the instance normally until the health check fails.

$ sudo iptables -I INPUT 1 -m state --state NEW -s 169.254.169.254 \
-p tcp --destination-port 80 -j REJECT  \
--reject-with tcp-reset # Network load balancing

For an TCP or HTTP load balancer instance, if you are using the default port (80) to check health, run the following command. If you are using a different port to check health, replace 80 in the command with your health check port.

If you are using the same port to check health and serve traffic, then this command prevents new connections to the instance even before the health check fails. If you are using different ports, then new connections can come in while the health check is failing.

$ sudo iptables -I INPUT 1 -m state --state NEW -s 35.191.0.0/16 \
-p tcp --destination-port 80 -j REJECT  \
--reject-with tcp-reset # HTTP load balancing
$ sudo iptables -I INPUT 1 -m state --state NEW -s 130.211.0.0/22 \
-p tcp --destination-port 80 -j REJECT  \
--reject-with tcp-reset # HTTP load balancing

For an SSL or HTTPS load balancer instance, if you are using the default port (443) to check health, run the following command. In addition to blocking health check connections, this command will prevent all new connections from the load balancer to the instance.

$ sudo iptables -I INPUT 1 -m state --state NEW -s 35.191.0.0/16 \
-p tcp --destination-port 443 -j REJECT  \
--reject-with tcp-reset # HTTPS load balancing
$ sudo iptables -I INPUT 1 -m state --state NEW -s 130.211.0.0/22 \
-p tcp --destination-port 443 -j REJECT  \
--reject-with tcp-reset # HTTPS load balancing

To restore the instance, run the same command again, but with -D INPUT instead of -I INPUT 1.

You can also add the relevant iptables command to a shutdown script so the instance shuts down more gracefully. For a graceful shutdown, have your script block the health check, then wait enough time for the health check to fail before shutting down other services. How long the script has to wait depends upon the settings of your health check.

For more information, see Running shutdown scripts.

Send feedback about...

Compute Engine Documentation