Managing Failovers

If a Cloud Bigtable cluster becomes unresponsive, replication makes it possible for incoming traffic to fail over to another cluster in the same instance. Failovers can be either manual or automatic, depending on the app profile an application is using and how the app profile is configured.

This page describes the steps for performing a failover between clusters.

Before you read this page, you should be familiar with the overview of Cloud Bigtable replication.

Performing a manual failover

Use a manual failover if an app profile routes all requests to a single cluster, and that cluster becomes unhealthy. See Manual failovers for examples of the criteria you could use to determine that a cluster is unhealthy.

To perform a manual failover, update your app profile so that it routes requests to a healthy cluster instead of the unhealthy cluster:

Console

  1. Open the list of Cloud Bigtable instances in the GCP Console.

    Open the instance list

  2. In the Application profiles column, click the app profile that is routing traffic to the unhealthy cluster.

    If you don't see the app profile you want to edit, you can view a complete list by clicking the name of the instance, then clicking Application profiles in the left pane.

  3. Under Cluster routing, select a healthy cluster in your instance.

  4. Click Save. A confirmation dialog appears.

  5. Carefully review the warnings in the confirmation dialog, then follow the instructions in the dialog and click Continue.

gcloud

  1. If you don't know the instance ID, use the bigtable instances list command to view a list of your project's instances:

    gcloud bigtable instances list
    
  2. If you don't know the instance's cluster IDs, use the bigtable clusters list command to view a list of clusters in the instance:

    gcloud bigtable clusters list --instances=INSTANCE_ID
    

    Replace INSTANCE_ID with the permanent identifier for the instance.

  3. If you don't know the app profile's ID, use the beta bigtable app-profiles list command to view a list of the instance's app profiles:

    gcloud beta bigtable app-profiles list --instance=INSTANCE_ID
    

    Replace INSTANCE_ID with the permanent identifier for the instance.

  4. Use the beta bigtable app-profiles update command to change which cluster the app profile uses:

    gcloud beta bigtable app-profiles update APP_PROFILE_ID \
        --instance=INSTANCE_ID \
        --route-to=CLUSTER_ID
    

    Provide the following values:

    • APP_PROFILE_ID: The permanent identifier for the app profile.
    • INSTANCE_ID: The permanent identifier for the instance.
    • CLUSTER_ID: The cluster ID that all requests should be routed to. This flag enables single-cluster routing.

    If you receive an error message, carefully review any warnings in the error message. If you want to override the error, run the command again with the --force flag.

Shortly after you update the app profile, any applications that use the app profile will start to direct all of their requests to the healthy cluster that you selected. The unhealthy cluster will continue using CPU to handle replication and other maintenance tasks.

After the unhealthy cluster recovers, you can follow the same steps to update your app profile so that it routes all requests to the recovered cluster.

Performing an automatic failover

With Cloud Bigtable, automatic failovers truly are automatic. If an app profile routes requests to multiple clusters, and one cluster becomes unhealthy, you don't need to take any action. Cloud Bigtable automatically fails over, even if a cluster is only briefly unhealthy, and uses the healthy clusters to handle requests until the unhealthy cluster has recovered.

What's next

Learn how to monitor a Cloud Bigtable instance.

Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Bigtable Documentation