Survivability mode

Distributed Cloud connected clusters use a local control plane deployed on your Distributed Cloud connected hardware. When the connection to Google Cloud is lost, such clusters enter survivability mode and your workloads continue to run for up to 7 days.

When you create a Distributed Cloud connected cluster, the following rules apply:

  • You must create Distributed Cloud connected clusters in their own Google Cloud project. Distributed Cloud connected clusters cannot coexist in the same Google Cloud project with any other type of clusters, including non-Distributed Cloud connected clusters. Mixing Distributed Cloud connected clusters with any other type of clusters in the same Google Cloud project can result in data loss.
  • If you reassign a node between Distributed Cloud connected clusters, that node is wiped clean and reset to the default configuration.
  • By default, the local control plane workloads run in high availability mode with three replicas that span across three nodes chosen automatically by Distributed Cloud. This is true unless there are fewer than three nodes in the cluster, or you specifically configure the cluster to use one node to run the local control plane workloads. You also have the option to specify the three nodes for high availability mode by using the --control-plane-machine-filter flag. No other node combinations are supported.
  • The nodes that run the local control plane workloads also run your application workloads.
  • The IP addresses of local control plane endpoints are accessible on your local network. You must ensure that your local network's security configuration prevents external access to those IP addresses.

When in survivability mode, a Distributed Cloud connected cluster operates as follows:

  • Control over workloads through the Google Cloud CLI, the kubectl CLI, and the Distributed Cloud Edge Container API is disabled.
  • Distributed Cloud software updates, SLOs, and hardware repair are unavailable.
  • Limited logs and metrics are synchronized with Google Cloud after the connection to Google Cloud is re-established.
  • By default, if a node reboots while the cluster is disconnected from Google Cloud, it cannot rejoin its cluster until the connection to Google Cloud is re-established because its authentication key cannot be refreshed. You have the option to specify an offline reboot window during which a node can rejoin a cluster after rebooting while the cluster is running in survivability mode. For more information, see Create a cluster.

Check the connection state of a cluster

You can check the state of your Distributed Cloud cluster's to Google Cloud by completing the steps in Get information about a cluster. The command returns the value for the connectionState field. This field can have one of the following values:

  • CONNECTED: The cluster is connected to and fully synchronized with Google Cloud.
  • DISCONNECTED: The cluster is not connected to Google Cloud.
  • CONNECTED_AND_SYNCING: The cluster has reconnected to Google Cloud and is synchronizing offline data with Google Cloud. Do not disconnect this cluster from Google Cloud until synchronization has completed.

What's next