Infrastructure incompatibility

vSphere feature incompatibility

This section describes the vSphere features that are incompatible with Google Distributed Cloud.

VM snapshot

VM snapshot is not compatible with Google Distributed Cloud. Taking snapshots of VMs created by Google Distributed Cloud will break many features including cluster upgrade, cluster update, node auto-repair and admin cluster control plane recovery. When Google Distributed Cloud tries to manage a VM with snapshots, you will see failures such as Invalid configuration for device '0' in the csi-controller-manager and vsphere-controller-manager logs. For more information on snapshots, see Best Practices for using VMware snapshots.

Don't create snapshots on Anthos VMs. To recover from VM or storage failures, see Recovery from VM failures and Recovery from storage failures.

Note: Using a third-party solution, such as Commvault, to back up VMs isn't supported by GKE Enterprise. Backing up VMs with a third-party solution may lead to unforeseen errors in cluster operations.

VM clone

Cloning a VM that is created and managed by Google Distributed Cloud may result in data and cluster state inconsistencies.

Don't clone Anthos VMs. To recover from VM or storage failures, see Recovery from VM failures and Recovery from storage failures.

vSAN File Service

Using vSAN File Service to Provision File Volumes supports ReadWriteMany or ReadOnlyMany persistent volumes that can be shared between multiple pods or applications.

Don't use that in Google Distributed Cloud as it may block node drain and fail the diagnose.

You can run the following command to list all PVCs:

kubectl --kubeconfig CLUSTER_KUBECONFIG get pvc --all-namespaces \
    -ojson | jq '.items[] | {namespace: .metadata.namespace, name: .metadata.name, accessModes: .spec.accessModes}'

If there is any PVC having ReadWriteMany or ReadOnlyMany in its accessModes, contact Google support for a workaround.

Storage vMotion

Storage vMotion lets you migrate the virtual disks of a VM from one datastore to another. This is different from the regular vMotion, which migrates a VM from one host to another. vMotion is supported by Google Distributed Cloud.

Storage vMotion is only compatible with Google Distributed Cloud when the clusters are configured with storage policies. Otherwise, operations such as cluster update, upgrade, user cluster creation and node auto-repair will fail when you use storage vMotion.

To recover from unplanned storage vMotion, or to plan a workaround, contact Google support.

Storage DRS

Storage DRS manages virtual machine disk placement and migration to balance the storage space or I/O resources between datastores in the datastore cluster.

Don't activate Storage DRS as it is not compatible with Google Distributed Cloud.

Changed Block Tracking(CBT)

Chnged Block Tracking(CBT) is a VMkernel feature that identifies blocks of data that have changed or are in use, which is enabled through VMware API calls by 3rd-party backup software or appliances.

Don't use 3rd-party backup software or appliances to backup Google Distributed Cloud VMs. They usually enable CBT through the VMware API that is not compatible with Google Distributed Cloud.

Networking incompatibility

This section applies to you if you are using the Seesaw load balancer.

Cisco ACI with IP Data-plane Learning

Cisco Application Centric Infrastructure (ACI) with Dataplane IP Learning is not compatible with Seesaw load balancer. We recommend that you use MetalLB as your load balancer as Seesaw is in maintenance mode, or disable Dataplane IP Learning when using Seesaw as your load balancer.

Stateful NSX-T Distributed Firewall (DFW)

Stateful NSX-T DFW is not compatible with Seesaw load balancer. We recommend that you use MetalLB as your load balancer as Seesaw is in maintenance mode, or configure a stateless NSX-T DFW policy for the Seesaw VMs when using Seesaw as your load balancer. For more information, see Configuring stateless NSX-T distributed firewall policies for use with Seesaw load balancer.