Version 1.7. This version is supported as outlined in the Anthos version support policy, offering the latest patches and updates for security vulnerabilities, exposures, and issues impacting Anthos clusters on VMware (GKE on-prem). Refer to the release notes for more details. This is the most recent version.

Using the vSphere Container Storage Interface driver

This page explains how to use the vSphere Container Storage Interface (CSI) storage driver with Anthos clusters on VMware (GKE on-prem) clusters.

Overview

Starting with Anthos clusters on VMware version 1.5, the vSphere CSI driver is automatically deployed in Anthos clusters on VMware clusters. For information about other CSI drivers, see Installing a CSI driver.

Before you begin

Make sure you have the following prerequisites:

If you are unsure whether your cluster meets the CSI prerequisites, run gkectl diagnose cluster before deploying any CSI workloads.

Note that Anthos clusters on VMware, does not support the vSphere CSI volume expansion feature.

Confirm driver availability

When the vSphere CSI driver is installed in your user cluster, a StorageClass called standard-rwo is created. To check that the driver is available on all nodes, run the following command:

kubectl get csinode -o="custom-columns=NAME:metadata.name,DRIVERS:spec.drivers[].name"

You should see the following output:

NAME     DRIVERS
node-0   csi.vsphere.vmware.com
node-1   csi.vsphere.vmware.com
node-2   csi.vsphere.vmware.com

Provisioning volumes

To provision volumes with the vSphere CSI driver, set the storageClassName field of your PersistentVolumeClaim to standard-rwo.

Setting the default storage class

To set standard-rwo as the default storage class, see Default storage class.

Creating additional StorageClasses

You can create additional vSphere CSI StorageClasses in your cluster using the datastoreurl or storagepolicyname parameters.

The following example uses the storagepolicyname parameter:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gold
parameters:
  csi.storage.k8s.io/fstype: ext4
  storagepolicyname: "Gold Policy"
provisioner: csi.vsphere.vmware.com

To find the correct value for the datastoreurl parameter, run the following command:

govc datastore.info DATASTORE_NAME

where DATASTORE_NAME is the name of your vSphere datastore.

Use the value for URL from the output for datastoreurl. An example output looks like the following:

Name:        DATASTORE_NAME
  Path:      /DATACENTER_NAME/datastore/DATASTORE_NAME
  Type:      VMFS
  URL:       ds:///vmfs/volumes/5d864fa0-7f2184d4-8eb4-3cfdfe0ce9a0/
  Capacity:  20479.8 GB
  Free:      10066.9 GB

Volume cleanup

Volumes provisioned by the vSphere CSI driver are not deleted when you delete a user cluster. You should delete all PVCs and StatefulSets before deleting the cluster. To delete your volumes, run the following command:

kubectl delete statefulsets,persistentvolumeclaims --namespace=MY_NAMESPACE --all

where MY_NAMESPACE is your Kubernetes namespace.

If you deleted a user cluster without deleting its volumes, you can find the volumes in vCenter:

  1. In vCenter, select a datastore.

  2. Navigate to Monitor > Cloud Native Storage > Container Volumes.

The Kubernetes cluster name is shown in the Details section of each volume.

Disabling vSphere CSI driver

To remove vSphere CSI support, add the following to your user cluster configuration:

# # (Optional) Storage specification for the cluster
storage:
# To disable the vSphere CSI driver, set this to true. The value is set to false by default and # the vSphere CSI driver is enabled by default on vSphere version >= 6.7U3.
  vSphereCSIDisabled: true

What's next