This page explains how to use the vSphere Container Storage Interface (CSI) storage driver with Anthos clusters on VMware (GKE on-prem) clusters.
Starting with Anthos clusters on VMware version 1.5, the vSphere CSI driver is automatically deployed in Anthos clusters on VMware clusters. For information about other CSI drivers, see Installing a CSI driver.
Before you begin
Make sure you have the following prerequisites:
vCenter and ESXi host both at version 6.7 update 3 or newer.
- For volume expansion:
- Offline expansion requires both vCenter and ESXi to be version 7.0 or newer.
- Online expansion requires both vCenter and ESXi to be version 7.0 update 2 or newer.
cns.searchableuser account privileges.
If you are unsure whether your cluster meets the CSI prerequisites, run
gkectl diagnose cluster before deploying any CSI workloads.
Confirm driver availability
When the vSphere CSI driver is installed in your user cluster, a StorageClass
standard-rwo is created. To check that the driver is available
on all nodes, run the following command:
kubectl get csinode -o="custom-columns=NAME:metadata.name,DRIVERS:spec.drivers.name"
You should see the following output:
NAME DRIVERS node-0 csi.vsphere.vmware.com node-1 csi.vsphere.vmware.com node-2 csi.vsphere.vmware.com
To provision volumes with the vSphere CSI driver, set the
of your PersistentVolumeClaim to
Setting the default storage class
standard-rwo as the default storage class, see Default storage class.
Creating additional StorageClasses
You can create additional vSphere CSI StorageClasses in your cluster using the
The following example uses the
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: gold parameters: csi.storage.k8s.io/fstype: ext4 storagepolicyname: "Gold Policy" provisioner: csi.vsphere.vmware.com
To find the correct value for the
datastoreurl parameter, run the following command:
govc datastore.info DATASTORE_NAME
where DATASTORE_NAME is the name of your vSphere datastore.
Use the value for URL from the output for
datastoreurl. An example
output looks like the following:
Name: DATASTORE_NAME Path: /DATACENTER_NAME/datastore/DATASTORE_NAME Type: VMFS URL: ds:///vmfs/volumes/5d864fa0-7f2184d4-8eb4-3cfdfe0ce9a0/ Capacity: 20479.8 GB Free: 10066.9 GB
Volumes provisioned by the vSphere CSI driver are not deleted when you delete a user cluster. You should delete all PVCs and StatefulSets before deleting the cluster. To delete your volumes, run the following command:
kubectl delete statefulsets,persistentvolumeclaims --namespace=MY_NAMESPACE --all
where MY_NAMESPACE is your Kubernetes namespace.
If you deleted a user cluster without deleting its volumes, you can find the volumes in vCenter:
In vCenter, select a datastore.
Navigate to Monitor > Cloud Native Storage > Container Volumes.
The Kubernetes cluster name is shown in the Details section of each volume.
Disabling vSphere CSI driver
To remove vSphere CSI support, add the following to your user cluster configuration:
# (Optional) Storage specification for the cluster storage: # To disable the vSphere CSI driver, set this to true. The value is set to false by default and # the vSphere CSI driver is enabled by default on vSphere version >= 6.7U3. vSphereCSIDisabled: true
- Read more about Anthos clusters on VMware storage concepts
- Set a default StorageClass for your cluster