Using the vSphere Container Storage Interface driver

This document explains how to use the vSphere Container Storage Interface (CSI) driver with GKE on VMware.

Overview

The vSphere CSI driver is deployed automatically in GKE on VMware. For information about using other CSI drivers, see Installing a CSI driver.

vSphere requirements

To use the vSphere CSI driver, you must have:

A Linux-based supported version of vCenter Server
A supported version of ESXI

Volume expansion

Offline volume expansion requires both vCenter Server and ESXi to be version 7.0 Update 1 or later
Online volume expansion requires both vCenter Server and ESXi to be version 7.0 Update 2 or later.

Volume snapshot

Volume snapshot requires both vCenter Server and ESXi to be version 7.0 Update 3 or later.

Cns.Searchable

Your vCenter user account must have the Cns.Searchable privilege on the root vCenter object.

If you are unsure whether your account meets the CSI prerequisites, run gkectl diagnose cluster before deploying any CSI workloads.

Verify that the driver is installed

When the vSphere CSI driver is installed in your user cluster, a StorageClass called standard-rwo is created.

List the StorageClass objects:

kubectl --kubeconfig CLUSTER_KUBECONFIG get storageclass

Replace CLUSTER_KUBECONFIG with the path of your cluster kubeconfig file.

Verify that standard-rwo appears in the output. For example:

standard                 kubernetes.io/vsphere-volume  Delete   Immediate     false      33m
standard-rwo (default)   csi.vsphere.vmware.com    Delete   Immediate     true       32m

Verify that the vSphere CSI driver is installed:

kubectl --kubeconfig CLUSTER_KUBECONFIG get csinode \
    -o="custom-columns=NAME:metadata.name,DRIVERS:spec.drivers[].name"

The output should indicate that the driver is available on all of your cluster nodes. For example:

NAME     DRIVERS
node-0   csi.vsphere.vmware.com
node-1   csi.vsphere.vmware.com
node-2   csi.vsphere.vmware.com

Provisioning volumes

To provision volumes with the vSphere CSI driver, set the storageClassName field of your PersistentVolumeClaim to standard-rwo.

Setting the default storage class

To set standard-rwo as the default storage class, see Default storage class.

Using a snapshot to restore a volume

See Restore a volume from a snapshot.

Creating additional StorageClasses

You can create additional vSphere CSI StorageClasses in your cluster using the datastoreurl or storagepolicyname parameters.

The following example uses the storagepolicyname parameter:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gold
parameters:
  csi.storage.k8s.io/fstype: ext4
  storagepolicyname: "Gold Policy"
provisioner: csi.vsphere.vmware.com

To find the correct value for the datastoreurl parameter, run the following command:

govc datastore.info DATASTORE_NAME

Replace DATASTORE_NAME with the name of your vSphere datastore.

Use the value for URL from the output for datastoreurl. An example output looks like the following:

Name:        DATASTORE_NAME
  Path:      /DATACENTER_NAME/datastore/DATASTORE_NAME
  Type:      VMFS
  URL:       ds:///vmfs/volumes/5d864fa0-7f2184d4-8eb4-3cfdfe0ce9a0/
  Capacity:  20479.8 GB
  Free:      10066.9 GB

Volume cleanup

Volumes provisioned by the vSphere CSI driver are not deleted when you delete a user cluster. You should delete all PVCs and StatefulSets before deleting the cluster. To delete your volumes, run the following command:

kubectl delete statefulsets,persistentvolumeclaims --namespace=MY_NAMESPACE --all

Replace MY_NAMESPACE with your Kubernetes namespace.

If you deleted a user cluster without deleting its volumes, you can find the volumes in vSphere user interface:

Select a datastore.
Navigate to Monitor > Cloud Native Storage > Container Volumes.
In the Details section of each volume, you can see the Kubernetes cluster name.

Disabling vSphere CSI driver

To remove vSphere CSI support, add the following to your user cluster configuration:

# (Optional) Storage specification for the cluster storage:
# To disable the vSphere CSI driver, set this to true. The value is set to false by default and 
# the vSphere CSI driver is enabled by default on vSphere version >= 6.7U3.
  vSphereCSIDisabled: true