This document explains how to use the StatefulSet CSI Migration Tool to
migrate stateful workloads from an in-tree vSphere volume plugin to the vSphere
CSI Driver in Google Distributed Cloud.
Overview
Google Distributed Cloud integrates with external block or file storage systems
through VMware vSphere storage, Kubernetes in-tree volume plugins (or
"drivers"), and Container Storage Interface (CSI) drivers.
With the Kubernetes CSI migration feature enabled by default in 1.15, a
PersistentVolume backed by the in-tree vSphere volume plugin continues
functioning in a CSI-only environment. The CSI migration feature redirects
in-tree plugin operation calls to the CSI driver. Because the PersistentVolume
spec is immutable, it is still backed by the in-tree plugin. The set of
available features is the same as for the in-tree volume plugin.
The full set of CSI features, like Volume Expansion and Volume Snapshot,
isn't available for such volumes. To take advantage of these features,
stateful workloads must be completely migrated to CSI by recreating the
PersistentVolumes backed by the vSphere CSI Driver. You can use the
CSI migration tool to migrate stateful workloads to CSI and use the full set of
CSI features.
This tool provides a way to rolling migrate a StatefulSet's PersistentVolume and
PersistentVolumeClaim to CSI and does not involve application downtime. This
tooling takes the backup of Kubernetes resources in a local directory and sets
ReclaimPolicy to Retain before migration. So there will be no data loss.
Limitation
The automated tooling is only supported on Google Distributed Cloud versions that
are fully supported.
It only works with StatefulSets. You can run preflight checks using
the tool to run a few safety checks before using the tools.
ADMIN_KUBECONFIG: the path of your admin cluster
kubeconfig file.
USER_CLUSTER_NAME: If the StatefulSet is
running on the user cluster, then provide the user cluster name.
Skip this flag if the workload is running on the admin cluster.
STS_NAME: name of the StatefulSet.
STS_NAMESPACE: namespace of the StatefulSet.
INTREE_STORAGECLASS: the in-tree StorageClass
name backing the StatefulSet's PersistentVolume.
CSI_STORAGECLASS: the CSI StorageClass
name backing the StatefulSet's PersistentVolume after migration.
Download
Download the tool at
gs://gke-on-prem-release/statefulset-csi-migration-tool/v0.1/statefulset-csi-migration-tool.
Note this tool is in preview.
Procedure
This section provides the steps needed for migrating StatefulSet from vSphere
in-tree vCP internal provisioner (kubernetes.io/vsphere-volume) to vSphere
CSI provisioner (csi.vsphere.vmware.com).
ADMIN_KUBECONFIG: the path of your admin cluster
kubeconfig file.
USER_CLUSTER_NAME: If the StatefulSet is
running on the user cluster, then provide the user cluster name.
Skip this flag if the workload is running on the admin cluster.
STS_NAME: name of the StatefulSet.
STS_NAMESPACE: namespace of the StatefulSet.
INTREE_STORAGECLASS: the in-tree StorageClass
name backing the StatefulSet's PersistentVolume.
CSI_STORAGECLASS: the CSI StorageClass
name backing the StatefulSet's PersistentVolume after migration.
WORKING_DIRECTORY: the local directory to
save the Kubernetes resource spec of StatefulSet and its Pod,
PersistentVolumeClaim and PersistentVolume. The directory name must be
unique for each StatefulSet. This directory should be empty or not exist.
It is ideal to not create this directory so that the tooling can create
one for you.
This command performs the following tasks:
Takes a backup of the StatefulSet and its dependencies like PersistentVolume,
PersistentVolumeClaim and Pod replica specs in the local working directory.
Deletes the StatefulSet with orphan deletion policy. This step only deletes
the StatefulSet but does not delete its dependencies like the Pod replicas,
PersistentVolume and PersistentVolumeClaim.
Migrates each Pod to CSI drivers (similar to Option 1), and does the
following:
a. Sets the PersistentVolume's ReclaimPolicy field as Retain.
b. Deletes the Pod, PersistentVolume and PersistentVolumeClaim.
c. Converts the existing VMDK to FCD.
d. Creates the PersistentVolume, PersistentVolumeClaim and Pod again.
Recreates the StatefulSet, but the PVCTemplate field in its spec points to
the CSI StorageClass. The StatefulSet controller should map to the orphaned
replicas again.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-09-04 UTC."],[[["\u003cp\u003eThis tool facilitates the migration of stateful workloads from in-tree vSphere volume plugins to the vSphere CSI Driver in Google Distributed Cloud, enabling the use of full CSI features.\u003c/p\u003e\n"],["\u003cp\u003eThe StatefulSet CSI Migration Tool supports rolling migration of a StatefulSet's PersistentVolume and PersistentVolumeClaim to CSI without application downtime, backing up resources and setting \u003ccode\u003eReclaimPolicy\u003c/code\u003e to \u003ccode\u003eRetain\u003c/code\u003e to prevent data loss.\u003c/p\u003e\n"],["\u003cp\u003eThe automated tooling is compatible only with fully supported Google Distributed Cloud versions and is exclusively designed for use with StatefulSets.\u003c/p\u003e\n"],["\u003cp\u003eBefore migration, the tool offers preflight checks to ensure safety, requiring parameters such as admin cluster kubeconfig, cluster name, StatefulSet name, namespace, in-tree, and CSI StorageClass names.\u003c/p\u003e\n"],["\u003cp\u003eThe migration process involves backing up resources, deleting the StatefulSet, migrating each Pod to CSI drivers, converting VMDKs to FCDs, recreating resources with the CSI StorageClass, and re-establishing the StatefulSet with the migrated components.\u003c/p\u003e\n"]]],[],null,["# Using the StatefulSet CSI Migration Tool\n\n\u003cbr /\u003e\n\n|\n| **Preview**\n|\n|\n| This feature is subject to the \"Pre-GA Offerings Terms\" in the General Service Terms section\n| of the [Service Specific Terms](/terms/service-terms#1).\n|\n| Pre-GA features are available \"as is\" and might have limited support.\n|\n| For more information, see the\n| [launch stage descriptions](/products#product-launch-stages).\n\nThis document explains how to use the StatefulSet CSI Migration Tool to\nmigrate stateful workloads from an in-tree vSphere volume plugin to the vSphere\nCSI Driver in Google Distributed Cloud.\n\nOverview\n--------\n\nGoogle Distributed Cloud integrates with external block or file storage systems\nthrough VMware vSphere storage, Kubernetes in-tree volume plugins (or\n\"drivers\"), and Container Storage Interface (CSI) drivers.\n\nWith the Kubernetes CSI migration feature enabled by default in 1.15, a\nPersistentVolume backed by the in-tree vSphere volume plugin continues\nfunctioning in a CSI-only environment. The CSI migration feature redirects\nin-tree plugin operation calls to the CSI driver. Because the PersistentVolume\nspec is immutable, it is still backed by the in-tree plugin. The set of\navailable features is the same as for the in-tree volume plugin.\n\nThe full set of CSI features, like Volume Expansion and Volume Snapshot,\nisn't available for such volumes. To take advantage of these features,\nstateful workloads must be completely migrated to CSI by recreating the\nPersistentVolumes backed by the vSphere CSI Driver. You can use the\nCSI migration tool to migrate stateful workloads to CSI and use the full set of\nCSI features.\n\nThis tool provides a way to rolling migrate a StatefulSet's PersistentVolume and\nPersistentVolumeClaim to CSI and does not involve application downtime. This\ntooling takes the backup of Kubernetes resources in a local directory and sets\n`ReclaimPolicy` to `Retain` before migration. So there will be no data loss.\n\nLimitation\n----------\n\n- The automated tooling is only supported on Google Distributed Cloud versions that\n are [fully supported](/anthos/clusters/docs/on-prem/1.16//version-history).\n\n- It only works with StatefulSets. You can run preflight checks using\n the tool to run a few safety checks before using the tools.\n\n```\n./statefulset-csi-migration-tool preflight \\\n --kubeconfig ADMIN_KUBECONFIG \\\n --cluster-name USER_CLUSTER_NAME \\\n --sts-name STS_NAME \\\n --sts-namespace STS_NAMESPACE \\\n --intree-storageclass INTREE_STORAGECLASS \\\n --csi-storageclass CSI_STORAGECLASS\n```\n\nReplace the following:\n\n- \u003cvar translate=\"no\"\u003eADMIN_KUBECONFIG\u003c/var\u003e: the path of your admin cluster\n kubeconfig file.\n\n- \u003cvar translate=\"no\"\u003eUSER_CLUSTER_NAME\u003c/var\u003e: If the StatefulSet is\n running on the user cluster, then provide the user cluster name.\n Skip this flag if the workload is running on the admin cluster.\n\n- \u003cvar translate=\"no\"\u003eSTS_NAME\u003c/var\u003e: name of the StatefulSet.\n\n- \u003cvar translate=\"no\"\u003eSTS_NAMESPACE\u003c/var\u003e: namespace of the StatefulSet.\n\n- \u003cvar translate=\"no\"\u003eINTREE_STORAGECLASS\u003c/var\u003e: the in-tree StorageClass\n name backing the StatefulSet's PersistentVolume.\n\n- \u003cvar translate=\"no\"\u003eCSI_STORAGECLASS\u003c/var\u003e: the CSI StorageClass\n name backing the StatefulSet's PersistentVolume after migration.\n\nDownload\n--------\n\nDownload the tool at\n`gs://gke-on-prem-release/statefulset-csi-migration-tool/v0.1/statefulset-csi-migration-tool`.\nNote this tool is in preview.\n\nProcedure\n---------\n\nThis section provides the steps needed for migrating StatefulSet from vSphere\nin-tree **vCP** internal provisioner (`kubernetes.io/vsphere-volume`) to vSphere\n**CSI** provisioner (`csi.vsphere.vmware.com`). \n\n```\n./statefulset-csi-migration-tool rolling-migration all \\\n --kubeconfig ADMIN_KUBECONFIG \\\n --cluster-name USER_CLUSTER_NAME \\\n --sts-name STS_NAME \\\n --sts-namespace STS_NAMESPACE \\\n --intree-storageclass INTREE_STORAGECLASS \\\n --csi-storageclass CSI_STORAGECLASS \\\n --working-directory WORKING_DIRECTORY\n```\n\nReplace the following:\n\n- \u003cvar translate=\"no\"\u003eADMIN_KUBECONFIG\u003c/var\u003e: the path of your admin cluster\n kubeconfig file.\n\n- \u003cvar translate=\"no\"\u003eUSER_CLUSTER_NAME\u003c/var\u003e: If the StatefulSet is\n running on the user cluster, then provide the user cluster name.\n Skip this flag if the workload is running on the admin cluster.\n\n- \u003cvar translate=\"no\"\u003eSTS_NAME\u003c/var\u003e: name of the StatefulSet.\n\n- \u003cvar translate=\"no\"\u003eSTS_NAMESPACE\u003c/var\u003e: namespace of the StatefulSet.\n\n- \u003cvar translate=\"no\"\u003eINTREE_STORAGECLASS\u003c/var\u003e: the in-tree StorageClass\n name backing the StatefulSet's PersistentVolume.\n\n- \u003cvar translate=\"no\"\u003eCSI_STORAGECLASS\u003c/var\u003e: the CSI StorageClass\n name backing the StatefulSet's PersistentVolume after migration.\n\n- \u003cvar translate=\"no\"\u003eWORKING_DIRECTORY\u003c/var\u003e: the local directory to\n save the Kubernetes resource spec of StatefulSet and its Pod,\n PersistentVolumeClaim and PersistentVolume. The directory name must be\n unique for each StatefulSet. This directory should be empty or not exist.\n It is ideal to not create this directory so that the tooling can create\n one for you.\n\nThis command performs the following tasks:\n\n1. Takes a backup of the StatefulSet and its dependencies like PersistentVolume,\n PersistentVolumeClaim and Pod replica specs in the local working directory.\n\n2. Deletes the StatefulSet with orphan deletion policy. This step only deletes\n the StatefulSet but does not delete its dependencies like the Pod replicas,\n PersistentVolume and PersistentVolumeClaim.\n\n3. Migrates each Pod to CSI drivers (similar to Option 1), and does the\n following:\n\n a. Sets the PersistentVolume's ReclaimPolicy field as Retain.\n\n b. Deletes the Pod, PersistentVolume and PersistentVolumeClaim.\n\n c. Converts the existing VMDK to FCD.\n\n d. Creates the PersistentVolume, PersistentVolumeClaim and Pod again.\n4. Recreates the StatefulSet, but the PVCTemplate field in its spec points to\n the CSI StorageClass. The StatefulSet controller should map to the orphaned\n replicas again."]]