This page describes how to manually delete a GKE On-Prem user cluster.
Before you begin
Before deleting a user cluster, make sure that you have removed any workloads that use PodDisruptionBudgets (PDBs), and all external objects, such as Persistent Volumes, that you have placed in the user cluster.
KUBECONFIGenvironment variable pointing to the kubeconfig of the user cluster that you want to delete:
where [USER_CLUSTER_KUBECONFIG] is the path of the user cluster's kubeconfig file.
Unregistering a user cluster
Manually unregister your user cluster if it is registered to Google Cloud.
Deleting the Cluster object
Each user cluster has a Cluster resource that is stored in the admin cluster. To manually delete a user cluster, you can delete its Cluster object:
Get all Cluster objects in the admin cluster, where [ADMIN_CLUSTER_KUBECONFIG] is the path to the admin cluster's kubeconfig file:
kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] get cluster --all-namespaces
In the command output, look for the Cluster object named after the user cluster.
Delete the Cluster:
kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] delete cluster [USER_CLUSTER_NAME] -n [USER_CLUSTER_NAMESPACE]
- [USER_CLUSTER_NAME] is the name of the user cluster.
- [USER_CLUSTER_NAMESPACE] shares the user cluster's name.
Verify that the user cluster has been removed by running the first command in this section again. Wait until the user cluster's name no longer appears in the list of Cluster object.
Verifying F5 partition is clean
Check that the VIP is down by running the following command:
ping -c 1 -W 1 [F5_LOAD_BALANCER_IP]; echo $?
which will return
1 if the VIP is down.
To check that the partition has been cleaned up from the F5 user interface, perform the following steps:
- From the upper-right corner, click the Partition drop-down menu. Select your user cluster's partition.
- From the left-hand Main menu, select Local Traffic > Network Map. There should be nothing listed below the Local Traffic Network Map.
- From Local Traffic > Virtual Servers, select Nodes, then select Nodes List. There should be nothing listed here as well.
If there are any entries remaining, delete them manually from the UI.
Cleaning up a user cluster's F5 partition
If there are any entries remaining in the user cluster's partition, then, from the F5 BIG-IP console:
- In the top-right corner of the console, switch to the partition to clean up.
- Select Local Traffic > Virtual Servers > Virtual Server List
- In the Virtual Servers menu, remove all the virtual IPs.
- Select Pools, then delete all the pools.
- Select Nodes, then delete all the nodes.
Deleting the user cluster namespace
To delete the user cluster namespace, run the following command:
kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] delete ns [USER_CLUSTER_NAMESPACE]
Then check that the user cluster namespace has been deleted by running the following command:
kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] get namespaces
Deleting the user control plane
To delete the user control plane, you need to delete the user cluster control plane's MachineDeployment resource using the admin cluster's kubeconfig.
To delete the MachineDeployment:
kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] delete machinedeployments -l kubernetes.googleapis.com/cluster-name=[USER_CLUSTER_NAME]
Wait for the user cluster control plane node machine to be deleted before moving on to the next step. You can check progress by running the following command:
kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] get machines -l kubernetes.googleapis.com/cluster-name=[USER_CLUSTER_NAME]
Delete the VMDK file
- From the vSphere Web Client's left-hand Navigator menu, click the Storage menu.
- From the datacenter list, select your datastore.
- In the main pane, click Files. Search for your VMDK file or find it in the list of files.
- Right-click the VMDK file, then select Delete File.
After you have finished
After you have finished deleting the user cluster and all user cluster related objects in the admin cluster, delete the user cluster kubeconfig.
For more information, refer to Troubleshooting.
Diagnosing cluster issues using
gkectl diagnosecommands to identify cluster issues
and share cluster information with Google. See
Diagnosing cluster issues.
gkectl commands verbosely
gkectl errors to