Manually deleting a user cluster

This page describes how to manually delete a GKE On-Prem user cluster.

Before you begin

• Before deleting a user cluster, make sure that you have removed any workloads that use PodDisruptionBudgets (PDBs), and all external objects, such as Persistent Volumes, that you have placed in the user cluster.

• Set a KUBECONFIG environment variable pointing to the kubeconfig of the user cluster that you want to delete:

  export KUBECONFIG=[USER_CLUSTER_KUBECONFIG]
  

  where [USER_CLUSTER_KUBECONFIG] is the path of the user cluster's kubeconfig file.

Unregistering a user cluster

Manually unregister your user cluster if it is registered to Google Cloud.

Deleting the Cluster object

Each user cluster has a Cluster resource that is stored in the admin cluster. To manually delete a user cluster, you can delete its Cluster object:

1. Get all Cluster objects in the admin cluster, where [ADMIN_CLUSTER_KUBECONFIG] is the path to the admin cluster's kubeconfig file:

   kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] get cluster --all-namespaces

   In the command output, look for the Cluster object named after the user cluster.

2. Delete the Cluster:

   kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] delete cluster [USER_CLUSTER_NAME] -n [USER_CLUSTER_NAMESPACE]
   

   • [USER_CLUSTER_NAME] is the name of the user cluster.
   • [USER_CLUSTER_NAMESPACE] shares the user cluster's name.

3. Verify that the user cluster has been removed by running the first command in this section again. Wait until the user cluster's name no longer appears in the list of Cluster object.

Note: This step may take several minutes. If any errors occur while deleting the cluster object, they are surfaced in the Cluster object's status. You can check the status by running the following from another shell:

kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] get cluster [USER_CLUSTER_NAME] -n [USER_CLUSTER_NAMESPACE] -o yaml

Verifying F5 partition is clean

CLI

Check that the VIP is down by running the following command:

ping -c 1 -W 1 [F5_LOAD_BALANCER_IP]; echo $?

which will return 1 if the VIP is down.

F5 UI

To check that the partition has been cleaned up from the F5 user interface, perform the following steps:

1. From the upper-right corner, click the Partition drop-down menu. Select your user cluster's partition.
2. From the left-hand Main menu, select Local Traffic > Network Map. There should be nothing listed below the Local Traffic Network Map.
3. From Local Traffic > Virtual Servers, select Nodes, then select Nodes List. There should be nothing listed here as well.

If there are any entries remaining, delete them manually from the UI.

Cleaning up a user cluster's F5 partition

If there are any entries remaining in the user cluster's partition, then, from the F5 BIG-IP console:

1. In the top-right corner of the console, switch to the partition to clean up.
2. Select Local Traffic > Virtual Servers > Virtual Server List
3. In the Virtual Servers menu, remove all the virtual IPs.
4. Select Pools, then delete all the pools.
5. Select Nodes, then delete all the nodes.

Deleting the user cluster namespace

To delete the user cluster namespace, run the following command:

kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] delete ns [USER_CLUSTER_NAMESPACE]

Then check that the user cluster namespace has been deleted by running the following command:

kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] get namespaces

Deleting the user control plane

To delete the user control plane, you need to delete the user cluster control plane's MachineDeployment resource using the admin cluster's kubeconfig.

To delete the MachineDeployment:

kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] delete machinedeployments -l kubernetes.googleapis.com/cluster-name=[USER_CLUSTER_NAME]

Note: If you are running a multi-control plane setup, this deletes multiple MachineDeployment objects.

Wait for the user cluster control plane node machine to be deleted before moving on to the next step. You can check progress by running the following command:

kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] get machines -l kubernetes.googleapis.com/cluster-name=[USER_CLUSTER_NAME]

Delete the VMDK file

1. From the vSphere Web Client's left-hand Navigator menu, click the Storage menu.
2. From the datacenter list, select your datastore.
3. In the main pane, click Files. Search for your VMDK file or find it in the list of files.
4. Right-click the VMDK file, then select Delete File.

After you have finished

After you have finished deleting the user cluster and all user cluster related objects in the admin cluster, delete the user cluster kubeconfig.

Troubleshooting

For more information, refer to Troubleshooting.

Diagnosing cluster issues using gkectl

Use gkectl diagnosecommands to identify cluster issues and share cluster information with Google. See Diagnosing cluster issues.