Anthos clusters on VMware cheatsheet

This topic provides an overview of commands called while using Anthos clusters on VMware (GKE on-prem). It is provided for convenience, and to supplement the Anthos clusters on VMware documentation.

Flags inside square brackets are optional. Placeholder variables are mutable.

`kubectl` commands

Set default kubeconfig file

export KUBECONFIG=[KUBECONFIG_PATH]

List clusters from default kubeconfig

kubectl get clusters

Pass in --kubeconfig [KUBECONFIG_PATH] to view clusters in a non-default kubeconfig.

List nodes in cluster from default kubeconfig

kubectl get nodes

Pass in --kubeconfig [KUBECONFIG_PATH] to view clusters in a non-default kubeconfig.

List all containers in all namespaces

kubectl get pods --all-namespaces -o jsonpath="{..image}" |\
tr -s '[[:space:]]' '\n' |\
sort |\
uniq -c

`gkectl` commands

Diagnosing cluster issues using `gkectl`

Use gkectl diagnosecommands to identify cluster issues and share cluster information with Google. See Diagnosing cluster issues.

Generate a Anthos clusters on VMware configuration file

gkectl create-config [--config [PATH]]

Validate a configuration file

gkectl check-config --config [PATH]

Push Anthos clusters on VMware images to your Docker registry, and initialize node OS image

gkectl prepare --config [CONFIG_FILE] [--validate-attestations]

Create clusters

gkectl create cluster --config [CONFIG_FILE]

Google Cloud service accounts

Create a service account

gcloud iam service-accounts create [SERVICE_ACCOUNT_NAME] --project [PROJECT_ID]

Grant an IAM role to a service account

gcloud projects add-iam-policy-binding \
    [PROJECT_ID] \
    --member="serviceAccount:[SERVICE_ACCOUNT_NAME]@[PROJECT_ID].iam.gserviceaccount.com" \
    --role="[ROLE_NAME]"

Create a private key for a service account

gcloud iam service-accounts keys create [KEY_FILE_NAME] \
--iam-account [SERVICE_ACCOUNT_NAME]@[PROJECT-ID].iam.gserviceaccount.com \
--project [PROJECT_ID]

Activate a service account and execute `gcloud`, `gsutil` commands as that account

gcloud auth activate-service-account --key-file=[SERVICE_ACCOUNT_KEY_FILE]

Admin workstation

SSH in to admin workstation

From the directory containing your Terraform configuration files:

ssh -i ~/.ssh/vsphere_workstation ubuntu@$(terraform output ip_address)

or, if you want to just use its address:

ssh -i ~/.ssh/vsphere_workstation ubuntu@[IP_ADDRESS]

Copy files to an admin workstation

scp -i ~./ssh/vsphere_workstation [SOURCE_PATH] ubuntu@$(terraform output ip_address)

Default logging behavior

For gkectl and gkeadm it is sufficient to use the default logging settings:

By default, log entries are saved as follows:
- For gkectl, the default log file is /home/ubuntu/.config/gke-on-prem/logs/gkectl-$(date).log, and the file is symlinked with the logs/gkectl-$(date).log file in the local directory where you run gkectl.
- For gkeadm, the default log file is logs/gkeadm-$(date).log in the local directory where you run gkeadm.
All log entries are saved in the log file, even if they are not printed in the terminal (when --alsologtostderr is false).
The -v5 verbosity level (default) covers all the log entries needed by the support team.
The log file also contains the command executed and the failure message.

We recommend that you send the log file to the support team when you need help.

Specifying a non-default location for the log file

To specify a non-default location for the gkectl log file, use the --log_file flag. The log file that you specify will not be symlinked with the local directory.

To specify a non-default location for the gkeadm log file, use the --log_file flag.

Locating Cluster API logs in the admin cluster

If a VM fails to start after the admin control plane has started, you can try debugging this by inspecting the Cluster API controllers' logs in the admin cluster:

Find the name of the Cluster API controllers Pod in the kube-system namespace, where [ADMIN_CLUSTER_KUBECONFIG] is the path to the admin cluster's kubeconfig file:
```
kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] -n kube-system get pods | grep clusterapi-controllers
```
Open the Pod's logs, where [POD_NAME] is the name of the Pod. Optionally, use grep or a similar tool to search for errors:
```
kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] -n kube-system logs [POD_NAME] vsphere-controller-manager
```

Clusters

Get IP addresses of an admin cluster's nodes

kubectl --kubeconfig [ADMIN_CLUSTER_KUBECONFIG] get nodes --output wide

Get IP addresses of a user cluster's nodes

kubectl --kubeconfig [USER_CLUSTER_KUBECONFIG] get nodes --output wide

SSH in to cluster nodes

See Using SSH to connect to a cluster node.