This page describes how to rotate keys for the following service accounts:
To rotate your service account keys:
Create a directory to store a backup of your current secrets:
mkdir backup
Note the following information for the relevant service account:
Component access
Cluster Secret Namespace Admin admin-cluster-creds kube-system Admin user-cluster-creds CLUSTER_NAME-gke-onprem-mgmt Admin private-registry-creds kube-system User private-registry-creds kube-system - If you aren't using a
private registry,
the
private-registry-creds
Secret holds the key for your component access service account. - If you are using a private registry, the
private-registry-creds
Secret holds the credentials for your private registry, not the component access service account key.
Connect-register
Cluster Secret Namespace Admin admin-cluster-creds kube-system Admin user-cluster-creds CLUSTER_NAME-gke-onprem-mgmt Logging-monitoring
Cluster Secret Namespace Admin admin-cluster-creds kube-system Admin user-cluster-creds CLUSTER_NAME-gke-onprem-mgmt User google-cloud-credentials kube-system User stackdriver-service-account-key knative-serving Audit logging
Cluster Secret Namespace Admin admin-cluster-creds kube-system Admin user-cluster-creds CLUSTER_NAME-gke-onprem-mgmt Admin kube-apiserver CLUSTER_NAME Usage Metering
Cluster Secret Namespace Admin user-cluster-creds CLUSTER_NAME-gke-onprem-mgmt User usage-metering-bigquery-service-account-key kube-system Stackdriver
Cluster Secret Namespace Admin admin-cluster-creds kube-system Admin user-cluster-creds CLUSTER_NAME-gke-onprem-mgmt User google-cloud-credentials kube-system User stackdriver-service-account-key knative-serving - If you aren't using a
private registry,
the
Create a backup of each secret using the following command:
kubectl get secret SECRET --namespace NAMESPACE \ --kubeconfig KUBECONFIG -o json > backup/SECRET-NAMESPACE.json
Replace the following:
NAMESPACE
: the namespace where the secret is located. For example,kube-system
.KUBECONFIG
: the path to the kubeconfig file for the admin or user cluster.SECRET
: the name of the secret. For example,admin-cluster-creds
.
For example, run the following commands for the audit logging service account:
kubectl get secret admin-cluster-creds --namespace kube-system \ --kubeconfig KUBECONFIG -o json > backup/admin-cluster-creds-kube-system.json kubectl get secret user-cluster-creds --namespace NAMESPACE \ --kubeconfig KUBECONFIG -o json > backup/user-cluster-creds-NAMESPACE.json kubectl get secret kube-apiserver --namespace NAMESPACE \ --kubeconfig KUBECONFIG -o json > backup/kube-apiserver-NAMESPACE.json
To create a new service account key file, run the following command:
gcloud iam service-accounts keys create NEW_KEY_FILE --iam-account IAM_ACCOUNT
Replace the following:
NEW_KEY_FILE
: the name for your new service account key fileIAM_ACCOUNT
: the email address of the service account
In the admin cluster configuration file, find the
componentAccessServiceAccountKeyPath
field, thegkeConnect
section, thestackdriver
section, and thecloudAuditLogging
section. In those places, replace the paths to the service account key files.In the user cluster configuration file, find the
componentAccessServiceAccountKeyPath
field, thegkeConnect
section, thestackdriver
section, thecloudAudigLogging
section, and theusageMetering
section. In those places, replace the paths to the service account key files.Save the changes you made by running the following commands. You can rotate the keys for one component at a time, or you can rotate all keys at once by setting the component to
sakeys
:gkectl update credentials COMPONENT \ --kubeconfig ADMIN_CLUSTER_KUBECONFIG \ --config ADMIN_CLUSTER_CONFIG \ --admin-cluster gkectl update credentials COMPONENT \ --kubeconfig ADMIN_CLUSTER_KUBECONFIG \ --config USER_CLUSTER_CONFIG
Replace the following;
COMPONENT
: one of:componentaccess
register
cloudauditlogging
usagemetering
stackdriver
sakeys
(Rotate keys for multiple components.)
ADMIN_CLUSTER_KUBECONFIG
: the path to the kubeconfig file for the admin cluster.ADMIN_CLUSTER_CONFIG
: the path to the admin cluster configuration file.USER_CLUSTER_CONFIG
: the path to the user cluster configuration file.
Node re-creation
Some service account key rotations may take longer time because node re-creation is required:
Service account | Nodes re-creation required |
---|---|
Component access | If using Container Registry: Yes If using a private registry: No |
Audit logging | Admin cluster: Yes but only control-plane nodes User cluster using kubeception: No User cluster using Controlplane V2: Yes but only control plane nodes |
Logging-monitoring | No |
Connect-register | No |
Usage metering | No |
For a key rotation that requires nodes to be re-created, the nodes are replaced in a rolling update process; that is, the nodes are re-created one by one.
The possible downtime during a key rotation is similar to the downtime for a cluster upgrade. For details, see Downtime during upgrades.
Restoring backups
If you need to restore the backups of the secrets you made earlier, run the following command:
kubectl apply -f backup/