Container Engine's IP Rotation feature lets you change the IP address that your cluster's Kubernetes master uses to serve requests from the Kubernetes API. It can be used to obfuscate the location of your running Kubernetes master. IP rotation also changes the SSL certificate and cluster certificate authority along with the master IP address, so there is no externally-visible connection between the previous address and the new address.
Performing an IP Rotation
An IP Rotation is a multi-step process. When you initiate a rotation, your
cluster master begins serving on a new IP address in addition to the original IP
address. Next, you must update your cluster's API clients (such as development
machines using the
kubectl command-line interface) to start communicating with
the master over the new IP address. Lastly, when you complete the rotation, the
master ceases serving traffic over the previous IP address.
You initiate an IP Rotation by using the
gcloud container clusters update
command and passing the
--start-ip-rotation option, as follows:
gcloud container clusters update CLUSTER --start-ip-rotation
The above command configures the cluster master to serve on two IP addresses, the original address, and a new address. This will cause a small amount of downtime of the cluster API.
Once the master has been reconfigured, Container Engine automatically updates your cluster's nodes in the background to point to the new IP address. Each node-pool is marked as "requires recreation," and Container Engine will not allow the IP Rotation to be completed until the automatic re-creation finishes.
If you would like to follow along as your node-pools are updated, you can find the running auto-upgrade operation by running:
gcloud container operations list | grep "AUTO_UPGRADE_NODES.*RUNNING"
Then, you can wait on the operation by running:
gcloud container operations wait OPERATION_ID
Node-pools will be re-created one-by-one, and each will have its own operation. If you have multiple node-pools, you can use the above instructions to wait on each auto-upgrade operation.
Once IP Rotation has been initiated, you must update all API clients outside of
the cluster (such as kubectl on developer machines) to point to the new address.
You can update your API clients by using the
gcloud container clusters
get-credentials command on the client machine as follows:
gcloud container clusters get-credentials CLUSTER
If you are using the Kubernetes Certificates API, you also must issue new certificates.
Lastly, you complete the IP rotation by using the
gcloud container clusters
update command and passing the
--complete-ip-rotation option, as follows:
gcloud container clusters update CLUSTER --complete-ip-rotation
The above command configures the cluster master to serve only on its new IP address. This will cause a small amount of downtime of the cluster API.