This page describes two different options for rolling out a new Game Servers config with a canary phase. In the canary phase, you deploy the new config to small set of customers. This helps you to identify potential problems without impacting all your users.
The first way is to use a canary region. The setup for this option is simpler, but a bad rollout affects the entire region. The other option is to create a canary deployment. This option is more flexible, but it's also a more complex process for you to manage.
Before you begin
Before you start, we recommend you familiarize yourself with key concepts in the Game Servers Overview. Make sure you have also performed the following tasks:
- Ensure that you have enabled the Game Services API. Enable Game Services API
- Either choose a shell with gcloud CLI installed, or use an API client:
Go to Google Cloud console.
From the upper-right corner of the console, click the Activate Cloud Shell button:
- Create a service account.
- Download a private key as a JSON file.
-
Set the environment variable
GOOGLE_APPLICATION_CREDENTIALS
to the path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again.
Cloud Shell
To launch Cloud Shell, perform the following steps:
A Cloud Shell session opens inside a frame lower on the console.
You use this shell to run gcloud
commands.
Local shell
Install the gcloud CLI.
Verify that you have set desired default project for Google Cloud CLI (otherwise you need to specify flag --project
explicitly for each command later):
gcloud config list project
If not you can run the following command to set a default project, replacing PROJECT_ID
with your desired project ID :
gcloud config set project PROJECT_ID
Run the following command to verify your version of the Google Cloud CLI.
Game Servers requires version 306.0.0
or higher of the gcloud CLI.
gcloud version
To update your installation, run the following command:
gcloud components update
curl / PowerShell
To use the REST API with curl
or Windows PowerShell, do the
following:
Client Library
Google Cloud Game Servers can be controlled programmatically using a client library. See Client Libraries Overview for instructions on using the library and authenticating.
Verifying the health of a rollout
Make sure you check the health of your servers at various points during the rollout. You can use the data you gather to help you decide whether it is safe to continue with the rollout. If the health checks reveal a problem, it's best to pause or cancel the rollout.
You can use the following recommendations to verify the health of a rollout:
Make sure that the state of individual game server instances is
Ready
.Verify that Agones can allocate and manage the full lifecycle of a
GameServer
. Ensure that both the main and canary deploymentGameServers
have the same labels that match to the expectedGameServerAllocation
required
selector, so that your matchmaker doesn't require a change to allocate from the canary deployment. Optionally, you could preferentially allocate from the canary fleet using the "preferred" attribute in aGameServerAllocation
specification.Monitor and make sure that any relevant game metrics and system metrics in your deployment don't indicate any degradation in performance.
Rolling out to a canary region
Game Servers lets you override a config in one or more realms. Make sure that the clusters in your canary region are in a separate realm.
In this example, the "australia" realm is the canary realm.
To set v2
as the active config in the australia
realm:
Copy the following code into a file:
- realmsSelector: realms: - projects/games/locations/australia-southeast1/realms/australia configVersion: v2
To apply the changes, run the following command:
gcloud game servers deployments update-rollout stk --config-overrides-file configOverrideFile --no-dry-run
where configOverrideFile is the path to the file with override config.
When you have confirmed that the canary region is healthy, run the following command to deploy the new version everywhere:
gcloud game servers deployments update-rollout stk --default-config "v2" --no-dry-run
When the operation completes, the v2
config is rolled out everywhere.
Now clean up the override by running the following command:
gcloud game servers deployments update-rollout stk --no-dry-run --clear-config-overrides
Rolling out to a canary deployment
You can use a separate deployment to test the config. Initially, there isn't a dedicated canary deployment.
Start by creating the canary deployment:
gcloud game servers deployments create canary
Make sure that the selector used in the GameServerAllocation is able to select GameServers from both your main fleet and the canary fleet, so that no changes are required to your current allocation strategies.
Next, run the following command to create a
v1
config that is identical to the active config in the non-canary deployment:gcloud game servers configs create v1 --deployment canary --fleet-configs-file fleetSpecFile --scaling-configs-file scalingConfigFile
where fleetSpecFile is the path of the file that contains the fleet specification. scalingConfigFile is the path of the file that contains all of the scaling configs.
We recommended setting the scaling configs so that the canary fleets are much smaller than the regular fleets.
Run the following command to update the canary rollout so that the
v1
config is rolled out everywhere:gcloud game servers deployments update-rollout canary --default-config v1 --no-dry-run
Verify that the canary fleets are healthy.
Run the following command to create the
v2
config in the canary deployment:gcloud game servers configs create v2 --deployment canary --fleet-configs-file fleetSpecFile --scaling-configs-file scalingConfigFile
where fleetSpecFile is the path of the file that contains the fleet specification. scalingConfigFile is the path of the file that contains all of the scaling configs. We recommend you configure the canary fleets to a small size.
Run the following command to roll out the
v2
config to the canary deployment:gcloud game servers deployments update-rollout canary --default-config v2 --no-dry-run
Verify that the canary fleets are healthy.
Run the following command to roll out the
v2
config to the main deployment:gcloud game servers deployments update-rollout stk --default-config v2 --no-dry-run
Verify that the fleets are healthy. At this stage, the
v2
is rolled out everywhere.Clean up the canary deployment.
Clear the rollout for the canary deployment by running:
gcloud game servers deployments update-rollout canary --clear-default-config --no-dry-run
Run the following command to delete the 'v1' config:
gcloud game servers configs delete v1 --deployment=canary
Delete the 'v2' config by running:
gcloud game servers configs delete v2 --deployment=canary
Remove the canary deployment by running:
gcloud game servers deployments delete canary
For more fine-grained control over a deployment, you can roll out a config to selected realms using overrides, before setting the config as the default. Follow the same procedure to roll out the canary more gradually.
You can skip the first three steps and the last step by maintaining a standing canary deployment. For frequent and regular rollouts, we recommend this approach.