After you create or update a stateful configuration for a stateful managed instance group (MIG), you can:
- Apply the stateful configuration for it to take effect.
- View the stateful configuration as well as the effective preserved state of your managed instances.
- Remove the stateful configuration.
Before you begin
- Review when to use stateful MIGs and how stateful MIGs work.
-
If you haven't already, then set up authentication.
Authentication is
the process by which your identity is verified for access to Google Cloud services and APIs.
To run code or samples from a local development environment, you can authenticate to
Compute Engine by selecting one of the following options:
Select the tab for how you plan to use the samples on this page:
Console
When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.
gcloud
-
Install the Google Cloud CLI, then initialize it by running the following command:
gcloud init
- Set a default region and zone.
REST
To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.
Install the Google Cloud CLI, then initialize it by running the following command:
gcloud init
For more information, see Authenticate for using REST in the Google Cloud authentication documentation.
-
Applying stateful configuration to managed instances
Your stateful configuration is effective after you or the MIG applies it.
- Configuration changes in a stateful policy are automatically applied to all managed instances.
- Configuration changes in a per-instance configuration can be applied manually or automatically.
For more conceptual information, read How stateful configuration is applied to managed instances.
Applying configuration from a stateful policy
All configuration changes in a stateful policy are automatically applied to all managed instances. Updates to a stateful policy do not disrupt running virtual machine (VM) instances.
Verifying whether a stateful policy has been applied
Verifying all VMs
Verify whether changes to a stateful policy have been applied to all VMs by checking if the MIG has become stable after the changes.
Verifying specific VMs
Verify if the changes to a stateful policy have been applied to a specific
VM in a MIG by
viewing the preserved state
of the managed instance and checking if all disks
from the stateful policy are present in either preservedStateFromPolicy
or
preservedStateFromConfig
for the managed instance.
Applying stateful configuration from per-instance configurations
You can apply new or updated per-instance configurations either manually or automatically. Use one of the following approaches:
- Selectively apply updated configurations to specific instances: Use this approach to control the disruption, the timing, and the sequence of the updates.
- Automatically apply updated configurations with rolling updates: Use this approach to apply configuration changes to managed instances in an automated, rolling fashion.
Both of these methods apply outstanding updates from both per-instance configurations and unapplied versions or instance template, if relevant.
Selectively apply updated configuration to specific instances
To manually apply per-instance configurations to specific instances, use the following steps:
- Configure an opportunistic update policy to prevent a race with automatic proactive updates.
- Create or update per-instance configurations.
- Apply updated configurations to specific VMs by selectively updating those VMs.
Alternatively, you can use the following approaches:
- When creating or updating a per-instance configuration with the
gcloud CLI, use the
--update-instance
flag to immediately apply the configuration to the associated managed instance. - When you manually create new instances, specify their names and per-instance configurations at creation time. The MIG applies configurations immediately on VM creation.
Automatically apply updated configuration with rolling updates
Configure proactive rolling updates for your MIG to automatically apply configuration updates to corresponding managed instances in a rolling fashion.
Stateful MIGs require the following configuration for a proactive update policy:
- You must set the
replacement method
to
RECREATE
. - You must set
max surge
to
0
. - For regional MIGs, you must disable proactive instance redistribution across zones.
This configuration ensures that the MIG recreates existing stateful instances and does not substitute them with different ones.
You can also ensure that changes in per-instance configurations are applied
without stopping instances, if possible, by setting the
minimal action
to REFRESH
in the group's update policy.
Verifying whether per-instance configurations have been applied
Verifying all per-instance configurations
To verify whether all per-instance configurations in a MIG have been applied,
view the MIG's stateful status
and check status.stateful.perInstanceConfigs.allEffective
:
true
: All per-instance configurations in the group have been applied and are effective, or the group has no per-instance configurations.false
: The group has at least one per-instance configuration that is not yet effective: either you have not yet applied it or it is in the process of being applied.
Verifying a specific per-instance configuration
To verify if the changes to a specific per-instance configuration have been applied to
its corresponding VM,
view all per-instance configurations
and check the specific per-instance configuration's status
field:
UNAPPLIED
: The per-instance configuration was created or updated and you need to apply it to the VM with a manual instance update.APPLYING
: The MIG is currently applying the new or updated per-instance configuration to the VM.EFFECTIVE
: The per-instance configuration has been applied successfully to the VM and is effective.UNAPPLIED_DELETION
: The per-instance configuration is set to be deleted. You must apply this update to the VM with a manual instance update for it to take effect.DELETING
: The per-instance configuration is being deleted and the change is being applied to the VM.
Viewing stateful configuration and preserved state
Get information about your stateful MIGs for the following tasks:
- Verifying whether a MIG has stateful configuration and whether this configuration is applied and effective.
- Viewing stateful configuration that is common to all instances in the MIG, stored in the stateful policy.
- Viewing stateful configuration that is specific to individual instances, stored in per-instance configurations, and checking whether this configuration is applied and effective.
- Viewing the effective preserved state for each instance in a MIG based on its stateful policy and applied per-instance configurations.
Viewing the status of a MIG's stateful configuration
A MIG is considered stateful if it has any stateful configuration, that is, either a stateful policy or at least one non-empty per-instance configuration.
If you create a stateful policy, the MIG automatically applies it to make it effective. If you create per-instance configurations, you might decide to apply them later.
To verify whether a MIG has stateful configuration and that all per-instance configurations are applied, view the MIG's stateful status by using the gcloud CLI or REST.
gcloud
To verify whether a MIG has stateful configuration, view its stateful status
by running the
gcloud compute instance-groups managed describe
command. For example:
gcloud compute instance-groups managed describe NAME
baseInstanceName node
...
name example-group
...
status:
isStable: true
stateful:
hasStatefulConfig: true
perInstanceConfigs:
allEffective: true
versionTarget:
isReached: true
targetSize: 3
...
In this output:
hasStatefulConfig
:true
: The group has stateful configuration, that is, the group has a non-empty stateful policy or at least one non-empty per-instance configuration.false
: No stateful configuration exists.
perInstanceConfigs.allEffective
:true
: All per-instance configurations in the group have been applied and are effective, or the group has no per-instance configurations.false
: The group has at least one per-instance configuration that is not yet effective: either you have not yet applied it or it is in the process of being applied.
REST
To verify whether a MIG has stateful configuration, use the
instanceGroupManagers.get
or regionInstanceGroupManagers.get
method and check the status.stateful
field. For example:
GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/NAME
The method returns the instanceGroupManagers
object:
{ "name": "example-group", "baseInstanceName": "node", ... "status": { "isStable": true, "stateful": { "hasStatefulConfig": true, "perInstanceConfigs": { "allEffective": true } }, "versionTarget": { "isReached": true } }, "targetSize": 3, ... }
In this output:
hasStatefulConfig
:true
: The group has stateful configuration, that is, the group has a non-empty stateful policy or at least one non-empty per-instance configuration.false
: No stateful configuration exists.
perInstanceConfigs.allEffective
:true
: All per-instance configurations in the group have been applied and are effective, or the group has no per-instance configurations.false
: The group has at least one per-instance configuration that is not yet effective: either you have not yet applied it or it is in the process of being applied.
Viewing a MIG's stateful policy
View a MIG's stateful policy by checking the group's details with the Google Cloud console, gcloud CLI or REST.
Console
In the Google Cloud console, go to the Instance groups page.
Click the name of the instance group of which you want to view the stateful policy.
Go to the Details tab.
Under Stateful resources, all the resources in the stateful policy are listed.
gcloud
To view a MIG's stateful policy, run the
gcloud compute instance-groups managed describe
command. For example:
gcloud compute instance-groups managed describe NAME
baseInstanceName node
...
name example-group
...
statefulPolicy:
preservedState:
disks:
data-disk:
autoDelete: NEVER
logs-disk:
autoDelete: ON_PERMANENT_INSTANCE_DELETION
internalIPs:
nic0:
autoDelete: ON_PERMANENT_INSTANCE_DELETION
externalIPs:
nic0:
autoDelete: NEVER
status:
isStable: true
stateful:
hasStatefulConfig: true
perInstanceConfigs:
allEffective: true
versionTarget:
isReached: true
targetSize: 3
...
REST
To view a MIG's
stateful policy
use the
instanceGroupManagers.get
or regionInstanceGroupManagers.get
method and check the statefulPolicy
field. For example:
GET https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/NAME
The method returns the instanceGroupManagers
object:
{ "name": "example-group", "baseInstanceName": "node", ... "status": { "isStable": true, "versionTarget": { "isReached": true }, "stateful": { "hasStatefulConfig": true, "perInstanceConfigs": { "allEffective": true } } ... "statefulPolicy": { "preservedState": { "disks": { "data-disk": { "autoDelete": "NEVER" }, "logs-disk": { "autoDelete": "ON_PERMANENT_INSTANCE_DELETION" } }, "internalIPs": { "nic0" : { "autoDelete": "ON_PERMANENT_INSTANCE_DELETION"} }, "externalIPs": { "nic0" : { "autoDelete": "NEVER"} }, } }, "targetSize": 3, ... }
Viewing a MIG's per-instance configurations
View per-instance configurations by listing all per-instance configurations in a MIG. Use the gcloud CLI or REST.
If you decide to apply per-instance configurations manually, some per-instance configurations might not yet be applied to the associated instances and, therefore, the instances' preserved states might not yet reflect your per-instance configurations. For more information, see Applying per-instance configurations and Verifying whether per-instance configurations have been applied
gcloud
List all per-instance configurations in a MIG by running the
gcloud compute instance-groups managed instance-configs list
command.
gcloud compute instance-groups managed instance-configs list NAME
Replace the following:
NAME
: the name of a MIG to list per-instance configurations from
Filter the list using the standard --filter
flag.
For example, to list all per-instance configurations in example-group
:
gcloud compute instance-groups managed instance-configs list example-group
---
fingerprint: JxPvsKOywuY=
name: node-1
preservedState:
disks:
data-disk:
autoDelete: NEVER
mode: rw
source: https://www.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/disks/data-disk-1
metadata:
role: primary
internalIPs:
nic0:
ipAddress:
address: /projects/example-project/regions/us-east1/addresses/node-1-int-ip
autoDelete: ON_PERMANENT_INSTANCE_DELETION
externalIPs:
nic0:
ipAddress:
literal: 130.211.181.55
autoDelete: NEVER
status: EFFECTIVE
---
fingerprint: IbGmJBqqEkk=
name: node-2
preservedState:
...
REST
List all per-instance configurations in a MIG by calling the
instanceGroupManagers.listPerInstanceConfigs
or regionInstanceGroupManagers.listPerInstanceConfigs
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/NAME/listPerInstanceConfigs
Replace the following:
PROJECT_ID
: the project ID for the requestZONE
: the zone where the MIG is located (applies to a zonal MIG)- For a regional MIG, replace
zones/ZONE
withregions/REGION
and specify the region of the MIG
- For a regional MIG, replace
NAME
: the name of a MIG to list per-instance configurations from
For example, to list all per-instance configurations in example-group
:
POST https://compute.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/instanceGroupManagers/example-group/listPerInstanceConfigs
The method returns the list of per-instance configurations in the group:
{ "items": [ { "fingerprint": "JxPvsKOywuY=", "name": "node-1", "preservedState" : { "disks": { "data-disk" : { "source": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/disks/data-disk-1", "mode": "READ_WRITE", "autoDelete": "NEVER" } }, "metadata": { "role": "primary" }, "internalIPs": { "nic0" : { "ipAddress" : { "address": "/projects/example-project/regions/us-east1/addresses/node-1-int-ip" }, "autoDelete": "ON_PERMANENT_INSTANCE_DELETION" } }, "externalIPs": { "nic0": { "ipAddress" : { "literal": "54.2.3.4" }, "autoDelete": "NEVER" } } }, "status": "EFFECTIVE" }, { "fingerprint": "IbGmJBqqEkk=", "name": "node-2", "preservedState" : { ... } ... }, ... ] }
Viewing the preserved states of managed instances
When stateful configuration is applied, the MIG generates and sets the effective preserved state of each managed instance in two fields:
preservedStateFromPolicy
: Contains the preserved state that is generated based on a stateful policy, excluding any stateful configuration that is overridden by per-instance configurations.preservedStateFromConfig
: Contains the preserved state that is generated based on a per-instance configuration that has already been applied to the managed instance.
To see the effective preserved state of each managed instance in a MIG, list the managed instances using the gcloud CLI or REST.
gcloud
Check which managed instances have preserved state by using the
gcloud compute instance-groups managed list-instances
command
and viewing values in the PRESERVED_STATE
column. For example:
gcloud compute instance-groups managed list-instances NAME
NAME ZONE STATUS HEALTH_STATE ACTION PRESERVED_STATE INSTANCE_TEMPLATE ...
node-1 us-east1-c RUNNING NONE POLICY,CONFIG example-template
node-2 us-east1-c RUNNING NONE POLICY,CONFIG example-template
The PRESERVED_STATE
column can contain the following values:
POLICY
: The managed instance has preserved state based on a stateful policy.CONFIG
: The managed instance has preserved state based on a per-instance configuration.- No value: The manage instance has no preserved state and is stateless.
View the preserved state of a specific managed instance by running the
gcloud compute instance-groups managed describe-instance
command.
gcloud compute instance-groups managed describe-instance INSTANCE_GROUP_NAME \ --instance INSTANCE_NAME
Replace the following:
INSTANCE_GROUP_NAME
: the name of a MIGINSTANCE_NAME
: the name of a managed instance in the group
For example, to view the preserved state of the node-1
managed instance in
example-group
, run:
gcloud compute instance-groups managed describe-instance example-group \
--instance node-1
instance: .../example-project/zones/us-east1-c/instances/node-1
instanceStatus: RUNNING
currentAction: NONE
id: 123456789012345678
version:
instanceTemplate: .../example-project/global/instanceTemplates/example-template
preservedStateFromPolicy:
disks:
data-disk:
autoDelete: NEVER
mode: rw
source: .../example-project/zones/us-east1-c/disks/data-disk-1
internalIPs:
nic0:
ipAddress:
literal: 10.0.0.1
address: .../example-project/regions/us-east1/addresses/node-1-int-ip
autoDelete: ON_PERMANENT_INSTANCE_DELETION
preservedStateFromConfig:
metadata:
role: primary
my-key: my-value
externalIPs:
nic0:
ipAddress:
literal: 130.211.181.55
address: .../example-project/regions/us-east1/addresses/node-1-nat-ip
autoDelete: NEVER
REST
List managed instances to view each instance's preserved state by
calling the
instanceGroupManagers.listManagedInstances
or regionInstanceGroupManagers.listManagedInstances
method.
For example, to list all managed instances:
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/NAME/listManagedInstances
The method returns the list of managed instances in the group, which contains information about their preserved states:
{ "managedInstances": [ { "instance": ".../example-project/zones/us-east1-c/instances/node-1", "instanceStatus": "RUNNING", "currentAction": "NONE", "id": "123456789012345678", "version": { "instanceTemplate":".../example-project/global/instanceTemplates/example-template" }, "preservedStateFromPolicy": { "disks": { "data-disk" : { "source": "https://www.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/disks/data-disk-1", "mode": "rw", "autoDelete": "NEVER" }, }, "internalIPs": { "nic0" : { "ipAddress": { "literal": "10.0.0.1", "address": "/projects/example-project/regions/us-east1/addresses/node-1-int-ip" }, "autoDelete": "ON_PERMANENT_INSTANCE_DELETION" } }, }, "preservedStateFromConfig": { "metadata": { "role": "primary", "my-key": "my-value" }, "externalIPs": { "nic0": { "ipAddress": { "literal": "130.211.181.55", "address": "/projects/example-project/regions/us-east1/addresses/node-1-nat-ip" }, "autoDelete": "NEVER" } }, } }, { "instance": ".../example-project/zones/us-east1-c/instances/node-2", "instanceStatus": "RUNNING", ... "preservedStateFromPolicy": { ... }, "preservedStateFromConfig": { ... } }, ... ] }
Viewing associated static IP address resources
To retrieve static IP addresses
(Address
resources)
that are assigned to managed instances in a MIG,
view the preserved states
of these instances. You can see static addresses in the ipAddress.address
fields.
If a MIG or the instances no longer exist, but the IP address reservations still exist, you can list the static internal or external address reservations.
Removing stateful configuration
Removing stateful configuration is useful in the following scenarios:
- You have migrated standalone VMs to a stateful MIG and now want to move the stateful configuration from per-instance configurations to a common stateful policy.
- You have rearchitected your workload and no longer need to keep any state on the VM instances.
- You created a stateful configuration for testing purposes and now want to clean it up.
This section describes how to fully remove a stateful policy on a per-instance configuration. If you only need to remove a subset of a stateful policy or per-instance configuration, see the following sections:
- Declaring previously stateful persistent disks as stateless for all VMs in a MIG.
- Declaring previously stateful network interfaces as stateless for all VMS in a MIG.
- Detaching a stateful disk or declaring it stateless for a specific VM in a MIG.
- Removing stateful metadata from a specific VM in a MIG
- Disassociating a static IP address from a specific VM in a MIG.
Removing a stateful policy
When you remove a stateful policy, you affect all VMs in a MIG. The MIG treats all stateful resources that were configured in the stateful policy as stateless, unless those resources are also configured in per-instance configurations for individual instances. On subsequent instance recreation, autohealing, or update operations, the now-stateless resources can lose their state:
- Disks can be deleted and recreated from the source, which is defined in the instance template.
- IP addresses might change.
For conceptual information, see how removing a resource from stateful policy affects preserved state.
gcloud
To delete all configuration from a stateful policy, run the
gcloud compute instance-groups managed update
command
with the applicable flags. The applicable flags are as follows:
--remove-stateful-disks
, and list the device names of all configured stateful disks.--remove-stateful-internal-ips
, and list the network interface names of all configured stateful internal IP addresses.--remove-stateful-external-ips
, and list the network interface names of all configured stateful external IP addresses.
For example, if your stateful policy contains configuration for two stateful
disks with device names data-disk
and logs-disk
, and contains
configuration for stateful internal and external IP addresses within the
nic0
network interface, run the following command to clear the policy:
gcloud compute instance-groups managed update example-group \ --zone us-east1-c \ --remove-stateful-disks data-disk,logs-disk \ --remove-stateful-internal-ips nic0 \ --remove-stateful-external-ips nic0
The MIG removes from its stateful policy the data-disk
and logs-disk
stateful configuration, as well as the stateful internal and external IP
configuration for the nic0
network interface. As a result, the MIG removes
those disks and static IP addresses from the preserved states of all managed
instances in the group automatically and asynchronously, unless the disks or
IP addresses are also configured in per-instance configurations.
REST
To delete all configuration in stateful policy, set the statefulPolicy
field to null
by using the
instanceGroupManagers.patch
or regionInstanceGroupManagers.patch
.
method.
For example, the following call removes stateful policy configuration:
PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/NAME { "statefulPolicy": null }
The MIG clears stateful policy configuration and removes the stateful resources from the preserved states of all managed instances in the group automatically and asynchronously, unless the resources are also configured in per-instance configurations.
Removing stateful configuration for a specific VM
Removing stateful configuration for a specific VM is done in two steps:
- Delete the associated per-instance configuration.
- Apply the change to the managed instance. You can choose whether the change should be applied manually or automatically.
After the change is applied, all stateful items that were previously configured in the per-instance configuration are now treated as stateless, unless they are also configured in the group's stateful policy. When the VM is subsequently recreated or updated, the state of the items is lost:
- Disks can be detached or recreated from their source in the instance template.
- Metadata is removed or reset to the values defined in the instance template.
- IP addresses might change.
For more information, see how removing items from per-instance configurations affects preserved state.
gcloud
To fully delete the per-instance configuration, run the
gcloud compute instance-groups managed instance-configs delete
command.
Use the optional --update-instance
flag to apply the changes immediately
to the instance (the default). If you set the --no-update-instance
flag,
the changes are
applied
when you next recreate or update the instance.
The --instance-update-minimal-action
flag can only be used together with
the --update-instance
flag.
gcloud compute instance-groups managed instance-configs delete INSTANCE_GROUP_NAME \ --instances INSTANCE_NAME[,INSTANCE_NAME,...] \ [--no-update-instance | --update-instance] \ [--instance-update-minimal-action MINIMAL_ACTION]
Replace the following:
INSTANCE_GROUP_NAME
: The name of the MIG.INSTANCE_NAME
: Names of the instances for which to delete per-instance configurationsMINIMAL_ACTION
: The minimal action to perform when applying the per-instance configuration update to the VM. This value must be one of the following:none
: No action.refresh
: Apply updates that are possible to apply without stopping the VM.restart
: Stop the VM and then start it again.replace
: Recreate the VM.
For example, to delete the per-instance configuration for node-1
in
example-group
, run the following command:
gcloud compute instance-groups managed instance-configs delete example-group \ --instances node-1 \ --update-instance
The MIG deletes the per-instance configuration for node-1
and, because the
VM was updated and configuration changes applied, the MIG no longer treats
those previously stateful items as stateful.
REST
To fully delete the per-instance configuration for one or multiple
instances, use the
instanceGroupManagers.deletePerInstanceConfigs
or regionInstanceGroupManagers.deletePerInstanceConfigs
method.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/INSTANCE_GROUP_NAME/deletePerInstanceConfigs { "names": ["INSTANCE_NAME",...] }
Replace the following:
PROJECT_ID
: the project ID for the requestZONE
: the zone where the MIG is located (applies to a zonal MIG)- For a regional MIG, replace
zones/ZONE
withregions/REGION
and specify the region of the MIG
- For a regional MIG, replace
INSTANCE_GROUP_NAME
: the name of the MIGINSTANCE_NAME
: names of the instances for which to delete per-instance configurations
The deletePerInstanceConfig
method deletes the specified per-instance
configurations but does not apply the changes to the associated VMs. The
changes are applied to a VM when you recreate or update the instance.
Update selected VMs manually
to apply the changes.
For example, to delete per-instance configuration for node-1
in example-group
,
call the following method:
POST https://compute.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/instanceGroupManagers/example-group/deletePerInstanceConfigs { "names": ["node-1"] }
The method deletes per-instance configuration for the node-1
instance from
example-group
. The change will be applied to the managed instance when you
recreate or update the instance.
To apply the configuration change, call the
instanceGroupManagers.applyUpdatesToInstances
method:
POST https://compute.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/instanceGroupManagers/example-group/applyUpdatesToInstances { "instances": ["/zones/us-east1-c/instances/node-1"] }
The method removes stateful items from the managed instance's preserved state.
Feedback
We want to learn about your use cases, challenges, and feedback about stateful MIGs. Please share your feedback with our team at mig-discuss@google.com.
What's next
- Configure a stateful MIG.
- Get info about a specific MIG and its managed instances, including VM status and properties.
- Learn more about about stateful MIGs.
- Learn more about about MIGs and working with managed instances.