Create VMs in bulk

Linux Windows

When you want to create a large number of virtual machine (VM) instances that are identical and independent from each other, create VMs in bulk using the Google Cloud CLI or the Compute Engine API. You can create these VMs across all zones in a region or distributed across certain zones.

For more details and associated limitations, see About bulk creation of VMs.

Before you begin

If you want to use the command-line examples in this guide, do the following:
1. Install or update to the latest version of the Google Cloud CLI.
2. Set a default region and zone.
If you want to use the API examples in this guide, set up API access.
For VMs and any related resources that you plan to create, make sure you have enough quota and the required permissions.

Create VMs in bulk in a region

To create VMs in bulk in a region, use the gcloud CLI or the Compute Engine API.

If you specify a machine type or support for additional hardware such as a GPU or a local SSD, Compute Engine places the VMs in a zone within the region that supports the machine type and the additional hardware.

Permissions required for this task

To perform this task, you must have the following permissions:

compute.instances.create on the project
compute.instances.updateShieldedVmConfig if you plan to create a Shielded VM instance and you want to be able to change any of the Shielded VM settings
compute.networks.use on the project if using a legacy network
compute.subnetworks.use either on the whole project or on the chosen subnet (VPC networks)
compute.networks.useExternalIp on the project if you need to assign an external IP address (either ephemeral or static) to the instance using a legacy network
compute.subnetworks.useExternalIp either on the whole project or on the chosen subnet if you need to assign an external IP address (either ephemeral or static) to the instance using a VPC network
compute.addresses.use on the project if specifying a static address in the project
compute.instances.setMetadata if setting metadata
compute.instances.setTags on the instance if setting tags
compute.instances.setLabels on the instance if setting labels
compute.instances.setServiceAccount on the instance if setting service account
compute.images.useReadOnly on the image if creating a new root persistent disk
compute.disks.create on the project if creating a new root persistent disk with this instance
compute.disks.useReadOnly on the disk if attaching an existing persistent disk in read-only mode
compute.disks.use on the disk if attaching an existing disk in read/write mode
compute.disks.setLabels on the disk if setting labels
compute.snapshots.create on the project to create a new snapshot if creating an instance from a snapshot
compute.snapshots.useReadOnly on the snapshot if creating an instance from a snapshot
compute.instanceTemplates.useReadOnly on the instance template if creating instance from instance template

gcloud

To create VMs in bulk in a region, use the following gcloud compute instances bulk create command:

gcloud compute instances bulk create \
    ( --name-pattern="NAME_PATTERN" | --predefined-names=[PREDEFINED_NAMES] ) \
    --region=REGION \
    --count=COUNT \
    [ --min-count=MIN_COUNT \ ]
    [--location-policy=LOCATION_POLICY \ ]
    [--target-shape=TARGET_SHAPE ]

Replace the following:

NAME_PATTERN: the name pattern for the VMs. Use a sequence of hash (#) characters for Compute Engine to replace with a sequence of numbers. For example, using vm-# for the name pattern generates VMs with names vm-1, vm-2, and so on, up to the number of VMs specified by --count, which must be less than or equal to the number of VMs that the name pattern allows.

When using a name pattern, Compute Engine tries to avoid name conflicts by checking the names of existing VMs created from previous requests.
PREDEFINED_NAMES: a list of predefined names for the VMs to create. If using this flag and specifying COUNT, COUNT must equal the number of names provided.
REGION: the region to create the VMs in.
COUNT: the number of VMs to create. This must be less than or equal to the number of VMs allowed by NAME_PATTERN. Or, if using --predefined-names, you do not have to specify COUNT, but if you do, it must be equal to the number of names provided.

MIN_COUNT: the minimum number of VMs to create. The following table describes the behavior of the request depending on how you set this flag:

Value	Description
Not set	Default value is `COUNT`. If Compute Engine can't create the number of VMs specified by `COUNT`, the request fails and no VMs are created.
`1`	Compute Engine creates as many VMs as possible, up to `COUNT`.
Greater than `1` and less than `COUNT`	Compute Engine creates at least `MIN_COUNT` VMs up to a maximum of `COUNT` VMs. If `MIN_COUNT` VMs can't be created, the request fails and no VMs are created.

LOCATION_POLICY: the zones to include or exclude within a region. Use a list of key-value pairs, with the zone as the key and the policy as the value. Valid values for the policy are allow, which is the default, and deny. The following is an example value for this flag:
```
--location-policy=us-east1-b=allow,us-east1-c=deny
```

TARGET_SHAPE: the distribution of the VMs across the specified zones. Use the --location-policy flag to specify the zones. The following table shows the valid values for this flag:

Value	Description
`any_single_zone`	Enforces VM placement in a single zone, and prioritizes utilization of unused reservations. Use this to avoid cross-zone network egress or to reduce network latency. This is the default value.
`balanced`	Attempts to distribute VMs evenly across all zones in the region.
`any`	Allows distribution of VMs across multiple zones in a region. Chooses zones that have available resources and that maximize unused zonal reservations.

API

To create VMs in bulk in a region, use the following instances.bulkInsert method:

POST https://www.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/instances/bulkInsert

{
  ...
  "namePattern": "NAME_PATTERN",
  "perInstanceProperties": {
    "PREDEFINED_NAME_1": {},
    "PREDEFINED_NAME_2": {},
    ...
  },
  "count": COUNT,
  "minCount": MIN_COUNT,
  "locationPolicy": {
    "LOCATION_POLICY"
    },
    "targetShape": "TARGET_SHAPE"
  },
  ...
}

Replace the following:

PROJECT_ID: the project ID.
REGION: the region to create the VMs in.
NAME_PATTERN: the name pattern for the VMs. Specify either this or perInstanceProperties. Use a sequence of hash (#) characters for Compute Engine to replace with a sequence of numbers. For example, using vm-# for the name pattern generates VMs with names vm-1, vm-2, and so on, up to the number of VMs specified by --count, which must be less than or equal to the number of VMs that the name pattern allows.

When using a name pattern, Compute Engine tries to avoid name conflicts by checking the names of existing VMs created from previous requests.
PREDEFINED_NAME_1, PREDEFINED_NAME_2, ...: a list of predefined names for the VMs to create. Specify either this or namePattern. If using this flag and specifying COUNT, COUNT must equal the number of names provided.
COUNT: the number of VMs to create. This must be less than or equal to the number of VMs allowed by NAME_PATTERN. Or, if using perInstanceProperties, you do not have to specify COUNT, but if you do, it must be equal to the number of names provided.

MIN_COUNT: the minimum number of VMs to create. The following table describes the behavior of the request depending on how you set this flag:

Value	Description
Not set	Default value is `COUNT`. If Compute Engine can't create the number of VMs specified by `COUNT`, the request fails and no VMs are created.
`1`	Compute Engine creates as many VMs as possible, up to `COUNT`.
Greater than `1` and less than `COUNT`	Compute Engine creates at least `MIN_COUNT` VMs up to a maximum of `COUNT` VMs. If `MIN_COUNT` VMs can't be created, the request fails and no VMs are created.

LOCATION_POLICY: the zones to include or exclude within a region. Use a list of key-value pairs, with the zone as the key and the policy as the value. Valid values for the policy are ALLOW, which is the default, and DENY. The following is an example value for this field:
```
"locations": {
  "zones/us-central1-a": { "preference": "ALLOW" },
  "zones/us-central1-c": { "preference": "DENY"  },
  ...
},
```

TARGET_SHAPE: the distribution of VMs across the specified zones. Use the locationPolicy field to specify the zones. The following table shows the valid values for this field:

Value	Description
`ANY_SINGLE_ZONE`	Enforces VM placement in a single zone, and prioritizes utilization of unused reservations. Use this to avoid cross-zone network egress or to reduce network latency. This is the default value.
`BALANCED`	Attempts to distribute VMs evenly across all zones in the region.
`ANY`	Allows distribution of VMs across multiple zones in a region. Chooses zones that have available resources and that maximize unused zonal reservations.

Create VMs in bulk in a zone

To create VMs in bulk in a zone, use the gcloud CLI or the Compute Engine API.

Permissions required for this task

To perform this task, you must have the following permissions:

compute.instances.create on the project
compute.instances.updateShieldedVmConfig if you plan to create a Shielded VM instance and you want to be able to change any of the Shielded VM settings
compute.networks.use on the project if using a legacy network
compute.subnetworks.use either on the whole project or on the chosen subnet (VPC networks)
compute.networks.useExternalIp on the project if you need to assign an external IP address (either ephemeral or static) to the instance using a legacy network
compute.subnetworks.useExternalIp either on the whole project or on the chosen subnet if you need to assign an external IP address (either ephemeral or static) to the instance using a VPC network
compute.addresses.use on the project if specifying a static address in the project
compute.instances.setMetadata if setting metadata
compute.instances.setTags on the instance if setting tags
compute.instances.setLabels on the instance if setting labels
compute.instances.setServiceAccount on the instance if setting service account
compute.images.useReadOnly on the image if creating a new root persistent disk
compute.disks.create on the project if creating a new root persistent disk with this instance
compute.disks.useReadOnly on the disk if attaching an existing persistent disk in read-only mode
compute.disks.use on the disk if attaching an existing disk in read/write mode
compute.disks.setLabels on the disk if setting labels
compute.snapshots.create on the project to create a new snapshot if creating an instance from a snapshot
compute.snapshots.useReadOnly on the snapshot if creating an instance from a snapshot
compute.instanceTemplates.useReadOnly on the instance template if creating instance from instance template

gcloud

To create VMs in bulk in a specific zone, use the following gcloud compute instances bulk create command.

gcloud compute instances bulk create \
    ( --name-pattern="NAME_PATTERN" | --predefined-names=[PREDEFINED_NAMES] ) \
    --zone=ZONE \
    --count=COUNT \
    [ --min-count=MIN_COUNT ]

Replace the following:

NAME_PATTERN: the name pattern for the VMs. Use a sequence of hash (#) characters for Compute Engine to replace with a sequence of numbers. For example, using vm-# for the name pattern generates VMs with names vm-1, vm-2, and so on, up to the number of VMs specified by --count, which must be less than or equal to the number of VMs that the name pattern allows.

When using a name pattern, Compute Engine tries to avoid name conflicts by checking the names of existing VMs created from previous requests.
PREDEFINED_NAMES: a list of predefined names for the VMs to create. If using this flag and specifying COUNT, COUNT must equal the number of names provided.
ZONE: the zone to create the VMs in.
COUNT: the number of VMs to create. This must be less than or equal to the number of VMs allowed by NAME_PATTERN. Or, if using --predefined-names, you do not have to specify COUNT, but if you do, it must be equal to the number of names provided.

MIN_COUNT: the minimum number of VMs to create. The following table describes the behavior of the request depending on how you set this flag:

Value	Description
Not set	Default value is `COUNT`. If Compute Engine can't create the number of VMs specified by `COUNT`, the request fails and no VMs are created.
`1`	Compute Engine creates as many VMs as possible, up to `COUNT`.
Greater than `1` and less than `COUNT`	Compute Engine creates at least `MIN_COUNT` VMs up to a maximum of `COUNT` VMs. If `MIN_COUNT` VMs can't be created, the request fails and no VMs are created.

API

To create VMs in bulk in a zone, use the following instances.bulkInsert method.

POST https://www.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/bulkInsert

{
  ...
  "namePattern": "NAME_PATTERN",
  "perInstanceProperties": {
    "PREDEFINED_NAME_1": {},
    "PREDEFINED_NAME_2": {},
    ...
  },
  "count": COUNT,
  "minCount": MIN_COUNT,
  ...
}

Replace the following:

PROJECT_ID: the project ID.
ZONE: the zone to create the VMs in.
NAME_PATTERN: the name pattern for the VMs. Specify either this or perInstanceProperties. Use a sequence of hash (#) characters for Compute Engine to replace with a sequence of numbers. For example, using vm-# for the name pattern generates VMs with names vm-1, vm-2, and so on, up to the number of VMs specified by --count, which must be less than or equal to the number of VMs that the name pattern allows.

When using a name pattern, Compute Engine tries to avoid name conflicts by checking the names of existing VMs created from previous requests.
PREDEFINED_NAME_1, PREDEFINED_NAME_2, ...: a list of predefined names for the VMs to create. Specify either this or namePattern. If using this flag and specifying COUNT, COUNT must equal the number of names provided.
COUNT: the number of VMs to create. This must be less than or equal to the number of VMs allowed by NAME_PATTERN. Or, if using perInstanceProperties, you do not have to specify COUNT, but if you do, it must be equal to the number of names provided.

MIN_COUNT: the minimum number of VMs to create. The following table describes the behavior of the request depending on how you set this flag:

Value	Description
Not set	Default value is `COUNT`. If Compute Engine can't create the number of VMs specified by `COUNT`, the request fails and no VMs are created.
`1`	Compute Engine creates as many VMs as possible, up to `COUNT`.
Greater than `1` and less than `COUNT`	Compute Engine creates at least `MIN_COUNT` VMs up to a maximum of `COUNT` VMs. If `MIN_COUNT` VMs can't be created, the request fails and no VMs are created.

Check the status of a request to create VMs in bulk

When checking the status of a request to create VMs in bulk, the operation status returns as DONE if Compute Engine either successfully creates the minimum number of specified VMs or rolls back the request.

For information about how to check the status of a request create VMs in bulk, see Handling API responses.

Check the status of a single VM

To check the status of a single VM created from a request to create VMs in bulk, use the gcloud CLI or the Compute Engine API.

gcloud

From the Operation returned by the request, get the value of the operationGroupId property.
Use the operationGroupId property as a filter with the gcloud compute operations list command to search across all operations and all zones in the project for VMs associated with the regional or zonal request:
```
gcloud compute operations list \
   --filter=(operationGroupId=OPERATION_GROUP_ID)
```
Get the rest of the VM's properties by doing any of the following:
- From the list of operations, the targetLink represents the path of the VM. Use the gcloud compute instances describe command with this path as the name of the VM to get the VM's properties:
```
gcloud compute instances describe VM_NAME
```
- Use the gcloud compute instances list command with a filter that includes the names of the VMs from the list of operations:
```
gcloud compute instances list VM_NAME \
   --filter=(name=VM_NAME_1) OR (name=VM_NAME_2)
```
- Use the gcloud compute instances list command to get the properties of VMs from across all zones and regions, and filter by either a label that is unique to the instances or by their names:
```
gcloud compute instances list \
   --filter=(name=VM_NAME_1) OR (name=VM_NAME_2)
```

API

From the Operation returned by the request, get the value of the operationGroupId property.
Use the operationGroupId property as a filter to get the list of VM operations associated with the regional or zonal request:
- If you sent a regional request, use the globalOperations.aggregatedList method to search across all operations and all zones in the project:
```
GET https://compute.googleapis.com/compute/v1/projects/PROJECT_NAME/aggregated/operations?filter=(operationGroupId=OPERATION_GROUP_ID)
```
- If you sent a zonal request, use the zoneOperations.get method to list the operations in that zone:
```
GET https://compute.googleapis.com/compute/v1/projects/PROJECT_NAME/zones/ZONE/instances/bulkInsert?filter=(operationGroupId=OPERATION_GROUP_ID)
```
Get the rest of the VM's properties by doing any of the following:
- From the list of operations, the targetLink represents the path of the VM. Use the instances.get method with this path as the name of the VM to get all of the VM's properties:
```
GET https://compute.googleapis.com/compute/v1/projects/PROJECT_NAME/zones/ZONE/instances/VM_NAME
```
- Use the instances.get method with a filter that includes the names of the VMs from the list of operations:
```
GET https://compute.googleapis.com/compute/v1/projects/PROJECT_NAME/zones/ZONE/instances?filter=(name=VM_NAME_1) OR (name=VM_NAME_2)
```
- Use the instances.aggregatedList method to get the properties of VMs from across all zones and regions, and filter by either a label that is unique to the instances or by their names:
```
GET https://compute.googleapis.com/compute/v1/projects/PROJECT_NAME/aggregated/instances?filter=(name=VM_NAME_1) OR (name=VM_NAME_2)
```

Pseudocode examples

The following pseudocode examples show how to customize requests for creating VMs in bulk.

Create VMs in bulk in one region from a set of regions

The following pseudocode describes how to create 1,000 VMs in one region from a set of regions. When attempting to create VMs in bulk in one region from a set of regions, the request first checks for capacity. If there is not enough capacity, the request fails immediately and tries again with the next region in the set.

Specify the number of VMs to create within a zone.
```
nTarget = 1000
```

Designate the regions to attempt to create the VMs in.

acceptableRegions = ["us-central1", "us-east1", "us-west1"]

Iterate through the regions, and attempt to create the VMs in each region until successful.

for region in acceptableRegions:
  call bulk API: region=region, location-policy=location-policy, count=nTarget
  if request succeeds and the operation succeeds:
    break

Create VMs in bulk in a zone on a machine type

The following pseudocode describes how to create multiple VMs in a zone on a specified machine type. When attempting to create VMs in bulk on the same machine type, the request first checks for availability of those machine types. If there is not enough of the machine type available, the request fails immediately and tries again with the next machine type.

Specify the number of VMs to create and the region to create them in.
```
nTarget = 1000
region = "us-central1"
```
Specify the machine families to attempt to create the VMs on.
```
acceptableMachineFamilies = ["n2","c2","e2","n1"]
```

Iterate through the set of machine types and attempt to create the VMs on the machine type until successful.

for family in acceptableMachineFamilies:
  call bulk APIs: region=region, count=nTarget, machineFamily=family
  if request succeeds and the operation succeeds:
    break

Create more than 1,000 VMs in a zone

When creating VMs in bulk, you can only create 1,000 VMs with each request. The following pseudocode describes how to create more than 1,000 VMs in a zone by issuing multiple requests.

Specify the number of VMs to create, a counter to keep track of the total number of created VMs, the region to create the VMs in, and a variable to store the zone that Compute Engine creates the VMs in.
```
nTarget = 10000
nCreated = 0
region = "us-central1"
targetZone = ""
```
Issue an initial request to create 1,000 VMs, save the zone returned by the request, and update the counter of the number of VMs created.
```
call bulk API: region=region, count=1000
targetZone = zone chosen by bulk API
nCreated += # of VMs created
```
Continue issuing requests to create up to 1,000 VMs at a time in the zone until Compute Engine creates the specified number of VMs.
```
while(nTarget - nCreated > 0):
  call bulk API: zone=targetZone, count=1000
  nCreated += # of VMs created
```

Create VMs in bulk and view their status

The following procedure shows you how to create a group of VMs that have predefined names and then view their status:

Specify the number of VMs to create, the zone to create them in, and a data structure to store the names in.
```
nTarget = 1000
targetZone = "us-central-1a"
names = []
```
Generate the patterned names for the VMs and add the names to the data structure.
```
for n in range(0, 1000):
  names.push("instance-%d".format(n))
```

Create the VMs, and use perInstanceProperties to specify the names.

call bulk API(zone=targetZone, count=nTarget, names=perInstanceProperties)

Get the details of the VMs by using the instances.list method with a filter for the names of VMs to return the details about.
```
instances.list(filter=(name = "instance-1") OR (name = "instance-2") ...)
```

What's next

After creating VMs in bulk, do either or both of the following to simplify management of those VMs:

Add the VMs to an unmanaged instance group. Adding VMs to an unmanaged instance group lets you use Cloud Monitoring. Unmanaged instance groups do not provide load balancing or management of the VM lifecycle.
Use labels. Labels let you organize resources using key-value pairs.