Google Cloud Hyperdisk is the newest generation of network block storage service in Google Cloud. Designed for the most demanding mission-critical applications, Hyperdisk offers a scalable, high-performance storage service with a comprehensive suite of data persistence and management capabilities. With Hyperdisk you can provision, manage, and scale your Compute Engine workloads without the cost and complexity of a typical on-premises storage area network (SAN).
Hyperdisk storage capacity is partitioned and made available to virtual machine (VM) instances as individual volumes. Hyperdisk volumes are decoupled from VMs enabling you to attach, detach, and move volumes between VMs. Data stored on Hyperdisk volumes are persistent over VM reboots and deletions.
Hyperdisk volumes have the following features:
- A Hyperdisk volume is mounted as a disk on a VM using an NVMe or SCSI interface, depending on the machine type of the VM.
- Hyperdisk volumes feature substantially better performance than Persistent Disk. With Hyperdisk, you get dedicated IOPS and throughput with each volume, as compared to Persistent Disk where performance is shared between volumes of the same type. You can add multiple Hyperdisk volumes to a single VM.
- Hyperdisk lets you scale performance and capacity dynamically. You can adjust provisioned IOPS, throughput, and the size of a volume to match your workload storage performance and capacity needs once every 4 hours. IOPS and throughput can be dialed up or down, but capacity can only be increased.
- The maximum Hyperdisk volume size is:
- 64 TiB for Hyperdisk Balanced (Preview)
- 64 TiB for Hyperdisk Extreme
- 32 TiB for Hyperdisk Throughput
You can provision up to 512 TiB total capacity for your VM or up to 257 TiB of Persistent Disk capacity for your VM.
When to use Hyperdisk
Hyperdisk volumes use the NVMe or SCSI storage interface, depending on the VM machine type.
Hyperdisk Balanced: (Preview) Hyperdisk Balanced is the best fit for most workloads. Hyperdisk Balanced is a good fit for a wide range of use cases such as LOB applications, web applications, and medium-tier databases that don't require the performance of Hyperdisk Extreme.
Hyperdisk Extreme: For performance-critical applications, where Extreme Persistent Disk does not provide enough performance, use Hyperdisk Extreme disks. Hyperdisk Extreme disks feature higher maximum IOPS and throughput, and offer high performance for the most demanding workloads, such as high performance databases.
Hyperdisk Throughput: Hyperdisk Throughput lets you flexibly provision capacity and throughput as needed for your scale-out workloads, such as Hadoop and Kafka. Hyperdisk Throughput offers increased efficiency and reduced TCO compared to Standard Persistent Disk volumes. Hyperdisk Throughput is recommended for scale-out analytics, data drives for cost sensitive apps, and cold storage.
How Hyperdisk storage works
Hyperdisk volumes are durable network storage devices that your VMs can access, similar to Persistent Disk volumes. The data on each Hyperdisk is distributed across several physical disks. Compute Engine manages the physical disks and the data distribution for you to ensure redundancy and optimal performance.
Hyperdisk volumes are located independently from your VMs, so you can detach or move Hyperdisk volumes to keep your data, even after you delete your VMs. Hyperdisk performance is decoupled from size, so you can dynamically update the performance, resize your existing Hyperdisk volumes or add more Hyperdisk volumes to a VM to meet your performance and storage space requirements.
Limitations for Hyperdisk
- You can attach a maximum of 8 Hyperdisk Balanced and Hyperdisk Extreme volumes per VM. For Hyperdisk Throughput, you can attach up to 30 volumes per VM.
- You can't create an image or machine image from a Hyperdisk volume.
- You can't clone a Hyperdisk volume.
- Hyperdisk volumes are zonal only. You can't create regional Hyperdisk volumes.
- You can't attach multiple VMs in read-only mode to a Hyperdisk volume.
- Hyperdisk volumes can't be used in multi-writer mode or attached to multiple VMs.
- Hyperdisk Extreme and Hyperdisk Throughput volumes can't be used as boot disks.
Hyperdisk capacity
Hyperdisk has the following capacity limits.
| Measurement | Hyperdisk Balanced limit | Hyperdisk Extreme limit | Hyperdisk Throughput limit | Persistent Disk limit |
|---|---|---|---|---|
| Disk size | 4 GiB to 64 TiB | 64 GiB to 64 TiB | 2 TiB to 32 TiB | 64 TiB |
| Max number of disks per VM | 8 | 8 | 30 | 128 |
| Max capacity per VM (TiB) |
|
|
|
257 TiB |
Machine type support
- Hyperdisk Balanced
- Hyperdisk Balanced (Preview) supports H3 machine types.
- Hyperdisk Extreme
Hyperdisk Extreme supports these machine types:
- C3 with 88 or more vCPUs
- M1 with 80 or more vCPUs
- M2
- M3 with 64 or more vCPUs
- N2 with 80 or more vCPUs
- Hyperdisk Throughput
Hyperdisk Throughput supports these machine types:
- N2
- N2D
- T2D
Hyperdisk performance limits
The following tables list the per VM Hyperdisk performance limits for the supported machine types.
For Persistent Disk performance limits, see performance limits for Persistent Disks.
The maximum IOPS rate is for read IOPS or write IOPS. If performing both read and write IOPS at the same time, the combined rate cannot exceed this limit.
Hyperdisk Balanced
This feature is in Preview.
| Machine type | Maximum IOPS - Read/write | Maximum throughput - Read/write |
|---|---|---|
h3-standard-88 |
15,000 | 240 MiBps |
Hyperdisk Extreme
| Machine type | Maximum IOPS Read/write |
Maximum throughput (MiBps) Read/write |
|---|---|---|
| C3 with 88 vCPUs* | 350,000 | 5,000 |
| C3 with 176 vCPUs * | 350,000 | 5,000 |
| C3 with 176 vCPUs * (Preview) | 500,000 | 10,000 |
| N2 VMs | 160,000 | 5,000 |
| M3 VMs with 64 vCPUs * | 350,000 | 5,000 |
| M3 VMs with 128 vCPUs * | 450,000 | 7,200 |
| M2 VMs | 100,000 | 4,000 |
| M1 VMs | 100,000 | 4,000 |
* If using Hyperdisk Extreme with a VM that uses Microsoft Windows, refer to the known issues for Windows VM instances.
Hyperdisk Throughput
| Maximum total throughput (MiBps) - Read and write * | |||
|---|---|---|---|
| vCPU count | N2 | N2D | T2D |
| 1 to 3 vCPUs | 200 | 200 | 200 |
| 4 to 7 vCPUs | 240 | 240 | 240 |
| 8 to 15 vCPUs | 800 | 800 | 800 |
| 16 to 31 vCPUs | 1,200 | 1,200 | 1,200 |
| 32 to 47 vCPUs | 1,800 | 1,800 | 1,800 |
| 48 to 63 vCPUs | 2,400 | 2,400 | 2,400 |
| 64 to 127 vCPUs | 3,000 | 2,400 | N/A |
| 128 or more vCPUs | 2,400 | 2,400 | N/A |
* Assuming at least 128K sequential IO or at least 256K random IO.
Higher performance for Hyperdisk Extreme on C3 instances
Hyperdisk Extreme now offers up to 500,000 IOPS and 10 GiBps on C3 VMs with 176 vCPUs. This improvement enables over 40% higher IOPS performance and 2 times higher throughput performance with Hyperdisk Extreme volumes.
Because the per disk limit is 350,000 IOPS and 5 GiBps for Hyperdisk Extreme, you must attach multiple Hyperdisk Extreme volumes to achieve this level of performance per VM.
Hyperdisk regional availability
Hyperdisk can be used in the following regions or zones:
Hyperdisk Balanced
This feature is in Preview.
- Eemshaven, Netherlands—
europe-west4 - Council Bluffs, Iowa—
us-central1
Hyperdisk Extreme
- Changhua County, Taiwan—
asia-east1 - Tokyo, Japan—
asia-northeast1 - Osaka, Japan—
asia-northeast2 - Seoul, South Korea—
asia-northeast3 - Mumbai, India—
asia-south1 - Delhi, India—
asia-south2 - Jurong West, Singapore—
asia-southeast1 - Jakarta, Indonesia—
asia-southeast2 - Sydney, Australia—
australia-southeast1 - Madrid, Spain—
europe-southwest1 - St. Ghislain, Belgium—
europe-west1 - London, England—
europe-west2 - Frankfurt, Germany—
europe-west3 - Eemshaven, Netherlands—
europe-west4 - Zurich, Switzerland—
europe-west6 - Milan, Italy—
europe-west8 - Paris, France—
europe-west9 - Tel Aviv, Israel—
me-west1 - Montréal, Québec—
northamerica-northeast1 - Toronto, Ontario—
northamerica-northeast2 - Osasco, São Paulo, Brazil—
southamerica-east1 - Council Bluffs, Iowa—
us-central1 - Moncks Corner, South Carolina—
us-east1 - Ashburn, Virginia—
us-east4 - The Dalles, Oregon—
us-west1 - Los Angeles, California—
us-west2 - Salt Lake City, Utah—
us-west3 - Las Vegas, Nevada—
us-west4
Hyperdisk Throughput
- Region: Council Bluffs, Iowa—
us-central1 - Region: Ashburn, Virginia—
us-east4 - Region: Jurong West, Singapore—
asia-southeast1 - Region: Eemshaven, Netherlands—
europe-west4 - Region: Moncks Corner, South Carolina—
us-east1 - Zone: Mumbai, India—
asia-south1-a
About IOPS and throughput provisioning for Hyperdisk
Unlike Persistent Disk, where performance scales automatically with size, with Hyperdisk you can provision performance directly. To provision performance, you select the target performance level for a given volume. Individual volumes have full performance isolation—each volume gets the performance provisioned to it.
About IOPS for Hyperdisk
You can modify the provisioned IOPS for Hyperdisk Balanced (Preview) and Hyperdisk Extreme volumes, but not for Hyperdisk Throughput volumes.
To reach maximum IOPS and throughput levels offered by Hyperdisk volumes, you must consider the following workload parameters:
- I/O size: Maximum IOPS limits assume that you are using an I/O size of 4 KB or 16 KB. Maximum throughput limits assume that you are using an I/O size of at least 64 KB.
- Queue length: Queue length is the number of pending requests for a volume. To reach maximum performance limits, you must tune your queue length according to the I/O size, IOPS, and latency sensitivity of your workload. Optimal queue length varies for each workload, but typically should be larger than 256.
- Working set size: Working set size is the amount of data of a volume being accessed within a short period of time. To achieve optimal performance, working set sizes must be greater than or equal to 32 GiB.
- Multiple attached disks: Hyperdisk volumes share the per-VM maximum IOPS and throughput limits with all Persistent Disk and Hyperdisk volumes attached to the same VM. With multiple attached disks, each disk has performance limits proportional to their share of IOPS in the total provisioned across all attached Hyperdisk volumes. When monitoring the performance of your Hyperdisk volumes, take into account any I/O requests that you are sending to other volumes that are attached to the same VM.
However, the IOPS for Hyperdisk volumes are ultimately capped by per-VM limits for the VM to which your volumes are attached. To review these limits, see Hyperdisk performance limits.
For more information about how to improve performance, see Optimize performance of Hyperdisk.
IOPS for Hyperdisk Balanced
If you don't specify a value for IOPS when creating a Hyperdisk Balanced volume, it uses the default value of 3,600 IOPS. You can provision custom IOPS levels for your Hyperdisk Balanced volumes. The provisioned IOPS must follow these rules:
- Minimum: The lesser of 3,000 IOPS or 500 IOPS per GiB of disk capacity.
- Maximum: 500 IOPS per GiB of disk capacity, but not more than 160,000.
IOPS for Hyperdisk Extreme
If you don't specify a value for IOPS when creating a Hyperdisk Extreme volume, a default value is used, which is the lesser of 100 IOPS per GiB of disk capacity or the maximum IOPS for the machine type. You can provision custom IOPS levels for your Hyperdisk Extreme volumes. The provisioned IOPS must follow these rules:
- At least 2 IOPS per GiB of disk capacity, but not more than 1000 IOPS per GiB of capacity
- At most 450,000 per volume, depending on the machine type
IOPS for Hyperdisk Throughput
For Hyperdisk Throughput volumes, the IOPS scales with the provisioned throughput, at a rate of 4 IOPS per MiBps for random I/O, or 8 IOPS per MiBps for sequential I/O. However, IOPS is ultimately limited by the machine type of the VM to which your Hyperdisk Throughput volumes are attached.
About throughput for Hyperdisk
You can modify the provisioned throughput for Hyperdisk Balanced (Preview) volumes and Hyperdisk Throughput volumes, but not for Hyperdisk Extreme volumes.
To reach maximum throughput levels offered by Hyperdisk volumes, you must consider the following workload parameters:
- I/O size: Maximum throughput limits assume that you are using a sequential I/O size of at least 128 KB, or a random I/O size of at least 256 KB.
- Queue length: Queue length is the number of pending requests for a volume. To reach maximum performance limits, you must tune your queue length according to the I/O size, IOPS, and latency sensitivity of your workload. Optimal queue length varies for each workload, but typically should be larger than 256.
- Multiple attached disks: If you attach more than one Hyperdisk volume to your VM, and the total throughput provisioned for all Hyperdisk volumes exceeds the limits documented for the machine type, the total disk performance won't exceed the limit for the machine type.
For more information, see Optimize performance of Hyperdisk.
Throughput for Hyperdisk Balanced
If you don't specify a value for throughput, the volume uses the default value of 140 MiBps.
You can provision custom throughput levels for your Hyperdisk Balanced volumes. The provisioned throughput for each disk must follow these rules:
- Minimum: The greater of 140 MiBps or the configured IOPS divided by 256.
- Maximum: The provisioned IOPS divided by 4, but not more than 2,400 MiBps.
Throughput for Hyperdisk Extreme
For Hyperdisk Extreme volumes, throughput scales with the number of IOPS you provision at a rate of 256 KiBps of throughput per I/O. However, throughput is ultimately capped by per-VM limits that depend on the number of vCPUs on the VM to which your Hyperdisk Extreme volumes are attached.
Throughput for Hyperdisk Extreme volumes is not full duplex. The maximum throughput limits listed in Hyperdisk performance limits apply to the sum total of read and write throughput.
Throughput for Hyperdisk Throughput
If you don't specify a value for throughput, a default value is used, which is 90 MiBps per TiB of disk capacity, but not more than the maximum supported throughput per disk.
You can provision custom throughput levels for your Hyperdisk Throughput volumes. The provisioned throughput must follow these rules:
- At least 10 MiBps per TiB of capacity, but not more than 90 MiBps per TiB of capacity.
- At most 600 MiBps per volume.
Pricing
You are billed for the total provisioned capacity of your Hyperdisk volumes until you delete them. You are charged per GiB per month. Additionally, you are billed for the following:
- Hyperdisk Balanced (Preview) charges a monthly rate for the provisioned IOPS and provisioned throughput (in MiBps) in excess of the baseline values of 3,000 IOPS and 140 MiBps throughput.
- Hyperdisk Extreme charges a monthly rate based on the provisioned IOPS.
- Hyperdisk Throughput charges a monthly rate based on the provisioned throughput (in MiBps).
For more pricing information, see Disk pricing.
Hyperdisk and committed use discounts
Hyperdisk volumes are not eligible for:
- Resource-based committed use discounts (CUDs)
- Sustained use discounts (SUDs)
Hyperdisk and preemptible VM instances
Hyperdisk can be used with Spot VMs (or preemptible VMs). However, there are no discounted spot prices for Hyperdisk.
What's next?
- Learn how to create a Hyperdisk volume.
- Review Disk pricing information.
- Learn how to optimize performance of Hyperdisk.