Google Cloud Hyperdisk is the newest generation of network block storage service in Google Cloud. Designed for the most demanding mission-critical applications, Hyperdisk offers a scalable, high-performance storage service with a comprehensive suite of data persistence and management capabilities. With Hyperdisk you can provision, manage, and scale your Compute Engine workloads without the cost and complexity of a typical on-premises storage area network (SAN).
Hyperdisk storage capacity is partitioned and made available to virtual machine (VM) instances as individual volumes. Hyperdisk volumes are decoupled from VMs enabling you to attach, detach, and move volumes between VMs. Data stored on Hyperdisk volumes are persistent over VM reboots and deletions.
Hyperdisk volumes have the following features:
- A Hyperdisk volume is mounted as a disk on a VM using an NVMe or SCSI interface, depending on the machine type of the VM.
- Hyperdisk volumes feature substantially better performance than Persistent Disk. With Hyperdisk, you get dedicated IOPS and throughput with each volume, as compared to Persistent Disk where performance is shared between volumes of the same type.
- Hyperdisk lets you scale storage performance and capacity to match your workload needs. IOPS and throughput can be adjusted up or down, but capacity can only be increased.
- To access static data from multiple VMs, you can attach the same disk in read-only mode to hundreds of VMs.
- The maximum total Hyperdisk capacity is 512 TiB for VMs with 32 or more vCPUs, and 257 TiB for VMs with 1 to 31 vCPUs.
- Synchronous replication is available with Hyperdisk Balanced High Availability, which synchronously replicates data between two zones in the same region. This ensures high availability (HA) for Hyperdisk Balanced High Availability disk data for up to one zonal failure.
- Hyperdisk Extreme, Hyperdisk Balanced, Hyperdisk ML are designed for sub-millisecond latencies.
The maximum number of Hyperdisk volumes you can attach to a VM varies by Hyperdisk type. See Hyperdisk limits per VM for details.
When to use Hyperdisk
Hyperdisk volumes use the NVMe or SCSI storage interface, depending on the VM machine type.
For most workloads, use Hyperdisk Balanced. Hyperdisk Balanced is a good fit for a wide range of use cases such as LOB applications, web applications, and medium-tier databases that don't require the performance of Hyperdisk Extreme. Use Hyperdisk Balanced for applications where multiple VMs in the same zone simultaneously require write access to the same disk.
To protect your applications from a zonal outage, use Hyperdisk Balanced High Availability to synchronously replicate the disk data across two zones in the same region. You can also use Hyperdisk Balanced High Availability for applications where multiple VMs in the same regions simultaneously require write access to the same disk.
For machine learning workloads that require the highest throughput, use Hyperdisk ML. Hyperdisk ML has the highest throughput available and the fastest data load times as a result. Faster data load times mean shorter accelerator idle times and lower compute costs. For large inference and training workloads, you can attach a single Hyperdisk ML volume to multiple VMs in read-only mode.
For performance-critical applications, use Hyperdisk Extreme if Extreme Persistent Disk isn't supported or doesn't provide enough performance. Hyperdisk Extreme disks feature higher maximum IOPS and throughput along with low sub-millisecond latencies, and offer high performance for the most demanding workloads, such as high performance databases.
For scale-out analytics workloads like Hadoop and Kafka, cold storage, and data drives for cost sensitive apps, use Hyperdisk Throughput. Hyperdisk Throughput lets you flexibly provision capacity and throughput as needed. Hyperdisk Throughput offers increased efficiency and reduced TCO compared to Standard Persistent Disk volumes. For details, see Throughput for Hyperdisk Throughput.
For detailed performance information, see Performance limits.
Feature summary
The following table summarizes the differences between the various Hyperdisk types.
Hyperdisk type | Customizable throughput | Customizable IOPS | Shareable between VMs | Boot disk support |
---|---|---|---|---|
Hyperdisk Balanced | Yes | Yes | Yes | Yes* |
Hyperdisk Balanced High Availability |
Yes | Yes | Yes | Yes* |
Hyperdisk Extreme | No | Yes | No | No |
Hyperdisk ML | Yes | No | Yes, in read-only mode |
No |
Hyperdisk Throughput | Yes | No | No | No |
How Hyperdisk storage works
Hyperdisk volumes are durable network storage devices that your VMs can access, similar to Persistent Disk volumes. The data on each Hyperdisk is distributed across several physical disks. Compute Engine manages the physical disks and the data distribution for you to ensure redundancy and optimal performance.
Hyperdisk volumes use Titanium to achieve higher IOPS and throughput rates. Titanium offloads processing from the host CPU onto silicon devices deployed throughout the data center.
Hyperdisk volumes are located independently from your VMs, so you can detach or move Hyperdisk volumes to keep your data, even after you delete your VMs. Hyperdisk performance is decoupled from size, so you can update the performance, resize your existing Hyperdisk volumes or add more Hyperdisk volumes to a VM to meet your performance and storage space requirements.
Share Hyperdisk volumes between VMs
You can share a Hyperdisk volume between multiple VMs by simultaneously attaching the same volume to multiple VMs.
The following scenarios are supported:
Concurrent read-write access to a single volume from multiple VMs. Recommended for clustered file systems and highly available workloads like SQL Server Failover Cluster Infrastructure. Supported for Hyperdisk Balanced and Hyperdisk Balanced High Availability volumes.
Concurrent read-only access to a single volume from multiple VMs. This is more cost effective than having multiple disks with the same data. Recommended for accelerator-optimized machine learning workloads. Supported for Hyperdisk ML volumes.
You can't attach a Hyperdisk Throughput or Hyperdisk Extreme volume to more than one VM.
To learn about disk sharing, see Share a disk between VMs.
Encryption for Hyperdisk volumes
By default, Compute Engine protects your Hyperdisk volumes with Google-managed encryption. You can also encrypt your Hyperdisk volumes with customer-managed encryption keys (CMEK).
For more information, see About disk encryption.
Confidential Computing with Hyperdisk volumes
You can add hardware-based encryption to a Hyperdisk Balanced disk by enabling Confidential mode for the disk when you create it. You can use Confidential mode only with Hyperdisk Balanced disks that are attached to Confidential VMs.
For more information, see Confidential mode for Hyperdisk Balanced volumes.
Limitations for Hyperdisk
- You can't create a machine image from a Hyperdisk volume.
- You can't back up a disk in multi-writer mode with snapshots or images. You must disable multi-writer mode first.
- You can't create an image from a Hyperdisk Extreme, Hyperdisk Throughput, or Hyperdisk Balanced High Availability volume.
- You can't create an instant snapshot from a Hyperdisk volume.
- You can't attach Hyperdisk Throughput or Hyperdisk Extreme volumes to more than one VM.
- Hyperdisk Extreme, Hyperdisk ML and Hyperdisk Throughput volumes can't be used as boot disks.
- You can attach a Hyperdisk ML volume to up to 100 VMs at most once every 30 seconds.
- You can't create a Hyperdisk ML disk in read-write mode from a snapshot or a disk image. You must create the disk in read-only mode.
- If you enable read-only mode for a Hyperdisk ML volume, you can't re-enable read-write mode.
- If you create a Hyperdisk Balanced volume in Confidential mode, see additional limitations.
Hyperdisk limits per disk
The following table shows the maximum and minimum values you can use for a single Hyperdisk volume.
For details about throughput and IOPS for Hyperdisk, see About IOPS and throughput provisioning for Hyperdisk.
Property | Hyperdisk Balanced | Hyperdisk Balanced HA | Hyperdisk Extreme | Hyperdisk ML | Hyperdisk Throughput |
---|---|---|---|---|---|
Min disk size | 4 GiB | 4 GiB | 64 GiB | 4 GiB | 2 TiB |
Max disk size | 64 TiB | 64 TiB | 64 TiB | 64 TiB | 32 TiB |
Min IOPS | 3,000 IOPS† | 3,000 IOPS† | 2 IOPS per GiB of capacity | 16 IOPS per MBps of provisioned throughput | 4 random IOPS or 8 sequential IOPS per MBps of throughput |
Max IOPS | 500 IOPS per GiB of disk capacity, but not more than 160,000 |
500 IOPS per GiB of disk capacity, but not more than 100,000 |
1,000 IOPS per GiB of capacity, but not more than 350,000 |
16 IOPS per MBps of provisioned throughput | 4 random IOPS or 8 sequential IOPS per MBps of throughput |
Min throughput | 140 MBps | 140 MBps | 256 KBps of throughput per provisioned IOPS | The greater of 0.12 MBps per GiB and 400 MBps per disk | The greater of 10 MBps per TiB, or 20 MBps per disk |
Max throughput | IOPS divided by 4, but not more than 2,400 MBps | IOPS divided by 4, but not more than 1,200 MBps | 256 KiBs of throughput per provisioned IOPS, but not more than 4,800 MBps | 1600 MBps per GiB, but not more than 1200000 MBps | The lesser of 90 MBps per TiB, or 600 MBps per disk |
Frequency for changes | Every 4 hours for capacity and performance |
Every 4 hours for capacity and performance |
Every 4 hours for capacity and performance |
Every 4 hours for capacity, every 6 hours for throughput |
Every 6 hours for capacity, every 4 hours for performance. |
† Hyperdisk Balanced and Hyperdisk Balanced HA volumes with sizes of 4 or 5 GiB can only be provisioned with exactly 2000 or 2500 IOPS, respectively.
Hyperdisk limits per VM
This section describes the capacity limits that apply to using Hyperdisk volumes with a VM. The limits discussed don't apply to any Local SSD disks attached to the same VM.
Maximum total capacity per VM
The maximum total disk capacity (in TiB) across all Hyperdisk and Persistent Disk types that you attach to a VM depends on the number of vCPUs the VM has. The capacity limits are as follows:
For machine types with less than 32 vCPUs:
- 257 TiB for all Hyperdisk or all Persistent Disk
- 257 TiB for a mixture of Hyperdisk and Persistent Disk
For machine types with 32 or more vCPUs:
- 512 TiB for all Hyperdisk
- 512 TiB for a mixture of Hyperdisk and Persistent Disk
- 257 TiB for all Persistent Disk
You can attach a combination of Hyperdisk and Persistent Disk volumes to a single VM but the total disk capacity for Persistent Disk can't exceed 257 TiB.
Maximum number of disks per VM, across all types
The maximum number of individual Persistent Disk and Hyperdisk volumes you can attach to a VM is 128. So, if you have attached 28 Hyperdisk volumes to a VM, you can still attach up to 100 more Persistent Disk volumes to the same VM.
Maximum Hyperdisk volumes per VM
The maximum number of Hyperdisk volumes that you can attach to a VM depends on the number of vCPUs that the VM has, as described in the following table:
Number of vCPUs | Max number of Hyperdisk, all types* | Max number of Hyperdisk Balanced disks† | Max number of Hyperdisk Balanced HA disks | Max number of Hyperdisk Extreme disks | Max number of Hyperdisk ML disks | Max number of Hyperdisk Throughput disks |
---|---|---|---|---|---|---|
1 to 3 | 20 | 16 | 0 | 0 | 20 | 20 |
4 to 7 | 24 | 16 | 16 | 0 | 24 | 24 |
8 to 15 | 32 | 32 | 16 | 0 | 32 | 32 |
16 to 31 | 48 | 32 | 32 | 0 | 48 | 48 |
32 to 63 | 64 | 32 | 32 | 0 | 64 | 64 |
64 or more | 64 | 32 | 32 | 8# | 64 | 64 |
† The maximum number of Hyperdisk Balanced and Hyperdisk Balanced High Availability disks that you can attach to a VM varies from 16 to 32 volumes, depending on the machine series and machine type. Refer to the specific machine series documentation to learn about its disk limits.
# N2 VMs require a minimum of 80 vCPUs to use Hyperdisk Extreme.
Summary of Hyperdisk per-VM limits
Overall, for an individual VM instance, there are the following limits for using Hyperdisk:
- A limit for the total number of all Persistent Disk and Hyperdisk volumes that you can attach to a VM, including the boot disk.
- A limit for the combined total capacity of all disks attached to a VM.
- A limit for the total number of Hyperdisk volumes that you can attach to a VM.
- A limit for the maximum number of each type of Hyperdisk volume that you can attach to a single VM.
When multiple limits apply, the most specific limit is enforced. For example, suppose you have a VM with 96 vCPUs, and you want to use a combination of Hyperdisk and Persistent Disk volumes. The following limits apply:
- Maximum number of Persistent Disk and Hyperdisk volumes that you can attach to the VM: 128
- Maximum number of Hyperdisk volumes, across all types: 64
- Maximum number of Hyperdisk Throughput volumes: 64
- Maximum number of Hyperdisk ML volumes: 64
- Maximum number of Hyperdisk Extreme volumes: 8
- Maximum number of Hyperdisk Balanced or Hyperdisk Balanced High Availability volumes: 32
The following examples illustrate these limits.
Maximum number of a single type of Hyperdisk per VM: You can only attach 8 Hyperdisk Extreme volumes to the VM. This is true even if you don't attach any other Persistent Disk or Hyperdisk volumes to the VM.
Maximum number of Hyperdisk volumes per VM: If you attach 8 Hyperdisk Extreme volumes to the VM, you can attach at most 56 other Hyperdisk volumes to the VM. This makes the combined number of Hyperdisk volumes equal to 64, which is the maximum number of Hyperdisk volumes that you can attach to a VM.
Maximum number of disks or volumes per VM, across all types: If you attach a combined total of 64 Hyperdisk volumes to the VM, then you can't attach any more Hyperdisk volumes. However, because the maximum number of disks of all types is 128, you can still attach up to 64 Persistent Disk volumes to the VM.
Machine type support
This section lists the machine types that each Hyperdisk type supports.
- Hyperdisk Balanced
Hyperdisk Balanced supports these machine types:
- A3 machine types with 8 GPUs
- C3 machine types (all machine types)
- C3D machine types (all machine types)
- C4 machine types (all machine types)
- C4A machine types (all machine types)
- H3 machine types with 88 vCPUs
- M1 machine types with 40, 80, 96, or 160 vCPUs
- M2 machine types with 208 or 416 vCPUs
- M3 machine types with 32, 64, or 128 vCPUs
- N4 machine types (all machine types)
- X4 machine types (all machine types)
- Hyperdisk Balanced High Availability
Hyperdisk Balanced High Availability supports these machine types:
- C4A machine types (all machine types)
- C3 machine types (all machine types)
- C3D machine types (all machine types)
- M3 machine types (all machine types)
- Hyperdisk Extreme
Hyperdisk Extreme supports these machine types:
- Hyperdisk ML
Hyperdisk ML supports these machine types:
- Hyperdisk Throughput
Hyperdisk Throughput supports these machine types:
Hyperdisk performance limits
The following tables list the per VM Hyperdisk performance limits for the supported machine types.
For Persistent Disk performance limits, see performance limits for Persistent Disks.
The maximum IOPS rate is for read IOPS or write IOPS. If performing both read and write IOPS at the same time, the combined rate cannot exceed this limit.
C4 Hyperdisk performance limits
The C4 machine series provides the option to see the steady state IOPS and maximum IOPS read/write as well as the steady state and maximum throughput read/write for every VM machine type.
Hyperdisk Balanced
Hyperdisk Balanced offers up to 320,000 IOPs and 10,000 MBps throughput on C4 VMs with 192 vCPUs. Because the per disk limit is 160,000 IOPs and 2,400 MBps throughput for Hyperdisk Balanced, you must attach multiple Hyperdisk Balanced volumes to achieve this level of performance per VM.
Machine type | Max disk count | Steady state IOPS - Read/write | Maximum IOPS - Read/write | Steady state throughput - Read/write | Maximum throughput - Read/write |
---|---|---|---|---|---|
C4 with 2 vCPUs | 8 | 6,200 | 50,000 | 120 MBps | 400 MBps |
C4 with 4 vCPUs | 16 | 12,500 | 50,000 | 240 MBps | 400 MBps |
C4 with 8 vCPUs | 32 | 25,000 | 50,000 | 480 MBps | 800 MBps |
C4 with 16 vCPUs | 32 | 50,000 | 100,000 | 1,000 MBps | 1,600 MBps |
C4 with 32 vCPUs | 32 | 100,000 | 100,000 | 1,600 MBps | 1,600 MBps |
C4 with 48 vCPUs | 32 | 160,000 | 160,000 | 2,400 MBps | 2,400 MBps |
C4 with 96 vCPUs | 64 | 240,000 | 240,000 | 4,800 MBps | 4,800 MBps |
C4 with 192 vCPUs | 128 | 320,000 | 320,000 | 10,000 MBps | 10,000 MBps |
Hyperdisk Extreme
Hyperdisk Extreme offers up to 500,000 IOPS and 10,000 MBps on C4 VMs with 192 vCPUs. Because the per disk limit is 350,000 IOPS and 5,000 MBps throughput for Hyperdisk Extreme, you must attach multiple Hyperdisk Extreme volumes to achieve this level of performance per VM.
Machine type | Max disk count | Steady state IOPS - Read/write | Maximum IOPS - Read/write | Steady state throughput - Read/write | Maximum throughput - Read/write |
---|---|---|---|---|---|
C4 with 2 vCPUs | 0 | 0 | 0 | 0 | 0 |
C4 with 4 vCPUs | 0 | 0 | 0 | 0 | 0 |
C4 with 8 vCPUs | 0 | 0 | 0 | 0 | 0 |
C4 with 16 vCPUs | 0 | 0 | 0 | 0 | 0 |
C4 with 32 vCPUs | 0 | 0 | 0 | 0 | 0 |
C4 with 48 vCPUs | 0 | 0 | 0 | 0 | 0 |
C4 with 96 vCPUs | 8 | 350,000 | 350,000 | 5,000 | 5,000 |
C4 with 192 vCPUs | 8 | 500,000 | 500,000 | 10,000 | 10,000 |
Performance limits for other VMs
Performance information for all other machine series are listed in the following table.
Hyperdisk Balanced
Hyperdisk Balanced offers up to 160,000 IOPs and 4,800 MBps throughput on some machine series. Because the per disk limit is 160,000 IOPs and 2,400 MBps throughput for Hyperdisk Balanced, you must attach multiple Hyperdisk Balanced volumes to achieve this level of performance per instance.
Machine type | Maximum IOPS - Read/write | Maximum throughput - Read/write |
---|---|---|
A3 High and Mega with 8 GPUs (using two volumes) | 160,000 | 4,800 MBps |
C4A with 1 vCPU | 25,000 | 400 MBps |
C4A with 2 or 4 vCPUs | 50,000 | 800 MBps |
C4A with 8 vCPUs | 50,000 | 1,000 MBps |
C4A with 16 vCPUs | 60,000 | 1,600 MBps |
C4A with 32 vCPUs | 120,000 | 2,400 MBps |
C4A with 48 vCPUs | 160,000 | 3,300 MBps |
C4A with 64 vCPUs | 240,000 | 4,400 MBps |
C4A with 72 vCPUs | 240,000 | 5,000 MBps |
C3 with 4 vCPUs | 25,000 | 400 MBps |
C3 with 8 vCPUs | 50,000 | 800 MBps |
C3 with 22 vCPUs | 120,000 | 1,800 MBps |
C3 with 44 vCPUs | 160,000 | 2,400 MBps |
C3 with 88 vCPUs | 160,000 | 4,800 MBps |
C3 with 176 or more vCPUs* | 160,000 | 10,000 MBps |
C3D with 4 vCPUs | 25,000 | 400 MBps |
C3D with 8 vCPUs | 50,000 | 800 MBps |
C3D with 16 to 30 vCPUs | 75,000 | 1,200 MBps |
C3D with 60 or more vCPUs | 160,000 | 2,400 MBps |
H3 with 88 vCPUs | 15,000 | 240 MBps |
M1 with 40 vCPUs | 60,000 | 1,200 MBps |
M1 with 80 vCPUs | 100,000 | 2,400 MBps |
M1 with 96 and 160 vCPUs | 100,000 | 4,000 MBps |
M2 with 208 vCPUs | 100,000 | 2,400 MBps |
M2 with 416 vCPUs | 100,000 | 4,000 MBps |
M3 with 32 vCPUs | 160,000 | 2,400 MBps |
M3 with 64 or more vCPUs | 160,000 | 4,800 MBps |
N4 with 2 and 4 vCPUs | 15,000 | 240 MBps |
N4 with 8 vCPUs | 15,000 | 480 MBps |
N4 with 16 vCPUs | 80,000 | 1,200 MBps |
N4 with 32 vCPUs | 100,000 | 1,600 MBps |
N4 with 48 or more vCPUs | 160,000 | 2,400 MBps |
X4 with 960 or more vCPUs | 160,000 | 4,800 MBps |
* Includes bare metal instances.
Hyperdisk Balanced HA
Machine type | Maximum IOPS - Read/write | Maximum throughput - Read/write |
---|---|---|
C4A with 1 vCPU | 25,000 | 400 MBps |
C4A with 2 or 4 vCPUs | 50,000 | 800 MBps |
C4A with 8 vCPUs | 50,000 | 1,000 MBps |
C4A with 16 vCPUs | 60,000 | 1,600 MBps |
C4A with 32 vCPUs | 120,000 | 2,400 MBps |
C4A with 48 vCPUs | 160,000 | 3,300 MBps |
C4A with 64 vCPUs | 240,000 | 4,400 MBps |
C4A with 72 vCPUs | 240,000 | 5,000 MBps |
C3 with 4 vCPUs | 25,000 | 400 MBps |
C3 with 8 vCPUs | 50,000 | 600 MBps |
C3 with 22 vCPUs | 100,000 | 600 MBps |
C3 with 44 vCPUs | 100,000 | 1,200 MBps |
C3 with 88 or more vCPUs | 100,000 | 2,500 MBps |
C3D with 4 vCPUs | 25,000 | 400 MBps |
C3D with 8 vCPUs | 50,000 | 600 MBps |
C3D with 16 to 30 vCPUs | 100,000 | 600 MBps |
C3D with 60 or more vCPUs | 100,000 | 1,200 MBps |
M3 with 32 vCPUs | 100,000 | 1,900 MBps |
M3 with 64 or more vCPUs | 100,000 | 2,500 MBps |
Hyperdisk Extreme
Hyperdisk Extreme offers up to 500,000 IOPS and 10,000 MBps on C3 VMs with 176 vCPUs. Because the per disk limit is 350,000 IOPS and 5,000 MBps throughput for Hyperdisk Extreme, you must attach multiple Hyperdisk Extreme volumes to achieve this level of performance per VM.
Machine type | Maximum IOPS Read/write |
Maximum throughput (MBps) Read/write |
---|---|---|
A3 High VMs with 1 GPU | N/A | N/A |
A3 High VMs with 2 GPUs | N/A | N/A |
A3 High VMs with 4 GPUs | 350,000 | 5,000 |
A3 High and Mega VMs with 8 GPUs (using two volumes) | 400,000 | 8,000 |
C3 with 88 vCPUs * | 350,000 | 5,000 |
C4A with 64 or more vCPUs | 350,000 | 5,000 |
C3 with 176 or more vCPUs *,† | 500,000 | 10,000 |
C3D with 60 or more vCPUs* | 350,000 | 5,000 |
M1 VMs | 100,000 | 4,000 |
M2 VMs | 100,000 | 4,000 |
M3 VMs with 64 vCPUs * | 350,000 | 5,000 |
M3 VMs with 128 vCPUs * | 450,000 | 7,200 |
N2 VMs with 80 or more vCPUs | 160,000 | 5,000 |
X4 with 960 or more vCPUs † | 400,000 | 10,000 |
Z3 with 88 vCPUs | 350,000 | 5,000 |
Z3 with 176 vCPUs | 350,000 | 5,000 |
* If using Hyperdisk Extreme with a VM that uses
Microsoft Windows, refer to the
known issues for Windows VM instances.
† Includes bare metal instances.
Hyperdisk ML
You can provision up to 1,200,000 MBps of throughput for each Hyperdisk ML disk. The provisioned throughput is distributed across each VM the disk is attached to, up to the maximum throughput level supported by the VM type.
For example, the maximum throughput for an A3 VM with 8 GPUs is 4,000 MBps (4 GBps). If you provision a Hyperdisk ML volume with 500,000 MBps of throughput, you must attach the disk to at least 125 A3 VMs to consume the provisioned 500,000 MBps of throughput. However, to use the same disk with A2 VMs with 1 GPU, you would need 278 A2 VMs because A2 VMs have a lower maximum throughput limit.
For more information about how throughput is distributed across VMs, see Throughput for Hyperdisk ML.
The following table outlines the maximum throughput each VM attached to a Hyperdisk ML volume can reach.
Each Hyperdisk ML volume gets 16 IOPS per MBps of provisioned throughput, and this can't be changed. The IOPS limits apply to sequential and random I/O.
Machine type | vCPU count | Maximum throughput (MBps) per VM | Max IOPS per instance |
---|---|---|---|
A2 with 1 GPU | 12 | 1,800 | 28,800 |
A2 with 2 GPUs | 24 | 2,400 | 38,400 |
A2 with 4 GPUs | 48 | 2,400 | 38,400 |
A2 with 8 GPUs | 96 | 2,400 | 38,400 |
A2 with 16 GPUs | 96 | 2,400 | 38,400 |
A3 High with 1 GPU | 26 | 1,800 | 28,800 |
A3 High with 2 GPUs | 52 | 2,400 | 38,400 |
A3 High with 4 GPUs | 104 | 2,400 | 38,400 |
A3 High and Mega with 8 GPUs (in read-only mode) | 208 | 4,000 | 64,000 |
A3 High and Mega with 8 GPUs (in read-write mode) | 208 | 2,400 | 38,400 |
C3 | 4 | 400 | 6,400 |
8 | 800 | 12,800 | |
22 | 1,800 | 28,800 | |
44 or more | 2,400 | 38,400 | |
C3D | 4 | 400 | 6,400 |
8 | 800 | 12,800 | |
16 | 1,200 | 19,200 | |
30 | 1,200 | 19,200 | |
60 or more | 2,400 | 38,400 | |
G2 | 4 | 800 | 12,800 |
8 | 1,200 | 19,200 | |
12 | 1,800 | 28,800 | |
16 or more | 2,400 | 38,400 |
For A3 VMs with 8 GPUs, performance depends on whether the disk is attached to the VM in read-only or read-write mode.
Hyperdisk Throughput
You can provision at most 600 MBps of throughput per Hyperdisk Throughput volume, but if you attach multiple Hyperdisk Throughput volumes to the same VM, then the throughput limits stated in the following table apply.
For example, as shown in the table, a C3 VM with 22 vCPUs can provide a total maximum throughput of 1200 MBps, regardless of how many Hyperdisk Throughput volumes you attach.
If you attach one Hyperdisk Throughput volume to a VM, the maximum throughput for the VM is 600 MBps. If you attach two or more Hyperdisk Throughput volumes to a VM, the maximum throughput for the VM is 1200 MBps.
Machine type | vCPU count | Maximum read/write throughput (MBps) per VM* |
Minimum number of disks needed to reach maximum throughput |
---|---|---|---|
A3 High with 1 GPU | 26 | 1200 | 2 |
A3 High with 2 GPUs | 52 | 2400 | 4 |
A3 High with 4 GPUs | 104 | 2400 | 4 |
A3 High and Mega with 8 GPUs | 208 | 3000 | 5 |
C3 | 4 | 240 | 1 |
8 | 800 | 2 | |
22 | 1200 | 2 | |
44 | 1800 | 3 | |
88 or more | 2400 | 4 | |
C3D | 4 | 400 | 1 |
8 | 800 | 2 | |
16 or 30 | 1200 | 2 | |
60 or more | 2400 | 4 | |
G2 | 4 | 240 | 1 |
8 or 12 | 800 | 2 | |
16 or 24 | 1200 | 2 | |
32 | 1800 | 3 | |
48 or more | 2400 | 4 | |
H3 | 88 | 240 | 1 |
M3 | 32 | 1800 | 3 |
64 or 128 | 2400 | 4 | |
N2 | 1-3 | 200 | 1 |
4-7 | 240 | 1 | |
8-15 | 800 | 2 | |
16-31 | 1200 | 2 | |
32-47 | 1800 | 3 | |
48-63 | 2400 | 4 | |
64-127 | 3000 | 5 | |
128 or more | 2400 | 4 | |
N2D | 2 | 200 | 1 |
4 | 240 | 1 | |
8 | 800 | 2 | |
16 | 1200 | 2 | |
32 | 1800 | 3 | |
48 to 63 | 2400 | 4 | |
T2D | 1-2 | 200 | 1 |
4 | 240 | 1 | |
8 | 800 | 2 | |
16 | 1200 | 2 | |
32 | 1800 | 3 | |
48 or more | 2400 | 4 | |
Z3 | 88 or more | 2400 | 4 |
Hyperdisk regional availability
Hyperdisk can be used in the following regions or zones:
Hyperdisk Balanced
The following table lists the specific zones within each region that supports Hyperdisk Balanced.
Region | Available Zones |
---|---|
Changhua County, Taiwan—asia-east1 |
asia-east1-a |
asia-east1-b |
|
asia-east1-c |
|
Tokyo, Japan—asia-northeast1 |
asia-northeast1-a |
asia-northeast1-b |
|
asia-northeast1-c |
|
Osaka, Japan—asia-northeast2 |
asia-northeast2-a |
asia-northeast2-b |
|
asia-northeast2-c |
|
Seoul, South Korea—asia-northeast3 |
asia-northeast3-a |
asia-northeast3-b |
|
asia-northeast3-c |
|
Jurong West, Singapore—asia-southeast1 |
asia-southeast1-a |
asia-southeast1-b |
|
asia-southeast1-c |
|
Jakarta, Indonesia—asia-southeast2 |
asia-southeast2-a |
asia-southeast2-c |
|
Mumbai, India—asia-south1 |
asia-south1-a |
asia-south1-b |
|
asia-south1-c |
|
Delhi, India—asia-south2 |
asia-south2-a |
asia-south2-b |
|
Sydney, Australia—australia-southeast1 |
australia-southeast1-a |
australia-southeast1-b |
|
australia-southeast1-c |
|
Melbourne, Australia—australia-southeast2 |
australia-southeast2-b |
australia-southeast2-c |
|
Warsaw, Poland—europe-central2 |
europe-central2-a |
europe-central2-b |
|
Madrid, Spain—europe-southwest1 |
europe-southwest1-a |
europe-southwest1-c |
|
St. Ghislain, Belgium—europe-west1 |
europe-west1-b |
europe-west1-c |
|
europe-west1-d |
|
London, England—europe-west2 |
europe-west2-a |
europe-west2-b |
|
europe-west2-c |
|
Frankfurt, Germany—europe-west3 |
europe-west3-a |
europe-west3-b |
|
europe-west3-c |
|
Eemshaven, Netherlands—europe-west4 |
europe-west4-a |
europe-west4-b |
|
europe-west4-c |
Zurich, Switzerland—europe-west6 |
europe-west6-b |
europe-west6-c |
|
Milan, Italy—europe-west8 |
europe-west8-a |
europe-west8-c |
Paris, France—europe-west9 |
europe-west9-a |
europe-west9-b |
|
europe-west9-c |
|
Turin, Italy—europe-west12 |
europe-west12-a |
europe-west12-b |
|
europe-west12-c |
|
Montréal, Québec—northamerica-northeast1 |
northamerica-northeast1-b |
northamerica-northeast1-c |
|
Toronto, Ontario—northamerica-northeast2 |
northamerica-northeast2-a |
northamerica-northeast2-b |
|
northamerica-northeast2-c |
|
Council Bluffs, Iowa—us-central1 |
us-central1-a |
us-central1-b |
|
us-central1-c |
|
us-central1-f |
|
Moncks Corner, South Carolina—us-east1 |
us-east1-b |
us-east1-c |
|
us-east1-d |
|
Ashburn, Virginia—us-east4 |
us-east4-a |
us-east4-b |
|
us-east4-c |
|
Columbus, Ohio—us-east5 |
us-east5-a |
us-east5-b |
|
Dallas, Texas—us-south1 |
us-south1-b |
The Dalles, Oregon—us-west1 |
us-west1-a |
us-west1-b |
|
Los Angeles, California—us-west2 |
us-west2-b |
us-west2-c |
|
Las Vegas, Nevada—us-west4 |
us-west4-a |
us-west4-b |
|
us-west4-c |
|
Osasco, São Paulo, Brazil—southamerica-east1 |
southamerica-east1-b |
southamerica-east1-c |
|
Santiago, Chile—southamerica-west1 |
southamerica-west1-b |
southamerica-west1-c |
|
Doha, Qatar—me-central1 |
me-central1-b |
me-central1-c |
|
Dammam, Saudi Arabia—me-central2 |
me-central2-a |
me-central2-c |
|
Tel Aviv, Israel—me-west1 |
me-west1-a |
me-west1-c |
Hyperdisk Balanced HA
The following table lists the regions where Hyperdisk Balanced High Availability is available.
- Jurong West, Singapore—
asia-southeast1
- St. Ghislain, Belgium—
europe-west1
- Eemshaven, Netherlands—
europe-west4
- Council Bluffs, Iowa—
us-central1
- Moncks Corner, South Carolina—
us-east1
- Ashburn, Virginia—
us-east4
Hyperdisk Extreme
- Changhua County, Taiwan—
asia-east1
- Tokyo, Japan—
asia-northeast1
- Osaka, Japan—
asia-northeast2
- Seoul, South Korea—
asia-northeast3
- Mumbai, India—
asia-south1
- Delhi, India—
asia-south2
- Jurong West, Singapore—
asia-southeast1
- Jakarta, Indonesia—
asia-southeast2
- Sydney, Australia—
australia-southeast1
- Madrid, Spain—
europe-southwest1
- St. Ghislain, Belgium—
europe-west1
- London, England—
europe-west2
- Frankfurt, Germany—
europe-west3
- Eemshaven, Netherlands—
europe-west4
- Zurich, Switzerland—
europe-west6
- Milan, Italy—
europe-west8
- Paris, France—
europe-west9
- Turin, Italy—
europe-west12-a
andeurope-west12-b
- Tel Aviv, Israel—
me-west1
- Montréal, Québec—
northamerica-northeast1
- Toronto, Ontario—
northamerica-northeast2
- Osasco, São Paulo, Brazil—
southamerica-east1
- Council Bluffs, Iowa—
us-central1
- Moncks Corner, South Carolina—
us-east1
- Ashburn, Virginia—
us-east4
- Columbus, Ohio—
us-east5-b
- The Dalles, Oregon—
us-west1
- Los Angeles, California—
us-west2
- Salt Lake City, Utah—
us-west3
- Las Vegas, Nevada—
us-west4
Hyperdisk ML
Region | Available Zones |
---|---|
Changhua County, Taiwan—asia-east1 |
asia-east1-a |
asia-east1-b |
|
asia-east1-c |
|
Tokyo, Japan—asia-northeast1 |
asia-northeast1-a |
asia-northeast1-b |
|
asia-northeast1-c |
|
Seoul, South Korea—asia-northeast3 |
asia-northeast3-a |
asia-northeast3-b |
|
Jurong West, Singapore—asia-southeast1 |
asia-southeast1-a |
asia-southeast1-b |
|
asia-southeast1-c |
|
Mumbai, India—asia-south1 |
asia-south1-b |
asia-south1-c |
|
St. Ghislain, Belgium—europe-west1 |
europe-west1-b |
europe-west1-c |
|
London, England—europe-west2 |
europe-west2-a |
europe-west2-b |
|
europe-west3-b |
|
Eemshaven, Netherlands—europe-west4 |
europe-west4-a |
europe-west4-b |
|
europe-west4-c |
|
Zurich, Switzerland—europe-west6 |
europe-west6-b |
europe-west6-c |
|
Tel Aviv, Israel—me-west1 |
me-west1-b |
me-west1-c |
|
Council Bluffs, Iowa—us-central1 |
us-central1-a |
us-central1-b |
|
us-central1-c |
|
us-central1-f |
|
Moncks Corner, South Carolina—us-east1 |
us-east1-a |
us-east1-b |
|
us-east1-c |
|
us-east1-d |
|
Ashburn, Virginia—us-east4 |
us-east4-a |
us-east4-b |
|
us-east4-c |
|
Columbus, Ohio—us-east5 |
us-east5-a |
us-east5-b |
|
The Dalles, Oregon—us-west1 |
us-west1-a |
us-west1-b |
|
us-west1-c |
|
Salt Lake City, Utah—us-west3 |
us-west3-b |
Las Vegas, Nevada—us-west4 |
us-west4-a |
us-west4-b |
|
us-west4-c |
Hyperdisk Throughput
- Zone: Mumbai, India—
asia-south1-a
- Region: Jurong West, Singapore—
asia-southeast1
- Region: Eemshaven, Netherlands—
europe-west4
- Region: Council Bluffs, Iowa—
us-central1
- Region: Moncks Corner, South Carolina—
us-east1
- Region: Ashburn, Virginia—
us-east4
About IOPS and throughput provisioning for Hyperdisk
Unlike Persistent Disk, where performance scales automatically with size, with Hyperdisk you can provision performance directly. To provision performance, you select the target performance level for a given volume. Individual volumes have full performance isolation—each volume gets the performance provisioned to it.
About IOPS for Hyperdisk
You can modify the provisioned IOPS for Hyperdisk Balanced, Hyperdisk Balanced High Availability, and Hyperdisk Extreme volumes, but not for Hyperdisk Throughput or Hyperdisk ML volumes.
To reach maximum IOPS and throughput levels offered by Hyperdisk volumes, you must consider the following workload parameters:
- I/O size: Maximum IOPS limits assume that you are using an I/O size of 4 KB or 16 KB. Maximum throughput limits assume that you are using an I/O size of at least 64 KB.
- Queue length: Queue length is the number of pending requests for a volume. To reach maximum performance limits, you must tune your queue length according to the I/O size, IOPS, and latency sensitivity of your workload. Optimal queue length varies for each workload, but typically should be larger than 256.
- Working set size: Working set size is the amount of data of a volume being accessed within a short period of time. To achieve optimal performance, working set sizes must be greater than or equal to 32 GiB.
Multiple attached disks: Hyperdisk volumes share the per-VM maximum IOPS and throughput limits with all Persistent Disk and Hyperdisk volumes attached to the same VM. With multiple attached disks, each disk has performance limits proportional to their share of IOPS in the total provisioned across all attached Hyperdisk volumes. When monitoring the performance of your Hyperdisk volumes, take into account any I/O requests that you are sending to other volumes that are attached to the same VM.
However, the IOPS for Hyperdisk volumes are ultimately capped by per-VM limits for the VM to which your volumes are attached. To review these limits, see Hyperdisk performance limits.
For more information about how to improve performance, see Optimize performance of Hyperdisk.
IOPS for Hyperdisk Balanced and Hyperdisk Balanced High Availability
If you don't specify a disk size or a value for IOPS when creating a Hyperdisk Balanced or Hyperdisk Balanced High Availability volume, the default IOPS value is 3,600 IOPS. If you specify a size for the disk, then the default value depends on the size:
- 6 GiB or less: 500 IOPS per GiB of disk size
- Larger than 6 GiB: The lesser of 3000 + 6 IOPS per GiB of disk size, or 160,000 for Hyperdisk Balanced and 100,000 for Hyperdisk Balanced High Availability
You can provision custom IOPS levels for your Hyperdisk Balanced and Hyperdisk Balanced High Availability volumes. The provisioned IOPS must follow these rules:
- Minimum: The lesser of 3,000 IOPS or 500 IOPS per GiB of disk capacity.
- Maximum: 500 IOPS per GiB of disk capacity, but not more than 160,000 for Hyperdisk Balanced and 100,000 for Hyperdisk Balanced High Availability.
IOPS for Hyperdisk ML
You can't specify a custom IOPS value for a Hyperdisk ML volume. The IOPS for a Hyperdisk ML volume scales with provisioned throughput, at a rate of 16 IOPS per MBps. For example, a Hyperdisk ML volume with 1,000 MBps of throughput can have at most 16,000 IOPS. However, IOPS is ultimately limited by the machine type of the VM to which the Hyperdisk ML volumes are attached.
IOPS for Hyperdisk Extreme
If you don't specify a value for IOPS when creating a Hyperdisk Extreme volume, a default value is used, which is the lesser of 100 IOPS per GiB of disk capacity or the maximum IOPS for the machine type. You can provision custom IOPS levels for your Hyperdisk Extreme volumes. The provisioned IOPS must follow these rules:
- At least 2 IOPS per GiB of disk capacity, but not more than 1000 IOPS per GiB of capacity
- At most 350,000 per volume, depending on the machine type
IOPS for Hyperdisk Throughput
For Hyperdisk Throughput volumes, the IOPS scales with the provisioned throughput, at a rate of 4 IOPS per MBps for random I/O, or 8 IOPS per MBps for sequential I/O. However, IOPS is ultimately limited by the machine type of the VM to which your Hyperdisk Throughput volumes are attached.
About throughput for Hyperdisk
You can modify the provisioned throughput for the following Hyperdisk volumes:
- Hyperdisk Balanced
- Hyperdisk Balanced High Availability
- Hyperdisk Throughput
- Hyperdisk ML
You can't modify the provisioned throughput for Hyperdisk Extreme volumes.
To reach maximum throughput levels offered by Hyperdisk volumes, you must consider the following workload parameters:
- I/O size: Maximum throughput limits assume that you are using the
following:
- Hyperdisk Throughput: a sequential I/O size of at least 128 KB, or a random I/O size of at least 256 KB.
- Hyperdisk Balanced or Hyperdisk Balanced High Availability: an I/O size of at least 64 KB
- Queue length: Queue length is the number of pending requests for a volume. To reach maximum performance limits, you must tune your queue length according to the I/O size, IOPS, and latency sensitivity of your workload. Optimal queue length varies for each workload, but typically should be larger than 256.
- Shared disks: If supported, you can attach the same disk to multiple VMs to reach the provisioned throughput for the disk. For example, suppose you have a Hyperdisk ML volume with a provisioned throughput limit of 50,000 MBps. If you attach the disk to one A3 VM, the maximum throughput achievable is the limit for an A3 VM: 4,000 MBps. To reach the disk's provisioned throughput limit, attach the same disk to 15 A3 VMs.
- Multiple attached disks: If you attach more than one Hyperdisk volume to your VM, and the total throughput provisioned for all Hyperdisk volumes exceeds the limits documented for the machine type, the total disk performance won't exceed the limit for the machine type. For example, to reach the maximum throughput for an A3 VM (3,000 MBps), you must attach at least 5 Hyperdisk Throughput volumes to the VM because each Hyperdisk Throughput volume has a maximum throughput limit of 600 MBps.
For more information, see Optimize performance of Hyperdisk.
Throughput for Hyperdisk Balanced and Hyperdisk Balanced High Availability
If you don't specify a value for throughput or the disk size, then the default value for throughput is 290 MBps. If you specify a size for the disk, then the default value depends on the size:
- 6 GiB or less: 140 MBps
- Larger than 6 GiB: The lesser of
((6 * disk size in GiB) / 4) + 140
, or 2,400 MBps for Hyperdisk Balanced or 1,200 MBps for Hyperdisk Balanced High Availability
You can provision custom throughput levels for your Hyperdisk Balanced and Hyperdisk Balanced High Availability volumes. The provisioned throughput for each disk must follow these rules:
- Minimum: The greater of 140 MBps or the configured IOPS divided by 256.
- Maximum: The greater of 2,400 MBps for Hyperdisk Balanced, 1,200 MBps for Hyperdisk Balanced High Availability, or the provisioned IOPS divided by 4.
Throughput for Hyperdisk ML
You can provision at most 1,200,000 MBps of throughput for a single Hyperdisk ML volume and at least 400 MBps of throughput. The provisioned throughput is shared among all attached VMs. This means the total throughput consumed by all VMs that the disk is attached to can't exceed the provisioned throughput.
The following rules apply to the provisioned throughput:
Number of VM attachments: If the volume is attached to more than 20 VMs, then you must provision at least 100 MBps of throughput for each VM. For example, if you attach a disk to 500 VMs, the disk must be provisioned with at least 50 GBps of throughput.
Default throughput: The default throughput is the greater of 24 MBps per GiB of disk size, or 400 MBps.
For example, a 10 GiB Hyperdisk ML volume is provisioned by default with 400 MBps of throughput, because
24 MBps * 10 = 240
, which is less than the 400 MBps minimum.Throughput limits per GiB: You can provision between 0.12 MBps and 1600 MBps per GiB of disk capacity. However, the provisioned throughput must be between 400 MBps and 1,200,000 MBps.
For example, for a 10 GiB Hyperdisk ML volume, the highest throughput for the disk is 16,000 MBps (
1,600 MBps * 10
). Because of the size of the disk, you can't provision the maximum throughput of 1,200,000 MBps.On the other hand, for a 20 TiB disk, the highest throughput is 1,200,000 MBps, because 32,000,000 MBps (
1,600 MBps * 20,000
) exceeds the 1,200,000 MBps per disk maximum.
Throughput for Hyperdisk Throughput
You can provision throughput levels for Hyperdisk Throughput volumes. The provisioned throughput must follow these rules:
- At most 600 MBps per volume.
- At least 10 MBps per TiB of capacity, but not more than 90 MBps per TiB of capacity.
If you don't specify a throughput value, Compute Engine provisions the disk with 90 MBps per TiB of disk capacity, up to a maximum of 600 MBps.
For Hyperdisk Throughput volumes, throughput doesn't automatically scale with the size of the volume or number of provisioned vCPUs. You must specify the throughput level you want for each Hyperdisk Throughput disk.
Throughput for Hyperdisk Extreme
For Hyperdisk Extreme volumes, throughput scales with the number of IOPS you provision at a rate of 256 KBps of throughput per I/O. However, throughput is ultimately capped by per-VM limits that depend on the number of vCPUs on the VM to which your Hyperdisk Extreme volumes are attached.
Throughput for Hyperdisk Extreme volumes is not full duplex. The maximum throughput limits listed in Hyperdisk performance limits apply to the sum total of read and write throughput.
Pricing
You are billed for the total provisioned capacity of your Hyperdisk volumes until you delete them. You are charged per GiB per month. Additionally, you are billed for the following:
- Hyperdisk Balanced charges a monthly rate for the provisioned IOPS and provisioned throughput (in MBps) in excess of the baseline values of 3,000 IOPS and 140 MBps throughput.
- Hyperdisk Extreme charges a monthly rate based on the provisioned IOPS.
- Hyperdisk ML charges a monthly rate based on the provisioned throughput (in MBps). There is no additional charge for attaching multiple VMs to a single Hyperdisk ML volume.
- Hyperdisk Throughput charges a monthly rate based on the provisioned throughput (in MBps).
Because the data for regional disks is written to two locations, the cost of Hyperdisk Balanced High Availability storage is twice the cost of Hyperdisk Balanced storage.
For more pricing information, see Disk pricing.
Hyperdisk and committed use discounts
Hyperdisk volumes are not eligible for:
- Resource-based committed use discounts (CUDs)
- Sustained use discounts (SUDs)
Hyperdisk and preemptible VM instances
Hyperdisk can be used with Spot VMs (or preemptible VMs). However, there are no discounted spot prices for Hyperdisk.
What's next?
- Learn how to create a Hyperdisk volume.
- Learn how to clone a Hyperdisk volume.
- Learn about synchronous disk replication with Hyperdisk Balanced High Availability.
- Learn about Hyperdisk Storage Pools.
- Review Disk pricing information.
- Learn how to optimize performance of Hyperdisk.
- Learn more about Cloud locations.