Block storage performance

Overview

This page discusses the many factors that determine the performance of the block storage volumes that you attach to your virtual machine (VM) instances. Before you begin, consider the following:

  • Persistent disks are networked storage and generally have higher latency compared to physical disks or local SSDs. To reach the maximum performance limits of your persistent disks, you must issue enough I/O requests in parallel. To check if you're using a high enough queue depth to reach your required performance levels, see I/O queue depth.

  • Make sure that your application is issuing enough I/Os to saturate your disk.

  • For workloads that primarily involve small (from 4 KB to 16 KB) random I/Os, the limiting performance factor is random input/output operations per second (IOPS).

  • For workloads that primarily involve sequential or large (256 KB to 1 MB) random I/Os, the limiting performance factor is throughput.

Choose a storage option

You can provide several different types of block storage for your instances to use. Attaching a persistent disk to multiple virtual machines does not affect aggregate performance or cost. Each machine gets a share of the per-disk performance limit.

When you configure a zonal or regional persistent disk, you can select one of the following disk types:

  • Standard persistent disks (pd-standard) are suited for large data processing workloads that primarily use sequential I/Os.
  • Balanced persistent disks (pd-balanced) are an alternative to SSD persistent disks that balance performance and cost. With the same maximum IOPS as SSD persistent disks and lower IOPS per GB, a balanced persistent disk offers performance levels suitable for most general-purpose applications at a price point between that of standard and SSD persistent disks.
  • SSD persistent disks (pd-ssd) are suited for enterprise applications and high-performance database needs that require lower latency and more IOPS than standard persistent disks provide. SSD persistent disks are designed for single-digit millisecond latencies; the observed latency is app specific.
  • Extreme persistent disks (pd-extreme) offer consistently high performance for both random access workloads and bulk throughput. They are designed for high-end database workloads, such as Oracle or SAP HANA. Unlike other disk types, you can provision your desired IOPS. For more information, see Extreme persistent disks.

If you create a disk in the Cloud Console, the default disk type is pd-balanced. If you create a disk using the gcloud tool or the Compute Engine API, the default disk type is pd-standard.

Zonal
standard
PD		 Regional
standard
PD		 Zonal
balanced
PD		 Regional
balanced
PD		 Zonal
SSD PD		 Regional
SSD PD		 Zonal
extreme PD		 Local SSD (SCSI) Local SSD (NVMe)
Maximum sustained IOPS
Read IOPS per GB 0.75 0.75 6 6 30 30 – – –
Write IOPS per GB 1.5 1.5 6 6 30 30 – – –
Read IOPS per instance 7,500* 7,500* 15,000–80,000* 15,000–60,000* 15,000–100,000* 15,000–60,000* 15,000–120,000* 900,000 2,400,000
Write IOPS per instance 15,000* 15,000* 15,000–80,000* 15,000–30,000* 15,000–100,000* 15,000–30,000* 15,000–120,000* 800,000 1,200,000
Maximum sustained throughput (MB/s)
Read throughput per GB 0.12 0.12 0.28 0.28 0.48 0.48 – – –
Write throughput per GB 0.12 0.12 0.28 0.28 0.48 0.48 – – –
Read throughput per instance 240–1,200* 240–1,200* 240–1,200* 240–1,200* 240–1,200* 240–1,200* 240–2,200** 9,360 9,360
Write throughput per instance 76–400** 38–200** 240–1,200* 120–600* 240–1,200* 120–600* 240–2,200** 4,680 4,680
* Persistent disk IOPS and throughput performance depends on disk size, instance vCPU count, and I/O block size, among other factors.
** Persistent disks can achieve greater throughput performance on instances with more vCPUs. Read Network egress caps on write throughput.
Attaching a disk to multiple virtual machine instances in read-only mode does not affect aggregate performance or cost. Each machine gets a share of the per-disk performance limit.

Persistent disk I/O operations share a common path with vNIC network traffic within your VM's hypervisor. Therefore, if your VM has significant network traffic, the actual read bandwidth and IOPS consistency might be less than the listed maximum limits. Some variability in the performance limits is to be expected, especially when operating near the maximum IOPS limits with an I/O size of 16 KB. For a summary of bandwidth expectations, see Bandwidth summary table.

Performance of persistent disks in multi-writer mode

Persistent disks created in multi-writer mode have specific IOPS and throughput limits.

Zonal SSD persistent disk multi-writer mode
Maximum sustained IOPS
Read IOPS per GB 30
Write IOPS per GB 30
Read IOPS per instance 15,000–60,000*
Write IOPS per instance 15,000–60,000*
Maximum sustained throughput (MB/s)
Read throughput per GB 0.48
Write throughput per GB 0.48
Read throughput per instance 240–1,200*
Write throughput per instance 240–1,200*
* Persistent disk IOPS and throughput performance depends on disk size, instance vCPU count, and I/O block size, among other factors.
Attaching a multi-writer disk to multiple virtual machine instances does not affect aggregate performance or cost. Each machine gets a share of the per-disk performance limit.

To learn how to share persistent disks between multiple VMs, see Sharing persistent disks between VMs.

Compare persistent disk to a physical hard drive

When you specify the size of your persistent disks, consider how these disks compare to traditional physical hard drives. The following tables compare standard persistent disks and SSD persistent disks to the typical performance that you would expect from a 7200 RPM SATA drive, which typically achieves 75 IOPS or 120 MB per second.

I/O type I/O pattern Size required to match a 7200 RPM SATA drive (GB)
Standard persistent disk Balanced persistent disk SSD persistent disk
Small random reads 75 small random reads 100 12 3
Small random writes 75 small random writes 50 12 3
Streaming large reads 120 MBps streaming reads 1,000 428 250
Streaming large writes 120 MBps streaming writes 1,000 428 250

Configure your persistent disks and instances

Persistent disk performance scales with the size of the disk and with the number of vCPUs on your VM instance.

Performance scales until it reaches either the limits of the disk or the limits of the VM instance to which the disk is attached. The machine type and the number of vCPUs on the instance determine the VM instance limits.

For example, consider a 1,000 GB SSD persistent disk attached to an instance with an N2 machine type and 4 vCPUs. The read limit based solely on the size of the disk is 30,000 IOPS. However, because the instance has 4 vCPUs, the read limit is restricted to 15,000 IOPS.

The following tables show performance by disk size in increments where performance changes significantly. However, you can specify a disk size in any 1 GB increment. Configure your disks and VM instances according to your performance requirements.

Performance by disk size

Standard persistent disk

IOPS Sustained throughput (MBps)
Disk size (GB) Read
(<=16 KB per I/O)		 Write
(<=8 KB per I/O)		 Write
(16 KB per I/O)		 Read Write
10 * * * * *
32 24 48 48 3 3
64 48 96 96 7 7
128 96 192 192 15 15
256 192 384 384 30 30
512 384 768 768 61 61
1,000 750 1,500 1,500 120 120
1,500 1,125 2,250 2,250 180 180
2,048 1,536 3,072 3,072 245 245
4,000 3,000 6,000 6,000 480 400
5,000 3,750 7,500 7,500 600 400
8,192 6,144 12,288 7,500 983 400
10,000–
65,536		 7,500 15,000 7,500 1,200 400

* Use this disk size only for boot disks, as the performance limits corresponding to this disk size are low but sufficient to quickly boot a VM.

SSD persistent disk

IOPS Sustained throughput (MBps)
Disk size (GB) Read
(<=8 KB per I/O)		 Read
(<=16 KB per I/O)		 Write
(<=8 KB per I/O)		 Write
(16 KB per I/O)		 Read Write
10 300 300 300 300 4.8 4.8
32 960 960 960 960 15 15
64 1,920 1,920 1,920 1,920 30 30
128 3,840 3,840 3,840 3,840 61 61
256 7,680 7,680 7,680 7,680 122 122
500 15,000 15,000 15,000 15,000 240 240
834 25,000 25,000 25,000 25,000 400 400
1,000 30,000 30,000 30,000 25,000 480 480
1,334 40,000 40,000 40,000 40,000 640 640
1,667 50,000 50,000 50,000 50,000 800 800
2,048 60,000 60,000 60,000 60,000 983 983
3,500–65,536 100,000 75,000 100,000 75,000 1,200 1,200

Machine type and vCPU count

Review performance and throttling metrics

You can review persistent disk performance metrics in Cloud Monitoring, Google Cloud's integrated monitoring solution.

If and when your disks are being throttled, several of these metrics are useful to understand. Throttling is intended to smooth out bursty I/Os. With throttling, bursty I/Os can be spread out over a period, so that the performance limits of your disk can be met but not exceeded at any given instant.

If your workload has a bursty I/O usage pattern, expect to see bursts in throttled bytes corresponding to bursts in read or written bytes. Similarly, expect to see bursts in throttled operations corresponding to bursts in read/write operations.

If your disk limit is 1,000 write IOPS, the disk accepts a write request every 1 millisecond. If you issue faster write requests, then a small delay is introduced to spread the requests 1 millisecond apart. The IOPS and throughput limits discussed in this document are enforced all the time, not on a per-minute or per-second basis.

Databases are a common example of bursty workloads. Databases tend to have short microbursts of I/O operations, which lead to temporary increases in queue depth. Higher queue depth can result in higher latency because more outstanding I/O operation requests are waiting in queue.

If your workload has a uniform I/O usage pattern and you are continuously reaching the performance limits of your disk, you can expect to see uniform levels of throttled bytes and operations.

To learn more, see Reviewing persistent disk performance metrics.

Optimize disk performance

To increase disk performance, start with the following steps:

  • Resize your persistent disks to increase the per-disk IOPS and throughput limits. Persistent disks do not have any reserved, unusable capacity, so you can use the full disk without performance degradation. However, certain file system and applications might perform worse as the disk becomes full, so you might need to consider increasing the size of your disk to avoid such situations.

  • Change the machine type and number of vCPUs on the instance to increase the per-instance IOPS and throughput limits.

After you ensure that any bottlenecks are not due to the disk size or machine type of the VM, your app and operating system might still need some tuning. See Optimizing persistent disk performance and Optimizing local SSD performance.

Other factors that affect performance

What's next