Block storage performance

Overview

This page discusses the many factors that determine the performance of the block storage volumes that you attach to your virtual machine (VM) instances. Before you begin, consider the following:

  • To reach the performance limits of your persistent disks, use a high I/O queue depth (32 or higher). Persistent disks are networked storage and generally have higher latency compared to physical disks or local SSDs.

  • Make sure that your application is issuing enough I/Os to saturate your disk.

  • For workloads that primarily involve small (4 KB to 16 KB) random I/Os, the limiting performance factor is random input/output operations per second (IOPS).

  • For workloads that primarily involve sequential or large (256 KB to 1 MB) random I/Os, the limiting performance factor is throughput.

Choose a storage option

You can provide several different types of block storage for your instances to use. Attaching a persistent disk to multiple virtual machines does not affect aggregate performance or cost. Each machine gets a share of the per-disk performance limit.

When you configure a zonal or regional persistent disk, you must select one of the following disk types:

  • Standard persistent disks (pd-standard) are suited for large data processing workloads that primarily use sequential I/Os.
  • Balanced persistent disks (pd-balanced) are an alternative to SSD persistent disks that balance performance and cost. With the same maximum IOPS as SSD persistent disks and lower IOPS per GB, balanced PD offers performance levels suitable for most general purpose applications at a price point between that of standard and SSD persistent disks.
  • SSD persistent disks (pd-ssd) are suited for enterprise applications and high-performance database needs that require lower latency and more IOPS than standard persistent disks provide. SSD persistent disks are designed for single-digit millisecond latencies; the observed latency is app specific.
Zonal
standard
PD		 Regional
standard
PD		 Zonal
balanced
PD		 Regional
balanced
PD		 Zonal
SSD PD		 Regional
SSD PD		 Local SSD (SCSI) Local SSD (NVMe)
Maximum sustained IOPS
Read IOPS per GB 0.75 0.75 6 6 30 30 – –
Write IOPS per GB 1.5 1.5 6 6 30 30 – –
Read IOPS per instance 7,500* 3,000* 15,000–80,000* 15,000–80,000* 15,000–100,000* 15,000–100,000* 900,000 2,400,000
Write IOPS per instance 15,000* 15,000* 15,000–30,000* 15,000–30,000* 15,000–100,000* 15,000–100,000* 800,000 1,200,000
Maximum sustained throughput (MB/s)
Read throughput per GB 0.12 0.12 0.28 0.28 0.48 0.48 – –
Write throughput per GB 0.12 0.12 0.28 0.28 0.48 0.48 – –
Read throughput per instance 240–1,200* 240–1,200* 240–1,200* 240–1,200* 240–1,200* 240–1,200* 9,360 9,360
Write throughput per instance 76–400** 38–200** 240–1,200* 120–600* 240–1,200* 120–600* 4,680 4,680
* Persistent disk IOPS and throughput performance depends on disk size, instance vCPU count, and I/O block size, among other factors.
** Persistent disks can achieve greater throughput performance on instances with more vCPUs. Read Network egress caps on write throughput.

Note that SSD read bandwidth and IOPS consistency near the maximum performance limits depend largely on network ingress utilization. Some variability in the performance limits is to be expected, especially when operating near the maximum IOPS limits with an I/O size of 16 KB.

Compare persistent disk to a physical hard drive

When you specify the size of your persistent disks, consider how these disks compare to traditional physical hard drives. The following tables compare standard persistent disks and SSD persistent disks to the typical performance that you would expect from a 7200 RPM SATA drive, which typically achieves 75 IOPS or 120 MB per second.

I/O type I/O pattern Size required to match a 7200 RPM SATA drive (GB)
Standard persistent disk Balanced persistent disk SSD persistent disk
Small random reads 75 small random reads 100 12 3
Small random writes 75 small random writes 50 12 3
Streaming large reads 120 MB/s streaming reads 1,000 428 250
Streaming large writes 120 MB/s streaming writes 1,000 428 250

Configure your persistent disks and instances

Persistent disk performance scales with the size of the disk and with the number of vCPUs on your VM instance.

Performance scales until it reaches either the limits of the disk or the limits of the VM instance to which the disk is attached. The VM instance limits are determined by the machine type and the number of vCPUs on the instance.

For example, consider a 1,000 GB SSD persistent disk attached to an instance with an N2 machine type and 4 vCPUs. The read limit based solely on the size of the disk is 30,000 IOPS. However, because the instance has 4 vCPUs, the read limit is restricted to 15,000 IOPS.

The following tables show performance by disk size in increments where performance changes significantly. However, you can specify a disk size in any 1 GB increment. Configure your disks and VM instances according to your performance requirements.

Performance by disk size

Standard persistent disk

IOPS Sustained throughput (MB/s)
Disk size (GB) Read
(<=16 KB per I/O)		 Write
(<=8 KB per I/O)		 Write
(16 KB per I/O)		 Read Write
10 * * * * *
32 24 48 48 3 3
64 48 96 96 7 7
128 96 192 192 15 15
256 192 384 384 30 30
512 384 768 768 61 61
1,000 750 1,500 1,500 120 120
1,500 1,125 2,250 2,250 180 180
2,048 1,536 3,072 3,072 245 245
4,000 3,000 6,000 6,000 480 400
5,000 3,750 7,500 7,500 600 400
8,192 6,144 12,288 7,500 983 400
10,000–
65,536		 7,500 15,000 7,500 1,200 400

* Use this disk size only for boot disks, as the performance limits corresponding to this disk size are very low but sufficient to quickly boot a VM.

SSD persistent disk

IOPS Sustained throughput (MB/s)
Disk size (GB) Read
(<=8 KB per I/O)		 Read
(<=16 KB per I/O)		 Write
(<=8 KB per I/O)		 Write
(16 KB per I/O)		 Read Write
10 300 300 300 300 4.8 4.8
32 960 960 960 960 15 15
64 1,920 1,920 1,920 1,920 30 30
128 3,840 3,840 3,840 3,840 61 61
256 7,680 7,680 7,680 7,680 122 122
500 15,000 15,000 15,000 15,000 240 240
834 25,000 25,000 25,000 25,000 400 400
1,000 30,000 30,000 30,000 25,000 480 480
1,334 40,000 40,000 40,000 40,000 640 640
1,667 50,000 50,000 50,000 50,000 800 800
2,048 60,000 60,000 60,000 60,000 983 983
3,500–65,536 100,000 75,000 100,000* 75,000 1,200 1,200

*To reach this limit on a regional SSD persistent disk, you must use a physical block size of 4 KB.

Machine type and vCPU count

Review performance and throttling metrics

You can review persistent disk performance metrics in Cloud Monitoring, Google Cloud's integrated monitoring solution.

Several of these metrics are useful for understanding if and when your disks are being throttled. Throttling is intended to smooth out bursty I/Os. With throttling, bursty I/Os can be spread out over a period of time, so that the performance limits of your disk can be met but not exceeded at any given instant.

If your workload has a bursty I/O usage pattern, expect to see bursts in throttled bytes corresponding to bursts in read or written bytes. Similarly, expect to see bursts in throttled operations corresponding to bursts in read/write operations.

If your disk limit is 1000 write IOPS, the disk will accept a write request every 1 millisecond. If you issue write requests faster than that, a small delay will be introduced to spread the requests 1 millisecond apart. The IOPS and throughput limits discussed on this page are enforced at all times, not on a per-minute or per-second basis.

Databases are a common example of bursty workloads. Databases tend to have short microbursts of I/O operations, which lead to temporary increases in queue depth. Higher queue depth can result in higher latency because more outstanding I/O operation requests are waiting in queue.

If your workload has a uniform I/O usage pattern and you are continuously reaching the performance limits of your disk, you can expect to see uniform levels of throttled bytes and operations.

To learn more, see Reviewing persistent disk performance metrics.

Optimize disk performance

To increase disk performance, start with the following steps:

  • Resize your persistent disks to increase the per-disk IOPS and throughput limits. Persistent disks do not have any reserved, unusable capacity, so you can use the full disk without performance degredation. However, certain file system and applications might perform worse as the disk becomes full, so you might need to consider increasing the size of your disk to avoid this.

  • Change the machine type and number of vCPUs on the instance to increase the per-instance IOPS and throughput limits.

After you ensure that any bottlenecks are not due to the disk size or machine type of the VM, your app and operating system might still need some tuning. See Optimizing persistent disk performance and Optimizing local SSD performance.

Other factors that affect performance

What's next