This page describes the expected average performance and the recommended performance settings for Filestore. It also shows you how you can test the performance of your Filestore instances.
Expected performance
Each Filestore service tier provides a different level of performance. The performance of any given instance might vary from the expected numbers due to various factors, such as the use of caching, the number of client VMs, the machine type of the client VMs, and the workload tested.
The following tables show the expected performance of Filestore instances based on its service tier and configured capacity:
| Performance | Capacity | Read and write IOPS | Read and write throughput (MiB/s) |
|---|---|---|---|
BASIC_HDD |
1 TiB to 10 TiB | 600/1,000 | 100/100 |
BASIC_HDD |
10 TiB to 63.9 TiB | 1,000/5,000 | 180/120 |
BASIC_SSD |
2.5 TiB to 63.9 TiB | 60,000/25,000 | 1,200/350 |
ZONAL |
1 TiB | 9,200/2,600 | 260/88 |
ZONAL |
9.75 TiB | 89,700/25,350 | 2,535/858 |
ZONAL |
10 TiB | 92,000/26,000 | 2,600/880 |
ZONAL |
100 TiB | 920,000/260,000 | 26,000/8,800 |
REGIONAL |
1 TiB | 12,000/4,000 | 120/100 |
REGIONAL |
9.75 TiB | 117,000/39,000 | 1,170/975 |
REGIONAL |
10 TiB | 92,000/26,000 | 2,600/880 |
REGIONAL |
100 TiB | 920,000/260,000 | 26,000/8,800 |
ENTERPRISE |
1 TiB | 12,000/4,000 | 120/100 |
ENTERPRISE |
10 TiB | 120,000/40,000 | 1,200/1,000 |
The previous table shows the expected performance at the minimum and maximum capacity for each service tier. Between these limits, performance scales linearly as the capacity scales. For example, if you double your enterprise instance capacity from 1 TiB to 2 TiB, the expected performance of the instance doubles from 12,000/4,000 read and write IOPS to 24,000/8,000 read and write IOPS.
In single- and few-client scenarios, you must increase the number of TCP
connections with the
nconnect
mount option to achieve maximum NFS performance. We recommend specifying up to
7 connections for the zonal service tier and up to 2 connections for the
regional and enterprise tiers. In general, the larger the file share capacity
and the fewer the connecting client VMs, the more performance you gain by
specifying additional connections with nconnect.
Recommended client machine type
We recommend having a Compute Engine machine type, such as n2-standard-8,
that provides an egress bandwidth of 16 Gbps. This egress bandwidth allows the
client to achieve approximately 16 Gbps read bandwidth for cache-friendly
workloads. For additional context, see Network bandwidth.
Linux client mount options
We recommend using the following NFS mount options, especially hard
mount, async, and the rsize and wsize options, to achieve the best
performance on Linux client VM instances. For more information on NFS mount
options, see nfs.
| Default option | Description |
|---|---|
hard |
The NFS client retries NFS requests indefinitely. |
timeo=600 |
The NFS client waits 600 deciseconds (60 seconds) before retrying an NFS request. |
retrans=3 |
The NFS client attempts NFS requests three times before taking further recovery action. |
rsize=262144 |
The NFS client can receive a maximum of 262,144 bytes from the NFS server per READ request. Note: For basic-tier instances, set the rsize value to 1048576. |
wsize=1048576 |
The NFS client can send a maximum of 1,048,576 bytes (1 MiB) to the NFS server per WRITE request. |
resvport |
The NFS client uses a privileged source port when communicating with the NFS server for this mount point. |
async |
The NFS client delays sending application writes to the NFS server until certain conditions are met. Caution: Using the sync option significantly reduces performance. |
Single and multiple client VM performance
Filestore's scalable service tiers are performance optimized for multiple client VMs, not a single client VM.
For zonal, regional, and enterprise instances, at least four client VMs are needed to take advantage of full performance. This ensures that all of the VMs in the underlying Filestore cluster are fully utilized.
For added context, the smallest scalable Filestore cluster has four
VMs. Each client VM communicates with just one Filestore cluster
VM, regardless of the number of NFS connections per client specified using the
nconnect
mount option. If using a single client VM, read and write operations are only
performed from a single Filestore cluster VM.
Improve performance across Google Cloud resources
Operations across multiple Google Cloud resources, such as copying data from Cloud Storage to a Filestore instance using the gcloud CLI, can be slow. To help mitigate performance issues, try the following:
Ensure the Cloud Storage bucket, client VM, and Filestore instance all reside in the same region.
Dual-regions provide a maximally-performant option for data stored in Cloud Storage. If using this option, ensure the other resources reside in one of the single regions contained in the dual-region. For example, if your Cloud Storage data resides in
us-central1,us-west1, ensure that your client VM and Filestore instance reside inus-central1.For a point of reference, verify the performance of a PD-attached VM and compare to the performance of a Filestore instance.
If the PD-attached VM is similar or slower in performance when compared to the Filestore instance, this might indicate a performance bottleneck unrelated to Filestore. To improve the baseline performance of your non-Filestore resources, you can adjust the gcloud CLI properties associated with parallel composite uploads. For more information see How tools and APIs use parallel composite uploads.
If the performance of the Filestore instance is notably slower than the PD-attached VM, try spreading the operation over multiple VMs.
This helps to improve performance of read operations from Cloud Storage.
For zonal, regional, and enterprise instances, at least four client VMs are needed to take advantage of full performance. This ensures that all of the VMs in the underlying Filestore cluster are fully utilized. For more information, see Single and multiple client VM performance.
Testing performance
If you are using Linux, you can use the fio tool to benchmark read and write throughput and IOPS for basic tier instances. Note that this method for benchmarking performance is not recommended for zonal, regional, and enterprise instances.
The examples in this section show common benchmarks you might want to run. You may need to run fio from multiple client VM instances to achieve maximum performance.
The following example benchmarks maximum write throughput:
fio --ioengine=libaio --filesize=32G --ramp_time=2s \ --runtime=5m --numjobs=16 --direct=1 --verify=0 --randrepeat=0 \ --group_reporting --directory=/mnt/nfs \ --name=write --blocksize=1m --iodepth=64 --readwrite=write
The following example benchmarks maximum write IOPS:
fio --ioengine=libaio --filesize=32G --ramp_time=2s \ --runtime=5m --numjobs=16 --direct=1 --verify=0 --randrepeat=0 \ --group_reporting --directory=/mnt/nfs \ --name=randwrite --blocksize=4k --iodepth=256 --readwrite=randwrite
The following example benchmarks maximum read throughput:
fio --ioengine=libaio --filesize=32G --ramp_time=2s \ --runtime=5m --numjobs=16 --direct=1 --verify=0 --randrepeat=0 \ --group_reporting --directory=/mnt/nfs \ --name=read --blocksize=1m --iodepth=64 --readwrite=read
The following example benchmarks maximum read IOPS:
fio --ioengine=libaio --filesize=32G --ramp_time=2s \ --runtime=5m --numjobs=16 --direct=1 --verify=0 --randrepeat=0 \ --group_reporting --directory=/mnt/nfs \ --name=randread --blocksize=4k --iodepth=256 --readwrite=randread