This document in the Google Cloud Architecture Framework provides recommendations to help you optimize the performance of your storage resources in Google Cloud.
Cloud Storage
This section provides best practices to help you optimize the performance of your Cloud Storage operations.
Assess bucket performance
Assess the performance of your Cloud Storage buckets by using the
gsutil perfdiag
command. This command tests the performance of the specified bucket by sending a
series of read and write requests with files of different sizes. You can tune
the test to match the usage pattern of your applications. Use the diagnostic
report that the command generates to set performance expectations and identify
potential bottlenecks.
Cache frequently accessed objects
To improve the read latency for frequently accessed objects that are publicly accessible, you can configure such objects to be cached. Although caching can improve performance, stale content could be served if a cache has the old version of an object.
Scale requests efficiently
As the request rate for a bucket increases, Cloud Storage automatically increases the I/O capacity for the bucket by distributing the request load across multiple servers. To achieve optimal performance when scaling requests, follow the best practices for ramping up request rates and distributing load evenly.
Enable multithreading and multiprocessing
When you use gsutil to upload numerous small files, you can improve the
performance of the operation by using the -m option. This option causes the
upload request to be implemented as a batched, parallel (that is, multithreaded
and multiprocessing) operation. Use this option only when you perform operations
over a fast network connection. For more information, use the command
gsutil help options to see the global command-line options for gsutil.
Upload large files as composites
To upload large files, you can use a strategy called parallel composite uploads. With this strategy, the large file is split into chunks, which are uploaded in parallel and then recomposed in the cloud. Parallel composite uploads can be faster than regular upload operations when network bandwidth and disk speed are not limiting factors. However, this strategy has some limitations and cost implications. For more information, see Parallel composite uploads.
Persistent disks and local SSDs
This section provides best practices to help you optimize the performance of your Persistent Disks and Local SSDs that are attached to Compute Engine VMs.
The performance of persistent disks and local SSDs depends on the disk type and size, VM machine type, and number of vCPUs. Use the following guidelines to manage the performance of your persistent disks and local SSDs:
- When you provision block storage for your VMs, choose disk types and disk sizes that are appropriate for your workload. For more information, see Configure disks to meet performance requirements.
- Benchmark the block storage performance. For more information, see the following documentation:
- Optimize the performance of your persistent disks and local SSDs. For more information, see the following documentation:
Filestore
This section provides best practices to help you optimize the performance of your Filestore instances. You can use Filestore to provision fully managed Network File System (NFS) file servers for Compute Engine VMs and GKE clusters.
- When you provision a Filestore instance, choose a service tier that meets the performance and capacity requirements of your workload.
- For client VMs that run cache-dependent workloads, use a machine type that helps optimize the network performance of the Filestore instance. For more information, see Recommended client machine type.
- To optimize the performance of Filestore instances for client VMs that run Linux, Google recommends specific NFS mount settings. For more information, see Linux client mount options.
- To minimize network latency, provision your Filestore instances in regions and zones that are close to where you plan to use the instances.
- Monitor the performance of your Filestore instances, and set up alerts by using Cloud Monitoring.
What's next
Review the best practices for optimizing the performance of your compute, networking, database, and analytics resources:
- Optimize compute performance.
- Optimize networking performance.
- Optimize database performance.
- Optimize analytics performance.