Optimize storage performance

Stay organized with collections Save and categorize content based on your preferences.

This document in the Google Cloud Architecture Framework provides recommendations to help you optimize the performance of your storage resources in Google Cloud.

Cloud Storage

This section provides best practices to help you optimize the performance of your Cloud Storage operations.

Assess bucket performance

Assess the performance of your Cloud Storage buckets by using the gsutil perfdiag command. This command tests the performance of the specified bucket by sending a series of read and write requests with files of different sizes. You can tune the test to match the usage pattern of your applications. Use the diagnostic report that the command generates to set performance expectations and identify potential bottlenecks.

Cache frequently accessed objects

To improve the read latency for frequently accessed objects that are publicly accessible, you can configure such objects to be cached. Although caching can improve performance, stale content could be served if a cache has the old version of an object.

Scale requests efficiently

As the request rate for a bucket increases, Cloud Storage automatically increases the I/O capacity for the bucket by distributing the request load across multiple servers. To achieve optimal performance when scaling requests, follow the best practices for ramping up request rates and distributing load evenly.

Enable multithreading and multiprocessing

When you use gsutil to upload numerous small files, you can improve the performance of the operation by using the -m option. This option causes the upload request to be implemented as a batched, parallel (that is, multithreaded and multiprocessing) operation. Use this option only when you perform operations over a fast network connection. For more information, see the documentation for the -m option in Global Command-Line Options.

Upload large files as composites

To upload large files, you can use a strategy called parallel composite uploads. With this strategy, the large file is split into chunks, which are uploaded in parallel and then recomposed in the cloud. Parallel composite uploads can be faster than regular upload operations when network bandwidth and disk speed are not limiting factors. However, this strategy has some limitations and cost implications. For more information, see Parallel composite uploads.

Persistent disks and local SSDs

This section provides best practices to help you optimize the performance of your Persistent Disks and Local SSDs that are attached to Compute Engine VMs.

The performance of persistent disks and local SSDs depends on the disk type and size, VM machine type, and number of vCPUs. Use the following guidelines to manage the performance of your persistent disks and local SSDs:

Filestore

This section provides best practices to help you optimize the performance of your Filestore instances. You can use Filestore to provision fully managed Network File System (NFS) file servers for Compute Engine VMs and GKE clusters.

  • When you provision a Filestore instance, choose a service tier that meets the performance and capacity requirements of your workload.
  • For client VMs that run cache-dependent workloads, use a machine type that helps optimize the network performance of the Filestore instance. For more information, see Recommended client machine type.
  • To optimize the performance of Filestore instances for client VMs that run Linux, Google recommends specific NFS mount settings. For more information, see Linux client mount options.
  • To minimize network latency, provision your Filestore instances in regions and zones that are close to where you plan to use the instances.
  • Monitor the performance of your Filestore instances, and set up alerts by using Cloud Monitoring.

What's next

Review the best practices for optimizing the performance of your compute, networking, database, and analytics resources: