What’s new with Filestore: Enhancing your stateful workloads on GKE

Many customers use Filestore for stateful applications running on Google Kubernetes Engine (GKE). For background on the benefits of using the two together, please see this blog.

Filestore, Google Cloud’s fully managed file storage service, is a multi-reader, multi-writer solution that is decoupled from compute VMs, making it resilient to VM changes/failures.

Filestore’s Enterprise and Zonal (formerly known as High Scale) tiers are built on Filestore’s modern platform that utilizes scale-out architecture, designed from the ground up for performance, availability and durability. Customer data is stored twice to ensure high durability within a zone, and Zonal offers a 99.9% availability SLA. Enterprise, a regional service, replicates the data synchronously across three zones within a region, offering 99.99% availability. Better yet, in October we reduced Filestore prices by up to 25% on Enterprise, Zonal, and Basic HDD tiers (Basic SSD pricing remains unchanged). The new lower prices are applied automatically with no action required.

Filestore is a fully managed service integrated into GKE’s CSI driver, and is continuously evolving with new features, functionality, and GKE integrations. Read on to learn about three new features, which have all recently become generally available.

1. CSI Driver support for Filestore Zonal High-capacity (100TiB)

The new CSI driver integration of our high-capacity, Zonal offering with GKE starts at 10TiB and scales capacity and performance linearly to meet your high-capacity and high performance needs up to 100TiB per instance. The high throughput of Filestore Zonal (up to 26GiB/s) is useful for large-scale AI/ML training frameworks like PyTorch/Tensorflow that expect a file interface. Additionally, it features non-disruptive upgrades, and 1,000 NFS connections per 10TiB. That’s up to 10,000 concurrent NFS connections supporting large GKE deployments and demanding multi-writer AI/ML workloads.

The addition of the CSI driver for Filestore Zonal complements our previously available offerings of Filestore Enterprise and Filestore Basic. This now enables you to choose the appropriate Filestore tier based on your performance, capacity and data protection (e.g., snapshot and backup) needs. Note that Filestore Enterprise supports non-disruptive upgrades while Filestore Basic tier does not.

2. Backups

We’re excited to announce the launch of the CSI-integrated backup for Filestore Enterprise.

Customers were already able to protect their data with the Volume Snapshots API on Filestore Basic volumes, and now they can protect Filestore Enterprise volumes the same way. It’s important to note the naming of the VolumeSnapshot API is a bit misleading as it’s actually a method to back up the data and is not a local file system snapshot as the name implies. The process of using the API to invoke a backup for Filestore Basic and Enterprise is identical.

You can now take backups of your existing Enterprise volumes, and create new volumes with the snapshot as a data source. The new feature enables you to back up and restore your instances directly from the CSI driver. This capability is available on the Enterprise tier in single-share configurations. The Volume Snapshots API is not supported with Enterprise Multishare configurations at this time.

3. Smaller capacity Filestore Enterprise multi-shares

GKE and Filestore Enterprise customers enjoy the efficiency of the multishare instances that we launched last year, enabling them to subdivide a 1TiB instance into multiple 100GiB persistent volumes to improve storage utilization.

Today, we’re announcing the next stage of our multi-shares feature, exclusively available on the Enterprise tier through the GKE CSI driver. With multishares, you can now subdivide your Enterprise instance into as many as 80 shares (up from 10 shares previously), and can start from a minimum share (persistent volume) capacity as low as 10GiB (down from 100GiB previously).

With Enterprise Multishares, persistent storage for your GKE stateful workloads is not only highly durable (99.99% SLA) and high-performance, it’s also highly efficient, offering comprehensive instance utilization of up to 80 shares per instance. Learn more about how you can increase your GKE storage efficacy with the improved, multishares feature here.

Stateful workloads, the Filestore way

Filestore is a fully managed, GKE integrated NFS storage service. With the addition of the Filestore Zonal CSI integration, Volume Snapshot API availability for Filestore Enterprise, and the more granular Filestore Enterprise Multishare persistent volume support, we continue to strive to support large-scale, demanding GKE stateful workloads such as AI/ML training. Get started here to use Filestore and the CSI driver with GKE.