IBM Spectrum Scale is now available in Google Cloud

Google Cloud and Sycomp are introducing Sycomp Storage Fueled by IBM Spectrum Scale. This launch is highly useful for customers looking to run HPC, AI/ML, and big data workloads on a proven file system, in a fully-persistent deployment, and with Google’s Cloud Storage integration for cost optimization.

Sycomp Storage Fueled by IBM Spectrum Scale optimizes performance for the cloud, deploys in minutes, supports auto-tiering to reduce costs, and allows customers to concurrently access data from thousands of VMs, with data access via NFS and the native Spectrum Scale client. The solution is available in the Google Cloud Marketplace, as a VM-based offering that deploys within a customer's project, and is supported and maintained by Sycomp. With Sycomp Storage Fueled by IBM Spectrum Scale, you can:

• Take advantage of auto-tiering technology: Automatically manage tiers of storage to optimize performance and cost to deliver the speed and reliability you need to succeed. 

• Deploy in minutes: Deploy Spectrum Scale in minutes via Google Cloud Marketplace, and enjoy seamless access to your data via NFS and the native Spectrum Scale client. 

• Maximize performance: With up to 320 GB/s of storage performance capability, you can select the VM and disk combination that best meets your requirements. 

About Spectrum Scale

Spectrum Scale is an industry-proven parallel file system that securely manages large volumes of data. The high performance, feature-rich filesystem is well-suited for HPC, AI/ML, big data and other applications requiring a POSIX compliant shared filesystem. With adaptable storage capacity and performance scaling it can support demanding workloads.

You can deploy an instance of Spectrum Scale using Terraform provided by Sycomp through the Marketplace. When deploying a cluster, you choose:

• How you access the data. You can use the Spectrum Scale clients deployed by Sycomp or access the data from your existing application VMs.

• The amount and type of storage

• Instance type and quantity of NSD Servers, NFS Servers, and Spectrum Scale Clients

Deployment models

Core considerations

Spectrum Scale offers multiple deployment models based on your performance and capacity needs. Customers can choose instance type, the number of nodes and select from Google's Persistent Disk and Local SSD options.

Storage type

Persistent Disk offers higher density and increased data protection. Customers can choose between Standard, Balanced, or SSD to optimize $/GB or $/Perf.

Local SSD (LSSD) offers reduced latency and high IOPS optimizing for $/Perf. With LSSD, Spectrum Scale leverages data replication across VMs to protect against data loss in the event of a VM failure, migration, or accidental deletion.

Protocols

Options include:

• NFS for protocol compliance. Deploy with converged (NFS and NSD Server in the same VM) or a separate NFS server VM 

• NSD native client for best per-client throughput

Hybrid deployment

Spectrum Scale allows you to integrate your on-premises systems with Google Cloud with bi-directional data flow, leveraging Spectrum Scale Active File Management (AFM).

You can use AFM to connect:

• Directly through two Spectrum Scale clusters

• through Cloud Storage to share data between your on-premises cluster and your Sycomp Storage deployment in Google Cloud.

AFM provides a seamless way to move data between your on-premises and Google Cloud-based clusters, and are deployed in a single Google Cloud project.

Key features

File system encryption

A deployment can be enhanced by enabling data encryption with an HA pair of VMs running IBM Security Guardium Key Lifecycle Manager (GKLM) to manage and protect your encryption keys. Tools provided by Sycomp automate the deployment of an HA pair of GKLM server VMs.

Integration with Cloud Storage

Once the cluster is deployed, you can enable Spectrum Scale AFM to cloud object storage (COS) to automatically tier your application data to Cloud Storage or dynamically hydrate the Sycomp Storage cluster from existing data in Cloud Storage

Cluster expansion

You can add storage, NFS servers and Spectrum Scale client VMs to an existing Sycomp Storage cluster. Adding storage capacity is achieved by adding NSD servers or disks to existing NSD servers. When you add NSD servers to the cluster, the storage solution implements best practices, as each new server inherits the disk configuration of the existing NSD servers. Storage is added in this manner to maintain balance across NSD servers. You can add NFS servers to increase capacity. Adding clients in groups allows you to customize the names and machine types of each set of clients giving you more control over your deployment.

Clusters have been expanded to a PiB and have been tested to demonstrate linear performance scaling to 250 TiB.

For a deeper dive, please see Sycomp’s architectural overview.

Get started or learn more

For more information, or to get started, see the following:

• Sycomp Storage Fueled by IBM Spectrum Scale on Google Cloud

• Sycomp Storage Fueled by IBM Spectrum Scale In Marketplace

^{Special thanks to Billy Kettler, Partner Engineer and Sean Derrington, Product Manager, for contributing to this post.}