Announcing public binaries for Spack on Google Cloud

Today, we are pleased to announce a new build cache for Spack, providing portable and optimized builds targeting select microarchitectures available on Google Cloud and the HPC VM Image (CentOS 7-based). 

Spack is a package management tool designed to support multiple versions and configurations of software across a wide variety of computer platforms and operating systems, helping to simplify deployments of complex software packages. Using a build cache with Spack can significantly reduce software installation time. 

Our build cache is hosted on Cloud Storage, providing a scalable storage back end to support large HPC deployments. Using this binary cache can replace a long compilation process (hours) into a significantly shorter installation process (minutes or seconds). 

What problem does Spack solve?

Near instantaneous deployment of infrastructure is a key benefit of running workloads in the cloud. However, deployments can sometimes be slowed by complex software builds and installation. This is especially true of scientific and engineering tools, libraries, and applications. Spack was originally created for supercomputing centers, where many users and application teams share complex software installations. Spack provides the ability to create a binary build cache to significantly reduce package installation times.

How to use Google’s Spack binary cache

If you are looking to use Spack on Google Cloud, we recommend starting with our tutorials that deploy common HPC applications using Spack and the Google Cloud HPC Toolkit. These tutorials use the Cloud HPC Toolkit’s spack-install module, which automates Spack installation in your environment.

The Spack binary cache is built against a specific version of spack. Today, our cache supports the v0.20.0 release of Spack, and in the future we will provide new caches built against major releases of Spack. 

If you already have Spack installed in your environment, Google’s v0.20.0 spack binary cache can be easily configured via the "spack mirror" command:

The name “latest” can also be replaced with the explicit version “v0.20.0”. Once spack is configured to read from this cache, the available packages can be viewed with the "buildcache list" command:

Packages (e.g., gcc v13.1.0) can be added with the install command:

While installing a package, the available binaries will be automatically pulled from the cache. If a package (or a dependency) is missing, Spack will fall back to building it from source. In order to forcibly select an optimized build, packages can be installed with the target specifier on the install command:

The arch command can be used to determine which target Spack will use on a given platform:

Supported platforms

Today there are optimized builds available for specific VM Families and the CentOS 7-based HPC VM Image (stay tuned for support for the Rocky Linux 8-based HPC VM Image). The SchedMD Slurm-GCP CentOS 7 HPC image is also supported.

How will the Spack build cache evolve?

This Spack build cache currently provides builds using Spack v0.20.0, which are supported by any minor release in the Spack v0.20.0 series. We aim to refresh packages with every new major Spack release. We also aim to include optimized builds for new VM families when they are released. Available build caches can be seen by using:

Spack can be configured to use a specific version of the build cache by using:

The version “<spack_version>” can also be replaced with “latest” to point at the latest build of the cache. 

This new Spack cache provides a selected list of packages, aimed at decreasing infrastructure deployment times. To request new packages be added to the build cache, please reach out to your account manager.

Conclusion

We’re excited to help introduce an additional degree of simplicity, optimization, and time savings by providing optimized builds to our customers via a Spack binary cache. If you’re interested in learning more about Google’s Cloud HPC Toolkit, check out our product documentation and the Github repository. If you have any questions or want to provide feedback, please reach out to your account manager or contact us.