Can cloud instances perform better than bare metal? Latest STAC-M3 benchmarks say yes

Who says you have to take a performance hit when you run in the cloud? We’ve always believed that you can run demanding, time-series databases on Google Cloud Platform as fast—or faster—than you can on bare metal, and now we have the data to prove it.

The Securities Technology Analysis Center (STAC) coordinates a community called the STAC Benchmark Council, comprised of over 300 financial institutions and more than 50 vendor organizations whose purpose it is to develop technology benchmark standards useful to financial organizations.  

STAC recently performed the STAC-M3 Shasta, Antuco and Kanaga suites of benchmarks with kdb+, the leading in-memory, time-series database technology from Kx, on two configurations of Google Cloud Platform.

Kx is a data analysis software vendor whose products are frequently used in financial modeling and data analysis by Wall Street investment banks and energy, government and telecommunications entities, and is a Google Cloud Technology Partner.

The STAC-M3 set of industry-standard enterprise tick-analytics benchmarks is used to evaluate database solutions that manage large time-series of market data (tick data). STAC-M3 Shasta is similar to the baseline STAC-M3 Antuco suite except that under Shasta, the System Under Test (SUT) may pre-load data into memory, and there is no requirement to clear the filesystem cache between benchmarks. In this test, the entire database was pre-loaded into memory and all database operations were performed on data in-memory.

STAC-M3 Shasta results

The first benchmark consisted of running Kx’s kdb+ 3.5 on a GCP n1-ultramem-160 instance containing 160 vCPUs and 3.97 TB of DRAM, with data loaded from 4TB of Google SSD Persistent Disk.

The results of the first benchmark are as follows:

• The GCP solution outperformed a bare-metal solution based on Broadwell EX and 6TB DRAM (SUT ID KDB160425) in eight of the 15 required benchmarks.

• The GCP solution outperformed a bare-metal solution based on Ivy Bridge EX and 6TB DRAM (SUT ID KDB140116) in 14 of the 15 required benchmarks.

• In five of these benchmarks, the GCP ultramem-based solution provided more than twice the speed of the bare-metal solution.

These results were achieved on a single n1-ultramem-160 VM, powered by four Broadwell EX CPUs. The node had a single Persistent SSD attached. The solution outperformed standard bare-metal solutions based on Ivy Bridge EX and Broadwell EX providing great support for for real-time in-memory databases.

Mark Sykes, Chief Operating Officer at Kx, shared the following about the benchmark results:

“We are delighted with the results of the two STAC-M3 benchmarks run by Google on GCP because they validate for our customers the viability of running significant time-series based kdb+ applications in the cloud. The combination of these two STAC-M3 benchmarks shows that not only does kdb+ execute well with in-memory real-time queries on a single GCP instance using a large memory footprint, but it also achieves good results with a widely distributed set of instances, using historical data contained on Google Cloud persistent disks, shared amongst all compute nodes. These results address many of our customers’ fundamental questions when migrating kdb+ market data and workloads from on-premise to the cloud.”

STAC-M3 Antuco and Kanaga results

This second STAC-M3 benchmark consisted of kdb+ 3.6 on a GCP n1-standard-32 instance with 32 vCPUs and 120GB DRAM, with Intel Scale Processor (Skylake), with data loaded from 4TB of a shared read-only Google SSD Persistent Disk.

The results of the second benchmark are as follows.

• The GCP solution outperformed a Lustre-based on-premises cluster (SUT ID KDB150528) in 14 of the 17 required benchmarks.

• The GCP solution outperformed a Lustre-based on-premises cluster (SUT ID KDB150528) in 16 of 16 benchmarks reported for KDB150528*

• In 19 of these benchmarks, the GCP scaled-out solution was more than six times faster than the Lustre-based solution.

The results were achieved on a cluster of thirteen nodes, (Intel Skylake n1-standard-32) deployed on a single subnetwork. Each node had a single Persistent SSD attached. With this configuration, there was a 1.6x to 12.6x speedup in all 16 STAC-M3 Kanaga benchmarks and a 1.3x to 7.8x speedup in 14 of the 17 STAC-M3 Antuco benchmarks versus the Lustre-based on-premises cluster.

A cloud without compromise

When we design our GCP compute infrastructure, we do so with the most demanding applications in mind, including real-time time-series databases like kdb+. If the fear of sacrificing performance has prevented you from migrating demanding workloads to the cloud, we hope that these benchmark results will help you reconsider. You can access the full STAC-M3 Shasta report here, and the STAC-M3 Antuco and Kanaga results here.