Managed Service for Apache Spark pricing
- Managed Service for Apache Spark on Compute Engine pricing
- Managed Service for Apache Spark on GKE pricing
- Managed Service for Apache Spark pricing
Managed Service for Apache Spark on Compute Engine pricing
Managed Service for Apache Spark on Compute Engine pricing is based on the size of Managed Service for Apache Spark clusters and the duration of time that they run. The size of a cluster is based on the aggregate number of virtual CPUs (vCPUs) across the entire cluster, including the master and worker nodes. The duration of a cluster is the length of time between cluster creation and cluster stopping or deletion.
The Dataproc pricing formula is: $0.010 * # of vCPUs * hourly duration.
Although the pricing formula is expressed as an hourly rate, Managed Service for Apache Spark is billed by the second, and all Managed Service for Apache Spark clusters are billed in one-second clock-time increments, subject to a 1-minute minimum billing. Usage is stated in fractional hours (for example, 30 minutes is expressed as 0.5 hours) in order to apply hourly pricing to second-by-second use.
Managed Service for Apache Spark pricing is in addition to the Compute Engine per-instance price for each virtual machine (see Use of other Google Cloud resources).
Accrued Charges
The following Managed Service for Apache Spark operations and scenarios cause charges to accrue:
Scaling and autoscaling: When VMs are added to the cluster, charges accrue while the VMs are active. These accrued charges continue until the VMs are removed.
Clusters in an Error state: When a Managed Service for Apache Spark cluster is in an error state, cluster VMs remain active and charges continue to accrue. These accrued charges continue until the cluster is deleted.
Pricing example
As an example, consider a cluster (with master and worker nodes) that has the following configuration:
| Item | Machine Type | Virtual CPUs | Attached persistent disk | Number in cluster |
|---|---|---|---|---|
| Master Node | n1-standard-4 | 4 | 500 GB | 1 |
| Worker Nodes | n1-standard-4 | 4 | 500 GB | 5 |
This Managed Service for Apache Spark cluster has 24 virtual CPUs, 4 for the master and 20 spread across the workers. For Managed Service for Apache Spark billing purposes, the pricing for this cluster would be based on those 24 virtual CPUs and the length of time the cluster ran (assuming no nodes are scaled down or preempted). If the cluster runs for 2 hours, the Managed Service for Apache Spark pricing would use the following formula:
Managed Service for Apache Spark charge = # of vCPUs * hours * Managed Service for Apache Spark price = 24 * 2 * $0.01 = $0.48
In this example, the cluster would also incur charges for Compute Engine and Standard Persistent Disk Provisioned Space in addition to the Managed Service for Apache Spark charge (see Use of other Google Cloud resources). The billing calculator can be used to determine separate Google Cloud resource costs.
Use of other Google Cloud resources
As a managed and integrated solution, Managed Service for Apache Spark is built on top of other Google Cloud technologies. Managed Service for Apache Spark clusters consume the following resources, each billed at its own pricing:
- Compute Engine—All Compute Engine instances for a Managed Service for Apache Spark cluster have a 1-minute clock-time minimum, and are billed based on per-second billing increments and sustained use price rules.
- Standard Persistent Disk provisioned space
- Cloud Monitoring—see Google Cloud Observability Pricing
Managed Service for Apache Spark clusters can optionally utilize the following resources, each billed at its own pricing, including but not limited to:
Managed Service for Apache Spark on GKE pricing
This section explains the charges that apply only to the virtual Managed Service for Apache Spark cluster that runs on a user-managed GKE. See GKE pricing to learn about the added charges that apply to the user-managed GKE cluster.
The Managed Service for Apache Spark on GKE pricing
formula, $0.010 * # of vCPUs * hourly duration, is the same as the
Managed Service for Apache Spark on Compute Engine pricing formula, and
is applied to the aggregate number of virtual CPUs running in VMs instances in
Managed Service for Apache Spark-created node pools
in the cluster. The duration of a virtual machine instance is the length of time
from its creation to its deletion. As with Managed Service for Apache Spark on Compute Engine,
Managed Service for Apache Spark on GKE is billed by the second, subject to a 1-minute minimum billing
per virtual machine instance. Other Google Cloud charges
are applied in addition to Managed Service for Apache Spark charges.
Managed Service for Apache Spark-created node pools continue to exist after deletion of the Managed Service for Apache Spark cluster since they may be shared by multiple clusters. If you delete the node pools or scale node pools down to zero instances, continued Managed Service for Apache Spark charges will not be incurred. Any remaining node pool VMs will continue to incur charges until you delete them.
Managed Service for Apache Spark pricing
See Managed Service for Apache Spark pricing.
What's next
- Read the Managed Service for Apache Spark documentation.
- Get started with Managed Service for Apache Spark.
- Try the Pricing calculator.
- Learn about Managed Service for Apache Spark solutions and use cases.