Increase developer productivity and get faster data insights
Operational simplicity through serverless Spark
Write Spark applications and pipelines that autoscale
without any manual infrastructure provisioning or
Write Spark applications and pipelines that autoscale without any manual infrastructure provisioning or tuning.
Seamless Spark for all data users
Flexibility of consumption
One size does not fit all. You can choose between
serverless, Kubernetes clusters, and compute clusters
for your Spark applications.
One size does not fit all. You can choose between serverless, Kubernetes clusters, and compute clusters for your Spark applications.
Run Spark jobs that autoscale, from the interface of your choice, in two clicks
Serverless Spark (General Availability)
Developers can spend all their time on code and logic, and use their chosen interface to submit Spark jobs which auto-provision and auto-scale. More details here.
Spark through BigQuery (Private Preview)
Unified SQL and Spark experience: enable data warehousing users to easily write and execute Spark on BigQuery data without exporting it. No infrastructure management required.
Spark through Vertex AI (Public Preview)
Spark for data science in one click: Data scientists can use Spark for development from Vertex AI Workbench seamlessly, with built-in security. Spark is integrated with Vertex AI's MLOps features, where users can execute Spark code through notebook executors that are integrated with Vertex AI Pipelines.
Spark through Dataplex (Private Preview)
Run auto-scaling Spark on data across Google Cloud from a single interface that has one-click access to SparkSQL, Notebooks, or PySpark. Also offers easy collaboration with the ability to save, share, search notebooks and scripts alongside data, and built-in governance across data lakes.
Flexible consumption options
In addition to serverless Spark for no-ops deployment, customers standardizing on Kubernetes for infrastructure management can run Spark on Google Kubernetes Engine (Private Preview) to improve resource utilization and simplify infrastructure management. Customers looking for Hadoop-style infrastructure management can run Spark on Compute Engine (GA).
Google Kubernetes Engine
Get the latest Spark on Google Cloud news, blogs, and events
to request early access to the new solutions for Spark
on Google Cloud
Register interest here to request early access to the new solutions for Spark on Google Cloud