Cloud Dataproc Version List

Google Cloud Dataproc uses image versions to bundle operating system, big data components, and Google Cloud Platform connectors into one package that is deployed on a cluster. For more information, see Cloud Dataproc Versioning.

Supported Cloud Dataproc versions

The following versions are supported in Cloud Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version Includes Released On Last Updated Notes
1.1 Apache Spark 2.0.2
Apache Hadoop 2.7.3
Apache Pig 0.16.0
Apache Hive 2.1.1
GCS connector 1.6.1-hadoop2
BigQuery connector 0.10.2-hadoop2
08/08/2016 04/28/2017 This is the default version for new clusters.
1.0 Apache Spark 1.6.2
Apache Hadoop 2.7.3
Apache Pig 0.15.0
Apache Hive 1.2.1
GCS connector 1.6.1-hadoop2
BigQuery connector 0.10.2-hadoop2
02/22/2016 04/28/2017 General availability release.

Unsupported Cloud Dataproc versions

The following versions are no longer supported in Cloud Dataproc clusters. You can create clusters with these image versions, however, these versions are no longer updated and are no longer supported. If you are using one of these versions you should consider upgrading to a more recent and supported version.

Version Includes Released On Last Updated Notes
0.2 Apache Spark 1.5.2
Apache Hadoop 2.7.1
Apache Pig 0.15.0
Apache Hive 1.2.1
GCS connector 1.5.1-hadoop2
BigQuery connector 0.7.7-hadoop2
11/18/2015 08/02/2016 Beta image second release.
0.1 Apache Spark 1.5.0
Apache Hadoop 2.7.1
Apache Pig 0.14.10
Apache Hive 1.0
GCS connector 1.5.1-hadoop2
BigQuery connector 0.7.7-hadoop2
09/23/2015 08/02/2016 Cloud Dataproc beta release.
Spark 1.5 has been compiled against Hive 1.2.

Other versions

These versions are not officially supported, and are provided for testing or preview purposes. The subcomponent (Hadoop, Spark, Hive, Pig, etc.) versions listed below are subject to change.

Version Includes Last Updated Notes
preview Apache Spark 2.1.0
Apache Hadoop 2.8.0-SNAPSHOT
Apache Pig 0.16.0
Apache Hive 2.1.1
GCS connector 1.6.1-hadoop2
BigQuery connector 0.10.2-hadoop2
04/28/2017 Preview of next version but components
in next stable version may differ.

Send feedback about...

Google Cloud Dataproc Documentation