Cloud Dataproc Version List

Google Cloud Dataproc uses image versions to bundle operating system, big data components, and Google Cloud Platform connectors into one package that is deployed on a cluster. For more information, see Cloud Dataproc Versioning.

Supported Cloud Dataproc versions

The following versions are supported in Cloud Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version Includes Base OS Released On Last Updated
(sub-minor version)
Notes
1.3 Apache Spark 2.3.0
Apache Hadoop 2.9.0
Apache Pig 0.17.0
Apache Hive 2.3.2
Apache Tez 0.9.0*
Cloud Storage connector 1.9.8-hadoop2
Debian 8 2018/06/29 2018/10/12
(1.3.12-deb8)
All releases on and after November 2, 2018 will be based on Debian 9.
1.3-deb9 Apache Spark 2.3.1
Apache Hadoop 2.9.0
Apache Pig 0.17.0
Apache Hive 2.3.2
Apache Tez 0.9.0*
Cloud Storage connector 1.9.8-hadoop2
Debian 9 2018/08/16 2018/10/12
(1.3.12-deb9)
1.2 Apache Spark 2.2.1
Apache Hadoop 2.8.4
Apache Pig 0.16.0
Apache Hive 2.1.1*
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
Debian 8 2017/07/18 2018/10/12
(1.2.52-deb8)
  • This is the default version for new clusters beginning 2017/08/08.
  • All releases on and after November 2, 2018 will be based on Debian 9.
1.2-deb9 Apache Spark 2.2.1
Apache Hadoop 2.8.4
Apache Pig 0.16.0
Apache Hive 2.1.1*
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
Debian 9 2018/08/16 2018/10/12
(1.2.52-deb9)
1.1 Apache Spark 2.0.2
Apache Hadoop 2.7.4
Apache Pig 0.16.0
Apache Hive 2.1.1*
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
Debian 8 2016/08/08 2018/10/12
(1.1.89-deb8)
  • This was the default version until 2017/08/08.
  • All releases on and after November 2, 2018 will be based on Debian 9.
1.1-deb9 Apache Spark 2.0.2
Apache Hadoop 2.7.4
Apache Pig 0.16.0
Apache Hive 2.1.1*
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
Debian 9 2018/08/16 2018/10/12
(1.1.89-deb9)
1.0 Apache Spark 1.6.2
Apache Hadoop 2.7.3
Apache Pig 0.15.0
Apache Hive 1.2.1*
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
Debian 8 2016/02/22 2018/10/12
(1.0.98-deb8)
  • General availability release.
  • All releases on and after November 2, 2018 will be based on Debian 9.
  • This version will be unsupported after March 31, 2019.
1.0-deb9 Apache Spark 1.6.2
Apache Hadoop 2.7.3
Apache Pig 0.15.0
Apache Hive 1.2.1*
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
Debian 9 2018/08/16 2018/10/12
(1.0.98-deb9)
  • This version will be unsupported after March 31, 2019.

Unsupported Cloud Dataproc versions

The following Cloud Dataproc versions are unsupported. Cloud Dataproc does not provide updates and support for clusters created with these versions. Although you can continue running a cluster that was created with an unsupported version, replacing the cluster with a new cluster that is created with a supported version is recommended.

Version Includes Released On Last Updated Notes
0.2 Apache Spark 1.5.2
Apache Hadoop 2.7.1
Apache Pig 0.15.0
Apache Hive 1.2.1
Cloud Storage connector 1.5.1-hadoop2
BigQuery connector 0.7.7-hadoop2
2015/11/18 2016/08/02 Beta image second release.
0.1 Apache Spark 1.5.0
Apache Hadoop 2.7.1
Apache Pig 0.14.10
Apache Hive 1.0
Cloud Storage connector 1.5.1-hadoop2
BigQuery connector 0.7.7-hadoop2
2015/09/23 2016/08/02 Cloud Dataproc beta release.
Spark 1.5 has been compiled against Hive 1.2.
Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Dataproc Documentation