Cloud Dataproc Version List

Google Cloud Dataproc uses image versions to bundle operating system, big data components, and Google Cloud Platform connectors into one package that is deployed on a cluster. For more information, see Cloud Dataproc Versioning.

Supported Cloud Dataproc versions

The following versions are supported in Cloud Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version	Includes	Base OS	Released On	Last Updated (sub-minor version)	Notes
1.3	Apache Spark 2.3.1 Apache Hadoop 2.9.0 Apache Pig 0.17.0 Apache Hive 2.3.2 Apache Tez 0.9.0^* Cloud Storage connector 1.9.9-hadoop2 Scala 2.11.8 Python 2.7	Debian 9	2018/06/29	2018/11/16 (1.3.16-deb9)
1.2	Apache Spark 2.2.1 Apache Hadoop 2.8.4 Apache Pig 0.16.0 Apache Hive 2.1.1^* Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2 Scala 2.11.8 Python 2.7	Debian 9	2017/07/18	2018/11/16 (1.2.56-deb9)	This is the default version for new clusters beginning 2017/08/08.
1.1	Apache Spark 2.0.2 Apache Hadoop 2.7.4 Apache Pig 0.16.0 Apache Hive 2.1.1^* Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2 Scala 2.11.8 Python 2.7	Debian 9	2016/08/08	2018/11/16 (1.1.93-deb9)	This was the default version until 2017/08/08.
1.0	Apache Spark 1.6.2 Apache Hadoop 2.7.4 Apache Pig 0.15.0 Apache Hive 1.2.1^* Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2 Scala 2.10.5 Python 2.7	Debian 9	2016/02/22	2018/11/16 (1.0.102-deb9)	General availability release. This version will be unsupported after March 31, 2019.

Currently, Apache Hive installations on Cloud Dataproc clusters do not include HCatalog and WebHCat. See the HCatalog initialization action to create a Cloud Dataproc cluster with an HCatalog installation.

Unsupported Cloud Dataproc versions

The following Cloud Dataproc versions are unsupported. Cloud Dataproc does not provide updates and support for clusters created with these versions. Although you can continue running a cluster that was created with an unsupported version, replacing the cluster with a new cluster that is created with a supported version is recommended.

Version	Includes	Released On	Last Updated	Notes
0.2	Apache Spark 1.5.2 Apache Hadoop 2.7.1 Apache Pig 0.15.0 Apache Hive 1.2.1 Cloud Storage connector 1.5.1-hadoop2 BigQuery connector 0.7.7-hadoop2	2015/11/18	2016/08/02	Beta image second release.
0.1	Apache Spark 1.5.0 Apache Hadoop 2.7.1 Apache Pig 0.14.10 Apache Hive 1.0 Cloud Storage connector 1.5.1-hadoop2 BigQuery connector 0.7.7-hadoop2	2015/09/23	2016/08/02	Cloud Dataproc beta release. Spark 1.5 has been compiled against Hive 1.2.