Cloud Dataproc Version List

Google Cloud Dataproc uses image versions to bundle operating system, big data components, and Google Cloud Platform connectors into one package that is deployed on a cluster. For more information, see Cloud Dataproc Versioning.

Supported Cloud Dataproc versions

The following versions are supported in Cloud Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version	Includes	Base OS	Released On	Last Updated (sub-minor version)	Notes
1.3	Apache Spark 2.3.0 Apache Hadoop 2.9.0 Apache Pig 0.17.0 Apache Hive 2.3.2 Apache Tez 0.9.0^* Cloud Storage connector 1.9.8-hadoop2	Debian 8	2018/06/29	2018/10/12 (1.3.12-deb8)	All releases on and after November 2, 2018 will be based on Debian 9.
1.3-deb9	Apache Spark 2.3.1 Apache Hadoop 2.9.0 Apache Pig 0.17.0 Apache Hive 2.3.2 Apache Tez 0.9.0^* Cloud Storage connector 1.9.8-hadoop2	Debian 9	2018/08/16	2018/10/12 (1.3.12-deb9)
1.2	Apache Spark 2.2.1 Apache Hadoop 2.8.4 Apache Pig 0.16.0 Apache Hive 2.1.1^* Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2	Debian 8	2017/07/18	2018/10/12 (1.2.52-deb8)	This is the default version for new clusters beginning 2017/08/08. All releases on and after November 2, 2018 will be based on Debian 9.
1.2-deb9	Apache Spark 2.2.1 Apache Hadoop 2.8.4 Apache Pig 0.16.0 Apache Hive 2.1.1^* Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2	Debian 9	2018/08/16	2018/10/12 (1.2.52-deb9)
1.1	Apache Spark 2.0.2 Apache Hadoop 2.7.4 Apache Pig 0.16.0 Apache Hive 2.1.1^* Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2	Debian 8	2016/08/08	2018/10/12 (1.1.89-deb8)	This was the default version until 2017/08/08. All releases on and after November 2, 2018 will be based on Debian 9.
1.1-deb9	Apache Spark 2.0.2 Apache Hadoop 2.7.4 Apache Pig 0.16.0 Apache Hive 2.1.1^* Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2	Debian 9	2018/08/16	2018/10/12 (1.1.89-deb9)
1.0	Apache Spark 1.6.2 Apache Hadoop 2.7.3 Apache Pig 0.15.0 Apache Hive 1.2.1^* Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2	Debian 8	2016/02/22	2018/10/12 (1.0.98-deb8)	General availability release. All releases on and after November 2, 2018 will be based on Debian 9. This version will be unsupported after March 31, 2019.
1.0-deb9	Apache Spark 1.6.2 Apache Hadoop 2.7.3 Apache Pig 0.15.0 Apache Hive 1.2.1^* Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2	Debian 9	2018/08/16	2018/10/12 (1.0.98-deb9)	This version will be unsupported after March 31, 2019.

Currently, Apache Hive installations on Cloud Dataproc clusters do not include HCatalog and WebHCat. See the HCatalog initialization action to create a Cloud Dataproc cluster with an HCatalog installation.

Unsupported Cloud Dataproc versions

The following Cloud Dataproc versions are unsupported. Cloud Dataproc does not provide updates and support for clusters created with these versions. Although you can continue running a cluster that was created with an unsupported version, replacing the cluster with a new cluster that is created with a supported version is recommended.

Version	Includes	Released On	Last Updated	Notes
0.2	Apache Spark 1.5.2 Apache Hadoop 2.7.1 Apache Pig 0.15.0 Apache Hive 1.2.1 Cloud Storage connector 1.5.1-hadoop2 BigQuery connector 0.7.7-hadoop2	2015/11/18	2016/08/02	Beta image second release.
0.1	Apache Spark 1.5.0 Apache Hadoop 2.7.1 Apache Pig 0.14.10 Apache Hive 1.0 Cloud Storage connector 1.5.1-hadoop2 BigQuery connector 0.7.7-hadoop2	2015/09/23	2016/08/02	Cloud Dataproc beta release. Spark 1.5 has been compiled against Hive 1.2.