Dataproc image version list

Google Dataproc uses image versions to bundle operating system, big data components, and Google Cloud Platform connectors into one package that is deployed on a cluster. For more information, see Dataproc Versioning.

Default Dataproc image version

Dataproc updates the default image version to the latest generally available Debian-based Dataproc image version 1 month after its GA date.

Supported Dataproc versions

Dataproc prevents the creation of clusters with image versions prior to 1.3.95, 1.4.77, 1.5.53, and 2.0.27, which were affected by Apache Log4j security vulnerabilities Dataproc also prevents cluster creation for Dataproc image versions 0.x, 1.0.x, 1.1.x, and 1.2.x. Dataproc advises that, when possible, you create Dataproc clusters with the latest sub-minor image versions.

See the Dataproc release notes for specific image and log4j update information.

Debian images

The following Debian 10-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

When creating a Debian-based cluster, you can omit the image version OS Distribution Code suffix, for example by specifying 2.0 to select the 2.0-debian10 image. The OS suffix must be used to select a Rocky Linux or Ubuntu-based image, for example by specifying 2.0-ubuntu18. See Selecting versions for image selection examples.

Version	Last Updated	Released On	Supported Until	Notes
2.0-debian10	2022/08/01	2021/01/22	2023/06/30	General availability release.
1.5-debian10	2022/08/01	2020/03/25	2023/03/31	General availability release.

Ubuntu images

The following Ubuntu 18.04 LTS-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version	Last Updated	Released On	Supported Until	Notes
2.0-ubuntu18	2022/08/01	2021/01/22	2023/06/30	General availability release.
1.5-ubuntu18	2022/08/01	2020/03/25	2023/03/31	General availability release.

Rocky Linux images

The following Rocky Linux-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version	Last Updated	Released On	Supported Until	Notes
2.0-rocky8	2022/08/01	2022/02/18	2023/06/30	General availability release.
1.5-rocky8	2022/08/01	2022/02/18	2023/03/31	General availability release.

Unsupported Dataproc versions

Click this link to download a script you can run to check if your project or organization is using an unsupported Dataproc image.

The following Dataproc versions are unsupported. Dataproc does not provide updates and support for clusters created with these versions. Although you can continue running a cluster that was created with an unsupported version, replacing the cluster with a new cluster that is created with a supported version is recommended.

Version	Includes	Released On	Last Updated	Notes
2.0-centos8	Apache Spark 3.1.2 Apache Hadoop 3.2.2 Apache Pig 0.18.0-SNAPSHOT Apache Hive 3.1.2 Cloud Storage connector 2.2.4-hadoop3 Python 3.8 Scala 2.12.14 Zookeeper 3.4.14	2021/03/16	2022/02/01	Unsupported as of 2022/02/01. 2.0.30-centos8 was the final released version.
1.5-centos8	Apache Spark 2.4.8 Apache Hadoop 2.10.1 Apache Pig 0.17.0 Apache Hive 2.3.7 Cloud Storage connector 2.1.5-hadoop2 Python 3.7 Scala 2.12.10 Zookeeper 3.4.14	2020/12/14	2022/02/01	Unsupported as of 2022/02/01. 1.5.56-centos8 was the final released version.
1.4-debian10/-ubuntu18	Apache Spark 2.4.8 Apache Hadoop 2.9.2 Apache Pig 0.17.0 Apache Hive 2.3.7 Cloud Storage connector 1.9.18-hadoop2 Python 3.6 Scala 2.11.12 Zookeeper 3.4.14	2019/03/22	2022/02/01	Unsupported as of 2022/02/01. 1.4.80-debian10/-ubuntu18 was the final released version.
1.3-debian10/-ubuntu18	Apache Spark 2.3.4 Apache Hadoop 2.9.2 Apache Pig 0.17.0 Apache Hive 2.3.7 Cloud Storage connector 1.9.18-hadoop2 Python 2.7 Scala 2.11.8 Zookeeper 3.4.13	2018/06/29	2021/12/22	Unsupported as of 2021/08/01. 1.3.95-debian10/-ubuntu18 was the final released version, which has log4j2 vulnerabilities addressed. Note: previously released versions are vulnerable and must be upgraded.
1.4-debian9	Apache Spark 2.4.5 Apache Hadoop 2.9.2 Apache Pig 0.17.0 Apache Hive 2.3.7 Cloud Storage connector 1.9.17-hadoop2 Python 3.6 Scala 2.11.12 Zookeeper 3.4.13	2019/03/22	2020/07/10	Unsupported as of 2020/07/10. 1.4.33-debian9 was the final released version.
1.3-debian9	Apache Spark 2.3.4 Apache Hadoop 2.9.2 Apache Pig 0.17.0 Apache Hive 2.3.7 Cloud Storage connector 1.9.17-hadoop2 Python 2.7 Scala 2.11.8 Zookeeper 3.4.13	2018/06/29	2020/07/10	Unsupported as of 2020/07/10. 1.3.62-debian9 was the final released version.
1.2-debian9	Apache Spark 2.2.3 Apache Hadoop 2.8.5 Apache Pig 0.16.0 Apache Hive 2.1.1 Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2 Python 2.7 Scala 2.11.8 Zookeeper 3.4.13	2017/07/21	2020/07/10	Unsupported as of 2020/07/10. 1.2.102-debian9 was the final released version.
1.1-debian9	Apache Spark 2.0.2 Apache Hadoop 2.7.7 Apache Pig 0.16.0 Apache Hive 2.1.1 Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2	2016/08/08	2019/09/26	Unsupported as of 2019/10/01. 1.1.121-debian9 is the final released version.
1.0-debian9	Apache Spark 1.6.2 Apache Hadoop 2.7.4 Apache Pig 0.15.0 Apache Hive 1.2.1 Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2	2016/02/22	2019/05/09	GA image first release. Unsupported as of 2019/04/01. 1.0.119-debian9 was the final released version.
0.2	Apache Spark 1.5.2 Apache Hadoop 2.7.1 Apache Pig 0.15.0 Apache Hive 1.2.1 Cloud Storage connector 1.5.1-hadoop2 BigQuery connector 0.7.7-hadoop2	2015/11/18	2016/08/02	Beta image second release.
0.1	Apache Spark 1.5.0 Apache Hadoop 2.7.1 Apache Pig 0.14.10 Apache Hive 1.0 Cloud Storage connector 1.5.1-hadoop2 BigQuery connector 0.7.7-hadoop2	2015/09/23	2016/08/02	Dataproc beta release. Spark 1.5 has been compiled against Hive 1.2.