Dataproc Image version list

Google Dataproc uses image versions to bundle operating system, big data components, and Google Cloud Platform connectors into one package that is deployed on a cluster. For more information, see Dataproc Versioning.

Default Dataproc image version

Dataproc updates the default image version to the latest generally available Debian-based Dataproc image version 1 month after its GA date.

Supported Dataproc versions

Dataproc image vulnerability:

Apache Log4j security vulnerabilities affect Dataproc images with subminor image version numbers prior to 1.4.77, 1.5.53, or 2.0.27. These prior affected images are scheduled to be deprecated on February 28, 2022. Dataproc strongly recommends that you create new clusters or recreate existing clusters with later released images.

Dataproc images 1.4.78, 1.5.54, and 2.0.28 were updated with log4j version 2.17.0. It is strongly recommended that your clusters use previously released images 1.4.77, 1.5.53, or 2.0.27, or higher. While not urgent nor strongly recommended, Dataproc advises you to create or recreate Dataproc clusters with the latest sub-minor image versions when possible.

See the Dataproc release notes for specific image and log4j update information.

Debian images

The following Debian 10-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

When creating a Debian-based cluster, you can omit the image version OS Distribution Code suffix, for example by specifying 2.0 to select the 2.0-debian10 image. The OS suffix must be used to select CentOS or Ubuntu-based image, for example by specifying 2.0-ubuntu18. See Selecting versions for image selection examples.

Version	Last Updated	Released On	Supported Until	Notes
2.0-debian10	2022/01/17	2021/01/22	2023/02	General availability release.
1.5-debian10	2022/01/17	2020/03/25	2022/12	General availability release.
1.4-debian10	2022/01/17	2019/03/22	2022/01	General availability release.

Ubuntu images

The following Ubuntu 18.04 LTS-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version	Last Updated	Released On	Supported Until	Notes
2.0-ubuntu18	2022/01/17	2021/01/22	2023/02	General availability release.
1.5-ubuntu18	2022/01/17	2020/03/25	2022/12	General availability release.
1.4-ubuntu18	2022/01/17	2019/03/22	2022/01	General availability release.

CentOS images

The following CentOS 8-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version	Last Updated	Released On	Supported Until	Notes
2.0-centos8	2022/01/17	2021/03/16	2023/02	General availability release.
1.5-centos8	2022/01/17	2020/12/14	2022/12	General availability release.

Unsupported Dataproc versions

The following Dataproc versions are unsupported. Dataproc does not provide updates and support for clusters created with these versions. Although you can continue running a cluster that was created with an unsupported version, replacing the cluster with a new cluster that is created with a supported version is recommended.

Version	Includes	Released On	Last Updated	Notes
1.3-debian10/-ubuntu18	Apache Spark 2.3.4 Apache Hadoop 2.9.2 Apache Pig 0.17.0 Apache Hive 2.3.7 Cloud Storage connector 1.9.18-hadoop2 Python 2.7 Scala 2.11.8 Zookeeper 3.4.13	2018/06/29	2021/12/22	Unsupported as of 2021/08/01. 1.3.95-debian10/-ubuntu18 was the final released version, which has log4j2 vulnerabilities addressed. Note: previously released versions are vulnerable and must be upgraded.
1.4-debian9	Apache Spark 2.4.5 Apache Hadoop 2.9.2 Apache Pig 0.17.0 Apache Hive 2.3.7 Cloud Storage connector 1.9.17-hadoop2 Python 3.6 Scala 2.11.12 Zookeeper 3.4.13	2019/03/22	2020/07/10	Unsupported as of 2020/07/10. 1.4.33-debian9 was the final released version.
1.3-debian9	Apache Spark 2.3.4 Apache Hadoop 2.9.2 Apache Pig 0.17.0 Apache Hive 2.3.7 Cloud Storage connector 1.9.17-hadoop2 Python 2.7 Scala 2.11.8 Zookeeper 3.4.13	2018/06/29	2020/07/10	Unsupported as of 2020/07/10. 1.3.62-debian9 was the final released version.
1.2-debian9	Apache Spark 2.2.3 Apache Hadoop 2.8.5 Apache Pig 0.16.0 Apache Hive 2.1.1 Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2 Python 2.7 Scala 2.11.8 Zookeeper 3.4.13	2017/07/21	2020/07/10	Unsupported as of 2020/07/10. 1.2.102-debian9 was the final released version.
1.1-debian9	Apache Spark 2.0.2 Apache Hadoop 2.7.7 Apache Pig 0.16.0 Apache Hive 2.1.1 Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2	2016/08/08	2019/09/26	Unsupported as of 2019/10/01. 1.1.121-debian9 is the final released version.
1.0-debian9	Apache Spark 1.6.2 Apache Hadoop 2.7.4 Apache Pig 0.15.0 Apache Hive 1.2.1 Cloud Storage connector 1.6.10-hadoop2 BigQuery connector 0.10.11-hadoop2	2016/02/22	2019/05/09	GA image first release. Unsupported as of 2019/04/01. 1.0.119-debian9 was the final released version.
0.2	Apache Spark 1.5.2 Apache Hadoop 2.7.1 Apache Pig 0.15.0 Apache Hive 1.2.1 Cloud Storage connector 1.5.1-hadoop2 BigQuery connector 0.7.7-hadoop2	2015/11/18	2016/08/02	Beta image second release.
0.1	Apache Spark 1.5.0 Apache Hadoop 2.7.1 Apache Pig 0.14.10 Apache Hive 1.0 Cloud Storage connector 1.5.1-hadoop2 BigQuery connector 0.7.7-hadoop2	2015/09/23	2016/08/02	Dataproc beta release. Spark 1.5 has been compiled against Hive 1.2.