Dataproc image version list

Google Dataproc uses image versions to bundle operating system, big data components, and Google Cloud Platform connectors into one package that is deployed on a cluster. For more information, see Dataproc Versioning.

Default Dataproc image version

Dataproc updates the default image version to the latest generally available Debian-based Dataproc image version 1 month after its GA date.

Supported Dataproc versions

Debian images

The following Debian 10-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version Last Updated Released On Supported Until Notes
2.0-debian10 2022/08/01 2021/01/22 2023/06/30 General availability release.
1.5-debian10 2022/08/01 2020/03/25 2023/03/31 General availability release.

Ubuntu images

The following Ubuntu 18.04 LTS-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version Last Updated Released On Supported Until Notes
2.0-ubuntu18 2022/08/01 2021/01/22 2023/06/30 General availability release.
1.5-ubuntu18 2022/08/01 2020/03/25 2023/03/31 General availability release.

Rocky Linux images

The following Rocky Linux-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version Last Updated Released On Supported Until Notes
2.0-rocky8 2022/08/01 2022/02/18 2023/06/30 General availability release.
1.5-rocky8 2022/08/01 2022/02/18 2023/03/31 General availability release.

Unsupported Dataproc versions

The following Dataproc versions are unsupported. Dataproc does not provide updates and support for clusters created with these versions. Although you can continue running a cluster that was created with an unsupported version, replacing the cluster with a new cluster that is created with a supported version is recommended.

Version Includes Released On Last Updated Notes
2.0-centos8 Apache Spark 3.1.2
Apache Hadoop 3.2.2
Apache Pig 0.18.0-SNAPSHOT
Apache Hive 3.1.2
Cloud Storage connector 2.2.4-hadoop3
Python 3.8
Scala 2.12.14
Zookeeper 3.4.14
2021/03/16 2022/02/01 Unsupported as of 2022/02/01.
2.0.30-centos8 was the final released version.
1.5-centos8 Apache Spark 2.4.8
Apache Hadoop 2.10.1
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 2.1.5-hadoop2
Python 3.7
Scala 2.12.10
Zookeeper 3.4.14
2020/12/14 2022/02/01 Unsupported as of 2022/02/01.
1.5.56-centos8 was the final released version.
1.4-debian10/-ubuntu18 Apache Spark 2.4.8
Apache Hadoop 2.9.2
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 1.9.18-hadoop2
Python 3.6
Scala 2.11.12
Zookeeper 3.4.14
2019/03/22 2022/02/01 Unsupported as of 2022/02/01.
1.4.80-debian10/-ubuntu18 was the final released version.
1.3-debian10/-ubuntu18 Apache Spark 2.3.4
Apache Hadoop 2.9.2
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 1.9.18-hadoop2
Python 2.7
Scala 2.11.8
Zookeeper 3.4.13
2018/06/29 2021/12/22 Unsupported as of 2021/08/01.
1.3.95-debian10/-ubuntu18 was the final released version, which has log4j2 vulnerabilities addressed. Note: previously released versions are vulnerable and must be upgraded.
1.4-debian9 Apache Spark 2.4.5
Apache Hadoop 2.9.2
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 1.9.17-hadoop2
Python 3.6
Scala 2.11.12
Zookeeper 3.4.13
2019/03/22 2020/07/10 Unsupported as of 2020/07/10.
1.4.33-debian9 was the final released version.
1.3-debian9 Apache Spark 2.3.4
Apache Hadoop 2.9.2
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 1.9.17-hadoop2
Python 2.7
Scala 2.11.8
Zookeeper 3.4.13
2018/06/29 2020/07/10 Unsupported as of 2020/07/10.
1.3.62-debian9 was the final released version.
1.2-debian9 Apache Spark 2.2.3
Apache Hadoop 2.8.5
Apache Pig 0.16.0
Apache Hive 2.1.1
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
Python 2.7
Scala 2.11.8
Zookeeper 3.4.13
2017/07/21 2020/07/10 Unsupported as of 2020/07/10.
1.2.102-debian9 was the final released version.
1.1-debian9 Apache Spark 2.0.2
Apache Hadoop 2.7.7
Apache Pig 0.16.0
Apache Hive 2.1.1
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
2016/08/08 2019/09/26 Unsupported as of 2019/10/01.
1.1.121-debian9 is the final released version.
1.0-debian9 Apache Spark 1.6.2
Apache Hadoop 2.7.4
Apache Pig 0.15.0
Apache Hive 1.2.1
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
2016/02/22 2019/05/09 GA image first release.
Unsupported as of 2019/04/01.
1.0.119-debian9 was the final released version.
0.2 Apache Spark 1.5.2
Apache Hadoop 2.7.1
Apache Pig 0.15.0
Apache Hive 1.2.1
Cloud Storage connector 1.5.1-hadoop2
BigQuery connector 0.7.7-hadoop2
2015/11/18 2016/08/02 Beta image second release.
0.1 Apache Spark 1.5.0
Apache Hadoop 2.7.1
Apache Pig 0.14.10
Apache Hive 1.0
Cloud Storage connector 1.5.1-hadoop2
BigQuery connector 0.7.7-hadoop2
2015/09/23 2016/08/02 Dataproc beta release.
Spark 1.5 has been compiled against Hive 1.2.