Dataproc cluster image version lists

Stay organized with collections Save and categorize content based on your preferences.

"Ubuntu & Debian Based Image Version Clusters | Google Cloud"

Google Dataproc uses Ubuntu, Debian, and Rocky Linux image versions to bundle operating system, big data components, and Google Cloud Platform connectors into one package that is deployed on a cluster. For more information, see Dataproc Versioning.

Default Dataproc image version

Dataproc updates the default image version to the latest generally available Debian-based Dataproc image version 1 month after its GA date.

Supported Dataproc versions

Debian images

The following Debian-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version Last Updated Released On Supported Until Notes
preview-debian11 2022/11/14 2022/10/28 TBD Preview release.
2.0-debian10 2022/11/14 2021/01/22 2023/12/31 General availability release.
1.5-debian10 2022/11/14 2020/03/25 2023/03/31 General availability release.

Ubuntu images

The following Ubuntu LTS-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version Last Updated Released On Supported Until Notes
preview-ubuntu20 2022/11/14 2022/10/28 TBD Preview release.
2.0-ubuntu18 2022/11/14 2021/01/22 2023/12/31 General availability release.
1.5-ubuntu18 2022/11/14 2020/03/25 2023/03/31 General availability release.

Rocky Linux images

The following Rocky Linux-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version Last Updated Released On Supported Until Notes
preview-rocky8 2022/11/14 2022/10/28 TBD Preview release.
2.0-rocky8 2022/11/14 2022/02/18 2023/12/31 General availability release.
1.5-rocky8 2022/11/14 2022/02/18 2023/03/31 General availability release.

Unsupported Dataproc versions

The following Dataproc versions are unsupported. Dataproc does not provide updates and support for clusters created with these versions. Although you can continue running a cluster that was created with an unsupported version, replacing the cluster with a new cluster that is created with a supported version is recommended.

Version Includes Released On Last Updated Notes
2.0-centos8 Apache Spark 3.1.2
Apache Hadoop 3.2.2
Apache Pig 0.18.0-SNAPSHOT
Apache Hive 3.1.2
Cloud Storage connector 2.2.4-hadoop3
Python 3.8
Scala 2.12.14
Zookeeper 3.4.14
2021/03/16 2022/02/01 Unsupported as of 2022/02/01.
2.0.30-centos8 was the final released version.
1.5-centos8 Apache Spark 2.4.8
Apache Hadoop 2.10.1
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 2.1.5-hadoop2
Python 3.7
Scala 2.12.10
Zookeeper 3.4.14
2020/12/14 2022/02/01 Unsupported as of 2022/02/01.
1.5.56-centos8 was the final released version.
1.4-debian10/-ubuntu18 Apache Spark 2.4.8
Apache Hadoop 2.9.2
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 1.9.18-hadoop2
Python 3.6
Scala 2.11.12
Zookeeper 3.4.14
2019/03/22 2022/02/01 Unsupported as of 2022/02/01.
1.4.80-debian10/-ubuntu18 was the final released version.
1.3-debian10/-ubuntu18 Apache Spark 2.3.4
Apache Hadoop 2.9.2
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 1.9.18-hadoop2
Python 2.7
Scala 2.11.8
Zookeeper 3.4.13
2018/06/29 2021/12/22 Unsupported as of 2021/08/01.
1.3.95-debian10/-ubuntu18 was the final released version, which has log4j2 vulnerabilities addressed. Note: previously released versions are vulnerable and must be upgraded.
1.4-debian9 Apache Spark 2.4.5
Apache Hadoop 2.9.2
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 1.9.17-hadoop2
Python 3.6
Scala 2.11.12
Zookeeper 3.4.13
2019/03/22 2020/07/10 Unsupported as of 2020/07/10.
1.4.33-debian9 was the final released version.
1.3-debian9 Apache Spark 2.3.4
Apache Hadoop 2.9.2
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 1.9.17-hadoop2
Python 2.7
Scala 2.11.8
Zookeeper 3.4.13
2018/06/29 2020/07/10 Unsupported as of 2020/07/10.
1.3.62-debian9 was the final released version.
1.2-debian9 Apache Spark 2.2.3
Apache Hadoop 2.8.5
Apache Pig 0.16.0
Apache Hive 2.1.1
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
Python 2.7
Scala 2.11.8
Zookeeper 3.4.13
2017/07/21 2020/07/10 Unsupported as of 2020/07/10.
1.2.102-debian9 was the final released version.
1.1-debian9 Apache Spark 2.0.2
Apache Hadoop 2.7.7
Apache Pig 0.16.0
Apache Hive 2.1.1
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
2016/08/08 2019/09/26 Unsupported as of 2019/10/01.
1.1.121-debian9 is the final released version.
1.0-debian9 Apache Spark 1.6.2
Apache Hadoop 2.7.4
Apache Pig 0.15.0
Apache Hive 1.2.1
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
2016/02/22 2019/05/09 GA image first release.
Unsupported as of 2019/04/01.
1.0.119-debian9 was the final released version.
0.2 Apache Spark 1.5.2
Apache Hadoop 2.7.1
Apache Pig 0.15.0
Apache Hive 1.2.1
Cloud Storage connector 1.5.1-hadoop2
BigQuery connector 0.7.7-hadoop2
2015/11/18 2016/08/02 Beta image second release.
0.1 Apache Spark 1.5.0
Apache Hadoop 2.7.1
Apache Pig 0.14.10
Apache Hive 1.0
Cloud Storage connector 1.5.1-hadoop2
BigQuery connector 0.7.7-hadoop2
2015/09/23 2016/08/02 Dataproc beta release.
Spark 1.5 has been compiled against Hive 1.2.