Dataproc cluster image version lists

"Ubuntu & Debian Based Image Version Clusters | Google Cloud"

Google Dataproc uses Ubuntu, Debian, and Rocky Linux image versions to bundle operating system, big data components, and Google Cloud Platform connectors into one package that is deployed on a cluster. For more information, see Dataproc Versioning.

Default Dataproc image version

Dataproc updates the default image version to the latest generally available Debian-based Dataproc image version one month after its General availability (GA) release date.

Supported Dataproc versions

Debian images

The following Debian-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version Last Updated Released On Supported Until Available Until Notes
2.2-debian12 2024/02/08 2023/12/08 2025/12/31 2027/12/31 General availability release.
2.1-debian11 2024/02/08 2022/12/12 2024/12/31 2026/12/31 General availability release.
2.0-debian10 2024/02/08 2021/01/22 2024/07/31 2026/07/31 General availability release.

Ubuntu images

The following Ubuntu LTS-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version Last Updated Released On Supported Until Available Until Notes
2.2-ubuntu22 2024/02/08 2023/12/08 2025/12/31 2027/12/31 General availability release.
2.1-ubuntu20 2024/02/08 2022/12/12 2024/12/31 2026/12/31 General availability release.
2.0-ubuntu18 2024/02/08 2021/01/22 2024/07/31 2026/07/31 General availability release.

Rocky Linux images

The following Rocky Linux-based image versions are supported in Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.

Version Last Updated Released On Supported Until Available Until Notes
2.2-rocky9 2024/02/08 2023/12/08 2025/12/31 2027/12/31 General availability release.
2.1-rocky8 2024/02/08 2022/12/12 2024/12/31 2026/12/31 General availability release.
2.0-rocky8 2024/02/08 2022/02/18 2024/07/31 2026/07/31 General availability release.

Unsupported Dataproc versions

The following Dataproc versions are unsupported. Dataproc does not provide updates and support for clusters created with these versions. Although you can continue running a cluster that was created with an unsupported version, replacing the cluster with a new cluster that is created with a supported version is recommended.

Version Includes Released On Last Updated Notes
1.5-debian10/-ubuntu18/-rocky8 Apache Spark 2.4.8
Apache Hadoop 2.10.2
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 2.1.9-hadoop2
Python 3.7
Scala 2.12.10
Zookeeper 3.4.14
2020/03/25:debian10/ubuntu18
2022/02/18:rocky8
2023/04/28 Unsupported as of 2023/04/28.
1.5.89-debian10/-ubuntu18/-rocky8 was the final released version.
2.0-centos8 Apache Spark 3.1.2
Apache Hadoop 3.2.2
Apache Pig 0.18.0-SNAPSHOT
Apache Hive 3.1.2
Cloud Storage connector 2.2.4-hadoop3
Python 3.8
Scala 2.12.14
Zookeeper 3.4.14
2021/03/16 2022/02/01 Unsupported as of 2022/02/01.
2.0.30-centos8 was the final released version.
1.5-centos8 Apache Spark 2.4.8
Apache Hadoop 2.10.1
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 2.1.5-hadoop2
Python 3.7
Scala 2.12.10
Zookeeper 3.4.14
2020/12/14 2022/02/01 Unsupported as of 2022/02/01.
1.5.56-centos8 was the final released version.
1.4-debian10/-ubuntu18 Apache Spark 2.4.8
Apache Hadoop 2.9.2
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 1.9.18-hadoop2
Python 3.6
Scala 2.11.12
Zookeeper 3.4.14
2019/03/22 2022/02/01 Unsupported as of 2022/02/01.
1.4.80-debian10/-ubuntu18 was the final released version.
1.3-debian10/-ubuntu18 Apache Spark 2.3.4
Apache Hadoop 2.9.2
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 1.9.18-hadoop2
Python 2.7
Scala 2.11.8
Zookeeper 3.4.13
2018/06/29 2021/12/22 Unsupported as of 2021/08/01.
1.3.95-debian10/-ubuntu18 was the final released version, which has log4j2 vulnerabilities addressed. Note: previously released versions are vulnerable and must be upgraded.
1.4-debian9 Apache Spark 2.4.5
Apache Hadoop 2.9.2
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 1.9.17-hadoop2
Python 3.6
Scala 2.11.12
Zookeeper 3.4.13
2019/03/22 2020/07/10 Unsupported as of 2020/07/10.
1.4.33-debian9 was the final released version.
1.3-debian9 Apache Spark 2.3.4
Apache Hadoop 2.9.2
Apache Pig 0.17.0
Apache Hive 2.3.7
Cloud Storage connector 1.9.17-hadoop2
Python 2.7
Scala 2.11.8
Zookeeper 3.4.13
2018/06/29 2020/07/10 Unsupported as of 2020/07/10.
1.3.62-debian9 was the final released version.
1.2-debian9 Apache Spark 2.2.3
Apache Hadoop 2.8.5
Apache Pig 0.16.0
Apache Hive 2.1.1
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
Python 2.7
Scala 2.11.8
Zookeeper 3.4.13
2017/07/21 2020/07/10 Unsupported as of 2020/07/10.
1.2.102-debian9 was the final released version.
1.1-debian9 Apache Spark 2.0.2
Apache Hadoop 2.7.7
Apache Pig 0.16.0
Apache Hive 2.1.1
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
2016/08/08 2019/09/26 Unsupported as of 2019/10/01.
1.1.121-debian9 is the final released version.
1.0-debian9 Apache Spark 1.6.2
Apache Hadoop 2.7.4
Apache Pig 0.15.0
Apache Hive 1.2.1
Cloud Storage connector 1.6.10-hadoop2
BigQuery connector 0.10.11-hadoop2
2016/02/22 2019/05/09 GA image first release.
Unsupported as of 2019/04/01.
1.0.119-debian9 was the final released version.
0.2 Apache Spark 1.5.2
Apache Hadoop 2.7.1
Apache Pig 0.15.0
Apache Hive 1.2.1
Cloud Storage connector 1.5.1-hadoop2
BigQuery connector 0.7.7-hadoop2
2015/11/18 2016/08/02 Beta image second release.
0.1 Apache Spark 1.5.0
Apache Hadoop 2.7.1
Apache Pig 0.14.10
Apache Hive 1.0
Cloud Storage connector 1.5.1-hadoop2
BigQuery connector 0.7.7-hadoop2
2015/09/23 2016/08/02 Dataproc beta release.
Spark 1.5 has been compiled against Hive 1.2.