Google Cloud Dataproc uses image versions to bundle operating system, big data components, and Google Cloud Platform connectors into one package that is deployed on a cluster. For more information, see Cloud Dataproc Versioning.
Supported Cloud Dataproc versions
Debian images
The following Debian9-based image versions are supported in Cloud Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.
| Version | Last Updated | Released On | Notes |
|---|---|---|---|
| 1.4-debian9 | 2019/03/22 | 2018/03/22 | General availability release. |
| 1.3-debian9 | 2019/03/22 | 2018/06/29 | General availability release. This is the default version for new clusters beginning 2019/01/22. |
| 1.2-debian9 | 2019/03/22 | 2017/7/21 | General availability release. This was the default version for new clusters beginning 2017/08/08 until 2019/01/22. |
| 1.1-debian9 | 2019/03/22 | 2016/08/08 | General availability release. This version will be unsupported after September 30, 2019. |
| 1.0-debian9 | 2019/03/22 | 2016/02/22 | General availability release. This version will be unsupported after March 31, 2019. |
Ubuntu images (preview)
The following Ubuntu 18.04 LTS-based image versions are supported in Cloud Dataproc clusters. Note that new clusters will be created to include any sub-minor patches that have been made to a version since its release.
| Version | Last Updated | Released On | Notes |
|---|---|---|---|
| 1.4-ubuntu18 | 2019/03/22 | 2018/03/22 | This version is provided for testing or preview purposes. Components in next stable version may differ. |
| 1.3-ubuntu18 | 2019/03/22 | 2018/03/22 | This version is provided for testing or preview purposes. Components in next stable version may differ. |
Unsupported Cloud Dataproc versions
The following Cloud Dataproc versions are unsupported. Cloud Dataproc does not provide updates and support for clusters created with these versions. Although you can continue running a cluster that was created with an unsupported version, replacing the cluster with a new cluster that is created with a supported version is recommended.
| Version | Includes | Released On | Last Updated | Notes |
|---|---|---|---|---|
0.2 |
Apache Spark 1.5.2 Apache Hadoop 2.7.1 Apache Pig 0.15.0 Apache Hive 1.2.1 Cloud Storage connector 1.5.1-hadoop2 BigQuery connector 0.7.7-hadoop2 |
2015/11/18 | 2016/08/02 | Beta image second release. |
0.1 |
Apache Spark 1.5.0 Apache Hadoop 2.7.1 Apache Pig 0.14.10 Apache Hive 1.0 Cloud Storage connector 1.5.1-hadoop2 BigQuery connector 0.7.7-hadoop2 |
2015/09/23 | 2016/08/02 | Cloud Dataproc beta release. Spark 1.5 has been compiled against Hive 1.2. |
Need help? Visit our