SDK version support status

This page lists the Dataflow support status for the Apache Beam SDK and Dataflow SDK releases:

Apache Beam SDK support-level definitions

The following are the Dataflow support levels for the Apache Beam SDK:

  • Supported: Apache Beam SDK versions are supported by Dataflow from their release date until their deprecation date.
  • Deprecated: When an SDK version is deprecated, it continues to work with Dataflow but might be decommissioned in the future. Before an SDK version is decommissioned, customers are notified and given time to upgrade their environments. Requests for jobs submitted from deprecated SDK versions result in a warning message. It's recommended that you update environments that use a deprecated version of an SDK to a supported version, but deprecated SDK versions continue to work with Dataflow.
  • Decommissioned: Decommissioned SDK versions no longer work with Dataflow. Requests for jobs submitted from decommissioned SDK versions result in an error. To use the SDK with Dataflow, you must update your environment to an SDK version that is not decommissioned. It's recommended that you update the SDK to a supported version. SDK versions are rarely decommissioned. Before an SDK version is decommissioned, customers are notified and given time to upgrade their environments.

The previous definitions don't apply to SDKs or features that have never been supported.

Apache Beam 2.x SDKs

Apache Beam is an open source, community-led project. Google and Google Cloud are part of the Apache Beam community, but we don't own the project or control the release process. We might open bugs or submit patches to the Apache Beam codebase on behalf of Dataflow customers. We cannot create hotfixes or official releases of Apache Beam on demand. See the Apache Beam policies page for more details about release policies.

Dataflow supports specific Apache Beam SDK components for the SDK releases listed on this page. These components have been tested thoroughly with Dataflow. Experimental features are not supported.

See the Apache Beam release notes for change information.

The following tables contain the support status for the Apache Beam 2.x SDKs, including the Apache Beam Python SDK, Apache Beam Java SDK, and the Apache Beam Go SDK:

Java

SDK version Status Supported components Details
2.60.0 Supported org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-iceberg
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-extensions-sql
org.apache.beam:beam-sdks-java-managed
This version will be deprecated on October 17, 2025.
2.59.0 Supported org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-iceberg
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-extensions-sql
org.apache.beam:beam-sdks-java-managed
This version will be deprecated on September 11, 2025.
2.58.1 Supported org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-iceberg
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-extensions-sql
org.apache.beam:beam-sdks-java-managed
This version will be deprecated on August 16, 2025.
2.58.0 Supported org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-iceberg
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-extensions-sql
org.apache.beam:beam-sdks-java-managed
This version will be deprecated on August 6, 2025.
2.57.0 Supported org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-iceberg
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-extensions-sql
org.apache.beam:beam-sdks-java-managed
This version will be deprecated on June 26, 2025.
2.56.0 Supported org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-iceberg
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-extensions-sql
org.apache.beam:beam-sdks-java-managed
This version will be deprecated on May 1, 2025.
2.55.1 Supported org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
This version will be deprecated on April 8, 2025.
2.55.0 Supported org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql

This version will be deprecated on March 25, 2025.

Known issue: The WriteToJson transform in cross-language pipelines fails with an error. To resolve this issue, upgrade to Apache Beam 2.55.1. For more information, see Issue #30776: Beam YAML WriteToJson fails.

2.54.0 Supported org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
This version will be deprecated on February 14, 2025.
2.53.0 Supported org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
This version will be deprecated on January 4, 2025.
2.52.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on November 17, 2024.
2.51.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on October 12, 2024.
2.50.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on August 30, 2024.
2.49.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on July 17, 2024.
2.48.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on May 31, 2024.
2.47.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on May 10, 2024.
2.46.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on March 10, 2024.
2.45.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on February 15, 2024.
2.44.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on January 13, 2024.
2.43.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on November 17, 2023.
2.42.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on October 17, 2023.
2.41.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on August 23, 2023.
2.40.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on June 27, 2023.
2.39.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on May 25, 2023.
2.38.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on April 20, 2023.
2.37.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on March 4, 2023.
2.36.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on February 7, 2023.
2.35.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-io-jdbc
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on December 29, 2022.
2.34.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
org.apache.beam:beam-sdks-java-extensions-sql
Deprecated on November 11, 2022.
2.33.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
Deprecated on October 7, 2022.
2.32.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
Deprecated on August 25, 2022.
2.31.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
Deprecated on July 8, 2022.
2.30.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka
Deprecated on June 10, 2022.
2.29.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on April 29, 2022.

Known issue: If you use Runner V2, by default Dataflow will use a Java 8 SDK container regardless of user Java version. To work around for Java 11, you can override the default SDK image by providing pipeline option `--worker_harness_container_image=gcr.io/cloud-dataflow/v1beta3/beam_java11_sdk:2.29.0`. See Apache Beam issue tracker for more information.

2.28.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on February 22, 2022.

2.27.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on January 7, 2022.

This release adds support for JDK 11.

2.26.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on December 11, 2021.

2.25.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on October 23, 2021.

2.24.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on September 18, 2021.

Known issue: If you use timers that reset, rather than timers that are set and only fired once, you shouldn't upgrade to 2.24.0. When a timer fires but is reset prior to being executed, a watermark hold can be leaked, causing your pipeline job to be stuck. See the Apache Beam issue tracker for more information.
2.23.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on July 29, 2021.

2.22.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
Google Cloud I/O connectors under module org.apache.beam:beam-runners-google-cloud-dataflow-java: bigquery, bigtable, datastore, healthcare, pubsub, spanner
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on June 8, 2021.

2.21.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on May 27, 2021.

2.20.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on April 15, 2021.

2.19.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on February 4, 2021.

2.18.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on January 23, 2021.

2.17.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on January 6, 2021.

2.16.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on October 7, 2020.

2.15.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on August 23, 2020.

Known issues:
  • Dataflow users who use schema features (including SQL transforms) shouldn't upgrade to 2.15.0 due to a known issue. See the Apache Beam issue tracker for more information.
2.14.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on August 1, 2020.

2.13.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on June 6, 2020.

This release adds experimental support for JDK 9 and later versions. See the Apache Beam issue tracker for more information.

2.12.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated on April 25, 2020.

Known issues:
  • The Dataflow runner has an incorrect logging configuration that might cause all logs to be missing. To work around this issue, add slf4j-jdk14 to your runtime dependencies.
2.11.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated as of March 1, 2020.

Known issues:
  • The Dataflow runner has an incorrect logging configuration that might cause all logs to be missing. To work around this issue, add slf4j-jdk14 to your runtime dependencies.
2.10.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated as of February 11, 2020.

Known issues:
  • SDK 2.10.0 depends on gcsio client library version 1.9.13, which has known issues:

    To work around these issues, either upgrade to SDK 2.11.0, or override the gcsio client library version to 1.9.16 or later.

  • The Dataflow runner has an incorrect logging configuration that might cause all logs to be missing. To work around this issue, add slf4j-jdk14 to your runtime dependencies.
2.9.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated as of December 13, 2019.

Known issues:
  • If you enable the Streaming Engine (Beta) experiment, don't upgrade to SDK 2.9.0 due to a known issue. If you choose to use SDK 2.9.0, you must also set the enable_conscrypt_security_provider experimental flag to enable Conscrypt, which has known stability issues.
  • In a specific case, users of Apache Beam Java SDKs 2.9.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.8.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated as of October 25, 2019.

Known issues:
  • Pipelines might become stuck due to an issue with the Conscrypt library. If you see errors in Stackdriver Logging with stack traces that include Conscrypt related calls, you might be affected by this issue. To resolve the issue, upgrade to SDK 2.9.0 or higher.
  • In a specific case, users of Apache Beam Java SDKs 2.9.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.7.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated as of October 3, 2019.

Known issues:
  • Pipelines might become stuck due to an issue with the Conscrypt library. If you see errors in Stackdriver Logging with stack traces that include Conscrypt related calls, you might be affected by this issue. To resolve the issue, upgrade to SDK 2.9.0 or higher.
  • In a specific case, users of Apache Beam Java SDKs 2.9.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.6.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated as of August 8, 2019.

Known issues:
  • Pipelines might become stuck due to an issue with the Conscrypt library. If you see errors in Stackdriver Logging with stack traces that include Conscrypt related calls, you might be affected by this issue. To resolve the issue, upgrade to SDK 2.9.0 or higher.
  • In a specific case, users of Apache Beam Java SDKs 2.9.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.5.0 Deprecated org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management
org.apache.beam:beam-sdks-java-io-kafka

Deprecated as of June 6, 2019.

Known issues:
  • Pipelines might become stuck due to an issue with the Conscrypt library. If you see errors in Stackdriver Logging with stack traces that include Conscrypt related calls, you might be affected by this issue. To resolve the issue, upgrade to SDK 2.9.0 or higher.
  • In a specific case, users of Apache Beam Java SDKs 2.9.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.4.0 Decommissioned org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management

Decommissioned as of Aug 12, 2020.

This version is decommissioned due to the discontinuation of support for JSON-RPC and Global HTTP Batch Endpoints.

Known issue:
  • In a specific case, users of Apache Beam Java SDKs 2.9.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.3.0 Decommissioned org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management

Decommissioned as of Aug 12, 2020.

This version is decommissioned due to the discontinuation of support for JSON-RPC and Global HTTP Batch Endpoints.

Known issue:
  • In a specific case, users of Apache Beam Java SDKs 2.9.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.2.0 Decommissioned org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management

Decommissioned as of Aug 12, 2020.

This version is decommissioned due to the discontinuation of support for JSON-RPC and Global HTTP Batch Endpoints.

Known issue:
  • In a specific case, users of Apache Beam Java SDKs 2.9.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.1.0 Decommissioned org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management

Decommissioned as of Aug 12, 2020.

This version is decommissioned due to the discontinuation of support for JSON-RPC and Global HTTP Batch Endpoints.

Known issue:
  • In a specific case, users of Apache Beam Java SDKs 2.9.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.0.0 Decommissioned org.apache.beam:beam-sdks-java-core
org.apache.beam:beam-sdks-java-io-google-cloud-platform
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
org.apache.beam:beam-sdks-java-extensions-protobuf
org.apache.beam:beam-runners-direct-java
org.apache.beam:beam-runners-google-cloud-dataflow-java
org.apache.beam:beam-model-pipeline
org.apache.beam:beam-runners-core-construction-java
org.apache.beam:beam-model-job-management

Decommissioned as of Aug 12, 2020.

This version is decommissioned due to the discontinuation of support for JSON-RPC and Global HTTP Batch Endpoints.

Known issue:
  • In a specific case, users of Apache Beam Java SDKs 2.9.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.

Python

SDK version Status Supported components Details
2.60.0 Supported Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql, yaml
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

This version will be deprecated on October 17, 2025.

2.59.0 Supported Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql, yaml
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

This version will be deprecated on September 11, 2025.

2.58.1 Supported Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql, yaml
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

This version will be deprecated on August 16, 2025.

Known issue: Pipelines that read data from Cloud Storage by using GcsIO, either directly or indirectly, such as with BigQueryIO, might experience data corruption on the Apache Beam Python SDK versions 2.53.0 to 2.58.1. For more information, see the Apache Beam GitHub issue #32169.

2.58.0 Supported Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql, yaml
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

This version will be deprecated on August 6, 2025.

Known issue: Pipelines that read data from Cloud Storage by using GcsIO, either directly or indirectly, such as with BigQueryIO, might experience data corruption on the Apache Beam Python SDK versions 2.53.0 to 2.58.1. For more information, see the Apache Beam GitHub issue #32169.

2.57.0 Supported Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql, yaml
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

This version will be deprecated on June 26, 2025.

Known issue: Pipelines that read data from Cloud Storage by using GcsIO, either directly or indirectly, such as with BigQueryIO, might experience data corruption on the Apache Beam Python SDK versions 2.53.0 to 2.58.1. For more information, see the Apache Beam GitHub issue #32169.

2.56.0 Supported Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql, yaml
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

This version will be deprecated on May 1, 2025.

Known issue: Pipelines that read data from Cloud Storage by using GcsIO, either directly or indirectly, such as with BigQueryIO, might experience data corruption on the Apache Beam Python SDK versions 2.53.0 to 2.58.1. For more information, see the Apache Beam GitHub issue #32169.

2.55.1 Supported Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql, yaml
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

This version will be deprecated on April 8, 2025.

Known issue: Python pipelines that use this SDK might experience occasional stuckness. For more information, see the Apache Beam GitHub issue #30867.

Known issue: Pipelines that read data from Cloud Storage by using GcsIO, either directly or indirectly, such as with BigQueryIO, might experience data corruption on the Apache Beam Python SDK versions 2.53.0 to 2.58.1. For more information, see the Apache Beam GitHub issue #32169.

2.55.0 Supported Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql, yaml
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

This version will be deprecated on March 25, 2025.

Known issue: The WriteToJson transform in cross-language pipelines fails with an error. To resolve this issue, upgrade to Apache Beam 2.55.1. For more information, see Issue #30776: Beam YAML WriteToJson fails.

Known issue: Python pipelines that use this SDK might experience occasional stuckness. For more information, see the Apache Beam GitHub issue #30867.

Known issue: Pipelines that read data from Cloud Storage by using GcsIO, either directly or indirectly, such as with BigQueryIO, might experience data corruption on the Apache Beam Python SDK versions 2.53.0 to 2.58.1. For more information, see the Apache Beam GitHub issue #32169.

2.54.0 Supported Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql, yaml
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

This version will be deprecated on February 14, 2025.

Known issue: Pipelines that read data from Cloud Storage by using GcsIO, either directly or indirectly, such as with BigQueryIO, might experience data corruption on the Apache Beam Python SDK versions 2.53.0 to 2.58.1. For more information, see the Apache Beam GitHub issue #32169.

2.53.0 Supported Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

This version will be deprecated on January 4, 2025.

Known issue: Pipelines that read data from Cloud Storage by using GcsIO, either directly or indirectly, such as with BigQueryIO, might experience data corruption on the Apache Beam Python SDK versions 2.53.0 to 2.58.1. For more information, see the Apache Beam GitHub issue #32169.

2.52.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on November 17, 2024.

2.51.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on October 12, 2024.

2.50.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on August 30, 2024.

2.49.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on July 17, 2024.

2.48.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on May 31, 2024.

2.47.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on May 10, 2024.

2.46.0 Supported Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

This version will be deprecated on March 10, 2025. The support timeframe for this version has been extended due to known issues discovered in later releases.

2.45.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on February 15, 2024.

2.44.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on January 13, 2024.

2.43.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on November 17, 2023.

2.42.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on October 17, 2023.

2.41.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on August 23, 2023.

2.40.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, ml.inference (excluding PytorchModelHandlerKeyedTensor and SklearnModelHandlerPandas), options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on June 27, 2023.

2.39.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on May 25, 2023.

2.38.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on April 20, 2023.

2.37.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on March 4, 2023.

Known issues:
  • When you use WriteToBigQuery with the ignore_unknown_columns parameter set to true, the parameter value is ignored. This issue is fixed in version 2.38.0. See BEAM-14039 for details.
2.36.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on February 7, 2023.

Known issues:
  • When you use WriteToBigQuery with the ignore_unknown_columns parameter set to true, the parameter value is ignored. This issue is fixed in version 2.38.0. See BEAM-14039 for details.
2.35.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub
The connectors under apache_beam.io.jdbc: ReadFromJdbc, WriteToJdbc.

Deprecated on December 29, 2022.

2.34.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints, sql
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub

Deprecated on November 11, 2022.

2.33.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub

Deprecated on October 7, 2022.

2.32.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub

Deprecated on August 25, 2022.

2.31.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe (preview), metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem, filesystems, fileio
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub

Deprecated on July 8, 2022.

2.30.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe (preview), metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub

Deprecated on June 10, 2022.

2.29.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe (preview), metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub

Deprecated on April 29, 2022.

2.28.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe (preview), metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub

Deprecated on February 22, 2022.

2.27.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe (preview), metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub

Deprecated on January 7, 2022.

2.26.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, dataframe (preview), metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub

Deprecated on December 11, 2021.

2.25.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub

Deprecated on October 23, 2021.

2.24.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub

Deprecated on September 18, 2021. Also, the Apache Beam SDK 2.24.0 version is the last to support Python 2 and Python 3.5.

Dataflow no longer supports pipelines using Python 2. Read more information on the Python 2 support on Google Cloud page.

2.23.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub

Deprecated on July 29, 2021.

Dataflow no longer supports pipelines using Python 2. Read more information on the Python 2 support on Google Cloud page.

2.22.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp: bigquery, datastore, pubsub

Deprecated on June 8, 2021.

Dataflow no longer supports pipelines using Python 2. Read more information on the Python 2 support on Google Cloud page.

2.21.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp

Deprecated on May 27, 2021.

Dataflow no longer supports pipelines using Python 2. Read more information on the Python 2 support on Google Cloud page.

2.20.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp

Deprecated on April 15, 2021.

Dataflow no longer supports pipelines using Python 2. Read more information on the Python 2 support on Google Cloud page.

2.19.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp

Deprecated on February 4, 2021.

Dataflow no longer supports pipelines using Python 2. Read more information on the Python 2 support on Google Cloud page.

2.18.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp

Deprecated on January 23, 2021.

Dataflow no longer supports pipelines using Python 2. Read more information on the Python 2 support on Google Cloud page.

2.17.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp

Deprecated on January 6, 2021.

Dataflow no longer supports pipelines using Python 2. Read more information on the Python 2 support on Google Cloud page.

2.16.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp
Deprecated on October 7, 2020.
2.15.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp
Deprecated on August 23, 2020.
2.14.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp

Deprecated on August 1, 2020.

Known issues:
  • The MongoDB source added in this release has a known issue that can result in data loss. See BEAM-7866 for details.
2.13.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp
Deprecated as of June 6, 2020.
2.12.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp
Deprecated as of April 25, 2020.
2.11.0 Deprecated Core Python SDK library under module apache_beam: submodules coders, metrics, options, portability, runners.dataflow, runners.direct, transforms, typehints
File-based sources and sinks and related modules under module apache_beam.io: textio, avroio, parquetio, tfrecordio, gcsfilesystem, localfilesystem
Google Cloud I/O connectors under module apache_beam.io.gcp
Deprecated as of March 1, 2020.
2.10.0 Deprecated Deprecated as of February 11, 2020.
2.9.0 Deprecated Deprecated as of December 13, 2019.
2.8.0 Deprecated Deprecated as of October 25, 2019.
2.7.0 Deprecated

Deprecated as of October 3, 2019.

2.6.0 Deprecated Deprecated as of August 8, 2019.
2.5.0 Deprecated Deprecated as of June 6, 2019.
2.4.0 Deprecated Deprecated as of March 20, 2019.
2.3.0 Deprecated Deprecated as of January 30, 2019.
2.2.0 Deprecated Deprecated as of December 2, 2018.
2.1.1
2.1.0
Deprecated Deprecated as of August 23, 2018.
2.0.0 Deprecated Deprecated as of May 17, 2018.

Go

SDK version Status Supported components Details
2.60.0 Supported Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
This version will be deprecated on October 17, 2025.
2.59.0 Supported Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
This version will be deprecated on September 11, 2025.
2.58.1 Supported Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
This version will be deprecated on August 16, 2025.
2.58.0 Supported Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
This version will be deprecated on August 6, 2025.
2.57.0 Supported Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
This version will be deprecated on June 26, 2025.
2.56.0 Supported Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
This version will be deprecated on May 1, 2025.
2.55.1 Supported Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
This version will be deprecated on April 8, 2025.
2.55.0 Supported Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
This version will be deprecated on March 25, 2025.
2.54.0 Supported Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
This version will be deprecated on February 14, 2025.
2.53.0 Supported Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
This version will be deprecated on January 4, 2025.
2.52.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
Deprecated on November 17, 2024.
2.51.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
Deprecated on October 12, 2024.
2.50.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
Deprecated on August 30, 2024.
2.49.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
Deprecated on July 17, 2024.
2.48.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
Deprecated on May 31, 2024.
2.47.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
Deprecated on May 10, 2024.
2.46.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
Deprecated on March 10, 2024.
2.45.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
Deprecated on February 15, 2024.
2.44.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
Deprecated on January 13, 2024.
2.43.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
Deprecated on November 17, 2023.
2.42.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
Deprecated on October 17, 2023.
2.41.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
Deprecated on August 23, 2023.
2.40.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
Deprecated on June 27, 2023.
2.39.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/xlang
Deprecated on May 25, 2023.
2.38.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/pubsubio
- beam/io/xlang
Deprecated on April 20, 2023.
2.37.0 Deprecated Packages under the following directories:
- Core Beam Go SDK package
- beam/transforms
- beam/io/textio
- beam/io/xlang
Deprecated on March 4, 2023.

Dataflow 2.x SDKs

The following tables contain the support status for the Dataflow 2.x SDKs, including the Dataflow Java SDK and the Dataflow Python SDK:

Java

See the Dataflow SDK 2.x for Java release notes for change information.

SDK version Status Details
2.5.0 Deprecated

Deprecated as of June 6, 2019.

Known issue:
  • In a specific case, users of Dataflow Java SDKs 2.5.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.4.0 Decommissioned

Decommissioned as of Aug 12, 2020.

This version is decommissioned due to the discontinuation of support for JSON-RPC and Global HTTP Batch Endpoints.

Known issue:
  • In a specific case, users of Dataflow Java SDKs 2.5.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.3.0 Decommissioned

Decommissioned as of Aug 12, 2020.

This version is decommissioned due to the discontinuation of support for JSON-RPC and Global HTTP Batch Endpoints.

Known issue:
  • In a specific case, users of Dataflow Java SDKs 2.5.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.2.0 Decommissioned

Decommissioned as of Aug 12, 2020.

This version is decommissioned due to the discontinuation of support for JSON-RPC and Global HTTP Batch Endpoints.

Known issue:
  • In a specific case, users of Dataflow Java SDKs 2.5.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.1.0 Decommissioned

Decommissioned as of Aug 12, 2020.

This version is decommissioned due to the discontinuation of support for JSON-RPC and Global HTTP Batch Endpoints.

Known issue:
  • In a specific case, users of Dataflow Java SDKs 2.5.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.0.0 Decommissioned

Decommissioned as of Aug 12, 2020.

This version is decommissioned due to the discontinuation of support for JSON-RPC and Global HTTP Batch Endpoints.

Known issue:
  • In a specific case, users of Dataflow Java SDKs 2.5.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.
2.0.0-beta3
2.0.0-beta2
2.0.0-beta1
Decommissioned

Decommissioned as of February 28, 2018.

Known issue:
  • In a specific case, users of Dataflow Java SDKs 2.5.0 and earlier might experience data duplication when reading files from Cloud Storage. Duplication might occur when all of the following conditions are true:
    • You're reading files with the content-encoding set to gzip, and the files are dynamically decompressive transcoded by Cloud Storage.
    • The file size (decompressed) is larger than 2.14 GB.
    • The input stream runs into an error (and is recreated) after 2.14 GB is read.
    As a workaround, don't set the content-encoding header, and store compressed files in Cloud Storage with the proper extension (for example, gz for gzip). For existing files, you can update the content-encoding header and filename with the gcloud storage commands.

Python

See the Dataflow SDK 2.x for Python release notes for change information.

SDK version Status Details
2.5.0 Deprecated Deprecated as of June 6, 2019.
2.4.0 Deprecated Deprecated as of March 20, 2019.
2.3.0 Deprecated Deprecated as of January 30, 2019.
2.2.0 Deprecated Deprecated as of December 2, 2018.
2.1.1 Deprecated Deprecated as of August 23, 2018.
Fixes a compatibility issue with the Python six package.
See the release notes for more information.
2.1.0 Deprecated Deprecated as of August 23, 2018.
This release has a compatibility issue with the Python six 1.11.0 package.
See the Release Notes for more information.
2.0.0 Deprecated Deprecated as of May 17, 2018.
This release has a compatibility issue with the Python six 1.11.0 package.
See the Release Notes for more information.
0.6.0
0.5.5
0.5.1
0.4.4
0.4.3
0.4.2
0.4.1
0.4.0
Decommissioned Decommissioned as of January 29, 2018.
0.2.7 and earlier versions Decommissioned Decommissioned as of March 23, 2017.