Join the Apache Beam community on July 18th-20th for the Beam Summit 2022 to learn more about Beam and share your expertise.

SDK and Worker Dependencies

The Apache Beam SDKs and Dataflow workers depend on common third-party components which then import additional dependencies. Version collisions can result in unexpected behavior in the service. If you are using any of these packages in your code, be aware that some libraries are not forward-compatible and you may need to pin to the listed versions that will be in scope during execution.

This page contains dependency and worker package information for Apache Beam and Dataflow SDK releases:

Apache Beam 2.x SDKs

SDK for Java

SDK for Python

Version 2.38.0

Dataflow workers have the following packages already installed.

Python 3.6.15

PackageVersion
absl-py0.15.0
astunparse1.6.3
atomicwrites1.4.0
attrs21.4.0
avro-python31.8.2
beautifulsoup44.10.0
bs40.0.1
cached-property1.5.2
cachetools4.2.4
certifi2021.10.8
charset-normalizer2.0.11
clang5.0
click8.0.3
cloudpickle2.0.0
crcmod1.7
Cython0.29.27
dataclasses0.8
dill0.3.1.1
docopt0.6.2
fastavro1.4.7
fasteners0.17.3
flatbuffers1.12
future0.18.2
gast0.4.0
google-api-core1.31.5
google-api-python-client2.36.0
google-apitools0.5.31
google-auth1.35.0
google-auth-httplib20.1.0
google-auth-oauthlib0.4.6
google-cloud-bigquery2.32.0
google-cloud-bigquery-storage2.11.0
google-cloud-bigtable1.7.0
google-cloud-core1.7.2
google-cloud-datastore1.15.3
google-cloud-dlp3.6.0
google-cloud-language1.3.0
google-cloud-profiler3.0.7
google-cloud-pubsub2.9.0
google-cloud-pubsublite1.3.0
google-cloud-recommendations-ai0.2.0
google-cloud-spanner1.19.1
google-cloud-videointelligence1.16.1
google-cloud-vision1.0.0
google-crc32c1.3.0
google-pasta0.2.0
google-python-cloud-debugger2.18
google-resumable-media2.2.0
googleapis-common-protos1.54.0
grpc-google-iam-v10.12.3
grpcio1.43.0
grpcio-gcp0.2.2
grpcio-status1.43.0
guppy33.1.2
h5py3.1.0
hdfs2.6.0
httplib20.19.1
idna3.3
importlib-metadata4.8.3
joblib1.1.0
keras2.6.0
Keras-Preprocessing1.1.2
libcst0.4.1
Markdown3.3.6
mmh33.0.0
more-itertools8.12.0
mypy-extensions0.4.3
nltk3.6.5
nose1.3.7
numpy1.19.5
oauth2client4.1.3
oauthlib3.2.0
opt-einsum3.3.0
orjson3.6.1
overrides6.1.0
packaging21.3
pandas1.1.5
Pillow7.2.0
pip21.3.1
pluggy0.13.1
proto-plus1.19.9
protobuf3.19.4
py1.11.0
pyarrow6.0.1
pyasn10.4.8
pyasn1-modules0.2.8
pydot1.4.2
pymongo3.12.3
pyparsing2.4.7
pytest4.6.11
python-dateutil2.8.2
python-snappy0.6.0
pytz2021.3
PyYAML6.0
regex2022.1.18
requests2.27.1
requests-oauthlib1.3.1
rsa4.8
scikit-learn0.24.2
scipy1.5.4
setuptools57.5.0
six1.15.0
soupsieve2.3.1
tensorboard2.6.0
tensorboard-data-server0.6.1
tensorboard-plugin-wit1.8.1
tensorflow2.6.2
tensorflow-estimator2.6.0
termcolor1.1.0
threadpoolctl3.1.0
tqdm4.62.3
typing-extensions3.7.4.3
typing-inspect0.7.1
typing-utils0.1.0
uritemplate4.1.1
urllib31.26.8
wcwidth0.2.5
Werkzeug2.0.3
wheel0.37.0
wrapt1.12.1
zipp3.6.0

Python 3.7.12

PackageVersion
absl-py1.0.0
astunparse1.6.3
atomicwrites1.4.0
attrs21.4.0
avro-python31.8.2
beautifulsoup44.10.0
bs40.0.1
cached-property1.5.2
cachetools4.2.4
certifi2021.10.8
charset-normalizer2.0.11
click8.0.3
cloudpickle2.0.0
crcmod1.7
Cython0.29.27
dill0.3.1.1
docopt0.6.2
fastavro1.4.9
fasteners0.17.3
flatbuffers2.0
future0.18.2
gast0.5.3
google-api-core1.31.5
google-api-python-client2.36.0
google-apitools0.5.31
google-auth1.35.0
google-auth-httplib20.1.0
google-auth-oauthlib0.4.6
google-cloud-bigquery2.32.0
google-cloud-bigquery-storage2.11.0
google-cloud-bigtable1.7.0
google-cloud-core1.7.2
google-cloud-datastore1.15.3
google-cloud-dlp3.6.0
google-cloud-language1.3.0
google-cloud-profiler3.0.7
google-cloud-pubsub2.9.0
google-cloud-pubsublite1.3.0
google-cloud-recommendations-ai0.2.0
google-cloud-spanner1.19.1
google-cloud-videointelligence1.16.1
google-cloud-vision1.0.0
google-crc32c1.3.0
google-pasta0.2.0
google-python-cloud-debugger2.18
google-resumable-media2.2.0
googleapis-common-protos1.54.0
grpc-google-iam-v10.12.3
grpcio1.43.0
grpcio-gcp0.2.2
grpcio-status1.43.0
guppy33.1.2
h5py3.6.0
hdfs2.6.0
httplib20.19.1
idna3.3
importlib-metadata4.10.1
joblib1.1.0
keras2.8.0
Keras-Preprocessing1.1.2
libclang13.0.0
libcst0.4.1
Markdown3.3.6
mmh33.0.0
more-itertools8.12.0
mypy-extensions0.4.3
nltk3.6.5
nose1.3.7
numpy1.21.5
oauth2client4.1.3
oauthlib3.2.0
opt-einsum3.3.0
orjson3.6.6
overrides6.1.0
packaging21.3
pandas1.3.5
Pillow7.2.0
pip22.0.3
pluggy0.13.1
proto-plus1.19.9
protobuf3.19.4
py1.11.0
pyarrow6.0.1
pyasn10.4.8
pyasn1-modules0.2.8
pydot1.4.2
pymongo3.12.3
pyparsing2.4.7
pytest4.6.11
python-dateutil2.8.2
python-snappy0.6.0
pytz2021.3
PyYAML6.0
regex2022.1.18
requests2.27.1
requests-oauthlib1.3.1
rsa4.8
scikit-learn1.0.2
scipy1.7.3
setuptools57.5.0
six1.16.0
soupsieve2.3.1
tensorboard2.8.0
tensorboard-data-server0.6.1
tensorboard-plugin-wit1.8.1
tensorflow2.8.0
tensorflow-io-gcs-filesystem0.24.0
termcolor1.1.0
tf-estimator-nightly2.8.0.dev2021122109
threadpoolctl3.1.0
tqdm4.62.3
typing-inspect0.7.1
typing-utils0.1.0
typing_extensions4.0.1
uritemplate4.1.1
urllib31.26.8
wcwidth0.2.5
Werkzeug2.0.3
wheel0.37.1
wrapt1.13.3
zipp3.7.0

Python 3.8.12

PackageVersion
absl-py1.0.0
astunparse1.6.3
atomicwrites1.4.0
attrs21.4.0
avro-python31.8.2
beautifulsoup44.10.0
bs40.0.1
cachetools4.2.4
certifi2021.10.8
charset-normalizer2.0.11
click8.0.3
cloudpickle2.0.0
crcmod1.7
Cython0.29.27
dill0.3.1.1
docopt0.6.2
fastavro1.4.9
fasteners0.17.3
flatbuffers2.0
future0.18.2
gast0.5.3
google-api-core1.31.5
google-api-python-client2.36.0
google-apitools0.5.31
google-auth1.35.0
google-auth-httplib20.1.0
google-auth-oauthlib0.4.6
google-cloud-bigquery2.32.0
google-cloud-bigquery-storage2.11.0
google-cloud-bigtable1.7.0
google-cloud-core1.7.2
google-cloud-datastore1.15.3
google-cloud-dlp3.6.0
google-cloud-language1.3.0
google-cloud-profiler3.0.7
google-cloud-pubsub2.9.0
google-cloud-pubsublite1.3.0
google-cloud-recommendations-ai0.2.0
google-cloud-spanner1.19.1
google-cloud-videointelligence1.16.1
google-cloud-vision1.0.0
google-crc32c1.3.0
google-pasta0.2.0
google-python-cloud-debugger2.18
google-resumable-media2.2.0
googleapis-common-protos1.54.0
grpc-google-iam-v10.12.3
grpcio1.43.0
grpcio-gcp0.2.2
grpcio-status1.43.0
guppy33.1.2
h5py3.6.0
hdfs2.6.0
httplib20.19.1
idna3.3
importlib-metadata4.10.1
joblib1.1.0
keras2.8.0
Keras-Preprocessing1.1.2
libclang13.0.0
libcst0.4.1
Markdown3.3.6
mmh33.0.0
more-itertools8.12.0
mypy-extensions0.4.3
nltk3.6.5
nose1.3.7
numpy1.21.5
oauth2client4.1.3
oauthlib3.2.0
opt-einsum3.3.0
orjson3.6.6
overrides6.1.0
packaging21.3
pandas1.4.0
Pillow7.2.0
pip22.0.3
pluggy0.13.1
proto-plus1.19.9
protobuf3.19.4
py1.11.0
pyarrow6.0.1
pyasn10.4.8
pyasn1-modules0.2.8
pydot1.4.2
pymongo3.12.3
pyparsing2.4.7
pytest4.6.11
python-dateutil2.8.2
python-snappy0.6.0
pytz2021.3
PyYAML6.0
regex2022.1.18
requests2.27.1
requests-oauthlib1.3.1
rsa4.8
scikit-learn1.0.2
scipy1.7.3
setuptools57.5.0
six1.16.0
soupsieve2.3.1
tensorboard2.8.0
tensorboard-data-server0.6.1
tensorboard-plugin-wit1.8.1
tensorflow2.8.0
tensorflow-io-gcs-filesystem0.24.0
termcolor1.1.0
tf-estimator-nightly2.8.0.dev2021122109
threadpoolctl3.1.0
tqdm4.62.3
typing-inspect0.7.1
typing-utils0.1.0
typing_extensions4.0.1
uritemplate4.1.1
urllib31.26.8
wcwidth0.2.5
Werkzeug2.0.3
wheel0.37.1
wrapt1.13.3
zipp3.7.0

Python 3.9.10

PackageVersion
absl-py1.0.0
astunparse1.6.3
atomicwrites1.4.0
attrs21.4.0
avro-python31.8.2
beautifulsoup44.10.0
bs40.0.1
cachetools4.2.4
certifi2021.10.8
charset-normalizer2.0.11
click8.0.3
cloudpickle2.0.0
crcmod1.7
Cython0.29.27
dill0.3.1.1
docopt0.6.2
fastavro1.4.9
fasteners0.17.3
flatbuffers2.0
future0.18.2
gast0.5.3
google-api-core1.31.5
google-api-python-client2.36.0
google-apitools0.5.31
google-auth1.35.0
google-auth-httplib20.1.0
google-auth-oauthlib0.4.6
google-cloud-bigquery2.32.0
google-cloud-bigquery-storage2.11.0
google-cloud-bigtable1.7.0
google-cloud-core1.7.2
google-cloud-datastore1.15.3
google-cloud-dlp3.6.0
google-cloud-language1.3.0
google-cloud-profiler3.0.7
google-cloud-pubsub2.9.0
google-cloud-pubsublite1.3.0
google-cloud-recommendations-ai0.2.0
google-cloud-spanner1.19.1
google-cloud-videointelligence1.16.1
google-cloud-vision1.0.0
google-crc32c1.3.0
google-pasta0.2.0
google-python-cloud-debugger2.18
google-resumable-media2.2.0
googleapis-common-protos1.54.0
grpc-google-iam-v10.12.3
grpcio1.43.0
grpcio-gcp0.2.2
grpcio-status1.43.0
guppy33.1.2
h5py3.6.0
hdfs2.6.0
httplib20.19.1
idna3.3
importlib-metadata4.10.1
joblib1.1.0
keras2.8.0
Keras-Preprocessing1.1.2
libclang13.0.0
libcst0.4.1
Markdown3.3.6
mmh33.0.0
more-itertools8.12.0
mypy-extensions0.4.3
nltk3.6.5
nose1.3.7
numpy1.21.5
oauth2client4.1.3
oauthlib3.2.0
opt-einsum3.3.0
orjson3.6.6
overrides6.1.0
packaging21.3
pandas1.4.0
Pillow7.2.0
pip22.0.3
pluggy0.13.1
proto-plus1.19.9
protobuf3.19.4
py1.11.0
pyarrow6.0.1
pyasn10.4.8
pyasn1-modules0.2.8
pydot1.4.2
pymongo3.12.3
pyparsing2.4.7
pytest4.6.11
python-dateutil2.8.2
python-snappy0.6.0
pytz2021.3
PyYAML6.0
regex2022.1.18
requests2.27.1
requests-oauthlib1.3.1
rsa4.8
scikit-learn1.0.2
scipy1.7.3
setuptools58.1.0
six1.16.0
soupsieve2.3.1
tensorboard2.8.0
tensorboard-data-server0.6.1
tensorboard-plugin-wit1.8.1
tensorflow2.8.0
tensorflow-io-gcs-filesystem0.24.0
termcolor1.1.0
tf-estimator-nightly2.8.0.dev2021122109
threadpoolctl3.1.0
tqdm4.62.3
typing-inspect0.7.1
typing-utils0.1.0
typing_extensions4.0.1
uritemplate4.1.1
urllib31.26.8
wcwidth0.2.5
Werkzeug2.0.3
wheel0.37.1
wrapt1.13.3
zipp3.7.0