- NAME
-
- gcloud beta dataflow jobs update-options - update pipeline options on-the-fly for running Dataflow jobs
- SYNOPSIS
-
-
gcloud beta dataflow jobs update-optionsJOB_ID[--max-num-workers=MAX_NUM_WORKERS] [--min-num-workers=MIN_NUM_WORKERS] [--region=REGION_ID] [--unset-worker-utilization-hint] [--worker-utilization-hint=WORKER_UTILIZATION_HINT] [GCLOUD_WIDE_FLAG …]
-
- DESCRIPTION
-
(BETA)This command can modify properties of running Dataflow jobs. Currently, only updating autoscaling settings for Streaming Engine jobs is supported.Adjust the autoscaling settings for Streaming Engine Dataflow jobs by providing at-least one of --min-num-workers or --max-num-workers or --worker-utilization-hint (or all 3), or --unset-worker-utilization-hint (which cannot be run at the same time as --worker-utilization-hint but works with the others). Allow a few minutes for the changes to take effect.
Note that autoscaling settings can only be modified on-the-fly for Streaming Engine jobs. Attempts to modify batch job or Streaming Appliance jobs will fail.
- EXAMPLES
-
Modify autoscaling settings to scale between 5-10 workers:
gcloud beta dataflow jobs update-options --min-num-workers=5 --max-num-workers=10Require a job to use at least 2 workers:
gcloud beta dataflow jobs update-options --min-num-workers=2Require a job to use at most 20 workers:
gcloud beta dataflow jobs update-options --max-num-workers=20Adjust the hint of target worker utilization to 70% for horizontal autoscaling:
gcloud beta dataflow jobs update-options --worker-utilization-hint=0.7"Unset" worker utilization hint so that horizontal scaling will rely on its default CPU utilization target:
gcloud beta dataflow jobs update-options --unset-worker-utilization-hint - POSITIONAL ARGUMENTS
-
JOB_ID- Job ID to operate on.
- FLAGS
-
--max-num-workers=MAX_NUM_WORKERS- Upper-bound for autoscaling, between 1-1000. Only supported for streaming-engine jobs.
--min-num-workers=MIN_NUM_WORKERS- Lower-bound for autoscaling, between 1-1000. Only supported for streaming-engine jobs.
--region=REGION_ID- Region ID of the job's regional endpoint. Defaults to 'us-central1'.
--unset-worker-utilization-hint- Unset --worker-utilization-hint. This causes the job autoscaling to fall back to internal tunings if they exist, or otherwise use the default hint value.
--worker-utilization-hint=WORKER_UTILIZATION_HINT- Target CPU utilization for autoscaling, ranging from 0.1 to 0.9. Only supported for streaming-engine jobs with autoscaling enabled.
- GCLOUD WIDE FLAGS
-
These flags are available to all commands:
--access-token-file,--account,--billing-project,--configuration,--flags-file,--flatten,--format,--help,--impersonate-service-account,--log-http,--project,--quiet,--trace-token,--user-output-enabled,--verbosity.Run
$ gcloud helpfor details. - NOTES
-
This command is currently in beta and might change without notice. This variant
is also available:
gcloud dataflow jobs update-options
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2024-03-05 UTC.