Service options are a type of pipeline option that allows you to specify
additional job modes and configurations for a Dataflow job. Set
these options by setting the dataflowServiceOptions
pipeline option. For more
information, see
Set Dataflow pipeline options.
Dataflow supports the following service options.
| Option | Description |
|---|---|
automatically_use_created_reservation |
Use Compute Engine reservations for the Dataflow workers. For more information, see Use Compute Engine reservations with Dataflow |
block-project-ssh-keys |
Prevents VMs from accepting SSH keys that are stored in project metadata. For more information, see Restrict SSH keys from VMs. |
enable_google_cloud_heap_sampling |
Enable heap profiling. For more information, see Monitoring pipeline performance using Cloud Profile. |
enable_google_cloud_profiler |
Enable performance profiling. For more information, see Monitoring pipeline performance using Cloud Profile. |
enable_prime |
Enable Dataflow Prime for this job. For more information, see Use Dataflow Prime. |
max_workflow_runtime_walltime_seconds |
The maximum number of seconds the job can run. If the job exceeds this limit, Dataflow cancels the job. This service option is currently supported for batch jobs only. Specify the number of seconds as a parameter to the flag. Example:
|