DataflowWorkerHarnessOptions (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. After August 12, 2020, Dataflow will not run jobs using Dataflow 1.x and below. See Migrating from Dataflow SDK 1.x for Java for migration guidance.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

com.google.cloud.dataflow.sdk.options

Interface DataflowWorkerHarnessOptions

• All Superinterfaces:

  ApplicationNameOptions, BigQueryOptions, CloudDebuggerOptions, DataflowPipelineDebugOptions, DataflowPipelineOptions, DataflowPipelineWorkerPoolOptions, DataflowProfilingOptions, DataflowWorkerLoggingOptions, GcpOptions, GcsOptions, GoogleApiDebugOptions, HasDisplayData, PipelineOptions, StreamingOptions

  ---

  
  

  @Hidden
  public interface DataflowWorkerHarnessOptions
  extends DataflowPipelineOptions

  Options that are used exclusively within the Dataflow worker harness. These options have no effect at pipeline creation time.

• • Nested Class Summary

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions

      DataflowPipelineOptions.JobNameFactory

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions

      DataflowPipelineDebugOptions.DataflowClientFactory, DataflowPipelineDebugOptions.PathValidatorFactory, DataflowPipelineDebugOptions.StagerFactory

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions

      DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType, DataflowPipelineWorkerPoolOptions.TeardownPolicy, DataflowPipelineWorkerPoolOptions.WorkerHarnessContainerImageFactory

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.GcpOptions

      GcpOptions.CredentialDirFactory, GcpOptions.DefaultProjectFactory, GcpOptions.GcpUserCredentialsFactory

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions

      GoogleApiDebugOptions.GoogleApiTracer

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.PipelineOptions

      PipelineOptions.AtomicLongFactory, PipelineOptions.CheckEnabled

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.GcsOptions

      GcsOptions.ExecutorServiceFactory

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions

      DataflowWorkerLoggingOptions.Level, DataflowWorkerLoggingOptions.WorkerLogLevelOverrides

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions

      DataflowProfilingOptions.DataflowProfilingAgentConfiguration

  • Method Summary

    All Methods Instance Methods Abstract Methods 

    Modifier and Type	Method and Description
    String	getJobId() The identity of the Dataflow job.
    Integer	getWorkerCacheMb() The size of the worker's in-memory cache, in megabytes.
    String	getWorkerId() The identity of the worker running this pipeline.
    void	setJobId(String value)
    void	setWorkerCacheMb(Integer value)
    void	setWorkerId(String value)

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions

      getJobName, getLabels, getProject, getServiceAccount, getStagingLocation, getUpdate, setJobName, setLabels, setProject, setServiceAccount, setStagingLocation, setUpdate

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions

      getApiRootUrl, getDataflowClient, getDataflowEndpoint, getDataflowJobFile, getDumpHeapOnOOM, getExperiments, getNumberOfWorkerHarnessThreads, getOverrideWindmillBinary, getPathValidator, getPathValidatorClass, getPubsubRootUrl, getStager, getStagerClass, getTransformNameMapping, getWindmillServiceEndpoint, getWindmillServicePort, setApiRootUrl, setDataflowClient, setDataflowEndpoint, setDataflowJobFile, setDumpHeapOnOOM, setExperiments, setNumberOfWorkerHarnessThreads, setOverrideWindmillBinary, setPathValidator, setPathValidatorClass, setPubsubRootUrl, setStager, setStagerClass, setTransformNameMapping, setWindmillServiceEndpoint, setWindmillServicePort

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions

      getAutoscalingAlgorithm, getDiskSizeGb, getFilesToStage, getMaxNumWorkers, getNetwork, getNumWorkers, getSubnetwork, getTeardownPolicy, getUsePublicIps, getWorkerDiskType, getWorkerHarnessContainerImage, getWorkerMachineType, getZone, setAutoscalingAlgorithm, setDiskSizeGb, setFilesToStage, setMaxNumWorkers, setNetwork, setNumWorkers, setSubnetwork, setTeardownPolicy, setUsePublicIps, setWorkerDiskType, setWorkerHarnessContainerImage, setWorkerMachineType, setZone

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.BigQueryOptions

      getTempDatasetId, setTempDatasetId

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.StreamingOptions

      isStreaming, setStreaming

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.ApplicationNameOptions

      getAppName, setAppName

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.GcpOptions

      getAuthorizationServerEncodedUrl, getCredentialDir, getCredentialFactoryClass, getCredentialId, getGcpCredential, getSecretsFile, getServiceAccountKeyfile, getServiceAccountName, getTokenServerUrl, setAuthorizationServerEncodedUrl, setCredentialDir, setCredentialFactoryClass, setCredentialId, setGcpCredential, setSecretsFile, setServiceAccountKeyfile, setServiceAccountName, setTokenServerUrl

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions

      getGoogleApiTrace, setGoogleApiTrace

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.PipelineOptions

      as, cloneAs, getOptionsId, getRunner, getStableUniqueNames, getTempLocation, outputRuntimeOptions, setOptionsId, setRunner, setStableUniqueNames, setTempLocation

    • Methods inherited from interface com.google.cloud.dataflow.sdk.transforms.display.HasDisplayData

      populateDisplayData

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.GcsOptions

      getExecutorService, getGcsEndpoint, getGcsUploadBufferSizeBytes, getGcsUtil, setExecutorService, setGcsEndpoint, setGcsUploadBufferSizeBytes, setGcsUtil

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.CloudDebuggerOptions

      getDebuggee, getEnableCloudDebugger, getMaxConditionCost, setDebuggee, setEnableCloudDebugger, setMaxConditionCost

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions

      getDefaultWorkerLogLevel, getWorkerLogLevelOverrides, setDefaultWorkerLogLevel, setWorkerLogLevelOverrides

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions

      getEnableProfilingAgent, getProfilingAgentConfiguration, getSaveProfilesToGcs, setEnableProfilingAgent, setProfilingAgentConfiguration, setSaveProfilesToGcs

• • Method Detail

    • getWorkerId

      String getWorkerId()

      The identity of the worker running this pipeline.

    • setWorkerId

      void setWorkerId(String value)

    • getJobId

      String getJobId()

      The identity of the Dataflow job.

    • setJobId

      void setJobId(String value)

    • getWorkerCacheMb

      @Default.Integer(value=100)
      Integer getWorkerCacheMb()

      The size of the worker's in-memory cache, in megabytes.

      Currently, this cache is used for storing read values of side inputs.

    • setWorkerCacheMb

      void setWorkerCacheMb(Integer value)