DataflowPipelineDebugOptions (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

com.google.cloud.dataflow.sdk.options

Interface DataflowPipelineDebugOptions

    • Method Detail

      • getExperiments

        @Experimental
        List<String> getExperiments()
        The list of backend experiments to enable.

        Dataflow provides a number of experimental features that can be enabled with this flag.

        Please sync with the Dataflow team before enabling any experiments.

      • setExperiments

        void setExperiments(List<String> value)
      • getApiRootUrl

        @Default.String(value="https://dataflow.googleapis.com/")
        String getApiRootUrl()
        The root URL for the Dataflow API. dataflowEndpoint can override this value if it contains an absolute URL, otherwise apiRootUrl will be combined with dataflowEndpoint to generate the full URL to communicate with the Dataflow API.
      • setApiRootUrl

        void setApiRootUrl(String value)
      • getDataflowEndpoint

        @Default.String(value="")
        String getDataflowEndpoint()
        Dataflow endpoint to use.

        Defaults to the current version of the Google Cloud Dataflow API, at the time the current SDK version was released.

        If the string contains "://", then this is treated as a URL, otherwise getApiRootUrl() is used as the root URL.

      • setDataflowEndpoint

        void setDataflowEndpoint(String value)
      • getDataflowJobFile

        String getDataflowJobFile()
        The path to write the translated Dataflow job specification out to at job submission time. The Dataflow job specification will be represented in JSON format.
      • setDataflowJobFile

        void setDataflowJobFile(String value)
      • getPathValidatorClass

        @Default.Class(value=com.google.cloud.dataflow.sdk.util.DataflowPathValidator.class)
        Class<? extends com.google.cloud.dataflow.sdk.util.PathValidator> getPathValidatorClass()
        The class of the validator that should be created and used to validate paths. If pathValidator has not been set explicitly, an instance of this class will be constructed and used as the path validator.
      • setPathValidatorClass

        void setPathValidatorClass(Class<? extends com.google.cloud.dataflow.sdk.util.PathValidator> validatorClass)
      • setPathValidator

        void setPathValidator(com.google.cloud.dataflow.sdk.util.PathValidator validator)
      • getStagerClass

        @Default.Class(value=com.google.cloud.dataflow.sdk.util.GcsStager.class)
        Class<? extends com.google.cloud.dataflow.sdk.util.Stager> getStagerClass()
        The class responsible for staging resources to be accessible by workers during job execution. If stager has not been set explicitly, an instance of this class will be created and used as the resource stager.
      • setStagerClass

        void setStagerClass(Class<? extends com.google.cloud.dataflow.sdk.util.Stager> stagerClass)
      • setStager

        void setStager(com.google.cloud.dataflow.sdk.util.Stager stager)
      • setDataflowClient

        void setDataflowClient(com.google.api.services.dataflow.Dataflow value)
      • getPubsubRootUrl

        @Default.String(value="https://pubsub.googleapis.com")
        String getPubsubRootUrl()
        Root URL for use with the Pubsub API.
      • setPubsubRootUrl

        void setPubsubRootUrl(String value)
      • setUpdate

        @Deprecated
        void setUpdate(boolean value)
        Deprecated. 
      • getTransformNameMapping

        Map<String,String> getTransformNameMapping()
        Mapping of old PTranform names to new ones, specified as JSON {"oldName":"newName",...}. To mark a transform as deleted, make newName the empty string.
      • setTransformNameMapping

        void setTransformNameMapping(Map<String,String> value)
      • getOverrideWindmillBinary

        String getOverrideWindmillBinary()
        Custom windmill_main binary to use with the streaming runner.
      • setOverrideWindmillBinary

        void setOverrideWindmillBinary(String value)
      • getWindmillServiceEndpoint

        String getWindmillServiceEndpoint()
        Custom windmill service endpoint.
      • setWindmillServiceEndpoint

        void setWindmillServiceEndpoint(String value)
      • setWindmillServicePort

        void setWindmillServicePort(int value)
      • getNumberOfWorkerHarnessThreads

        int getNumberOfWorkerHarnessThreads()
        Number of threads to use on the Dataflow worker harness. If left unspecified, the Dataflow service will compute an appropriate number of threads to use.
      • setNumberOfWorkerHarnessThreads

        void setNumberOfWorkerHarnessThreads(int value)
      • getDumpHeapOnOOM

        boolean getDumpHeapOnOOM()
        If true, save a heap dump before killing a thread or process which is GC thrashing or out of memory. The location of the heap file will either be echoed back to the user, or the user will be given the opportunity to download the heap file.

        CAUTION: Heap dumps can of comparable size to the default boot disk. Consider increasing the boot disk size before setting this flag to true.

      • setDumpHeapOnOOM

        void setDumpHeapOnOOM(boolean dumpHeapBeforeExit)


Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Dataflow
Need help? Visit our support page.