GcsOptions (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. After August 12, 2020, Dataflow will not run jobs using Dataflow 1.x and below. See Migrating from Dataflow SDK 1.x for Java for migration guidance.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

com.google.cloud.dataflow.sdk.options

Interface GcsOptions

• All Superinterfaces:

  ApplicationNameOptions, GcpOptions, GoogleApiDebugOptions, HasDisplayData, PipelineOptions

  All Known Subinterfaces:

  BlockingDataflowPipelineOptions, DataflowPipelineOptions, DataflowWorkerHarnessOptions, DirectPipelineOptions, TestDataflowPipelineOptions

  ---

  
  

  public interface GcsOptions
  extends ApplicationNameOptions, GcpOptions, PipelineOptions

  Options used to configure Google Cloud Storage.

• • Nested Class Summary

    Nested Classes 

    Modifier and Type	Interface and Description
    static class	GcsOptions.ExecutorServiceFactory Returns the default ExecutorService to use within the Dataflow SDK.

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.GcpOptions

      GcpOptions.CredentialDirFactory, GcpOptions.DefaultProjectFactory, GcpOptions.GcpUserCredentialsFactory

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions

      GoogleApiDebugOptions.GoogleApiTracer

    • Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.options.PipelineOptions

      PipelineOptions.AtomicLongFactory, PipelineOptions.CheckEnabled

  • Method Summary

    All Methods Instance Methods Abstract Methods 

    Modifier and Type	Method and Description
    ExecutorService	getExecutorService() The ExecutorService instance to use to create threads, can be overridden to specify an ExecutorService that is compatible with the users environment.
    String	getGcsEndpoint() GCS endpoint to use.
    Integer	getGcsUploadBufferSizeBytes() The buffer size (in bytes) to use when uploading files to GCS.
    com.google.cloud.dataflow.sdk.util.GcsUtil	getGcsUtil() The GcsUtil instance that should be used to communicate with Google Cloud Storage.
    void	setExecutorService(ExecutorService value)
    void	setGcsEndpoint(String value)
    void	setGcsUploadBufferSizeBytes(Integer bytes)
    void	setGcsUtil(com.google.cloud.dataflow.sdk.util.GcsUtil value)

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.ApplicationNameOptions

      getAppName, setAppName

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.GcpOptions

      getAuthorizationServerEncodedUrl, getCredentialDir, getCredentialFactoryClass, getCredentialId, getGcpCredential, getProject, getSecretsFile, getServiceAccountKeyfile, getServiceAccountName, getTokenServerUrl, setAuthorizationServerEncodedUrl, setCredentialDir, setCredentialFactoryClass, setCredentialId, setGcpCredential, setProject, setSecretsFile, setServiceAccountKeyfile, setServiceAccountName, setTokenServerUrl

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions

      getGoogleApiTrace, setGoogleApiTrace

    • Methods inherited from interface com.google.cloud.dataflow.sdk.options.PipelineOptions

      as, cloneAs, getOptionsId, getRunner, getStableUniqueNames, getTempLocation, outputRuntimeOptions, setOptionsId, setRunner, setStableUniqueNames, setTempLocation

    • Methods inherited from interface com.google.cloud.dataflow.sdk.transforms.display.HasDisplayData

      populateDisplayData

• • Method Detail

    • getGcsUtil

      @Default.InstanceFactory(value=com.google.cloud.dataflow.sdk.util.GcsUtil.GcsUtilFactory.class)
       @Hidden
      com.google.cloud.dataflow.sdk.util.GcsUtil getGcsUtil()

      The GcsUtil instance that should be used to communicate with Google Cloud Storage.

    • setGcsUtil

      void setGcsUtil(com.google.cloud.dataflow.sdk.util.GcsUtil value)

    • getExecutorService

      @Default.InstanceFactory(value=GcsOptions.ExecutorServiceFactory.class)
       @Hidden
      ExecutorService getExecutorService()

      The ExecutorService instance to use to create threads, can be overridden to specify an ExecutorService that is compatible with the users environment. If unset, the default is to create an ExecutorService with an unbounded number of threads; this is compatible with Google AppEngine.

    • setExecutorService

      void setExecutorService(ExecutorService value)

    • getGcsEndpoint

      @Hidden
      String getGcsEndpoint()

      GCS endpoint to use. If unspecified, uses the default endpoint.

    • setGcsEndpoint

      void setGcsEndpoint(String value)

    • getGcsUploadBufferSizeBytes

      Integer getGcsUploadBufferSizeBytes()

      The buffer size (in bytes) to use when uploading files to GCS. Please see the documentation for AbstractGoogleAsyncWriteChannel.setUploadBufferSize(int) for more information on the restrictions and performance implications of this value.

    • setGcsUploadBufferSizeBytes

      void setGcsUploadBufferSizeBytes(Integer bytes)