Class ParallelCompositeUploadBlobWriteSessionConfig (2.34.0) 

public final class ParallelCompositeUploadBlobWriteSessionConfig extends BlobWriteSessionConfig implements BlobWriteSessionConfig.GrpcCompatible

Immutable config builder to configure BlobWriteSession instances to perform Parallel Composite Uploads.

Parallel Composite Uploads can yield higher throughput when uploading large objects. However, there are some things which must be kept in mind when choosing to use this strategy.

  1. Performing parallel composite uploads costs more money. Class A operations are performed to create each part and to perform each compose. If a storage tier other than STANDARD is used, early deletion fees apply to deletion of the parts.

    An illustrative example. Upload a 5GiB object using 64MiB as the max size per part.

    1. 80 Parts will be created (Class A)
    2. 3 compose calls will be performed (Class A)
    3. Delete 80 Parts along with 2 intermediary Compose objects (Free tier as long as STANDARD class)
    Once the parts and intermediary compose objects are deleted, there will be no storage charges related to those temporary objects.
  2. The service account/credentials used to perform the parallel composite upload require storage.objects.delete in order to cleanup the temporary part and intermediary compose objects.
    To handle handle part and intermediary compose object deletion out of band passing PartCleanupStrategy#never() to ParallelCompositeUploadBlobWriteSessionConfig#withPartCleanupStrategy(PartCleanupStrategy) will prevent automatic cleanup.
  3. Please see the Parallel composite uploads documentation for a more in depth explanation of the limitations of Parallel composite uploads.
  4. A failed upload can leave part and intermediary compose objects behind which will count as storage usage, and you will be billed for it.
    By default if an upload fails, an attempt to cleanup the part and intermediary compose will be made. However if the program were to crash there is no means for the client to perform the cleanup.
    Every part and intermediary compose object will be created with a name which ends in .part. An Object Lifecycle Management rule can be setup on your bucket to automatically cleanup objects with the suffix after some period of time. See Object Lifecycle Management for full details and a guide on how to setup a Delete rule with a suffix match condition.
  5. Using parallel composite uploads are not a a one size fits all solution. They have very real overhead until uploading a large enough object. The inflection point is dependent upon many factors, and there is no one size fits all value. You will need to experiment with your deployment and workload to determine if parallel composite uploads are useful to you.

In general if you object sizes are smaller than several hundred megabytes it is unlikely parallel composite uploads will be beneficial to overall throughput. See Also: BlobWriteSessionConfigs#parallelCompositeUpload(), Storage#blobWriteSession(BlobInfo, BlobWriteOption...), GrpcStorageOptions.Builder#setBlobWriteSessionConfig(BlobWriteSessionConfig), https://cloud.google.com/storage/docs/parallel-composite-uploads

Inheritance

java.lang.Object > BlobWriteSessionConfig > ParallelCompositeUploadBlobWriteSessionConfig

Implements

com.google.cloud.storage.BlobWriteSessionConfig.GrpcCompatible

Methods

withBufferAllocationStrategy(ParallelCompositeUploadBlobWriteSessionConfig.BufferAllocationStrategy bufferAllocationStrategy)

public ParallelCompositeUploadBlobWriteSessionConfig withBufferAllocationStrategy(ParallelCompositeUploadBlobWriteSessionConfig.BufferAllocationStrategy bufferAllocationStrategy)

Specify a specific buffering strategy which will dictate how buffers are allocated and used when performing a parallel composite upload.

Default: BufferAllocationStrategy#simple(16MiB)

Parameter
Name Description
bufferAllocationStrategy ParallelCompositeUploadBlobWriteSessionConfig.BufferAllocationStrategy
Returns
Type Description
ParallelCompositeUploadBlobWriteSessionConfig

withExecutorSupplier(ParallelCompositeUploadBlobWriteSessionConfig.ExecutorSupplier executorSupplier)

public ParallelCompositeUploadBlobWriteSessionConfig withExecutorSupplier(ParallelCompositeUploadBlobWriteSessionConfig.ExecutorSupplier executorSupplier)

Specify a specific executor supplier where work will be submitted when performing a parallel composite upload.

Default: ExecutorSupplier#cachedPool()

Parameter
Name Description
executorSupplier ParallelCompositeUploadBlobWriteSessionConfig.ExecutorSupplier
Returns
Type Description
ParallelCompositeUploadBlobWriteSessionConfig

withPartCleanupStrategy(ParallelCompositeUploadBlobWriteSessionConfig.PartCleanupStrategy partCleanupStrategy)

public ParallelCompositeUploadBlobWriteSessionConfig withPartCleanupStrategy(ParallelCompositeUploadBlobWriteSessionConfig.PartCleanupStrategy partCleanupStrategy)

Specify a specific cleanup strategy which will dictate what cleanup operations are performed automatically when performing a parallel composite upload.

Default: PartCleanupStrategy#always()

Parameter
Name Description
partCleanupStrategy ParallelCompositeUploadBlobWriteSessionConfig.PartCleanupStrategy
Returns
Type Description
ParallelCompositeUploadBlobWriteSessionConfig

withPartNamingStrategy(ParallelCompositeUploadBlobWriteSessionConfig.PartNamingStrategy partNamingStrategy)

public ParallelCompositeUploadBlobWriteSessionConfig withPartNamingStrategy(ParallelCompositeUploadBlobWriteSessionConfig.PartNamingStrategy partNamingStrategy)

Specify a specific naming strategy which will dictate how individual part and intermediary compose objects will be named when performing a parallel composite upload.

Default: PartNamingStrategy#noPrefix()

Parameter
Name Description
partNamingStrategy ParallelCompositeUploadBlobWriteSessionConfig.PartNamingStrategy
Returns
Type Description
ParallelCompositeUploadBlobWriteSessionConfig