compose - Concatenate a sequence of objects into a new composite object.

Synopsis

gsutil compose gs://bucket/obj1 [gs://bucket/obj2 ...] gs://bucket/composite

Description

The compose command creates a new object whose content is the concatenation of a given sequence of source objects under the same bucket. gsutil uses the content type of the first source object to determine the destination object's content type. For more information, please see: https://cloud.google.com/storage/docs/composite-objects

Note also that the gsutil cp command can automatically split uploads for large files into multiple component objects, upload them in parallel, and compose them into a final object. This will still perform all uploads from a single machine. For extremely large files and/or very low per-machine bandwidth, you may want to split the file and upload it from multiple machines, and later compose these parts of the file manually.

Appending simply entails uploading your new data to a temporary object, composing it with the growing append-target, and deleting the temporary object:

$ echo 'new data' | gsutil cp - gs://bucket/data-to-append
$ gsutil compose gs://bucket/append-target gs://bucket/data-to-append \
    gs://bucket/append-target
$ gsutil rm gs://bucket/data-to-append

Note that there is a limit (currently 32) to the number of components that can be composed in a single operation.