compose - Concatenate a sequence of objects into a new composite object.
gsutil compose gs://bucket/obj1 [gs://bucket/obj2 ...] gs://bucket/composite
The compose command creates a new object whose content is the concatenation of a given sequence of source objects under the same bucket. gsutil uses the content type of the first source object to determine the destination object's content type. For more information, please see: https://cloud.google.com/storage/docs/composite-objects
Note also that the
gsutil cp command can automatically split uploads
for large files into multiple component objects, upload them in parallel,
and compose them into a final object. This will still perform all uploads
from a single machine. For extremely large files and/or very low
per-machine bandwidth, you may want to split the file and upload it from
multiple machines, and later compose these parts of the file manually.
Appending simply entails uploading your new data to a temporary object, composing it with the growing append-target, and deleting the temporary object:
$ echo 'new data' | gsutil cp - gs://bucket/data-to-append $ gsutil compose gs://bucket/append-target gs://bucket/data-to-append \ gs://bucket/append-target $ gsutil rm gs://bucket/data-to-append
Note that there is a limit (currently 32) to the number of components that can be composed in a single operation.