Composing Objects

Go to concepts

This page shows you how to compose source objects to create a composite object. You can compose between 1 and 32 source objects in a single request.

Caution: The source objects are unaffected by the composition process. If they are meant to be temporary, you must delete them once you've successfully completed the composition. Because other storage classes are subject to early deletion fees, you should always use Standard Storage for temporary objects.

To create a composite object:

Console

The Cloud Console does not support performing object compositions. Use gsutil instead.

gsutil

Use the gsutil compose command:

gsutil compose gs://BUCKET_NAME/SOURCE_OBJECT_1 gs://BUCKET_NAME/SOURCE_OBJECT_2 gs://BUCKET_NAME/COMPOSITE_OBJECT_NAME

Where:

• BUCKET_NAME is the name of the bucket that contains the source objects.
• SOURCE_OBJECT_1 and SOURCE_OBJECT_2 are the names of the source objects to use in the object composition.
• COMPOSITE_OBJECT_NAME is the name you are giving to the result of the object composition.

Note: You can also use a wildcard to compose objects with a common prefix without having to list each object separately in the compose command.

Code samples

C++

For more information, see the Cloud Storage C++ API reference documentation.

View on GitHub Feedback

namespace gcs = google::cloud::storage;
using ::google::cloud::StatusOr;
[](gcs::Client client, std::string const& bucket_name,
   std::string const& destination_object_name,
   std::vector<gcs::ComposeSourceObject> const& compose_objects) {
  StatusOr<gcs::ObjectMetadata> composed_object = client.ComposeObject(
      bucket_name, compose_objects, destination_object_name);

  if (!composed_object) {
    throw std::runtime_error(composed_object.status().message());
  }

  std::cout << "Composed new object " << composed_object->name()
            << " in bucket " << composed_object->bucket()
            << "\nFull metadata: " << *composed_object << "\n";
}

Go

For more information, see the Cloud Storage Go API reference documentation.

View on GitHub Feedback

import (
	"context"
	"fmt"
	"io"
	"time"

	"cloud.google.com/go/storage"
)

// composeFile composes source objects to create a composite object.
func composeFile(w io.Writer, bucket, object1, object2, toObject string) error {
	// bucket := "bucket-name"
	// object1 := "object-name-1"
	// object2 := "object-name-2"
	// toObject := "object-name-3"

	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("storage.NewClient: %v", err)
	}
	defer client.Close()

	ctx, cancel := context.WithTimeout(ctx, time.Second*10)
	defer cancel()

	src1 := client.Bucket(bucket).Object(object1)
	src2 := client.Bucket(bucket).Object(object2)
	dst := client.Bucket(bucket).Object(toObject)

	// ComposerFrom takes varargs, so you can put as many objects here
	// as you want.
	_, err = dst.ComposerFrom(src1, src2).Run(ctx)
	if err != nil {
		return fmt.Errorf("ComposerFrom: %v", err)
	}
	fmt.Fprintf(w, "New composite object %v was created by combining %v and %v\n", toObject, object1, object2)
	return nil
}

Java

For more information, see the Cloud Storage Java API reference documentation.

View on GitHub Feedback

import com.google.cloud.storage.Blob;
import com.google.cloud.storage.BlobInfo;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;

public class ComposeObject {
  public static void composeObject(
      String bucketName,
      String firstObjectName,
      String secondObjectName,
      String targetObjectName,
      String projectId) {
    // The ID of your GCP project
    // String projectId = "your-project-id";

    // The ID of your GCS bucket
    // String bucketName = "your-unique-bucket-name";

    // The ID of the first GCS object to compose
    // String firstObjectName = "your-first-object-name";

    // The ID of the second GCS object to compose
    // String secondObjectName = "your-second-object-name";

    // The ID to give the new composite object
    // String targetObjectName = "new-composite-object-name";

    Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();

    Storage.ComposeRequest composeRequest =
        Storage.ComposeRequest.newBuilder()
            // addSource takes varargs, so you can put as many objects here as you want, up to the
            // max of 32
            .addSource(firstObjectName, secondObjectName)
            .setTarget(BlobInfo.newBuilder(bucketName, targetObjectName).build())
            .build();

    Blob compositeObject = storage.compose(composeRequest);

    System.out.println(
        "New composite object "
            + compositeObject.getName()
            + " was created by combining "
            + firstObjectName
            + " and "
            + secondObjectName);
  }
}

Python

For more information, see the Cloud Storage Python API reference documentation.

View on GitHub Feedback

from google.cloud import storage


def compose_file(bucket_name, sources, destination_blob_name):
    """Concatenate source blobs into destination blob."""
    # bucket_name = "your-bucket-name"
    # sources = [blob_1, blob_2]
    # destination_blob_name = "destination-object-name"

    storage_client = storage.Client()
    bucket = storage_client.bucket(bucket_name)
    destination = bucket.blob(destination_blob_name)
    destination.content_type = "text/plain"
    destination.compose(sources)

    print(
        "Composed new object {} in the bucket {}".format(
            destination_blob_name, bucket.name
        )
    )
    return destination

C#

For more information, see the Cloud Storage C# API reference documentation.

You cannot currently compose objects using the C# client library.

Node.js

For more information, see the Cloud Storage Node.js API reference documentation.

To compose an object using NodeJS, see the Bucket reference documentation.

PHP

For more information, see the Cloud Storage PHP API reference documentation.

To compose an object using PHP, see the Google\Cloud\Storage\Bucket reference documentation.

Ruby

For more information, see the Cloud Storage Ruby API reference documentation.

To compose an object using Ruby, see the Google::Cloud::Storage reference documentation.

REST APIs

JSON API

1. Get an authorization access token from the OAuth 2.0 Playground. Configure the playground to use your own OAuth credentials.

2. Create a .json file that contains the following information:

   {
     "sourceObjects": [
       {
         "name": "SOURCE_OBJECT_1"
       },
       {
         "name": "SOURCE_OBJECT_2"
       }
     ],
     "destination": {
       "contentType": "COMPOSITE_OBJECT_CONTENT_TYPE"
     }
   }

   Where:

   • SOURCE_OBJECT_1 and SOURCE_OBJECT_2 are the names of the source objects to use in the object composition.
   • COMPOSITE_OBJECT_CONTENT_TYPE is the Content-Type of the resulting composite object.

3. Use cURL to call the JSON API with a POST Object request:

   curl -X POST --data-binary @JSON_FILE_NAME.json \
     -H "Authorization: Bearer OAUTH2_TOKEN" \
     -H "Content-Type: application/json" \
     "https://storage.googleapis.com/storage/v1/b/BUCKET_NAME/o/COMPOSITE_OBJECT_NAME/compose"

   Where:

   • JSON_FILE_NAME is the name of the file you created in the previous step.
   • OAUTH2_TOKEN is the access token you generated earlier in the guide.
   • BUCKET_NAME is the name of the bucket that contains the source objects.
   • COMPOSITE_OBJECT_NAME is the name you are giving to the result of the object composition.

If successful, the response is an object resource for the resulting composite object.

XML API

1. Get an authorization access token from the OAuth 2.0 Playground. Configure the playground to use your own OAuth credentials.

2. Create a .xml file that contains the following information:

     <ComposeRequest>
       <Component>
         <Name>SOURCE_OBJECT_1</Name>
       </Component>
       <Component>
         <Name>SOURCE_OBJECT_2</Name>
       </Component>
     </ComposeRequest>

   Where:

   • SOURCE_OBJECT_1 and SOURCE_OBJECT_2 are the names of the source objects to use in the object composition.

3. Use cURL to call the XML API with a PUT Object request and compose query string parameter:

   curl -X PUT --data-binary @XML_FILE_NAME.xml \
     -H "Authorization: Bearer OAUTH2_TOKEN" \
     -H "Content-Type: COMPOSITE_OBJECT_CONTENT_TYPE" \
     "https://storage.googleapis.com/BUCKET_NAME/COMPOSITE_OBJECT_NAME?compose"

   Where:

   • XML_FILE_NAME is the name of the file you created in the previous step.
   • OAUTH2_TOKEN is the access token you generated earlier in the guide.
   • COMPOSITE_OBJECT_CONTENT_TYPE is the Content-Type of the resulting composite object.
   • BUCKET_NAME is the name of the bucket that contains the source objects.
   • COMPOSITE_OBJECT_NAME is the name you are giving to the result of the object composition.

If successful, an empty response body is returned.

Note: The XML API supports including a generation number for each <Component>, using either a <Generation> element or an <IfGenerationMatch> element.

What's next

• Learn more about object composition.