Upload objects from a filesystem

This page shows you how to upload objects to your Cloud Storage bucket from your local filesystem. An uploaded object consists of the data you want to store along with any associated metadata. For a conceptual overview, including how to choose the optimal upload method based on your file size, see Uploads and downloads.

For instructions on uploading from memory, see Upload objects from memory.

Required permissions

Console

In order to complete this guide using the Google Cloud console, you must have the proper IAM permissions. If the bucket you want to upload to exists in a project that you did not create, you might need the project owner to give you a role that contains the necessary permissions.

For a list of permissions required for specific actions, see IAM permissions for the Google Cloud console.

For a list of relevant roles, see Cloud Storage roles. Alternatively, you can create a custom role that has specific, limited permissions.

Command line

In order to complete this guide using a command-line utility, you must have the proper IAM permissions. If the bucket you want to upload to exists in a project that you did not create, you might need the project owner to give you a role that contains the necessary permissions.

For a list of permissions required for specific actions, see IAM permissions for gsutil commands.

For a list of relevant roles, see Cloud Storage roles. Alternatively, you can create a custom role that has specific, limited permissions.

Client libraries

In order to complete this guide using the Cloud Storage client libraries, you must have the proper IAM permissions. If the bucket you want to upload to exists in a project that you did not create, you might need the project owner to give you a role that contains the necessary permissions.

Unless otherwise noted, client library requests are made through the JSON API and require permissions as listed in IAM permissions for JSON methods. To see which JSON API methods are invoked when you make requests using a client library, log the raw requests.

For a list of relevant IAM roles, see Cloud Storage roles. Alternatively, you can create a custom role that has specific, limited permissions.

REST APIs

JSON API

In order to complete this guide using the JSON API, you must have the proper IAM permissions. If the bucket you want to upload to exists in a project that you did not create, you might need the project owner to give you a role that contains the necessary permissions.

For a list of permissions required for specific actions, see IAM permissions for JSON methods.

For a list of relevant roles, see Cloud Storage roles. Alternatively, you can create a custom role that has specific, limited permissions.

Upload an object to a bucket

Complete the following steps to upload an object to a bucket:

Console

1. In the Google Cloud console, go to the Cloud Storage Buckets page.
   

   Go to Buckets

2. In the list of buckets, click on the name of the bucket that you want to upload an object to.

3. In the Objects tab for the bucket, either:

   • Drag and drop the desired files from your desktop or file manager to the main pane in the Google Cloud console.

   • Click the Upload Files button, select the files you want to upload in the dialog that appears, and click Open.

   Note: If you are using the Chrome browser, folder uploads are also supported.

To learn how to get detailed error information about failed Cloud Storage operations in the Google Cloud console, see Troubleshooting.

Command line

gcloud

Use the gcloud storage cp command:

gcloud storage cp OBJECT_LOCATION gs://DESTINATION_BUCKET_NAME/

Where:

• OBJECT_LOCATION is the local path to your object. For example, Desktop/dog.png.

• DESTINATION_BUCKET_NAME is the name of the bucket to which you are uploading your object. For example, my-bucket.

If successful, the response looks like the following example:

Completed files 1/1 | 164.3kiB/164.3kiB

Note: Use the --recursive flag to recursively upload directories.

You can set fixed-key and custom object metadata as part of your object upload by using command flags.

gsutil

Use the gsutil cp command:

gsutil cp OBJECT_LOCATION gs://DESTINATION_BUCKET_NAME/

Where:

• OBJECT_LOCATION is the local path to your object. For example, Desktop/dog.png.

• DESTINATION_BUCKET_NAME is the name of the bucket to which you are uploading your object. For example, my-bucket.

If successful, the response looks like the following example:

Operation completed over 1 objects/58.8 KiB.

Note: Use the -r flag to recursively upload directories.

You can set fixed-key and custom object metadata as part of your object upload in the headers of the request by using the -h global option.

Client libraries

C++

For more information, see the Cloud Storage C++ API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

namespace gcs = ::google::cloud::storage;
using ::google::cloud::StatusOr;
[](gcs::Client client, std::string const& file_name,
   std::string const& bucket_name, std::string const& object_name) {
  // Note that the client library automatically computes a hash on the
  // client-side to verify data integrity during transmission.
  StatusOr<gcs::ObjectMetadata> metadata = client.UploadFile(
      file_name, bucket_name, object_name, gcs::IfGenerationMatch(0));
  if (!metadata) throw std::move(metadata).status();

  std::cout << "Uploaded " << file_name << " to object " << metadata->name()
            << " in bucket " << metadata->bucket()
            << "\nFull metadata: " << *metadata << "\n";
}

C#

For more information, see the Cloud Storage C# API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

using Google.Cloud.Storage.V1;
using System;
using System.IO;

public class UploadFileSample
{
    public void UploadFile(
        string bucketName = "your-unique-bucket-name",
        string localPath = "my-local-path/my-file-name",
        string objectName = "my-file-name")
    {
        var storage = StorageClient.Create();
        using var fileStream = File.OpenRead(localPath);
        storage.UploadObject(bucketName, objectName, null, fileStream);
        Console.WriteLine($"Uploaded {objectName}.");
    }
}

Go

For more information, see the Cloud Storage Go API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

import (
	"context"
	"fmt"
	"io"
	"os"
	"time"

	"cloud.google.com/go/storage"
)

// uploadFile uploads an object.
func uploadFile(w io.Writer, bucket, object string) error {
	// bucket := "bucket-name"
	// object := "object-name"
	ctx := context.Background()
	client, err := storage.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("storage.NewClient: %w", err)
	}
	defer client.Close()

	// Open local file.
	f, err := os.Open("notes.txt")
	if err != nil {
		return fmt.Errorf("os.Open: %w", err)
	}
	defer f.Close()

	ctx, cancel := context.WithTimeout(ctx, time.Second*50)
	defer cancel()

	o := client.Bucket(bucket).Object(object)

	// Optional: set a generation-match precondition to avoid potential race
	// conditions and data corruptions. The request to upload is aborted if the
	// object's generation number does not match your precondition.
	// For an object that does not yet exist, set the DoesNotExist precondition.
	o = o.If(storage.Conditions{DoesNotExist: true})
	// If the live object already exists in your bucket, set instead a
	// generation-match precondition using the live object's generation number.
	// attrs, err := o.Attrs(ctx)
	// if err != nil {
	// 	return fmt.Errorf("object.Attrs: %w", err)
	// }
	// o = o.If(storage.Conditions{GenerationMatch: attrs.Generation})

	// Upload an object with storage.Writer.
	wc := o.NewWriter(ctx)
	if _, err = io.Copy(wc, f); err != nil {
		return fmt.Errorf("io.Copy: %w", err)
	}
	if err := wc.Close(); err != nil {
		return fmt.Errorf("Writer.Close: %w", err)
	}
	fmt.Fprintf(w, "Blob %v uploaded.\n", object)
	return nil
}

Java

For more information, see the Cloud Storage Java API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.BlobInfo;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;
import java.io.IOException;
import java.nio.file.Paths;

public class UploadObject {
  public static void uploadObject(
      String projectId, String bucketName, String objectName, String filePath) throws IOException {
    // The ID of your GCP project
    // String projectId = "your-project-id";

    // The ID of your GCS bucket
    // String bucketName = "your-unique-bucket-name";

    // The ID of your GCS object
    // String objectName = "your-object-name";

    // The path to your file to upload
    // String filePath = "path/to/your/file"

    Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();
    BlobId blobId = BlobId.of(bucketName, objectName);
    BlobInfo blobInfo = BlobInfo.newBuilder(blobId).build();

    // Optional: set a generation-match precondition to avoid potential race
    // conditions and data corruptions. The request returns a 412 error if the
    // preconditions are not met.
    Storage.BlobWriteOption precondition;
    if (storage.get(bucketName, objectName) == null) {
      // For a target object that does not yet exist, set the DoesNotExist precondition.
      // This will cause the request to fail if the object is created before the request runs.
      precondition = Storage.BlobWriteOption.doesNotExist();
    } else {
      // If the destination already exists in your bucket, instead set a generation-match
      // precondition. This will cause the request to fail if the existing object's generation
      // changes before the request runs.
      precondition =
          Storage.BlobWriteOption.generationMatch(
              storage.get(bucketName, objectName).getGeneration());
    }
    storage.createFrom(blobInfo, Paths.get(filePath), precondition);

    System.out.println(
        "File " + filePath + " uploaded to bucket " + bucketName + " as " + objectName);
  }
}

Node.js

For more information, see the Cloud Storage Node.js API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// The ID of your GCS bucket
// const bucketName = 'your-unique-bucket-name';

// The path to your file to upload
// const filePath = 'path/to/your/file';

// The new ID for your GCS file
// const destFileName = 'your-new-file-name';

// Imports the Google Cloud client library
const {Storage} = require('@google-cloud/storage');

// Creates a client
const storage = new Storage();

async function uploadFile() {
  const options = {
    destination: destFileName,
    // Optional:
    // Set a generation-match precondition to avoid potential race conditions
    // and data corruptions. The request to upload is aborted if the object's
    // generation number does not match your precondition. For a destination
    // object that does not yet exist, set the ifGenerationMatch precondition to 0
    // If the destination object already exists in your bucket, set instead a
    // generation-match precondition using its generation number.
    preconditionOpts: {ifGenerationMatch: generationMatchPrecondition},
  };

  await storage.bucket(bucketName).upload(filePath, options);
  console.log(`${filePath} uploaded to ${bucketName}`);
}

uploadFile().catch(console.error);

PHP

For more information, see the Cloud Storage PHP API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

use Google\Cloud\Storage\StorageClient;

/**
 * Upload a file.
 *
 * @param string $bucketName The name of your Cloud Storage bucket.
 *        (e.g. 'my-bucket')
 * @param string $objectName The name of your Cloud Storage object.
 *        (e.g. 'my-object')
 * @param string $source The path to the file to upload.
 *        (e.g. '/path/to/your/file')
 */
function upload_object(string $bucketName, string $objectName, string $source): void
{
    $storage = new StorageClient();
    $file = fopen($source, 'r');
    $bucket = $storage->bucket($bucketName);
    $object = $bucket->upload($file, [
        'name' => $objectName
    ]);
    printf('Uploaded %s to gs://%s/%s' . PHP_EOL, basename($source), $bucketName, $objectName);
}

Python

For more information, see the Cloud Storage Python API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

from google.cloud import storage


def upload_blob(bucket_name, source_file_name, destination_blob_name):
    """Uploads a file to the bucket."""
    # The ID of your GCS bucket
    # bucket_name = "your-bucket-name"
    # The path to your file to upload
    # source_file_name = "local/path/to/file"
    # The ID of your GCS object
    # destination_blob_name = "storage-object-name"

    storage_client = storage.Client()
    bucket = storage_client.bucket(bucket_name)
    blob = bucket.blob(destination_blob_name)

    # Optional: set a generation-match precondition to avoid potential race conditions
    # and data corruptions. The request to upload is aborted if the object's
    # generation number does not match your precondition. For a destination
    # object that does not yet exist, set the if_generation_match precondition to 0.
    # If the destination object already exists in your bucket, set instead a
    # generation-match precondition using its generation number.
    generation_match_precondition = 0

    blob.upload_from_filename(source_file_name, if_generation_match=generation_match_precondition)

    print(
        f"File {source_file_name} uploaded to {destination_blob_name}."
    )

Ruby

For more information, see the Cloud Storage Ruby API reference documentation.

To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

View on GitHub Feedback

def upload_file bucket_name:, local_file_path:, file_name: nil
  # The ID of your GCS bucket
  # bucket_name = "your-unique-bucket-name"

  # The path to your file to upload
  # local_file_path = "/local/path/to/file.txt"

  # The ID of your GCS object
  # file_name = "your-file-name"

  require "google/cloud/storage"

  storage = Google::Cloud::Storage.new
  bucket  = storage.bucket bucket_name, skip_lookup: true

  file = bucket.create_file local_file_path, file_name

  puts "Uploaded #{local_file_path} as #{file.name} in bucket #{bucket_name}"
end

Terraform

You can use a Terraform resource to upload an object. Either content or source must be specified (you can also specify both).

View on GitHub Feedback

# Upload files
# Discussion about using tf to upload a large number of objects
# https://stackoverflow.com/questions/68455132/terraform-copy-multiple-files-to-bucket-at-the-same-time-bucket-creation

# The text object in Cloud Storage
resource "google_storage_bucket_object" "default" {
  name = "new-object"
  # Uncomment and add valid path to an object.
  #  source       = "/path/to/an/object"
  content      = "Data as string to be uploaded"
  content_type = "text/plain"
  bucket       = google_storage_bucket.static.id
}

REST APIs

JSON API

The JSON API distinguishes between media uploads, in which only object data is included in the request, and JSON API multipart uploads, in which both object data and object metadata are included in the request.

Media upload (a single-request upload without object metadata)

1. Get an authorization access token from the OAuth 2.0 Playground. Configure the playground to use your own OAuth credentials. For instructions, see API authentication.

2. Use cURL to call the JSON API with a POST Object request:

   curl -X POST --data-binary @OBJECT_LOCATION \
       -H "Authorization: Bearer OAUTH2_TOKEN" \
       -H "Content-Type: OBJECT_CONTENT_TYPE" \
       "https://storage.googleapis.com/upload/storage/v1/b/BUCKET_NAME/o?uploadType=media&name=OBJECT_NAME"

   Where:

   • OBJECT_LOCATION is the local path to your object. For example, Desktop/dog.png.
   • OAUTH2_TOKEN is the access token you generated in Step 1.
   • OBJECT_CONTENT_TYPE is the content type of the object. For example, image/png.
   • BUCKET_NAME is the name of the bucket to which you are uploading your object. For example, my-bucket.
   • OBJECT_NAME is the URL-encoded name you want to give your object. For example, pets/dog.png, URL-encoded as pets%2Fdog.png.

JSON API multipart upload (a single-request upload that includes object metadata)

Note: To add or change metadata for an existing object in Cloud Storage, see Viewing and editing object metadata.

1. Get an authorization access token from the OAuth 2.0 Playground. Configure the playground to use your own OAuth credentials. For instructions, see API authentication.

2. Create a multipart/related file that contains the following information:

   --BOUNDARY_STRING
   Content-Type: application/json; charset=UTF-8
   
   OBJECT_METADATA
   
   --BOUNDARY_STRING
   Content-Type: OBJECT_CONTENT_TYPE
   
   OBJECT_DATA
   --BOUNDARY_STRING--

   Where:

   • BOUNDARY_STRING is a string you define that identifies the different parts of the multipart file. For example, my-boundary.
   • OBJECT_METADATA is metadata you want to include for the file, in JSON format. At a minimum, this section should include a name attribute for the object, for example {"name": "myObject"}.
   • OBJECT_CONTENT_TYPE is the content type of the object. For example, image/png.
   • OBJECT_DATA is the data for the object.

3. Use cURL to call the JSON API with a POST Object request:

   curl -X POST --data-binary @MULTIPART_FILE_LOCATION \
       -H "Authorization: Bearer OAUTH2_TOKEN" \
       -H "Content-Type: multipart/related; boundary=BOUNDARY_STRING" \
       -H "Content-Length: MULTIPART_FILE_SIZE" \
       "https://storage.googleapis.com/upload/storage/v1/b/BUCKET_NAME/o?uploadType=multipart"

   Where:

   • MULTIPART_FILE_LOCATION is the local path to the multipart file you created in step 2. For example, Desktop/my-upload.multipart.
   • OAUTH2_TOKEN is the access token you generated in Step 1.
   • BOUNDARY_STRING is the boundary string you defined in Step 2. For example, my-boundary.
   • MULTIPART_FILE_SIZE is the total size, in bytes, of the multipart file you created in Step 2. For example, 2000000.
   • BUCKET_NAME is the name of the bucket to which you are uploading your object. For example, my-bucket.

If the request succeeds, the server returns the HTTP 200 OK status code along with the file's metadata.

XML API

1. Get an authorization access token from the OAuth 2.0 Playground. Configure the playground to use your own OAuth credentials. For instructions, see API authentication.

2. Use cURL to call the XML API with a PUT Object request:

   curl -X PUT --data-binary @OBJECT_LOCATION \
       -H "Authorization: Bearer OAUTH2_TOKEN" \
       -H "Content-Type: OBJECT_CONTENT_TYPE" \
       "https://storage.googleapis.com/BUCKET_NAME/OBJECT_NAME"

   Where:

   • OBJECT_LOCATION is the local path to your object. For example, Desktop/dog.png.
   • OAUTH2_TOKEN is the access token you generated in Step 1.
   • OBJECT_CONTENT_TYPE is the content type of the object. For example, image/png.
   • BUCKET_NAME is the name of the bucket to which you are uploading your object. For example, my-bucket.
   • OBJECT_NAME is the URL-encoded name you want to give your object. For example, pets/dog.png, URL-encoded as pets%2Fdog.png.

You can set additional object metadata as part of your object upload in the headers of the request in the same way the above example sets Content-Type. When working with the XML API, metadata can only be set at the time the object is written, such as when uploading, copying, or replacing the object. For more information, see Editing object metadata.

What's next

• Learn about naming requirements for objects.
• Transfer objects from your Compute Engine instance.
• Transfer data from cloud providers or other online sources, such as URL lists.
• Control who has access to your objects and buckets.
• View your object's metadata, including the URL for the object.

Try it for yourself

If you're new to Google Cloud, create an account to evaluate how Cloud Storage performs in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

Try Cloud Storage free