Uploads and downloads

This page discusses concepts related to uploading and downloading objects. You can upload and store any MIME type of data up to 5 TiB in size.

Uploads

You can send upload requests to Cloud Storage in the following ways:

  • Single-request upload. An upload method where an object is uploaded as a single request. Use this if the file is small enough to upload in its entirety if the connection fails. See Upload object from file or Upload object from memory for guides to single-request uploads.

    • JSON API multipart upload. An upload method that lets you upload both object data and its metadata in a single request. The XML API, however, allows setting object metadata in all types of uploads.

  • Resumable upload. An upload method that provides a more reliable transfer, which is especially important with large files. Resumable uploads are a good choice for most applications, since they also work for small files at the cost of one additional HTTP request per upload. You can also use resumable uploads to perform streaming transfers, which lets you upload an object of unknown size.

  • XML API multipart upload. An upload method that is compatible with Amazon S3 multipart uploads. Files are uploaded in parts and assembled into a single object with the final request. XML API multipart uploads allow you to upload the parts in parallel, potentially reducing the time to complete the overall upload.

Using these basic upload types, more advanced upload strategies are possible:

  • Parallel composite upload. An upload strategy in which you chunk a file and upload the chunks in parallel. Unlike XML API multipart uploads, parallel composite uploads use the compose operation, and the final object is stored as a composite object.

  • Streaming upload. An upload method that lets you upload data without requiring that the data first be saved to a file, which is useful when you don't know the total size of the upload at the start of the upload.

  • Chunked transfer encoding upload. A single-request upload that includes the header Transfer-Encoding: chunked, allowing the upload to be sent in multiple, sequential chunks of data.

Upload size considerations

When choosing whether to use a single-request upload instead of a resumable upload or XML API multipart upload, consider the amount of time that you're willing to lose should a network failure occur and you need to restart the upload from the beginning. For faster connections, your cutoff size can typically be larger.

For example, say you're willing to tolerate 30 seconds of lost time:

  • If you upload from a local system with an average upload speed of 8 Mbps, you can use single-request uploads for files as large as 30 MB.

  • If you upload from an in-region service that averages 500 Mbps for its upload speed, the cutoff size for files is almost 2 GB.

Downloads

All downloads from Cloud Storage have the same basic behavior: an HTTP or HTTPS GET request that can include an optional Range header, which defines a specific portion of the object to download.

Using this basic download behavior, you can resume interrupted downloads, and you can utilize more advanced download strategies, such as sliced object downloads and streaming downloads.

Support per tool

Console

You can perform the following types of upload and downloads when using the Google Cloud console:

Command line

You can perform the following types of uploads and downloads when using the Google Cloud CLI:

Client libraries

C++

You can perform the following types of uploads and downloads when using the C++ client library:

C#

You can perform the following types of uploads and downloads when using the C# client library:

Go

You can perform the following types of uploads and downloads when using the Go client library:

Java

You can perform the following types of uploads and downloads when using the Java client library:

Node.js

You can perform the following types of uploads and downloads when using the Node.js client library:

PHP

You can perform the following types of uploads and downloads when using the PHP client library:

Python

You can perform the following types of uploads and downloads when using the Python client library:

Ruby

You can perform the following types of upload and download when using the Ruby client library:

Terraform

You can perform single-request uploads when using Terraform. Resumable uploads occur automatically when the file being uploaded is larger than 16 MiB.

REST APIs

JSON API

You can perform the following types of uploads and downloads when using the JSON API:

XML API

You can perform the following types of uploads and downloads when using the XML API:

If you use REST APIs to upload and download, see Request endpoints for a complete discussion on the request endpoints you can use.

What's next