Perform resumable uploads

Overview

This page describes how to make a resumable upload request in the Cloud Storage JSON and XML APIs. This protocol lets you resume an upload operation after a communication failure interrupts the flow of data.

For information on using resumable uploads in the Google Cloud CLI and client libraries, see How tools and APIs use resumable uploads.

Required roles

To get the permissions that you need to perform a resumable upload, ask your administrator to grant you one of the following roles:

For uploads that include an Object Retention Lock, ask your administrator to grant you the Storage Object Admin (roles/storage.objectAdmin) IAM role for the bucket.
For all other cases, ask your administrator to grant you the Storage Object User (roles/storage.objectUser) IAM role for the bucket.

These predefined roles contain the permissions required to upload an object to a bucket for their respective cases. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

storage.objects.create
storage.objects.delete

This permission is only required for uploads that overwrite an existing object.

storage.objects.setRetention
- This permission is only required for uploads that include an Object Retention Lock.

You can also get these permissions with other predefined roles or custom roles.

For information about granting roles on buckets, see Use IAM with buckets.

Initiate a resumable upload session

To initiate a resumable upload session:

JSON API

Have gcloud CLI installed and initialized , which lets you generate an access token for the Authorization header.

Optionally, create a JSON file that contains the metadata you want to set on the object that you're uploading. For example, the following JSON file sets the contentType metadata of the object you want to upload to image/png:
```
{
    "contentType": "image/png"
}
```
Use cURL to call the JSON API with a POST Object request:
```
curl -i -X POST --data-binary @METADATA_LOCATION \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "Content-Type: application/json" \
    -H "Content-Length: INITIAL_REQUEST_LENGTH" \
    "https://storage.googleapis.com/upload/storage/v1/b/BUCKET_NAME/o?uploadType=resumable&name=OBJECT_NAME"
```
Where:
- METADATA_LOCATION is the local path to the JSON file containing the optional metadata you specified in the previous step. If you are not including a metadata file, exclude this, along with --data-binary @ and the Content-Type header.
- INITIAL_REQUEST_LENGTH is the number of bytes in the body of this initial request, for example 79.
- BUCKET_NAME is the name of the bucket to which you are uploading your object. For example, my-bucket.
- OBJECT_NAME is the URL-encoded name you want to give your object. For example, pets/dog.png, URL-encoded as pets%2Fdog.png. This is not required if you included a name in the object metadata file in Step 2.
If you have enabled Cross-Origin Resource Sharing, you should also include an Origin header in both this and subsequent upload requests.

Optional headers that you can add to the request include X-Upload-Content-Type and X-Upload-Content-Length.

If successful, the response includes a 200 status code.
Save the resumable session URI given in the Location header of the response to your POST Object request.

This URI is used in subsequent requests to upload the object data.

Caution: Be careful when sharing the resumable session URI, because it can be used by anyone to upload data to the target bucket without any further authentication.

XML API

Have gcloud CLI installed and initialized , which lets you generate an access token for the Authorization header.

Use cURL to call the XML API with a POST Object request that has an empty body:
```
curl -i -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "Content-Length: 0" \
    -H "Content-Type: OBJECT_CONTENT_TYPE" \
    -H "x-goog-resumable: start" \
    "https://storage.googleapis.com/BUCKET_NAME/OBJECT_NAME"
```
Where:
- OBJECT_CONTENT_TYPE is the content type of the object. For example, image/png. If you do not specify a content type, the Cloud Storage system sets the Content-Type metadata for the object to be application/octet-stream.
- BUCKET_NAME is the name of the bucket to which you are uploading your object. For example, my-bucket.
- OBJECT_NAME is the URL-encoded name you want to give your object. For example, pets/dog.png, URL-encoded as pets%2Fdog.png.
If you have enabled Cross-Origin Resource Sharing, you should also include an Origin header in both this and subsequent upload requests.

If successful, the response includes a 201 status message.
Save the resumable session URI given in the Location header of the response to your POST Object request.

This URI is used in subsequent requests to upload the object data.

Caution: Be careful when sharing the resumable session URI, because it can be used by anyone to upload data to the target bucket without any further authentication.

Upload the data

Once you have initiated a resumable upload, there are two ways to upload the object's data:

In a single chunk: This approach is usually best, since it requires fewer requests and thus has better performance.
In multiple chunks: Use this approach if you need to reduce the amount of data transferred in any single request, such as when there is a fixed time limit for individual requests, or if you don't know the total size of the upload at the time the upload begins.

Single chunk upload

To upload the data in a single chunk:

JSON API

Use cURL to call the JSON API with a PUT Object request:
```
curl -i -X PUT --data-binary @OBJECT_LOCATION \
    -H "Content-Length: OBJECT_SIZE" \
    "SESSION_URI"
```
Where:
- OBJECT_LOCATION is the local path to your object. For example, Desktop/dog.png.
- OBJECT_SIZE is the number of bytes in your object. For example, 20000000.
- SESSION_URI is the value returned in the Location header when you initiated the resumable upload.
Optionally, you can include headers prefixed with X-Goog-Meta- to add custom metadata for the object as part of this request.

XML API

Use cURL to call the XML API with a PUT Object request:
```
curl -i -X PUT --data-binary @OBJECT_LOCATION \
    -H "Content-Length: OBJECT_SIZE" \
    "SESSION_URI"
```
Where:
- OBJECT_LOCATION is the local path to your object. For example, Desktop/dog.png.
- OBJECT_SIZE is the number of bytes in your object. For example, 20000000.
- SESSION_URI is the value returned in the Location header when you initiated the resumable upload.

If the upload completes in its entirety, you receive a 200 OK or 201 Created response, along with any metadata associated with the resource.

If the upload request is interrupted, or if you receive a 5xx response, follow the procedure in Resuming an interrupted upload.

Multiple chunk upload

To upload the data in multiple chunks:

JSON API

Create a chunk of data from the overall data you want to upload.

The chunk size should be a multiple of 256 KiB (256 x 1024 bytes), unless it's the last chunk that completes the upload. Larger chunk sizes typically make uploads faster, but note that there's a tradeoff between speed and memory usage. It's recommended that you use at least 8 MiB for the chunk size.
Use cURL to call the JSON API with a PUT Object request:
```
curl -i -X PUT --data-binary @CHUNK_LOCATION \
    -H "Content-Length: CHUNK_SIZE" \
    -H "Content-Range: bytes CHUNK_FIRST_BYTE-CHUNK_LAST_BYTE/TOTAL_OBJECT_SIZE" \
    "SESSION_URI"
```
Where:
- CHUNK_LOCATION is the local path to the chunk that you're currently uploading.
- CHUNK_SIZE is the number of bytes you're uploading in the current request. For example, 8388608.
- CHUNK_FIRST_BYTE is the starting byte in the overall object that the chunk you're uploading contains.
- CHUNK_LAST_BYTE is the ending byte in the overall object that the chunk you're uploading contains.
- TOTAL_OBJECT_SIZE is the total size of the object you are uploading.
- SESSION_URI is the value returned in the Location header when you initiated the resumable upload.
An example Content-Range is Content-Range: bytes 0-8388607/20000000. See Content-Range for more information about this header.

If the request succeeds, the server responds with 308 Resume Incomplete. The response contains a Range header.
Repeat the above steps for each remaining chunk of data that you want to upload, using the upper value contained in the Range header of each response to determine where to start each successive chunk; you should not assume that the server received all bytes sent in any given request.

Optionally, in the final request of the resumable upload you can include headers prefixed with X-Goog-Meta- to add custom metadata for the object.

XML API

Create a chunk of data from the overall data you want to upload.

The chunk size should be a multiple of 256 KiB (256 x 1024 bytes), unless it's the last chunk that completes the upload. Larger chunk sizes typically make uploads faster, but note that there's a tradeoff between speed and memory usage. It's recommended that you use at least 8 MiB for the chunk size.
Use cURL to call the XML API with a PUT Object request:
```
curl -i -X PUT --data-binary @CHUNK_LOCATION \
    -H "Content-Length: CHUNK_SIZE" \
    -H "Content-Range: bytes CHUNK_FIRST_BYTE-CHUNK_LAST_BYTE/TOTAL_OBJECT_SIZE" \
    "SESSION_URI"
```
Where:
- CHUNK_LOCATION is the local path to the chunk that you're currently uploading.
- CHUNK_SIZE is the number of bytes you're uploading in the current request. For example, 8388608.
- CHUNK_FIRST_BYTE is the starting byte in the overall object that the chunk you're uploading contains.
- CHUNK_LAST_BYTE is the ending byte in the overall object that the chunk you're uploading contains.
- TOTAL_OBJECT_SIZE is the total size of the object you are uploading.
- SESSION_URI is the value returned in the Location header when you initiated the resumable upload.
An example Content-Range is Content-Range: bytes 0-8388607/20000000. See Content-Range for more information about this header.

If the request succeeds, the server responds with 308 Resume Incomplete. The response contains a Range header.
Repeat the above steps for each remaining chunk of data that you want to upload, using the upper value contained in the Range header of each response to determine where to start each successive chunk; you should not assume that the server received all bytes sent in any given request.

Once the upload completes in its entirety, you receive a 200 OK or 201 Created response, along with any metadata associated with the resource.

If any of the chunk uploads are interrupted, or if you receive a 5xx response, you should either resend the interrupted chunk in its entirety or resume the interrupted upload from where it left off.

Check the status of a resumable upload

If your resumable upload is interrupted, or you're not sure the upload completed, you can check the status of the upload:

JSON API

Use cURL to call the JSON API with a PUT Object request:
```
curl -i -X PUT \
    -H "Content-Length: 0" \
    -H "Content-Range: bytes */OBJECT_SIZE" \
    "SESSION_URI"
```
Where:
- OBJECT_SIZE is the total number of bytes in your object. If you don't know the full size of your object, use * for this value.
- SESSION_URI is the value returned in the Location header when you initiated the resumable upload.

XML API

Use cURL to call the XML API with a PUT Object request:
```
curl -i -X PUT \
    -H "Content-Length: 0" \
    -H "Content-Range: bytes */OBJECT_SIZE" \
    "SESSION_URI"
```
Where:
- OBJECT_SIZE is the total number of bytes in your object. If you don't know the full size of your object, use * for this value.
- SESSION_URI is the value returned in the Location header when you initiated the resumable upload.

A 200 OK or 201 Created response indicates that the upload was completed, and no further action is necessary.

A 308 Resume Incomplete response indicates that you need to continue uploading the data.

If Cloud Storage has not yet persisted any bytes, the 308 response does not have a Range header. In this case, you should start your upload from the beginning.
Otherwise, the 308 response has a Range header, which specifies which bytes Cloud Storage has persisted so far. Use this value when resuming an interrupted upload.

Resume an interrupted upload

If an upload request is terminated before receiving a response, or if you receive a 503 or 500 response, then you need to resume the interrupted upload from where it left off. To resume an interrupted upload:

JSON API

Check the status of your resumable upload.
Save the upper value of the Range header contained in the response to your status check. If the response does not have a Range header, Cloud Storage has not yet persisted any bytes, and you should upload from the beginning.
Ensure that that object data you're about to upload begins at the byte following the upper value in the Range header.
If the interrupted request contained headers prefixed with X-Goog-Meta-, include those headers in the request to resume your upload.
Use cURL to call the JSON API with a PUT Object request that picks up at the byte following the value in the Range header:
```
curl -i -X PUT --data-binary @PARTIAL_OBJECT_LOCATION \
    -H "Content-Length: UPLOAD_SIZE_REMAINING" \
    -H "Content-Range: bytes NEXT_BYTE-LAST_BYTE/TOTAL_OBJECT_SIZE" \
    "SESSION_URI"
```
Where:
- PARTIAL_OBJECT_LOCATION is the local path to the remaining portion of data that you want to upload.
- UPLOAD_SIZE_REMAINING is the number of bytes you're uploading in the current request. For example, uploading the rest of an object with a total size of 20000000 that was interrupted after bytes 0-42 uploaded would have an UPLOAD_SIZE_REMAINING of 19999957.
- NEXT_BYTE is the next integer after the value you saved in step 2. For example, if 42 is the upper value in step 2, the value for NEXT_BYTE is 43.
- LAST_BYTE is the ending byte contained in this PUT request. For example, to finish uploading an object whose total size is 20000000, the value for LAST_BYTE is 19999999.
- TOTAL_OBJECT_SIZE is the total size of the object you are uploading. For example, 20000000.
- SESSION_URI is the value returned in the Location header when you initiated the resumable upload.

XML API

Check the status of your resumable upload.
Save the upper value of the Range header contained in the response to your status check. If the response does not have a Range header, Cloud Storage has not yet persisted any bytes, and you should upload from the beginning.
Ensure that that object data you're about to upload begins at the byte following the upper value in the Range header.
Use cURL to call the XML API with a PUT Object request that picks up at the byte following the value in the Range header:
```
curl -i -X PUT --data-binary @PARTIAL_OBJECT_LOCATION \
    -H "Content-Length: UPLOAD_SIZE_REMAINING" \
    -H "Content-Range: bytes NEXT_BYTE-LAST_BYTE/TOTAL_OBJECT_SIZE" \
    "SESSION_URI"
```
Where:
- PARTIAL_OBJECT_LOCATION is the local path to the remaining portion of data that you want to upload.
- UPLOAD_SIZE_REMAINING is the number of bytes you're uploading in the current request. For example, uploading the rest of an object with a total size of 20000000 that was interrupted after bytes 0-42 uploaded would have an UPLOAD_SIZE_REMAINING of 19999957.
- NEXT_BYTE is the next integer after the value you saved in step 2. For example, if 42 is the upper value in step 2, the value for NEXT_BYTE is 43.
- LAST_BYTE is the ending byte contained in this PUT request. For example, to finish uploading an object whose total size is 20000000, the value for LAST_BYTE is 19999999.
- TOTAL_OBJECT_SIZE is the total size of the object you are uploading. For example, 20000000.
- SESSION_URI is the value returned in the Location header when you initiated the resumable upload.

You can resume uploads as many times as necessary while the session URI is active; the session URI expires after a week. When the data is successfully uploaded, Cloud Storage responds with a 200 OK or 201 created status code.

Cancel an upload

To cancel an incomplete resumable upload and prevent any further action on it:

JSON API

Use cURL to call the JSON API with a DELETE request:
```
curl -i -X DELETE -H "Content-Length: 0" \
  "SESSION_URI"
```
Where:
- SESSION_URI is the value returned in the Location header when you initiated the resumable upload.

If successful, the response contains a 499 status code. Future attempts to query or resume the upload result in a 4xx response.

XML API

Use cURL to call the XML API with a DELETE request:
```
curl -i -X DELETE -H "Content-Length: 0" \
  "SESSION_URI"
```
Where:
- SESSION_URI is the value returned in the Location header when you initiated the resumable upload.

If successful, the response contains a 204 status code, and future attempts to query or resume the upload also result in a 204 response.

Failure Handling

Under rare circumstances, a request to resume an interrupted upload might fail with a non-retriable '4xx' error because permissions on the bucket have changed, or because the integrity check on the final uploaded object detected a mismatch. If this occurs, retry the upload by initiating a new resumable upload session.

What's next

Learn more about resumable uploads.
Read an overview for the JSON API and XML API.
Stream uploads of unknown size.
Batch multiple requests to the Cloud Storage JSON API.