This page describes how to make a resumable upload request in the Cloud Storage JSON and XML APIs. This protocol lets you resume an upload operation after a communication failure interrupts the flow of data.
For information on using resumable uploads in the Google Cloud CLI and client libraries, see How tools and APIs use resumable uploads.
Required roles
To get the permissions that you need to perform a resumable upload, ask your administrator to grant you one of the following roles:
For uploads that include an Object Retention Lock, ask your administrator to grant you the Storage Object Admin (
roles/storage.objectAdmin
) IAM role for the bucket.For all other cases, ask your administrator to grant you the Storage Object User (
roles/storage.objectUser
) IAM role for the bucket.
These predefined roles contain the permissions required to upload an object to a bucket for their respective cases. To see the exact permissions that are required, expand the Required permissions section:
Required permissions
storage.objects.create
storage.objects.delete
- This permission is only required for uploads that overwrite an existing object.
storage.objects.setRetention
- This permission is only required for uploads that include an Object Retention Lock.
You can also get these permissions with other predefined roles or custom roles.
For information about granting roles on buckets, see Use IAM with buckets.
Initiate a resumable upload session
To initiate a resumable upload session:
JSON API
Have gcloud CLI installed and initialized, which lets you generate an access token for the
Authorization
header.Optionally, create a JSON file that contains the metadata you want to set on the object that you're uploading. For example, the following JSON file sets the
contentType
metadata of the object you want to upload toimage/png
:{ "contentType": "image/png" }
Use
cURL
to call the JSON API with aPOST
Object request:curl -i -X POST --data-binary @METADATA_LOCATION \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ -H "Content-Length: INITIAL_REQUEST_LENGTH" \ "https://storage.googleapis.com/upload/storage/v1/b/BUCKET_NAME/o?uploadType=resumable&name=OBJECT_NAME"
Where:
METADATA_LOCATION
is the local path to the JSON file containing the optional metadata you specified in the previous step. If you are not including a metadata file, exclude this, along with--data-binary @
and theContent-Type
header.INITIAL_REQUEST_LENGTH
is the number of bytes in the body of this initial request, for example79
.BUCKET_NAME
is the name of the bucket to which you are uploading your object. For example,my-bucket
.OBJECT_NAME
is the URL-encoded name you want to give your object. For example,pets/dog.png
, URL-encoded aspets%2Fdog.png
. This is not required if you included aname
in the object metadata file in Step 2.
If you have enabled Cross-Origin Resource Sharing, you should also include an
Origin
header in both this and subsequent upload requests.Optional headers that you can add to the request include
X-Upload-Content-Type
andX-Upload-Content-Length
.If successful, the response includes a
200
status code.Save the resumable session URI given in the
Location
header of the response to yourPOST
Object request.This URI is used in subsequent requests to upload the object data.
XML API
Have gcloud CLI installed and initialized, which lets you generate an access token for the
Authorization
header.Use
cURL
to call the XML API with aPOST
Object request that has an empty body:curl -i -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Length: 0" \ -H "Content-Type: OBJECT_CONTENT_TYPE" \ -H "x-goog-resumable: start" \ "https://storage.googleapis.com/BUCKET_NAME/OBJECT_NAME"
Where:
OBJECT_CONTENT_TYPE
is the content type of the object. For example,image/png
. If you do not specify a content type, the Cloud Storage system sets theContent-Type
metadata for the object to beapplication/octet-stream
.BUCKET_NAME
is the name of the bucket to which you are uploading your object. For example,my-bucket
.OBJECT_NAME
is the URL-encoded name you want to give your object. For example,pets/dog.png
, URL-encoded aspets%2Fdog.png
.
If you have enabled Cross-Origin Resource Sharing, you should also include an
Origin
header in both this and subsequent upload requests.If successful, the response includes a
201
status message.Save the resumable session URI given in the
Location
header of the response to yourPOST
Object request.This URI is used in subsequent requests to upload the object data.
Upload the data
Once you have initiated a resumable upload, there are two ways to upload the object's data:
- In a single chunk: This approach is usually best, since it requires fewer requests and thus has better performance.
- In multiple chunks: Use this approach if you need to reduce the amount of data transferred in any single request, such as when there is a fixed time limit for individual requests, or if you don't know the total size of the upload at the time the upload begins.
Single chunk upload
To upload the data in a single chunk:
JSON API
Use
cURL
to call the JSON API with aPUT
Object request:curl -i -X PUT --data-binary @OBJECT_LOCATION \ -H "Content-Length: OBJECT_SIZE" \ "SESSION_URI"
Where:
OBJECT_LOCATION
is the local path to your object. For example,Desktop/dog.png
.OBJECT_SIZE
is the number of bytes in your object. For example,20000000
.SESSION_URI
is the value returned in theLocation
header when you initiated the resumable upload.
Optionally, you can include headers prefixed with
X-Goog-Meta-
to add custom metadata for the object as part of this request.
XML API
Use
cURL
to call the XML API with aPUT
Object request:curl -i -X PUT --data-binary @OBJECT_LOCATION \ -H "Content-Length: OBJECT_SIZE" \ "SESSION_URI"
Where:
OBJECT_LOCATION
is the local path to your object. For example,Desktop/dog.png
.OBJECT_SIZE
is the number of bytes in your object. For example,20000000
.SESSION_URI
is the value returned in theLocation
header when you initiated the resumable upload.
If the upload completes in its entirety, you receive a 200 OK
or 201 Created
response, along with any metadata associated with the resource.
If the upload request is interrupted, or if you receive a 5xx
response, follow the procedure in Resuming an interrupted upload.
Multiple chunk upload
To upload the data in multiple chunks:
JSON API
Create a chunk of data from the overall data you want to upload.
The chunk size should be a multiple of 256 KiB (256 x 1024 bytes), unless it's the last chunk that completes the upload. Larger chunk sizes typically make uploads faster, but note that there's a tradeoff between speed and memory usage. It's recommended that you use at least 8 MiB for the chunk size.
Use
cURL
to call the JSON API with aPUT
Object request:curl -i -X PUT --data-binary @CHUNK_LOCATION \ -H "Content-Length: CHUNK_SIZE" \ -H "Content-Range: bytes CHUNK_FIRST_BYTE-CHUNK_LAST_BYTE/TOTAL_OBJECT_SIZE" \ "SESSION_URI"
Where:
CHUNK_LOCATION
is the local path to the chunk that you're currently uploading.CHUNK_SIZE
is the number of bytes you're uploading in the current request. For example,8388608
.CHUNK_FIRST_BYTE
is the starting byte in the overall object that the chunk you're uploading contains.CHUNK_LAST_BYTE
is the ending byte in the overall object that the chunk you're uploading contains.TOTAL_OBJECT_SIZE
is the total size of the object you are uploading.SESSION_URI
is the value returned in theLocation
header when you initiated the resumable upload.
An example
Content-Range
isContent-Range: bytes 0-8388607/20000000
. SeeContent-Range
for more information about this header.If the request succeeds, the server responds with
308 Resume Incomplete
. The response contains aRange
header.Repeat the above steps for each remaining chunk of data that you want to upload, using the upper value contained in the
Range
header of each response to determine where to start each successive chunk; you should not assume that the server received all bytes sent in any given request.Optionally, in the final request of the resumable upload you can include headers prefixed with
X-Goog-Meta-
to add custom metadata for the object.
XML API
Create a chunk of data from the overall data you want to upload.
The chunk size should be a multiple of 256 KiB (256 x 1024 bytes), unless it's the last chunk that completes the upload. Larger chunk sizes typically make uploads faster, but note that there's a tradeoff between speed and memory usage. It's recommended that you use at least 8 MiB for the chunk size.
Use
cURL
to call the XML API with aPUT
Object request:curl -i -X PUT --data-binary @CHUNK_LOCATION \ -H "Content-Length: CHUNK_SIZE" \ -H "Content-Range: bytes CHUNK_FIRST_BYTE-CHUNK_LAST_BYTE/TOTAL_OBJECT_SIZE" \ "SESSION_URI"
Where:
CHUNK_LOCATION
is the local path to the chunk that you're currently uploading.CHUNK_SIZE
is the number of bytes you're uploading in the current request. For example,8388608
.CHUNK_FIRST_BYTE
is the starting byte in the overall object that the chunk you're uploading contains.CHUNK_LAST_BYTE
is the ending byte in the overall object that the chunk you're uploading contains.TOTAL_OBJECT_SIZE
is the total size of the object you are uploading.SESSION_URI
is the value returned in theLocation
header when you initiated the resumable upload.
An example
Content-Range
isContent-Range: bytes 0-8388607/20000000
. SeeContent-Range
for more information about this header.If the request succeeds, the server responds with
308 Resume Incomplete
. The response contains aRange
header.Repeat the above steps for each remaining chunk of data that you want to upload, using the upper value contained in the
Range
header of each response to determine where to start each successive chunk; you should not assume that the server received all bytes sent in any given request.
Once the upload completes in its entirety, you receive a 200 OK
or
201 Created
response, along with any metadata associated with the resource.
If any of the chunk uploads are interrupted, or if you receive a 5xx
response, you should either resend the interrupted chunk in its entirety or
resume the interrupted upload from where it left off.
Check the status of a resumable upload
If your resumable upload is interrupted, or you're not sure the upload completed, you can check the status of the upload:
JSON API
Use
cURL
to call the JSON API with aPUT
Object request:curl -i -X PUT \ -H "Content-Length: 0" \ -H "Content-Range: bytes */OBJECT_SIZE" \ "SESSION_URI"
Where:
OBJECT_SIZE
is the total number of bytes in your object. If you don't know the full size of your object, use*
for this value.SESSION_URI
is the value returned in theLocation
header when you initiated the resumable upload.
XML API
Use
cURL
to call the XML API with aPUT
Object request:curl -i -X PUT \ -H "Content-Length: 0" \ -H "Content-Range: bytes */OBJECT_SIZE" \ "SESSION_URI"
Where:
OBJECT_SIZE
is the total number of bytes in your object. If you don't know the full size of your object, use*
for this value.SESSION_URI
is the value returned in theLocation
header when you initiated the resumable upload.
A 200 OK
or 201 Created
response indicates that the upload was
completed, and no further action is necessary.
A 308 Resume Incomplete
response indicates that you need to continue
uploading the data.
- If Cloud Storage has not yet persisted any bytes, the
308
response does not have aRange
header. In this case, you should start your upload from the beginning. - Otherwise, the
308
response has aRange
header, which specifies which bytes Cloud Storage has persisted so far. Use this value when resuming an interrupted upload.
Resume an interrupted upload
If an upload request is terminated before receiving a response, or if you
receive a 503
or 500
response, then you need to resume
the interrupted upload from where it left off. To resume an interrupted upload:
JSON API
Check the status of your resumable upload.
Save the upper value of the
Range
header contained in the response to your status check. If the response does not have aRange
header, Cloud Storage has not yet persisted any bytes, and you should upload from the beginning.Ensure that that object data you're about to upload begins at the byte following the upper value in the
Range
header.If the interrupted request contained headers prefixed with
X-Goog-Meta-
, include those headers in the request to resume your upload.Use
cURL
to call the JSON API with aPUT
Object request that picks up at the byte following the value in theRange
header:curl -i -X PUT --data-binary @PARTIAL_OBJECT_LOCATION \ -H "Content-Length: UPLOAD_SIZE_REMAINING" \ -H "Content-Range: bytes NEXT_BYTE-LAST_BYTE/TOTAL_OBJECT_SIZE" \ "SESSION_URI"
Where:
PARTIAL_OBJECT_LOCATION
is the local path to the remaining portion of data that you want to upload.UPLOAD_SIZE_REMAINING
is the number of bytes you're uploading in the current request. For example, uploading the rest of an object with a total size of 20000000 that was interrupted after bytes 0-42 uploaded would have anUPLOAD_SIZE_REMAINING
of19999957
.NEXT_BYTE
is the next integer after the value you saved in step 2. For example, if42
is the upper value in step 2, the value forNEXT_BYTE
is43
.LAST_BYTE
is the ending byte contained in thisPUT
request. For example, to finish uploading an object whose total size is20000000
, the value forLAST_BYTE
is19999999
.TOTAL_OBJECT_SIZE
is the total size of the object you are uploading. For example,20000000
.SESSION_URI
is the value returned in theLocation
header when you initiated the resumable upload.
XML API
Check the status of your resumable upload.
Save the upper value of the
Range
header contained in the response to your status check. If the response does not have aRange
header, Cloud Storage has not yet persisted any bytes, and you should upload from the beginning.Ensure that that object data you're about to upload begins at the byte following the upper value in the
Range
header.Use
cURL
to call the XML API with aPUT
Object request that picks up at the byte following the value in theRange
header:curl -i -X PUT --data-binary @PARTIAL_OBJECT_LOCATION \ -H "Content-Length: UPLOAD_SIZE_REMAINING" \ -H "Content-Range: bytes NEXT_BYTE-LAST_BYTE/TOTAL_OBJECT_SIZE" \ "SESSION_URI"
Where:
PARTIAL_OBJECT_LOCATION
is the local path to the remaining portion of data that you want to upload.UPLOAD_SIZE_REMAINING
is the number of bytes you're uploading in the current request. For example, uploading the rest of an object with a total size of 20000000 that was interrupted after bytes 0-42 uploaded would have anUPLOAD_SIZE_REMAINING
of19999957
.NEXT_BYTE
is the next integer after the value you saved in step 2. For example, if42
is the upper value in step 2, the value forNEXT_BYTE
is43
.LAST_BYTE
is the ending byte contained in thisPUT
request. For example, to finish uploading an object whose total size is20000000
, the value forLAST_BYTE
is19999999
.TOTAL_OBJECT_SIZE
is the total size of the object you are uploading. For example,20000000
.SESSION_URI
is the value returned in theLocation
header when you initiated the resumable upload.
You can resume uploads as many times as necessary while the session URI is
active; the session URI expires after a week. When the data is successfully
uploaded, Cloud Storage responds with a 200 OK
or 201 created
status code.
Cancel an upload
To cancel an incomplete resumable upload and prevent any further action on it:
JSON API
Use
cURL
to call the JSON API with aDELETE
request:curl -i -X DELETE -H "Content-Length: 0" \ "SESSION_URI"
Where:
SESSION_URI
is the value returned in theLocation
header when you initiated the resumable upload.
If successful, the response contains a 499
status code. Future attempts
to query or resume the upload result in a 4xx
response.
XML API
Use
cURL
to call the XML API with aDELETE
request:curl -i -X DELETE -H "Content-Length: 0" \ "SESSION_URI"
Where:
SESSION_URI
is the value returned in theLocation
header when you initiated the resumable upload.
If successful, the response contains a 204
status code, and future
attempts to query or resume the upload also result in a 204
response.
Failure Handling
Under rare circumstances, a request to resume an interrupted upload might fail with a non-retriable '4xx' error because permissions on the bucket have changed, or because the integrity check on the final uploaded object detected a mismatch. If this occurs, retry the upload by initiating a new resumable upload session.
What's next
- Learn more about resumable uploads.
- Read an overview for the JSON API and XML API.
- Stream uploads of unknown size.
- Batch multiple requests to the Cloud Storage JSON API.