This document covers some techniques you can use to improve the performance of your application. In some cases, examples from other APIs or generic APIs are used to illustrate the ideas presented. However, the same concepts are applicable to the Cloud Storage JSON API.
Using gzip
An easy and convenient way to reduce the bandwidth needed for each request is to enable gzip compression. Although this requires additional CPU time to uncompress the results, the trade-off with network costs usually makes it very worthwhile.
An object that was uploaded in gzip format can generally be served in gzip format when certain criteria are met. If the criteria are not met, the content is likely to get decompressed at serving time. For a full discussion on the behavior of gzipped content, see the Transcoding page.
To specify that multiple content encodings are acceptable, concatenate them as shown in RFC 7231.
Working with partial resources
Another way to improve the performance of your API calls is by sending and receiving only the portion of the data that you're interested in. This lets your application avoid transferring, parsing, and storing unneeded fields, so it can use resources including network, CPU, and memory more efficiently.
There are two types of partial requests:
- Partial response: A request where you specify which
fields to include in the response (use the
fields
request parameter). - Patch: An update request where you send only the fields you want
to change (use the
PATCH
HTTP verb).
More details on making partial requests are provided in the following sections.
Partial response
By default, the server sends back the full representation of a resource after processing requests. For better performance, you can ask the server to send only the fields you really need and get a partial response instead.
To request a partial response, use the fields
request parameter to specify
the fields you want returned. You can use this parameter with any request
that returns response data.
Note that the fields
parameter only affects the response data; it does not
affect the data that you need to send, if any. To reduce the amount of data
you send when modifying resources, use a patch request.
Example
The following example shows the use of the fields
parameter with the Cloud Storage
JSON API.
Simple request: This HTTP GET
request omits the fields
parameter and
returns the full resource.
https://storage.googleapis.com/storage/v1/b/YOUR-BUCKET/o
Full resource response: The full resource data includes the following fields, along with many others that have been omitted for brevity.
{ "kind": "storage#objects", "items": [ { "kind": "storage#object", "id": "YOUR-BUCKET/object1.png/123123123123", "selfLink": "https://www.googleapis.com/storage/v1/b/YOUR-BUCKET/o/object1.png", "name": "object1.png", "bucket": "YOUR-BUCKET", "generation": "123123123123", ... }, { "kind": "storage#object", "id": "YOUR-BUCKET/object2.png/456456456456", "selfLink": "https://www.googleapis.com/storage/v1/b/YOUR-BUCKET/o/object2.png", "name": "object2.png", "bucket": "YOUR-BUCKET", "generation": "456456456456", "metadata": { "key1": "val1", "key2": "val2" }, ... }, ... ] }
Request for a partial response: The following request for this same resource
uses the fields
parameter to significantly reduce the amount of data returned.
https://storage.googleapis.com/storage/v1/b/YOUR-BUCKET/o?fields=kind,items(id, name, metadata/key1)
Partial response: In response to the request above, the server sends back a response that contains only the kind information along with a pared-down items array that includes only the id, name, and the metadata/key property in each item, if present.
200 OK
{ "kind": "storage#objects", "items": [ { "id": "YOUR-BUCKET/object1.png/123123123123", "name": "object1.png" }, { "id": "YOUR-BUCKET/object2.png/456456456456", "name": "object2.png", "metadata": { "key1": "val1" } }, ... ] }
Note that the response is a JSON object that includes only the selected fields and their enclosing parent objects.
Details on how to format the fields
parameter is covered next, followed by
more details about what exactly gets returned in the response.
Fields parameter syntax summary
The format of the fields
request parameter value is loosely based on XPath
syntax. The supported syntax is summarized below, and additional examples are
provided in the following section.
Use a comma-separated list to select multiple fields.
Use
a/b
to select a fieldb
that is nested within fielda
; usea/b/c
to select a fieldc
nested withinb
.Use a sub-selector to request a set of specific sub-fields of arrays or objects by placing expressions in parentheses "
( )
".For example:
fields=items(id,author/email)
returns only the item ID and author's email for each element in the items array. You can also specify a single sub-field, wherefields=items(id)
is equivalent tofields=items/id
.
More examples of using the fields parameter
The examples below include descriptions of how the fields
parameter value
affects the response.
- Identify the fields you want returned, or make field selections.
The
fields
request parameter value is a comma-separated list of fields, and each field is specified relative to the root of the response. Thus, if you are performing alist
operation, the response is a collection, and it generally includes an array of resources. If you are performing an operation that returns a single resource, fields are specified relative to that resource. If the field you select is (or is part of) an array, the server returns the selected portion of all elements in the array.Here are some collection-level examples:
Example Effect items
Returns all elements in the items array, including all fields in each element, but no other fields. etag,items
Returns both the etag
field and all elements in the items array.items/id
Returns only the id
field for all elements in the items array.
Whenever a nested field is returned, the response includes the enclosing parent objects. The parent fields do not include any other child fields unless they are also selected explicitly.items/metadata/key
Returns only the key
field for all members of themetadata
object, which is itself nested under theitems
array.Here are some resource-level examples:
Example Effect id
Returns the id
field of the requested resource.metadata/key
Returns the key
sub-field of themetadata
object in the requested resource.- Request only parts of specific fields using sub-selections.
By default, if your request specifies particular fields, the server returns the objects or array elements in their entirety. You can specify a response that includes only certain sub-fields. You do this using "
( )
" sub-selection syntax, as in the example below.Example Effect items(id,metadata/key)
Returns only the values of the id
and metadatakey
for each element in the items array.
Handling partial responses
After a server processes a valid request that includes the fields
query
parameter, it sends back an HTTP 200 OK
status code, along with the requested
data. If the fields
query parameter has an error or is otherwise invalid, the
server returns an HTTP 400 Bad Request
status code, along with an error
message telling the user what was wrong with their fields selection (for
example, "Invalid field selection a/b"
).
Patch (partial update)
You can also avoid sending unnecessary data when modifying resources. To send
updated data only for the specific fields that you’re changing, use the HTTP
PATCH
verb.
The short example below shows how using patch minimizes the data you need to send to make a small update.
Example
This example shows a simple patch request to update only the metadata of a
Cloud Storage JSON API object. The object also has an id, a name,
generation, and many other fields, but this request only sends the metadata
field, since that's the only field being modified:
PATCH https://storage.googleapis.com/storage/v1/b/YOUR-BUCKET/o/YOUR-OBJECT Authorization: Bearer your_auth_token Content-Type: application/json { "metadata": {"new-key" : "new-value"} }
Response:
200 OK
{ "id": "your-object-id...", "name": "YOUR-OBJECT", "bucket": "YOUR-BUCKET", "metadata": { "existing-key" : "existing-value", "new-key" : "new-value" }, ... }
The server returns a 200 OK
status code, along with the full representation
of the updated resource. Since only the metadata
field was included in the
patch request, that's the only value that is different from before.
Semantics of a patch request
The body of the patch request includes only the resource fields you want to modify. When you specify a field, you must include any enclosing parent objects, just as the enclosing parents are returned with a partial response. The modified data you send is merged into the data for the parent object, if there is one.
- Add: To add a field that doesn't already exist, specify the new field and its value.
- Modify: To change the value of an existing field, specify the field and set it to the new value.
- Delete: To delete a field, specify the field and set it to
null
. For example,"metadata": null
. You can also delete an entire object (if it is mutable) by setting it tonull
. If you are using the Java API Client Library, useData.NULL_STRING
instead; for details, see JSON null.
Note about arrays: Patch requests that contain arrays replace the existing array with the one you provide. You cannot modify, add, or delete items in an array in a piecemeal fashion.
Handling the response to a patch
After processing a valid patch request, the API returns a 200 OK
HTTP response
code along with the complete representation of the modified resource.
The patch request returns the entire resource representation unless you use the
fields
parameter to reduce the amount of data it returns.
If a patch request results in a new resource state that is syntactically or
semantically invalid, the server returns a 400 Bad Request
or
422 Unprocessable Entity
HTTP status code, and the resource state remains
unchanged. For example, if you attempt to delete the value for a required
field, the server returns an error.
Alternate notation when PATCH HTTP verb is not supported
If your firewall does not allow HTTP PATCH
requests, then you can send an HTTP
POST
request and set the override header to PATCH
, as shown below:
POST https://storage.googleapis.com/... X-HTTP-Method-Override: PATCH ...
Difference between patch and update
In practice, when you send data for an update request that uses the HTTP PUT
verb, you only need to send those fields which are either required or optional;
if you send values for fields that are set by the server, they are ignored.
Although this might seem like another way to do a partial update, this approach
has some limitations. With updates that use the HTTP PUT
verb, the request
fails if you don't supply required parameters, and it clears previously set data
if you don't supply optional parameters.
It's much safer to use patch for this reason. You only supply data for the fields you want to change; fields that you omit are not cleared. The only exception to this rule occurs with repeated elements or arrays: If you omit all of them, they stay just as they are; if you provide any of them, the whole set is replaced with the set that you provide.