Reading and Writing to Google Cloud Storage

This document describes how to store and retrieve data using Cloud Storage in an App Engine app using the Google Cloud Storage client library. It assumes that you completed the tasks described in Setting Up for Google Cloud Storage to activate a Cloud Storage bucket and download the client libraries. It also assumes that you know how to build an App Engine application, as described in the Quickstart for Java App Engine standard environment.

Required imports

The following snippet shows the imports you need for Cloud Storage acccess via the client library:

import com.google.appengine.tools.cloudstorage.GcsFileOptions;
import com.google.appengine.tools.cloudstorage.GcsFilename;
import com.google.appengine.tools.cloudstorage.GcsInputChannel;
import com.google.appengine.tools.cloudstorage.GcsOutputChannel;
import com.google.appengine.tools.cloudstorage.GcsService;
import com.google.appengine.tools.cloudstorage.GcsServiceFactory;
import com.google.appengine.tools.cloudstorage.RetryParams;

Specifying the Cloud Storage bucket

The following snippet shows one way to allow the user to specify a bucket name when writing a file to a bucket:

function uploadFile() {
  var bucket = document.forms["putFile"]["bucket"].value;
  var filename = document.forms["putFile"]["fileName"].value;
  if (bucket == null || bucket == "" || filename == null || filename == "") {
    alert("Both Bucket and FileName are required");
    return false;
  } else {
    var postData = document.forms["putFile"]["content"].value;
    document.getElementById("content").value = null;

    var request = new XMLHttpRequest();
    request.open("POST", "/gcs/" + bucket + "/" + filename, false);
    request.setRequestHeader("Content-Type", "text/plain;charset=UTF-8");
    request.send(postData);
  }
}

The snippet prepends the required /gcs/ to the user supplied bucket and file name.

Writing to Cloud Storage

To write a file to Cloud Storage:

@Override
public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException {
  GcsFileOptions instance = GcsFileOptions.getDefaultInstance();
  GcsFilename fileName = getFileName(req);
  GcsOutputChannel outputChannel;
  outputChannel = gcsService.createOrReplace(fileName, instance);
  copy(req.getInputStream(), Channels.newOutputStream(outputChannel));
}

This sample writes a new file to Cloud Storage, or, if a file with the same name already exists, it overwrites it. This is useful because once you write a file to Cloud Storage, you cannot modify it. To change the file, you must make your modifications to a copy of the file, then overwrite the old file.

Reading from Cloud Storage

To read a file from Cloud Storage:

@Override
public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
  GcsFilename fileName = getFileName(req);
  if (SERVE_USING_BLOBSTORE_API) {
    BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();
    BlobKey blobKey = blobstoreService.createGsBlobKey(
        "/gs/" + fileName.getBucketName() + "/" + fileName.getObjectName());
    blobstoreService.serve(blobKey, resp);
  } else {
    GcsInputChannel readChannel = gcsService.openPrefetchingReadChannel(fileName, 0, BUFFER_SIZE);
    copy(Channels.newInputStream(readChannel), resp.getOutputStream());
  }
}

In the line containing gcsService.openPrefetchingReadChannel, notice the use of prefetching. This buffers data in memory and prefetches it before it is required to avoid blocking on the read call.