Google Cloud Platform

Using Files API with Google Cloud Storage

Python |Java |PHP |Go

Deprecation Notice: The Files API feature documented on this page, is deprecated as of May 21, 2013. This feature is sometimes referred to as the Google Cloud Storage API, and is schedule for shutdown as documented in the Files API Service Turndown page. It is replaced by the Google Cloud Storage Client Library. The documentation for the deprecated API is retained for the convenience of developers still using the Files API.

Note: The Files API was also known as the Google Cloud Storage API.

The Files API allows your application to store and serve your data as opaque blobs known as "objects". While Cloud Storage offers a RESTful API, this documentation describes how to use Cloud Storage on App Engine.

  1. Introducing Google Cloud Storage
  2. Prerequisites
  3. Using Cloud Storage
  4. Complete sample app
  5. Quotas and limits

Introducing Google Cloud Storage

Google App Engine provides an easier way to read from and write to Cloud Storage objects , which allows applications to create and serve data objects. These objects are stored within buckets in Cloud Storage but can be additionally accessed by Google App Engine applications through the Files API. You can interact with the Files API using the RESTful interface or through the Files Java API for Google App Engine applications, which is discussed in this document.

Cloud Storage is useful for storing and serving large files because it does not have a size limit for your objects. Additionally, Cloud Storage offers the use of access control lists (ACLs), the ability to resume upload operations if they're interrupted, use of the POST policy document, and many other features.

Once you have created an object, it cannot be modified. To modify an existing object, you need to overwrite the object with a new object that contains your desired changes.

For more information, see the Cloud Storage pricing.


To use the Files API to access Cloud Storage, you must complete the following prerequisites:

  1. The Files API for Java is available for App Engine SDKs 1.6.0 and later. Also, you need the Google Plugin For Eclipse to run some of the samples described in this page.

    For instructions on updating or downloading the Google Plugin for Eclipse and the App Engine SDK, see the downloads page for Google Plugin for Eclipse.

  2. Activate Cloud Storage.

    Activate the Cloud Storage service in the Google Developers Console.

  3. Set up billing.

    Cloud Storage requires that you provide billing information. For details about charges, see the Cloud Storage pricing page.

  4. Create a bucket.

    Cloud Storage is made up of buckets, basic containers that hold your data. Buckets hold all your data in the form of objects, individual pieces of data that you upload to Cloud Storage. Before you can use the Files API, you need to create a bucket(s) where you would like to store your data. The easiest way to do so is using the gsutil tool or the online Google Storage browser that is accessible through the Google Developers Console.

  5. Give permissions to your bucket or objects.

    To enable your app to create new objects in a bucket, you need to do the following:

    1. Go to the Developers Console Permissions page.
    2. Select the application you want to authorize for your Cloud Storage bucket.
    3. Scroll down to the service accounts section.
    4. Copy the value under Service Account Name. This is the service account name of your application, in the format If you are using an App Engine Premier Account, the service account name for your application is in the format
    5. Grant access permissions using one of the following methods:
      • The easiest way to grant app access to a bucket is to use the Google Developers Console to add the service account name of the app as a team member to the project that contains the bucket. You can do this under Permissions in the left sidebar of the Google Developers Console. The app should have edit permissions if it needs to write to the bucket. For information about permissions in Cloud Storage, see Scopes and Permissions. Add more apps to the project team if desired.

        Note: In some circumstances, you might not be able to add the service account as a team member. If you cannot add the service account, use the alternative method, bucket ACLs, as described next.

      • An alternate way to grant app access to a bucket is manually edit and set the bucket ACL and the default object ACL, using the gsutil utility:
        1. Get the ACL for the bucket and save it to a file for editing: gsutil acl get gs://mybucket > myAcl.txt
        2. Add the following Entry to the ACL file you just retrieved:
            <Scope type="UserByEmail">
        3. If you are adding multiple apps to the ACL, repeat the above entry for each app, changing only the email address to reflect each app's service name.
        4. Set the modified ACL on your bucket: gsutil acl set myAcl.txt gs://mybucket
        5. You also need to set the default object ACL on the bucket. Many App Engine features require access to objects in the bucket. To set the default object ACL on the bucket:
          1. Get the default object ACL for the bucket and save it to a file for editing: gsutil defacl get gs://mybucket > myDefAcl.txt
          2. Add the same entries that you added above to the bucket ACL, but replacing WRITE with FULL_CONTROL.
          3. Set the modified default object ACL on your bucket: gsutil defacl set myDefAcl.txt gs://mybucket
      • If you need to prevent non-authorized applications from reading certain objects, you can:
        1. Set the predefined project-private ACL on your objects manually
        2. Store your objects in a bucket with the default project-private object ACL
        3. Then, add the service account as a project viewer in the Google Developers Console, giving your App Engine application READ-ONLY access to your objects. You can also set the predefined public-read ACL, which allows all users, whether or not they are authenticated, access to read your object. You won't need to add your application's service account name as a project viewer, but if you have data you do not want to be publicly accessible, do not set the public-read ACL on your object.

          Note: Setting READ permission only on your buckets does not provide READ permission to any objects inside those buckets. You must set READ permission on objects individually or by setting the default ACL before creating any objects in the bucket.

Using Cloud Storage

Applications can use Cloud Storage to read from and write to large files of any type, or to upload, store, and serve large files from users. Files are called "objects" in Cloud Storage once they're uploaded.

Before you begin

You can use the Files API from your App Engine application to read and write data to Cloud Storage. In order to use the Cloud Storage Java API, you must include the following import statements at the beginning of your code:


Creating an object

To create an object:

// Get the file service
FileService fileService = FileServiceFactory.getFileService();

 * Set up properties of your new object
 * After finalizing objects, they are accessible
 * through Cloud Storage with the URL:
GSFileOptionsBuilder optionsBuilder = new GSFileOptionsBuilder()

// Create your object
AppEngineFile writableFile = fileService.createNewGSFile(;

In this example, we also pass in the public-read parameter in the setAcl() method. This allows everyone to read the object. If you do not set this parameter, Cloud Storage sets this parameter as null and uses the default object ACL for that bucket (by default, this is project-private).

You can use the following optional methods to set ACLs and HTTP headers on your object:

Method Description Usage Default
setAcl Set a predefined ACL on your object. setAcl("public-read")
If you do not set this parameter, Cloud Storage sets this parameter as null and uses the default object ACL for that bucket (by default, this is project-private).
setMimeType() Set a Content-Type header on your object. setMimeType("text/html") binary/octet-stream
setCacheControl() Set a Cache-Control header on your object. setCacheControl("no-cache") See Cache-Control
setContentEncoding() If your object is compressed, specify the compression method using the Content-Encoding header. setContentEncoding("gzip") None
setContentDisposition() Set the Content-Disposition header for your object. setContentDisposition("attachment;filename=filename.ext") None
addUserMetadata() Create and apply Map of custom headers and values. addUserMetadata('header1', 'value1', 'header2', 'value2')
These custom headers are served in the following format when accessing objects through

x-goog-meta-custom_header: <em>custom_value

The following code snippet provides an example using some of these optional methods:

GSFileOptionsBuilder optionsBuilder = new GSFileOptionsBuilder()
  .setUserMetadata("date-created", "092011", "owner", "Jon");

AppEngineFile writableFile = fileService.createNewGSFile(;

Opening and writing to the object

To write to an object, you must open a channel for writing:

// Open a channel for writing
boolean lockForWrite = false; // Do you want to exclusively lock this object?
FileWriteChannel writeChannel = fileService.openWriteChannel(writableFile, lockForWrite);

Next, you can write to the object using standard Java ways:

// For this example, we write to the object using the PrintWriter
PrintWriter out = new PrintWriter(Channels.newWriter(writeChannel, "UTF8"));
out.println("The woods are lovely and deep.");
out.println("But I have promises too keep.");

// Close the object without finalizing.

 * You can write to this file again, later, by saving the file path.
 * It is not possible to read the file until you finalize it.
 * Once you finalize a file, it is not possible to write to it.

// Save the file path
String path = writableFile.getFullPath();

// Lock the file so no one else can access it at the same time
lockForWrite = true;

// Write to the unfinalized file again in a separate request
writableFile = new AppEngineFile(path);
writeChannel = fileService.openWriteChannel(writableFile, lockForWrite);
writeChannel.write(ByteBuffer.wrap("And miles to go before I sleep.".getBytes()));

Finalizing the object

Once you are done writing to the object, you need to finalize the file before you can read from it:

// Finalize the object

Your object is now visible to App Engine with the file name /gs/my_bucket/my_object. If you set your object to be publicly accessible, your object can be accessed using the URL

Once you finalize an object, you can no longer write to it. If you want to modify your file, you will need to create a new object with the same name to overwrite it.

Reading the object

Before you can read an object, you must finalize the object.

To read an object:

boolean lockForRead = false;
String filename = "/gs/my_bucket/my_object";
AppEngineFile readableFile = new AppEngineFile(filename);
FileReadChannel readChannel = fileService.openReadChannel(readableFile, lockForRead);

// Read the file in whichever way you'd like
BufferedReader reader = new BufferedReader(Channels.newReader(readChannel, "UTF8"));
String line = reader.readLine();
resp.getWriter().println("READ:" + line);


The maximum size of an object that can be read by an application with one API call is 32 megabytes.

Complete sample app

The following is a sample application that demonstrates one use of the Files API within an App Engine application. In order to successfully run the application, you must be an owner or have WRITE access to a bucket.

To set up and run this application:

  1. Follow the instructions on Creating a New Web Application, to set up a new blank App Engine project.
  2. Copy the following imports to the beginning of your file:

    import java.nio.ByteBuffer;
    import java.nio.channels.Channels;
    import javax.servlet.http.*;
  3. Override the doGet() method in your file with the following code, replacing BUCKETNAME and FILENAME with your bucket and file names:

     * Copyright 2011 Google Inc. All Rights Reserved.
     * Create, Write, Read, and Finalize Cloud Storage objects.
     * Access your app at:
    public class GuestbookServlet extends HttpServlet {
       public static final String BUCKETNAME = "YOUR_BUCKET_NAME";
       public static final String FILENAME = "YOUR_FILE_NAME";
      public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
        resp.getWriter().println("Hello, world from java");
        FileService fileService = FileServiceFactory.getFileService();
        GSFileOptionsBuilder optionsBuilder = new GSFileOptionsBuilder()
           .addUserMetadata("myfield1", "my field value");
        AppEngineFile writableFile =
        // Open a channel to write to it
         boolean lock = false;
         FileWriteChannel writeChannel =
             fileService.openWriteChannel(writableFile, lock);
         // Different standard Java ways of writing to the channel
         // are possible. Here we use a PrintWriter:
         PrintWriter out = new PrintWriter(Channels.newWriter(writeChannel, "UTF8"));
         out.println("The woods are lovely dark and deep.");
         out.println("But I have promises to keep.");
         // Close without finalizing and save the file path for writing later
         String path = writableFile.getFullPath();
         // Write more to the file in a separate request:
         writableFile = new AppEngineFile(path);
         // Lock the file because we intend to finalize it and
         // no one else should be able to edit it
         lock = true;
         writeChannel = fileService.openWriteChannel(writableFile, lock);
         // This time we write to the channel directly
                   ("And miles to go before I sleep.".getBytes()));
         // Now finalize
         resp.getWriter().println("Done writing...");
         // At this point, the file is visible in App Engine as:
         // "/gs/BUCKETNAME/FILENAME"
         // and to anybody on the Internet through Cloud Storage as:
         // (
         // We can now read the file through the API:
         String filename = "/gs/" + BUCKETNAME + "/" + FILENAME;
         AppEngineFile readableFile = new AppEngineFile(filename);
         FileReadChannel readChannel =
             fileService.openReadChannel(readableFile, false);
         // Again, different standard Java ways of reading from the channel.
         BufferedReader reader =
                 new BufferedReader(Channels.newReader(readChannel, "UTF8"));
         String line = reader.readLine();
         resp.getWriter().println("READ:" + line);
        // line = "The woods are lovely, dark, and deep."
  4. Deploy your application
    From Eclipse:

    1. Click the Google icon on the toolbar in Eclipse.
    2. In the drop down menu, click Deploy to App Engine.
    3. Click App Engine project settings....
    4. Enter your Application ID in the Deployment section of the dialog, and click OK.
    5. Enter your email address and password, and click Deploy.

    From commandline:

    You can upload your application code and files using a command included in the SDK named appcfg.cmd (Windows) or (Mac OS X, Linux).

    AppCfg is a multi-purpose tool for interacting with your app on App Engine. The command takes the name of an action, the path to your application's war/ directory, and other options. To upload the app code and files to App Engine, you use the update action.

    To upload the app using Windows:

    ..\appengine-java-sdk\bin\appcfg.cmd update war

    To upload the app using Mac OS X or Linux:

    ../appengine-java-sdk/bin/ update war

    Enter your Google Account username and password at the prompts.

  5. View your application at

Quotas and limits

Cloud Storage is a pay-to-use service; you will be charged according to the Cloud Storage price sheet.