App Engine and Google Cloud Storage Sample

This tutorial shows everything you need to do to enable your App Engine Go app to access Cloud Storage. This non-interactive sample show to create, write, read, and list files in the Cloud Storage bucket.

When this sample runs, it executes a script that does file writes, reads, copies, and writes the output to the browser.


  • Deploy the sample app to Google App Engine.
  • Run the app.
  • Learn how to create a file and write the file to a bucket.
  • Learn how to read files.
  • Learn how to copy files.
  • Learn how to list files in the bucket.


App Engine has free a level of usage. If your total usage of App Engine is less than the limits specified in the App Engine free quota, there is no charge for doing this tutorial.

Before you begin

  1. Complete the Quickstart for Go App Engine standard environment.
  2. Sign in to your Google account.

    If you don't already have one, sign up for a new account.

  3. Select or create a Cloud Platform Console project.

    Go to the Projects page

  4. Make a note of the project ID, which might be different from the project name. The project ID is used in commands and in the app.yaml configuration file.
  5. Downloaded and install the App Engine SDK for Go.
  6. Activate the default Cloud Storage bucket
  7. Download and install the Google Cloud Storage Client Library.

Downloading the sample project

To download the project, choose either of these methods:

  • Clone the project containing the sample code:

    git clone
  • Download the sample as a zip file and extract it.

Specify the project ID in the sample

The deployment utility deploys the app to the project ID specified in the app.yaml file.

To specify the project ID in the sample code:

  1. Change directory to google-cloud-go/examples/storage/appengine.

  2. Edit app.yaml replacing the value <your-app-id-here> with the project ID you obtained previously.

  3. Save your changes.

Deploying and running the sample

To deploy the sample app to App Engine and run it:

  1. In the subdirectory google-cloud-go/examples/storage/appengine, invoke goapp as follows:

    [PATH_TO_APP_ENGINE_SDK_FOR_GO]/goapp deploy
  2. Follow the prompts to authorize the app deployment. Typically, you only need to supply these the first time you deploy an app to a project.

  3. Wait for the app to compile and deploy.

  4. In your browser, visit https://<your-project-id>; the application executes on page load, writing to and reading from the Cloud Storage bucket.

  5. The results are displayed in the browser:

    demo sample

As indicated in the results shown above, the app:

Code walkthrough

This walkthrough only covers how to read and write to a Cloud Storage bucket, and list the bucket contents, even though the sample code contains additional Cloud Storage functionality.

app.yaml walkthrough

The configuration in the app.yaml file is typical for nearly any app. Like any other App Engine app, it allows you to specify application configuration details, such as the project ID (also called application ID in App Engine) and the application version:

application: <your-app-id-here>
version: v1
runtime: go
api_version: go1

- url: /.*
  script: _go_app

For more information on configuration options available in this file, see the app.yaml reference.

app.go imports walkthrough

The app.go file contains these imports:

import (


The imports in the file required for App Engine and for Cloud Storage are:


Specifying the Cloud Storage bucket

Before you can execute any Cloud Storage operation, you must supply the bucket name. The easiest way to do this is to use the default bucket for your project, which can be obtained from the App Engine context:

ctx := appengine.NewContext(r)
if bucket == "" {
	var err error
	if bucket, err = file.DefaultBucketName(ctx); err != nil {
		log.Errorf(ctx, "failed to get default GCS bucket name: %v", err)

Writing a file to Cloud Storage

To write a file to Cloud Storage:

// createFile creates a file in Google Cloud Storage.
func (d *demo) createFile(fileName string) {
	fmt.Fprintf(d.w, "Creating file /%v/%v\n", bucket, fileName)

	wc := d.bucket.Object(fileName).NewWriter(d.ctx)
	wc.ContentType = "text/plain"
	wc.Metadata = map[string]string{
		"x-goog-meta-foo": "foo",
		"x-goog-meta-bar": "bar",
	d.cleanUp = append(d.cleanUp, fileName)

	if _, err := wc.Write([]byte("abcde\n")); err != nil {
		d.errorf("createFile: unable to write data to bucket %q, file %q: %v", bucket, fileName, err)
	if _, err := wc.Write([]byte(strings.Repeat("f", 1024*4) + "\n")); err != nil {
		d.errorf("createFile: unable to write data to bucket %q, file %q: %v", bucket, fileName, err)
	if err := wc.Close(); err != nil {
		d.errorf("createFile: unable to close bucket %q, file %q: %v", bucket, fileName, err)

When the file is created, the sample specifies Cloud Storage headers (x-goog-meta-foo and x-goog-meta-bar). This optional code introduces the notion of using Cloud Storage headers, which you can apply to

  • affect request behavior
  • specify access to the file in the bucket different from the defaults (see x-goog-acl)
  • write file metadata.

The x-goog-meta- headers shown above are custom file metadata that you can set; these headers are always returned with the file. Note that the space available for custom headers and their data is limited to a few kilobytes, so use these carefully.

Because the code sample doesn't set x-goog-acl, the default Cloud Storage ACL of public read is applied to the object when it is written to the bucket.

Finally, notice the call to Close the file after you finish the write. If you don't do this, the file is not written to Cloud Storage. Be aware that after you call Close, you cannot append to the file.

Reading a file from Cloud Storage

To read a file from Cloud Storage:

// readFile reads the named file in Google Cloud Storage.
func (d *demo) readFile(fileName string) {
	io.WriteString(d.w, "\nAbbreviated file content (first line and last 1K):\n")

	rc, err := d.bucket.Object(fileName).NewReader(d.ctx)
	if err != nil {
		d.errorf("readFile: unable to open file from bucket %q, file %q: %v", bucket, fileName, err)
	defer rc.Close()
	slurp, err := ioutil.ReadAll(rc)
	if err != nil {
		d.errorf("readFile: unable to read data from bucket %q, file %q: %v", bucket, fileName, err)

	fmt.Fprintf(d.w, "%s\n", bytes.SplitN(slurp, []byte("\n"), 2)[0])
	if len(slurp) > 1024 {
		fmt.Fprintf(d.w, "...%s\n", slurp[len(slurp)-1024:])
	} else {
		fmt.Fprintf(d.w, "%s\n", slurp)

Listing bucket contents

This sample code shows how to list the contents of the bucket:

// listBucket lists the contents of a bucket in Google Cloud Storage.
func (d *demo) listBucket() {
	io.WriteString(d.w, "\nListbucket result:\n")

	query := &storage.Query{Prefix: "foo"}
	for query != nil {
		objs, err := d.bucket.List(d.ctx, query)
		if err != nil {
			d.errorf("listBucket: unable to list bucket %q: %v", bucket, err)
		query = objs.Next

		for _, obj := range objs.Results {

Copying Cloud Storage files

This sample code shows how to list the contents of the bucket:

// copyFile copies a file in Google Cloud Storage.
func (d *demo) copyFile(fileName string) {
	copyName := fileName + "-copy"
	fmt.Fprintf(d.w, "Copying file /%v/%v to /%v/%v:\n", bucket, fileName, bucket, copyName)

	obj, err := d.bucket.Object(fileName).CopyTo(d.ctx, d.bucket.Object(copyName), nil)
	if err != nil {
		d.errorf("copyFile: unable to copy /%v/%v to bucket %q, file %q: %v", bucket, fileName, bucket, copyName, err)
	d.cleanUp = append(d.cleanUp, copyName)


What's next

Learn more about the following topics:

Send feedback about...

App Engine standard environment for Go