Using Cloud Storage with Go

This part of the Go Bookshelf tutorial shows how the sample app stores images in Google Cloud Storage.

This page is part of a multi-page tutorial. To start from the beginning and see instructions for setting up, go to Go Bookshelf App.

Creating a Cloud Storage bucket

The following instructions show how to create a Cloud Storage bucket.

To create a bucket:

  1. Invoke the following command in a terminal window:

    gsutil mb gs://[YOUR-BUCKET-NAME]

  2. Set the bucket's default ACL to public-read, which enables users to see their uploaded images:

    gsutil defacl set public-read gs://[YOUR-BUCKET-NAME]

Configuring settings

  1. Go to the directory that contains the sample code:

    Linux/Mac OS X

    cd $GOPATH/src/github.com/GoogleCloudPlatform/golang-samples/getting-started/bookshelf
    

    Windows

    cd %GOPATH%\src\github.com\GoogleCloudPlatform\golang-samples\getting-started\bookshelf
    

  2. Open config.go for editing.

  3. Uncomment these lines:

    // StorageBucketName = "<your-storage-bucket>"
    // StorageBucket, err = configureStorage(StorageBucketName)
    
  4. Replace <your-storage-bucket> with the name of your Cloud Storage bucket.

  5. Save and close config.go.

Running the app on your local machine

  1. Run the app to start a local web server:

    cd app
    go run app.go auth.go template.go
    
  2. In your web browser, enter this address:

    http://localhost:8080

Now you can browse the app's web pages, add books with cover images, edit books, and delete books.

Press Control+C to exit the local web server.

Deploying the app to the App Engine flexible environment

  1. In the app directory, enter this command to deploy the sample:

    gcloud app deploy
    
  2. In your web browser, enter this address. Replace [YOUR_PROJECT_ID] with your project ID:

    https://[YOUR_PROJECT_ID].appspot.com
    

If you update your app, you can deploy the updated version by entering the same command you used to deploy the app the first time. The new deployment creates a new version of your app and promotes it to the default version. The older versions of your app remain, as do their associated VM instances. Be aware that all of these app versions and VM instances are billable resources.

You can reduce costs by deleting the non-default versions of your app.

To delete an app version:

  1. In the Cloud Platform Console, go to the App Engine Versions page.

    Go to the Versions page

  2. Click the checkbox next to the non-default app version you want to delete.
  3. Click the Delete button at the top of the page to delete the app version.

For complete information about cleaning up billable resources, see the Cleaning up section in the final step of this tutorial.

Application structure

Binary data sample structure

The application uses Cloud Storage to store binary data, which are pictures in this case,while continuing to use a structured database for the book information, which is Cloud Datastore, Cloud SQL, or MongoDB.

Understanding the code

This section walks you through the application code and explains how it works.

Handling user uploads

To allow users to upload images, the add/edit form sets the enctype to multipart/form-data. The template action tags specify the target of the HTTP POST request. For a new book, which does not have an ID, the target is /books. For an existing book, it is /books/$ID:

<form method="post" enctype="multipart/form-data" action="/books{{if .}}/{{.ID}}{{end}}">

The image field accepts a file as its input:

<div class="form-group">
  <label for="image">Cover Image</label>
  <input class="form-control" name="image" id="image" type="file">
</div>

Uploading to Cloud Storage

The uploadFileFromForm function checks whether the image form field is present, and if it is, uploads the file to Cloud Storage. Finally, the function returns the public URL of the uploaded file.

// uploadFileFromForm uploads a file if it's present in the "image" form field.
func uploadFileFromForm(r *http.Request) (url string, err error) {
	f, fh, err := r.FormFile("image")
	if err == http.ErrMissingFile {
		return "", nil
	}
	if err != nil {
		return "", err
	}

	if bookshelf.StorageBucket == nil {
		return "", errors.New("storage bucket is missing - check config.go")
	}

	// random filename, retaining existing extension.
	name := uuid.NewV4().String() + path.Ext(fh.Filename)

	ctx := context.Background()
	w := bookshelf.StorageBucket.Object(name).NewWriter(ctx)
	w.ACL = []storage.ACLRule{{Entity: storage.AllUsers, Role: storage.RoleReader}}
	w.ContentType = fh.Header.Get("Content-Type")

	// Entries are immutable, be aggressive about caching (1 day).
	w.CacheControl = "public, max-age=86400"

	if _, err := io.Copy(w, f); err != nil {
		return "", err
	}
	if err := w.Close(); err != nil {
		return "", err
	}

	const publicURL = "https://storage.googleapis.com/%s/%s"
	return fmt.Sprintf(publicURL, bookshelf.StorageBucketName, name), nil
}

The uploadFileFromForm function is called by bookFromForm, which extracts all of the expected form values into a Book struct. The bookFromForm function stores the public URL of the image in the database as a string.

// bookFromForm populates the fields of a Book from form values
// (see templates/edit.html).
func bookFromForm(r *http.Request) (*bookshelf.Book, error) {
	imageURL, err := uploadFileFromForm(r)
	if err != nil {
		return nil, fmt.Errorf("could not upload file: %v", err)
	}
	if imageURL == "" {
		imageURL = r.FormValue("imageURL")
	}

	book := &bookshelf.Book{
		Title:         r.FormValue("title"),
		Author:        r.FormValue("author"),
		PublishedDate: r.FormValue("publishedDate"),
		ImageURL:      imageURL,
		Description:   r.FormValue("description"),
		CreatedBy:     r.FormValue("createdBy"),
		CreatedByID:   r.FormValue("createdByID"),
	}

Serving images from Cloud Storage

Because we have the public URL for the image, serving it is straightforward. Serving directly from Cloud Storage is helpful because the requests leverage Google’s global serving infrastructure and the application doesn’t have to respond to requests for images, freeing up CPU cycles for other requests.

<img src="{{if .ImageURL}}{{.ImageURL}}{{else}}https://placekitten.com/g/200/300{{end}}">

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...