Using Cloud Storage with Node.js

This part of the Node.js Bookshelf tutorial shows how the sample app stores images in Google Cloud Storage.

This page is part of a multi-page tutorial. To start from the beginning and see instructions for setting up, go to Node.js Bookshelf App.

Creating a Cloud Storage bucket

The following instructions show how to create a Cloud Storage bucket.

To create a bucket:

  1. Invoke the following command in a terminal window:

    gsutil mb gs://[YOUR-BUCKET-NAME]

  2. Set the bucket's default ACL to public-read, which enables users to see their uploaded images:

    gsutil defacl set public-read gs://[YOUR-BUCKET-NAME]

Configuring settings

Copy yourconfig.json file from the Structured Data part of this tutorial to the nodejs-getting-started/3-binary-data directory. Add this line to the copied file:

"CLOUD_BUCKET": "[YOUR_CLOUD_BUCKET]"

Replace [YOUR_CLOUD_BUCKET] with the name of your Cloud Storage bucket.

Installing dependencies

Install dependencies in the nodejs-getting-started/3-binary-data directory:

  • Using npm:

    npm install
    
  • Using yarn:

    yarn install
    

Running the app on your local machine

  1. Start a local web server using npm or yarn:

    • Using npm:

      npm start
      
    • Using yarn:

      yarn start
      
  2. In your web browser, enter this address.

    http://localhost:8080

Now you can browse the app's web pages, add books with cover images, edit books, and delete books.

Deploying the app to the App Engine flexible environment

  1. Deploy the sample app:

    gcloud app deploy
    

  2. In your web browser, enter this address. Replace [YOUR_PROJECT_ID] with your project ID:

    https://[YOUR_PROJECT_ID].appspot.com
    

If you update your app, you can deploy the updated version by entering the same command you used to deploy the app the first time. The new deployment creates a new version of your app and promotes it to the default version. The older versions of your app remain, as do their associated VM instances. Be aware that all of these app versions and VM instances are billable resources.

You can reduce costs by deleting the non-default versions of your app.

To delete an app version:

  1. In the Cloud Platform Console, go to the App Engine Versions page.

    Go to the Versions page

  2. Click the checkbox next to the non-default app version you want to delete.
  3. Click the Delete button at the top of the page to delete the app version.

For complete information about cleaning up billable resources, see the Cleaning up section in the final step of this tutorial.

Application structure

Binary data sample structure

The application uses Cloud Storage to store binary data (pictures in this case), while continuing to use a structured database for the book information (either Cloud Datastore, Cloud SQL, or MongoDB).

Understanding the code

This section walks you through the application code and explains how it works.

Handling user uploads

To allow users to upload images, the add/edit form has been modified to allow file uploads by setting the enctype to multipart/form-data and a new field has been added for the image:

block content
  h3 #{action} book
  form(method="POST", enctype="multipart/form-data")
    .form-group
      label(for="title") Title
      input.form-control(type="text", name="title", id="title", value=book.title)
    .form-group
      label(for="author") Author
      input.form-control(type="text", name="author", id="author", value=book.author)
    .form-group
      label(for="publishedDate") Date Published
      input.form-control(type="text", name="publishedDate", id="publishedDate", value=book.publishedDate)
    .form-group
      label(for="description") Description
      input.form-control(type="text", name="description", id="description", value=book.description)
    .form-group
      label(for="image") Cover Image
      input.form-control(type="file", name="image", id="image")
    .form-group.hidden
      label(for="imageUrl") Cover Image URL
      input.form-control(type="text", name="imageUrl", id="imageUrl", value=book.imageUrl)
    button.btn.btn-success(type="submit") Save

The application uses the Express multer middleware to handle parsing file upload requests. The application stores the uploaded files temporarily in memory because it will upload them directly to Cloud Storage:

const Multer = require('multer');
const multer = Multer({
  storage: Multer.MemoryStorage,
  limits: {
    fileSize: 5 * 1024 * 1024 // no larger than 5mb
  }
});

Uploading to Cloud Storage

Next, the application uses the sendUploadToGCS middleware to handle uploading the in-memory files to Cloud Storage:

function sendUploadToGCS (req, res, next) {
  if (!req.file) {
    return next();
  }

  const gcsname = Date.now() + req.file.originalname;
  const file = bucket.file(gcsname);

  const stream = file.createWriteStream({
    metadata: {
      contentType: req.file.mimetype
    }
  });

  stream.on('error', (err) => {
    req.file.cloudStorageError = err;
    next(err);
  });

  stream.on('finish', () => {
    req.file.cloudStorageObject = gcsname;
    req.file.cloudStoragePublicUrl = getPublicUrl(gcsname);
    next();
  });

  stream.end(req.file.buffer);
}

The middleware checks each file in the request and uploads them to Cloud Storage via a standard writable stream. Once uploaded, the files are made public and the middleware sets the cloudStoragePublicUrl property on the file. The public URL can be used to serve the image directly from Cloud Storage:

function getPublicUrl (filename) {
  return `https://storage.googleapis.com/${CLOUD_BUCKET}/${filename}`;
}

Then, the application uses this property to save the public URL of the image to the database:

router.post(
  '/add',
  images.multer.single('image'),
  images.sendUploadToGCS,
  (req, res, next) => {
    let data = req.body;

    // Was an image uploaded? If so, we'll use its public URL
    // in cloud storage.
    if (req.file && req.file.cloudStoragePublicUrl) {
      data.imageUrl = req.file.cloudStoragePublicUrl;
    }

    // Save the data to the database.
    getModel().create(data, (err, savedData) => {
      if (err) {
        next(err);
        return;
      }
      res.redirect(`${req.baseUrl}/${savedData.id}`);
    });
  }
);

Serving images from Cloud Storage

Because the app has the public URL for the image, serving it is straightforward. Serving directly from Cloud Storage is helpful because the requests leverage Google's global serving infrastructure and the application doesn't have to respond to requests for images, freeing up CPU cycles for other requests.

extends ../base.jade

block content
  h3 Book
    small

  .btn-group
    a(href="/books/#{book.id}/edit", class='btn btn-primary btn-sm')
      i.glyphicon.glyphicon-edit
      span  Edit book
    a(href="/books/#{book.id}/delete", class='btn btn-danger btn-sm')
      i.glyphicon.glyphicon-trash
      span  Delete book

  .media
    .media-left
      img(src=book.imageUrl || "http://placekitten.com/g/128/192")
    .media-body
      h4= book.title
        |  
        small= book.publishedDate
      h5 By #{book.author||'unknown'}
      p= book.description

Send feedback about...