Using Cloud Storage with Node.js

This part of the Node.js Bookshelf tutorial shows how the sample app stores images in Cloud Storage.

This page is part of a multipage tutorial. To start from the beginning and read the setup instructions, go to Node.js Bookshelf app.

Creating a Cloud Storage bucket

The following instructions show how to create a Cloud Storage bucket. Buckets are the basic containers that hold your data in Cloud Storage.

  1. In a terminal window, enter the following command:

    gsutil mb gs://[YOUR-BUCKET-NAME]

    [YOUR-BUCKET-NAME] represents the name of your Cloud Storage bucket.

  2. To view uploaded images in the bookshelf app, set the bucket's default access control list (ACL) to public-read.

    gsutil defacl set public-read gs://[YOUR-BUCKET-NAME]

Configuring settings

In the nodejs-getting-started/3-binary-data directory, create a config.json file with this content:

{
  "CLOUD_BUCKET": "[YOUR_BUCKET_NAME]"
}

Replace [YOUR_BUCKET_NAME] with the name of your Cloud Storage bucket.

Installing dependencies

Install dependencies in the nodejs-getting-started/3-binary-data directory by using npm:

npm install

Running the app on your local machine

  1. Start a local web server using npm:

    npm start
    
  2. In your web browser, enter the following address:

    http://localhost:8080

Now you can browse the app's web pages, add books with cover images, edit books, and delete books.

Deploying the app to the App Engine standard environment

  1. Deploy the sample app from the nodejs-getting-started/3-binary-data directory:

    gcloud app deploy
    
  2. In your web browser, enter this address:

    https://[YOUR_PROJECT_ID].appspot.com
    

    Replace [YOUR_PROJECT_ID] with your GCP project ID.

If you update your app, you deploy the updated version by entering the same command that you used to deploy the app. The new deployment creates a new version of your app and promotes it to the default version. The older versions of your app remain. By default the App Engine standard environment scales to 0 instances when there is no incoming traffic to a version. Thus, unused versions shouldn't cost anything. However, all of these app versions are still billable resources.

See the Cleaning up section in the final step of this tutorial for more information on cleaning up billable resources, including non-default app versions.

App structure

Binary data sample structure

The sample app uses Cloud Storage to store binary data (pictures in this case) while using a structured database for the book information (either Cloud Datastore or Cloud SQL).

Understanding the code

This section walks you through the app's code and explains how it works.

Handle user uploads

To let users upload images, you modify the add or edit form to allow file uploads. Set the enctype to multipart/form-data and add a field for the image:

block content
  h3 #{action} book
  form(method="POST", enctype="multipart/form-data")
    .form-group
      label(for="title") Title
      input.form-control(type="text", name="title", id="title", value=book.title)
    .form-group
      label(for="author") Author
      input.form-control(type="text", name="author", id="author", value=book.author)
    .form-group
      label(for="publishedDate") Date Published
      input.form-control(type="text", name="publishedDate", id="publishedDate", value=book.publishedDate)
    .form-group
      label(for="description") Description
      input.form-control(type="text", name="description", id="description", value=book.description)
    .form-group
      label(for="image") Cover Image
      input.form-control(type="file", name="image", id="image")
    .form-group.hidden
      label(for="imageUrl") Cover Image URL
      input.form-control(type="text", name="imageUrl", id="imageUrl", value=book.imageUrl)
    button.btn.btn-success(type="submit") Save

The app uses Multer middleware to handle parsing file upload requests. The app stores the uploaded files temporarily in memory because it uploads them directly to Cloud Storage:

const Multer = require('multer');
const multer = Multer({
  storage: Multer.MemoryStorage,
  limits: {
    fileSize: 5 * 1024 * 1024 // no larger than 5mb
  }
});

Upload to Cloud Storage

Next, the app uses the sendUploadToGCS middleware to handle uploading the in-memory files to Cloud Storage:

function sendUploadToGCS (req, res, next) {
  if (!req.file) {
    return next();
  }

  const gcsname = Date.now() + req.file.originalname;
  const file = bucket.file(gcsname);

  const stream = file.createWriteStream({
    metadata: {
      contentType: req.file.mimetype
    },
    resumable: false
  });

  stream.on('error', (err) => {
    req.file.cloudStorageError = err;
    next(err);
  });

  stream.on('finish', () => {
    req.file.cloudStorageObject = gcsname;
    file.makePublic().then(() => {
      req.file.cloudStoragePublicUrl = getPublicUrl(gcsname);
      next();
    });
  });

  stream.end(req.file.buffer);
}

The middleware checks each file in the request and uploads it to Cloud Storage by using a standard writable stream. Once uploaded, the files are made public and the middleware sets the cloudStoragePublicUrl property on the file. The public URL serves the image directly from Cloud Storage:

function getPublicUrl (filename) {
  return `https://storage.googleapis.com/${CLOUD_BUCKET}/${filename}`;
}

The app then uses the following property to save the public URL of the image to the database:

router.post(
  '/add',
  images.multer.single('image'),
  images.sendUploadToGCS,
  (req, res, next) => {
    let data = req.body;

    // Was an image uploaded? If so, we'll use its public URL
    // in cloud storage.
    if (req.file && req.file.cloudStoragePublicUrl) {
      data.imageUrl = req.file.cloudStoragePublicUrl;
    }

    // Save the data to the database.
    getModel().create(data, (err, savedData) => {
      if (err) {
        next(err);
        return;
      }
      res.redirect(`${req.baseUrl}/${savedData.id}`);
    });
  }
);

Serve images from Cloud Storage

Because the app has the public URL for the image, serving it is straightforward. Serving directly from Cloud Storage is helpful because the requests use Google's global serving infrastructure. The app doesn't have to respond to requests for images, which frees CPU cycles for other requests.

block content
  h3 Book
    small

  .btn-group
    a(href=`/books/${book.id}/edit`, class='btn btn-primary btn-sm')
      i.glyphicon.glyphicon-edit
      span  Edit book
    a(href=`/books/${book.id}/delete`, class='btn btn-danger btn-sm')
      i.glyphicon.glyphicon-trash
      span  Delete book

  .media
    .media-left
      img(src=book.imageUrl || "http://placekitten.com/g/128/192")
    .media-body
      h4= book.title
        |  
        small= book.publishedDate
      h5 By #{book.author||'unknown'}
      p= book.description
Var denne siden nyttig? Si fra hva du synes:

Send tilbakemelding om ...