Using Cloud Storage

Python |Java |Node.js |Go |Ruby |PHP |.NET

You can use Cloud Storage to store and serve files, such as movies or images or other static content.

This document describes how to use the Google Cloud Client Library in a flexible environment app to store data in and retrieve data from Cloud Storage.

Before you begin

• Follow the instructions in "Hello, World!" for Node.js on App Engine to set up your environment and project, and to understand how Node.js apps are structured in App Engine. Write down and save your project ID, because you will need it to run the sample application described in this document.

• Make sure you create a Cloud Storage bucket for your application by invoking the following command:

  gsutil mb gs://[YOUR_BUCKET_NAME]
  

• Make the bucket publically readable so it can serve files:

  gsutil defacl set public-read gs://[YOUR_BUCKET_NAME]
  

Download the sample

To clone the repository:

git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples/
cd nodejs-docs-samples/appengine/storage

Edit project configuration and install dependencies

In app.yaml, a GCLOUD_PROJECT for your project. This value is your project ID. Then set GCLOUD_STORAGE_BUCKET; this value is the name of the Cloud Storage bucket you created previously.

appengine/storage/app.yaml

View on GitHub

# Copyright 2015-2016, Google, Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# [START app_yaml]
runtime: nodejs
env: flex

# [START env]
env_variables:
  GCLOUD_STORAGE_BUCKET: YOUR_BUCKET_NAME
# [END env]
# [END app_yaml]

In package.json, set gcloud as a dependency, which provides the functions to use Cloud Storage.

appengine/storage/package.json

View on GitHub

{
  "name": "appengine-storage",
  "description": "Node.js Google Cloud Storage sample for Google App Engine",
  "scripts": {
    "start": "node app.js",
    "test": "ava system-test/*.test.js -T 30s"
  },
  "engines": {
    "node": ">=4.3.2"
  },
  "dependencies": {
    "@google-cloud/storage": "1.2.1",
    "body-parser": "1.17.2",
    "express": "4.15.4",
    "multer": "1.3.0",
    "pug": "2.0.0-rc.3"
  },
  "devDependencies": {
    "@google-cloud/nodejs-repo-tools": "1.4.17",
    "ava": "0.22.0"
  },
  "cloud-repo-tools": {
    "requiresProjectId": true,
    "requiresKeyFile": true,
    "test": {
      "app": {
        "requiredEnvVars": [
          "GCLOUD_STORAGE_BUCKET"
        ],
        "msg": "<input type=\"file\" name=\"file\">",
        "substitutions": "YOUR_BUCKET_NAME=$GCLOUD_STORAGE_BUCKET"
      },
      "build": {
        "requiredEnvVars": [
          "GCLOUD_STORAGE_BUCKET"
        ]
      }
    }
  }
}

Application code

The sample application presents a web page prompting the user to supply a file to be stored in Cloud Storage. When the user selects a file and clicks submit, the upload handler loads the file content into a blob and writes it to Cloud Storage.

Notice that after the file is uploaded to Cloud Storage, the public URL to this file is returned, which you can use to serve the file directly from Cloud Storage. You should store this value in your app for future use.

appengine/storage/app.js

View on GitHub

'use strict';

const format = require('util').format;
const express = require('express');
const Multer = require('multer');
const bodyParser = require('body-parser');
const process = require('process'); // Required to mock environment variables

// By default, the client will authenticate using the service account file
// specified by the GOOGLE_APPLICATION_CREDENTIALS environment variable and use
// the project specified by the GCLOUD_PROJECT environment variable. See
// https://googlecloudplatform.github.io/gcloud-node/#/docs/google-cloud/latest/guides/authentication
// These environment variables are set automatically on Google App Engine
const Storage = require('@google-cloud/storage');

// Instantiate a storage client
const storage = Storage();

const app = express();
app.set('view engine', 'pug');
app.use(bodyParser.json());

// Multer is required to process file uploads and make them available via
// req.files.
const multer = Multer({
  storage: Multer.memoryStorage(),
  limits: {
    fileSize: 5 * 1024 * 1024 // no larger than 5mb, you can change as needed.
  }
});

// A bucket is a container for objects (files).
const bucket = storage.bucket(process.env.GCLOUD_STORAGE_BUCKET);

// Display a form for uploading files.
app.get('/', (req, res) => {
  res.render('form.pug');
});

// Process the file upload and upload to Google Cloud Storage.
app.post('/upload', multer.single('file'), (req, res, next) => {
  if (!req.file) {
    res.status(400).send('No file uploaded.');
    return;
  }

  // Create a new blob in the bucket and upload the file data.
  const blob = bucket.file(req.file.originalname);
  const blobStream = blob.createWriteStream();

  blobStream.on('error', (err) => {
    next(err);
  });

  blobStream.on('finish', () => {
    // The public URL can be used to directly access the file via HTTP.
    const publicUrl = format(`https://storage.googleapis.com/${bucket.name}/${blob.name}`);
    res.status(200).send(publicUrl);
  });

  blobStream.end(req.file.buffer);
});

const PORT = process.env.PORT || 8080;
app.listen(PORT, () => {
  console.log(`App listening on port ${PORT}`);
  console.log('Press Ctrl+C to quit.');
});

For more information

For complete information on Cloud Storage, see the Cloud Storage documentation.