Using Cloud Storage

Python |Java |Node.js |Go |Ruby |PHP |.NET

You can use Cloud Storage to store and serve files, such as movies or images or other static content.

This document describes how to use the Google Cloud Client Library in your app to store data in and retrieve data from Cloud Storage.

Before you begin

• Follow the instructions in "Hello, World!" for Node.js on App Engine to set up your environment and project, and to understand how Node.js apps are structured in App Engine. Write down and save your project ID, because you will need it to run the sample application described in this document.

• Make sure you create a Cloud Storage bucket for your application by invoking the following command:

  gsutil mb gs://[YOUR_BUCKET_NAME]
  

• Make the bucket publicly readable so it can serve files:

  gsutil defacl set public-read gs://[YOUR_BUCKET_NAME]
  

Download the sample

To clone the repository:

git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples/
cd nodejs-docs-samples/appengine/storage/flexible

Edit project configuration and install dependencies

In app.yaml, add your project ID to the GOOGLE_CLOUD_PROJECT environment value. Then set the GCLOUD_STORAGE_BUCKET environment value to the name of the Cloud Storage bucket you created previously.

appengine/storage/flexible/app.yaml

View on GitHub

runtime: nodejs
env: flex

env_variables:
  GCLOUD_STORAGE_BUCKET: YOUR_BUCKET_NAME

In package.json, add @google-cloud/storage as a dependency, which provides the functions to use Cloud Storage.

appengine/storage/flexible/package.json

View on GitHub

{
  "name": "appengine-storage",
  "description": "Node.js Google Cloud Storage sample for Google App Engine",
  "scripts": {
    "start": "node app.js",
    "test": "mocha system-test/*.test.js --exit --timeout=30000"
  },
  "engines": {
    "node": ">=8.0.0"
  },
  "dependencies": {
    "@google-cloud/storage": "^3.0.0",
    "body-parser": "^1.18.3",
    "express": "^4.16.4",
    "multer": "^1.4.1",
    "pug": "^2.0.3"
  },
  "devDependencies": {
    "@google-cloud/nodejs-repo-tools": "^3.3.0",
    "mocha": "^6.0.0"
  },
  "cloud-repo-tools": {
    "requiresProjectId": true,
    "requiresKeyFile": true,
    "test": {
      "app": {
        "requiredEnvVars": [
          "GCLOUD_STORAGE_BUCKET"
        ],
        "msg": "<input type=\"file\" name=\"file\">",
        "substitutions": "YOUR_BUCKET_NAME=$GCLOUD_STORAGE_BUCKET"
      },
      "build": {
        "requiredEnvVars": [
          "GCLOUD_STORAGE_BUCKET"
        ]
      }
    }
  }
}

Application code

The sample application presents a web page prompting the user to supply a file to be stored in Cloud Storage. When the user selects a file and clicks submit, the upload handler loads the file content into a blob and writes it to Cloud Storage.

Notice that after the file is uploaded to Cloud Storage, the public URL to this file is returned, which you can use to serve the file directly from Cloud Storage. You should store this value in your app for future use.

appengine/storage/flexible/app.js

View on GitHub

const {format} = require('util');
const express = require('express');
const Multer = require('multer');
const bodyParser = require('body-parser');

// By default, the client will authenticate using the service account file
// specified by the GOOGLE_APPLICATION_CREDENTIALS environment variable and use
// the project specified by the GOOGLE_CLOUD_PROJECT environment variable. See
// https://github.com/GoogleCloudPlatform/google-cloud-node/blob/master/docs/authentication.md
// These environment variables are set automatically on Google App Engine
const {Storage} = require('@google-cloud/storage');

// Instantiate a storage client
const storage = new Storage();

const app = express();
app.set('view engine', 'pug');
app.use(bodyParser.json());

// Multer is required to process file uploads and make them available via
// req.files.
const multer = Multer({
  storage: Multer.memoryStorage(),
  limits: {
    fileSize: 5 * 1024 * 1024, // no larger than 5mb, you can change as needed.
  },
});

// A bucket is a container for objects (files).
const bucket = storage.bucket(process.env.GCLOUD_STORAGE_BUCKET);

// Display a form for uploading files.
app.get('/', (req, res) => {
  res.render('form.pug');
});

// Process the file upload and upload to Google Cloud Storage.
app.post('/upload', multer.single('file'), (req, res, next) => {
  if (!req.file) {
    res.status(400).send('No file uploaded.');
    return;
  }

  // Create a new blob in the bucket and upload the file data.
  const blob = bucket.file(req.file.originalname);
  const blobStream = blob.createWriteStream();

  blobStream.on('error', err => {
    next(err);
  });

  blobStream.on('finish', () => {
    // The public URL can be used to directly access the file via HTTP.
    const publicUrl = format(
      `https://storage.googleapis.com/${bucket.name}/${blob.name}`
    );
    res.status(200).send(publicUrl);
  });

  blobStream.end(req.file.buffer);
});

const PORT = process.env.PORT || 8080;
app.listen(PORT, () => {
  console.log(`App listening on port ${PORT}`);
  console.log('Press Ctrl+C to quit.');
});

For more information

For complete information on Cloud Storage, see the Cloud Storage documentation.