Testing and CI/CD

This tutorial covers best practices for testing and deploying Cloud Functions. The Types of tests section discusses the types of tests you should use. Example testing scenarios provides some example testing scenarios and instructions on how to run them. The Continuous testing and deployment section contains instructions on how to automatically run your tests (and optionally redeploy your functions) using a Continuous Integration and Deployment (CI/CD) platform such as Cloud Container Builder.

Prerequisites

Before starting this tutorial, make sure you do the following:

In this tutorial, we'll use AVA to run our tests and Sinon to mock external dependencies. An external dependency is any dependency your functions rely on that isn't part of your functions' code itself. Common examples of such dependencies are other Google Cloud Platform services and language-specific packages like those from NPM.

Types of tests

There are three types of tests you can use when working with Google Cloud Functions, each of which tests a different aspect of your code. They are listed below, from least to most thorough.

  • Unit tests
  • Integration tests
  • System tests

In general, tests take more time to complete as they become more thorough. This document discusses these test types in detail, as well as how to balance between speed and thoroughness.

Unit tests

Unit tests are small tests that provide highly specific information on which parts of your code work. These tests are good for quickly verifying assumptions made during the development process.

By design, unit tests do not depend on (or test integration with) any external dependencies, such as Cloud Functions itself or other Google Cloud Platform (GCP) components. You can use frameworks such as Sinon to mock any such dependencies, including Google Cloud Functions itself.

Integration tests

Integration tests provide information on how well the various parts of your code work together, and typically take a moderate amount of time to complete.

The Cloud Functions Emulator

In this tutorial, we'll use the Cloud Functions Emulator to create and run these tests locally. Unlike Cloud Functions itself, the Emulator does not require you to redeploy your code whenever you make changes. The Cloud Functions Emulator also lets you mock Pub/Sub messages and Storage notifications locally.

To configure and start the Emulator, run the following commands. Replace [YOUR_GCP_PROJECT_ID] and [YOUR_GCF_REGION] with your GCP project ID and Cloud Functions region respectively.

functions config set project [YOUR_GCP_PROJECT_ID]
functions config set region [YOUR_GCF_REGION]
functions start

See the Emulator documentation to learn more about the Cloud Functions Emulator.

System tests

System tests are larger, more complex tests that thoroughly test your code’s interaction with GCP itself and other external dependencies. These tests are good for determining whether or not your code “actually works” at a high level. Their primary downside is that they can take a long time to complete, especially when large amounts of network and/or disk activity are required.

If you're interacting with other GCP components, system tests help you test your function’s interaction with those components before deploying it to production. We recommend deploying your Cloud Function to a test environment and testing its functionality using HTTP requests, Pub/Sub messages, or Cloud Storage changes. You can verify your function's correctness by either reading the logs or checking for desired behavior.

Example testing scenarios

The three scenarios below cover the different ways of triggering Cloud Functions. The structure of a function's tests typically depends heavily on which Cloud Platform resources a function uses. In turn, a function's resource use often depends on the how that function is triggered.

HTTP-triggered functions

Unit tests for HTTP-triggered functions generally involve mocking the wrapping HTTP framework and supplying a mock request to the function itself. All other types of tests usually send a request to a URL pointing to the function.

In both cases, the tests then compare the function's response against an expected value.

System tests and integration tests for HTTP-triggered functions can sometimes be combined into one file by changing the URL used by their tests dynamically. This technique is shown in the tests below.

Here is an example HTTP-triggered function:

/**
 * HTTP Cloud Function.
 *
 * @param {Object} req Cloud Function request context.
 * @param {Object} res Cloud Function response context.
 */
exports.helloHttp = (req, res) => {
  res.send(`Hello ${req.body.name || 'World'}!`);
};

Unit tests

These tests act as unit tests for the above function. ExpressJS' req and res parameters are mocked using Sinon, and tested using AVA.

const test = require(`ava`);
const sinon = require(`sinon`);
const uuid = require(`uuid`);

const helloHttp = require(`..`).helloHttp;

test(`helloHttp: should print a name`, t => {
  // Initialize mocks
  const name = uuid.v4();
  const req = {
    body: {
      name: name
    }
  };
  const res = { send: sinon.stub() };

  // Call tested function
  helloHttp(req, res);

  // Verify behavior of tested function
  t.true(res.send.calledOnce);
  t.deepEqual(res.send.firstCall.args, [`Hello ${name}!`]);
});

test(`helloHttp: should print hello world`, t => {
  // Initialize mocks
  const req = {
    body: {}
  };
  const res = { send: sinon.stub() };

  // Call tested function
  helloHttp(req, res);

  // Verify behavior of tested function
  t.true(res.send.calledOnce);
  t.deepEqual(res.send.firstCall.args, [`Hello World!`]);
});

Use the following command to run the unit tests:

ava test/sample.unit.http.test.js

Integration tests

These tests act as integration tests for the above function:

const test = require(`ava`);
const Supertest = require(`supertest`);
const supertest = Supertest(process.env.BASE_URL);

test.cb(`helloHttp: should print a name`, (t) => {
  supertest
    .post(`/helloHttp`)
    .send({ name: 'John' })
    .expect(200)
    .expect((response) => {
      t.is(response.text, 'Hello John!');
    })
    .end(t.end);
});

test.cb(`helloHttp: should print hello world`, (t) => {
  supertest
    .get(`/helloHttp`)
    .expect(200)
    .expect((response) => {
      t.is(response.text, `Hello World!`);
    })
    .end(t.end);
});

To run integration tests for HTTP functions, start the Cloud Functions Emulator and use the following commands. Replace [YOUR_GCP_PROJECT_ID] and [YOUR_GCF_REGION] with your GCP project ID and Cloud Functions region respectively.

export BASE_URL=http://localhost:8010/[YOUR_GCP_PROJECT_ID]/[YOUR_GCF_REGION]
functions deploy helloHttp --trigger-http
ava test/sample.integration.http.test.js

System tests

These tests act as system tests for the above function. Note that these are identical to the function's integration tests:

const test = require(`ava`);
const Supertest = require(`supertest`);
const supertest = Supertest(process.env.BASE_URL);

test.cb(`helloHttp: should print a name`, (t) => {
  supertest
    .post(`/helloHttp`)
    .send({ name: 'John' })
    .expect(200)
    .expect((response) => {
      t.is(response.text, 'Hello John!');
    })
    .end(t.end);
});

test.cb(`helloHttp: should print hello world`, (t) => {
  supertest
    .get(`/helloHttp`)
    .expect(200)
    .expect((response) => {
      t.is(response.text, `Hello World!`);
    })
    .end(t.end);
});

To run integration tests for HTTP functions, first deploy your functions using the following command:

gcloud beta functions deploy helloHttp --trigger-http

Use the following commands to test your deployed HTTP function. Note that the primary difference between system tests and integration tests for HTTP Cloud Functions is the URL the functions are accessible from.

export BASE_URL=https://[YOUR_GCF_REGION]-[YOUR_GCP_PROJECT_ID].cloudfunctions.net/helloHttp
ava test/sample.system.http.test.js

Pub/Sub-triggered functions

Pub/Sub-triggered function tests are structured differently depending on where the tested function resides.

Unit tests should mock the Pub/Sub message itself, and provide a callback to be run once the tested function finishes. You can confirm the function's behavior by combining testing and dependency mocking frameworks (such as AVA and Sinon respectively) and comparing your function's results to expected values.

Integration tests can use the Cloud Functions Emulator to run functions locally. Because the Emulator does not support listening to actual Pub/Sub topics, integration tests often call the Emulator's CLI directly using Node's child_process module. To validate the function's behavior, confirm that the desired effects in other GCP products occur or check the Emulator's logs. You can access these logs via the Cloud Functions Emulator's command-line interface (CLI) using the functions logs read command.

System tests should publish to actual Pub/Sub topics. You can validate their behavior by confirming that desired side effects occur, or by reading their logs via the gcloud beta functions logs read command.

Here is an example Pub/Sub-triggered function:

/**
 * Background Cloud Function to be triggered by Pub/Sub.
 *
 * @param {object} event The Cloud Functions event.
 * @param {function} callback The callback function.
 */
exports.helloPubSub = (event, callback) => {
  const pubsubMessage = event.data;
  const name = pubsubMessage.data ? Buffer.from(pubsubMessage.data, 'base64').toString() : 'World';

  console.log(`Hello, ${name}!`);

  callback();
};

Unit tests

These tests act as unit tests for the above function:

const test = require(`ava`);
const uuid = require(`uuid`);
const sinon = require(`sinon`);

const helloPubSub = require(`..`).helloPubSub;
const consoleLog = sinon.stub(console, 'log');

test.cb(`helloPubSub: should print a name`, t => {
  t.plan(1);

  // Initialize mocks
  const name = uuid.v4();
  const event = {
    data: {
      data: Buffer.from(name).toString(`base64`)
    }
  };

  // Call tested function and verify its behavior
  helloPubSub(event, () => {
    t.true(consoleLog.calledWith(`Hello, ${name}!`));
    t.end();
  });
});

test.cb(`helloPubSub: should print hello world`, t => {
  t.plan(1);

  // Initialize mocks
  const event = {
    data: {}
  };

  // Call tested function and verify its behavior
  helloPubSub(event, () => {
    t.true(consoleLog.calledWith(`Hello, World!`));
    t.end();
  });
});

Use the following command to run the unit tests:

ava test/sample.unit.pubsub.test.js

Integration tests

These tests act as integration tests for this function:

const childProcess = require(`child_process`);
const test = require(`ava`);
const uuid = require(`uuid`);

test(`helloPubSub: should print a name`, async (t) => {
  t.plan(1);
  const startTime = new Date(Date.now()).toISOString();
  const name = uuid.v4();

  // Mock Pub/Sub call, as the emulator doesn't listen to Pub/Sub topics
  const encodedName = Buffer.from(name).toString(`base64`);
  const data = JSON.stringify({ data: encodedName });
  childProcess.execSync(`functions call helloPubSub --data '${data}'`);

  // Check the emulator's logs
  const logs = childProcess.execSync(`functions logs read helloPubSub --start-time ${startTime}`).toString();
  t.true(logs.includes(`Hello, ${name}!`));
});

test(`helloPubSub: should print hello world`, async (t) => {
  t.plan(1);
  const startTime = new Date(Date.now()).toISOString();

  // Mock Pub/Sub call, as the emulator doesn't listen to Pub/Sub topics
  childProcess.execSync(`functions call helloPubSub --data {}`);

  // Check the emulator's logs
  const logs = childProcess.execSync(`functions logs read helloPubSub --start-time ${startTime}`).toString();
  t.true(logs.includes(`Hello, World!`));
});

To run the integration tests for this function, start the Cloud Functions Emulator and use the following commands. Replace [YOUR_PUBSUB_TOPIC] with the name of the Pub/Sub topic you want to subscribe to.

functions deploy helloPubSub --trigger-topic [YOUR_PUBSUB_TOPIC]
ava test/sample.integration.pubsub.test.js

System tests

These tests act as system tests for this function:

const childProcess = require(`child_process`);
const test = require(`ava`);
const uuid = require(`uuid`);
const Pubsub = require(`@google-cloud/pubsub`);
const pubsub = Pubsub();

const topicName = process.env.FUNCTIONS_TOPIC;
const baseCmd = `gcloud beta functions`;

test(`helloPubSub: should print a name`, async (t) => {
  t.plan(1);
  const startTime = new Date(Date.now()).toISOString();
  const name = uuid.v4();

  // Publish to pub/sub topic
  const topic = pubsub.topic(topicName);
  const publisher = topic.publisher();
  await publisher.publish(Buffer.from(name));

  // Wait for logs to become consistent
  await new Promise(resolve => setTimeout(resolve, 15000));

  // Check logs after a delay
  const logs = childProcess.execSync(`${baseCmd} logs read helloPubSub --start-time ${startTime}`).toString();
  t.true(logs.includes(`Hello, ${name}!`));
});

test(`helloPubSub: should print hello world`, async (t) => {
  t.plan(1);
  const startTime = new Date(Date.now()).toISOString();

  // Publish to pub/sub topic
  const topic = pubsub.topic(topicName);
  const publisher = topic.publisher();
  await publisher.publish(Buffer.from(''), { a: 'b' });

  // Wait for logs to become consistent
  await new Promise(resolve => setTimeout(resolve, 15000));

  // Check logs after a delay
  const logs = childProcess.execSync(`${baseCmd} logs read helloPubSub --start-time ${startTime}`).toString();
  t.true(logs.includes('Hello, World!'));
});

To run the system tests, first select one of your GCP project's Pub/Sub topics to subscribe to. If you provide the name of a Pub/Sub topic that does not currently exist, it will be created automatically.

Then, deploy your functions using the following command. Replace [YOUR_PUBSUB_TOPIC] with the name of the Pub/Sub topic you want to subscribe to.

gcloud beta functions deploy helloPubSub --trigger-topic [YOUR_PUBSUB_TOPIC]

Use the following commands to run the system tests, replacing [YOUR_PUBSUB_TOPIC] appropriately.

export FUNCTIONS_TOPIC=[YOUR_PUBSUB_TOPIC]
ava test/sample.system.pubsub.test.js

Storage-triggered functions

Tests for Storage-triggered functions are fairly similar in structure to their Pub/Sub-triggered counterparts. Like Pub/Sub-triggered function tests, Storage-triggered function tests are structured differently depending on where the tested function is hosted.

Unit tests should mock the Storage event itself, and provide a callback to be run once the tested function finishes. You can verify the function's behavior using dependency mocking frameworks such as Sinon and comparing the function's results to expected values.

Integration tests can use the Cloud Functions Emulator to run functions locally. Because the Emulator does not support listening to actual Cloud Storage buckets, integration tests often call the Emulator's CLI directly using Node's child_process module. You can determine the function's behavior by watching for desired side effects in other GCP products, or by checking the Emulator's logs. One way to access these logs is via the Cloud Functions Emulator's command-line interface (CLI) using the functions logs read command.

System tests should manipulate actual Cloud Storage objects. You can determine function behavior by checking for desired side effects, or by reading function logs via the gcloud beta functions logs read command.

Here is an example Storage-triggered function:

/**
 * Background Cloud Function to be triggered by Cloud Storage.
 *
 * @param {object} event The Cloud Functions event.
 * @param {function} callback The callback function.
 */
exports.helloGCS = (event, callback) => {
  const file = event.data;

  if (file.resourceState === 'not_exists') {
    console.log(`File ${file.name} deleted.`);
  } else if (file.metageneration === '1') {
    // metageneration attribute is updated on metadata changes.
    // on create value is 1
    console.log(`File ${file.name} uploaded.`);
  } else {
    console.log(`File ${file.name} metadata updated.`);
  }

  callback();
};

Unit tests

These tests act as unit tests for the above function:

const test = require(`ava`);
const uuid = require(`uuid`);
const sinon = require(`sinon`);

const helloGCS = require(`..`).helloGCS;
const consoleLog = sinon.stub(console, 'log');

test.cb(`helloGCS: should print uploaded message`, t => {
  t.plan(1);

  // Initialize mocks
  const filename = uuid.v4();
  const event = {
    data: {
      name: filename,
      resourceState: 'exists',
      metageneration: '1'
    }
  };

  // Call tested function and verify its behavior
  helloGCS(event, () => {
    t.true(consoleLog.calledWith(`File ${filename} uploaded.`));
    t.end();
  });
});

test.cb(`helloGCS: should print metadata updated message`, t => {
  t.plan(1);

  // Initialize mocks
  const filename = uuid.v4();
  const event = {
    data: {
      name: filename,
      resourceState: 'exists',
      metageneration: '2'
    }
  };

  // Call tested function and verify its behavior
  helloGCS(event, () => {
    t.true(consoleLog.calledWith(`File ${filename} metadata updated.`));
    t.end();
  });
});

test.cb(`helloGCS: should print deleted message`, t => {
  t.plan(1);

  // Initialize mocks
  const filename = uuid.v4();
  const event = {
    data: {
      name: filename,
      resourceState: 'not_exists',
      metageneration: '3'
    }
  };

  // Call tested function and verify its behavior
  helloGCS(event, () => {
    t.true(consoleLog.calledWith(`File ${filename} deleted.`));
    t.end();
  });
});

Use the following command to run the unit tests:

ava test/sample.unit.storage.test.js

Integration tests

These tests act as integration tests for the above function:

const childProcess = require(`child_process`);
const test = require(`ava`);
const uuid = require(`uuid`);

test(`helloGCS: should print uploaded message`, async (t) => {
  t.plan(1);
  const startTime = new Date(Date.now()).toISOString();
  const filename = uuid.v4(); // Use a unique filename to avoid conflicts

  // Mock GCS call, as the emulator doesn't listen to GCS buckets
  const data = JSON.stringify({
    name: filename,
    resourceState: 'exists',
    metageneration: '1'
  });

  childProcess.execSync(`functions call helloGCS --data '${data}'`);

  // Check the emulator's logs
  const logs = childProcess.execSync(`functions logs read helloGCS --start-time ${startTime}`).toString();
  t.true(logs.includes(`File ${filename} uploaded.`));
});

test(`helloGCS: should print metadata updated message`, async (t) => {
  t.plan(1);
  const startTime = new Date(Date.now()).toISOString();
  const filename = uuid.v4(); // Use a unique filename to avoid conflicts

  // Mock GCS call, as the emulator doesn't listen to GCS buckets
  const data = JSON.stringify({
    name: filename,
    resourceState: 'exists',
    metageneration: '2'
  });

  childProcess.execSync(`functions call helloGCS --data '${data}'`);

  // Check the emulator's logs
  const logs = childProcess.execSync(`functions logs read helloGCS --start-time ${startTime}`).toString();
  t.true(logs.includes(`File ${filename} metadata updated.`));
});

test(`helloGCS: should print deleted message`, async (t) => {
  t.plan(1);
  const startTime = new Date(Date.now()).toISOString();
  const filename = uuid.v4(); // Use a unique filename to avoid conflicts

  // Mock GCS call, as the emulator doesn't listen to GCS buckets
  const data = JSON.stringify({
    name: filename,
    resourceState: 'not_exists',
    metageneration: '3'
  });

  childProcess.execSync(`functions call helloGCS --data '${data}'`);

  // Check the emulator's logs
  const logs = childProcess.execSync(`functions logs read helloGCS --start-time ${startTime}`).toString();
  t.true(logs.includes(`File ${filename} deleted.`));
});

Use the following commands to run the integration tests. Replace [YOUR_GCS_BUCKET_NAME] with the name of a Cloud Storage bucket you want to monitor.

functions deploy helloGCS --trigger-bucket [YOUR_GCS_BUCKET_NAME]
ava test/sample.integration.storage.test.js

System tests

These tests act as system tests for the above function:

const Storage = require(`@google-cloud/storage`);
const storage = Storage();
const uuid = require(`uuid`);
const test = require(`ava`);
const path = require(`path`);
const childProcess = require(`child_process`);
const localFileName = `test.txt`;

// Use unique GCS filename to avoid conflicts between concurrent test runs
const gcsFileName = `test-${uuid.v4()}.txt`;

const bucketName = process.env.BUCKET_NAME;
const bucket = storage.bucket(bucketName);
const baseCmd = `gcloud beta functions`;

test.serial(`helloGCS: should print uploaded message`, async (t) => {
  t.plan(1);
  const startTime = new Date(Date.now()).toISOString();

  // Upload file
  const filepath = path.join(__dirname, localFileName);
  await bucket.upload(filepath, {
    destination: gcsFileName
  });

  // Wait for consistency
  await new Promise(resolve => setTimeout(resolve, 15000));

  // Check logs
  const logs = childProcess.execSync(`${baseCmd} logs read helloGCS --start-time ${startTime}`).toString();
  t.true(logs.includes(`File ${gcsFileName} uploaded`));
});

test.serial(`helloGCS: should print metadata updated message`, async (t) => {
  t.plan(1);
  const startTime = new Date(Date.now()).toISOString();

  // Update file metadata
  const file = bucket.file(gcsFileName);
  await file.setMetadata(gcsFileName, { foo: `bar` });

  // Wait for consistency
  await new Promise(resolve => setTimeout(resolve, 15000));

  // Check logs
  const logs = childProcess.execSync(`${baseCmd} logs read helloGCS --start-time ${startTime}`).toString();
  t.true(logs.includes(`File ${gcsFileName} metadata updated`));
});

test.serial(`helloGCS: should print deleted message`, async (t) => {
  t.plan(1);
  const startTime = new Date(Date.now()).toISOString();

  // Delete file
  bucket.deleteFiles();

  // Wait for consistency
  await new Promise(resolve => setTimeout(resolve, 15000));

  // Check logs
  const logs = childProcess.execSync(`${baseCmd} logs read helloGCS --start-time ${startTime}`).toString();
  t.true(logs.includes(`File ${gcsFileName} deleted`));
});

Use the following commands to run the system tests. Replace [YOUR_GCS_BUCKET_NAME] with the name of a Cloud Storage bucket you want to monitor. Note that this must reference a bucket that exists within the same GCP project that the function will be deployed to.

export BUCKET_NAME=[YOUR_GCS_BUCKET_NAME]
gcloud beta functions deploy helloGCS --trigger-bucket [YOUR_GCS_BUCKET_NAME]
ava test/sample.system.storage.test.js

Continuous testing and deployment

As you're developing your function, you can run the various test types mentioned above to ensure that your functions work both locally and in a test environment on Google Cloud Platform.

Once you've finished developing locally, you can configure a continuous integration and deployment (CI/CD) platform such as Cloud Container Builder to run your existing Cloud Functions tests on an ongoing basis. Continuous testing helps ensure that your code continues to work as intended and that your dependencies remain up-to-date. As Cloud Functions are not updated automatically, you can also configure continuous integration platforms (including Cloud Container Builder) to automatically test and redeploy your functions from a source repository such as GitHub, Bitbucket, or Cloud Source Repositories.

Follow the build config file instructions in this tutorial using the cloudbuild.yaml file below to configure Cloud Container Builder to automatically test and deploy your function. Replace [YOUR_FUNCTION_NAME] with the name of your Cloud Function and [YOUR_FUNCTION_TRIGGER] with the appropriate trigger value.

steps:
- name: 'gcr.io/cloud-builders/yarn'
  args: ['install']
  dir: 'functions/autodeploy'
- name: 'gcr.io/cloud-builders/npm'
  args: ['test']
  dir: 'functions/autodeploy'
- name: 'gcr.io/cloud-builders/gcloud'
  args: ['beta', 'functions', 'deploy', '[YOUR_FUNCTION_NAME]', '[YOUR_FUNCTION_TRIGGER]']
  dir: 'functions/autodeploy'

Remember to grant any permissions to the [YOUR_PROJECT_ID_NUMBER]@cloudbuild.gserviceaccount.com service account that your build will need to run (such as the Cloud Functions Developer permission, which is required when deploying Cloud Functions).

If Cloud Container Builder doesn't suit your needs, see this page for a list of continuous integration platforms.

Send feedback about...

Cloud Functions Documentation