Scheduling Compute Instances with Cloud Scheduler

This tutorial shows how to use Cloud Scheduler and Cloud Functions to automatically start and stop Compute Engine instances on a regular schedule using resource labels.

Objectives

  • Write and deploy a set of functions with Cloud Functions that start and stop Compute Engine instances.
  • Create a set of jobs with Cloud Scheduler that schedule instances with a dev resource label to run 09:00-17:00, Monday-Friday to match typical business hours.

Costs

This tutorial uses the following billable components of Google Cloud Platform:

  • Cloud Scheduler
  • Cloud Functions
  • Cloud Pub/Sub
  • Compute Engine

You can use the pricing calculator to generate a cost estimate based on your projected usage. New GCP users might be eligible for a free trial.

Before you begin

  1. Set up your environment for Cloud Scheduler.

    Setting Up Your Environment

  2. Enable the Cloud Functions, Cloud Pub/Sub, and Compute Engine APIs.

    Enable the APIs

Application architecture

This solution includes the following GCP components:

  1. Compute Engine Instance: A Compute Engine instance we want to run on a schedule.
  2. Cloud Functions functions: Functions to start and stop the instance we want to schedule.
  3. Cloud Pub/Sub messages: Messages sent and received for each start and stop event.
  4. Cloud Scheduler jobs: Jobs to make calls on a set schedule to start and stop the instance.

System architecture diagram showing Cloud Scheduler scheduling a Compute Engine instance via Cloud Pub/Sub

Location Requirements

Some components are only supported in certain regions:

  1. Compute Engine Instance: supported in any region listed under the Compute Engine Regions and Zones.
  2. Cloud Functions functions: supported in the regions listed as Cloud Functions Locations.
  3. Cloud Pub/Sub messages: supported globally as Cloud Pub/Sub is a global service.
  4. Cloud Scheduler jobs: supported in any of the current App Engine locations.

Best Practice: Why not HTTP instead of Cloud Pub/Sub?

You may want to simplify this architecture by using Cloud Functions HTTP Triggers instead of Cloud Pub/Sub Triggers.

System architecture diagram showing Cloud Scheduler scheduling a Compute Engine instance via HTTP

System architecture diagram showing that scheduling via HTTP allows anyone to schedule your Compute Engine instance

To create a more secure setup, we recommend you use Cloud Pub/Sub functions instead.

Set up the Compute Engine instance

Console

  1. Go to the VM instances page in the GCP Console.
    Go to the VM instances page.
  2. Click Create instance.
  3. Set the Name to dev-instance.
  4. For Region select us-west1.
  5. For Zone select us-west1-b.
  6. Expand the Management, security, disks, networking, sole tenancy section.
  7. Under Management, click Add label. Enter env for Key and dev for Value.
  8. Click Create at the bottom of the page.

gcloud

gcloud compute instances create dev-instance \
    --network default \
    --zone us-west1-b \
    --labels=env=dev

Set up the Cloud Functions functions with Cloud Pub/Sub

Create and deploy the functions

Console

Create the start function.

  1. Go to the Cloud Functions page in the GCP Console.
    Go to the Cloud Functions page.
  2. Click Create Function.
  3. Set the Name to startInstancePubSub.
  4. Leave Memory allocated at its default value.
  5. For Trigger, select Cloud Pub/Sub.
  6. For Topic, select Create new topic....
  7. A New pub/sub topic dialog box should appear.
    1. Under Name, enter start-instance-event.
    2. Click Create to finish the dialog box.
  8. For Runtime, select Node.js 8.
  9. Above the code text block, select the index.js tab.
  10. Replace the starter code with the following code:

    const {Buffer} = require('safe-buffer');
    const Compute = require('@google-cloud/compute');
    const compute = new Compute();
    
    /**
     * Starts a Compute Engine instance.
     *
     * Expects a PubSub message with JSON-formatted event data containing the
     * following attributes:
     *  zone - the GCP zone the instances are located in.
     *  label - the label of instances to start.
     *
     * @param {!object} event Cloud Function PubSub message event.
     * @param {!object} callback Cloud Function PubSub callback indicating
     *  completion.
     */
    exports.startInstancePubSub = (event, context, callback) => {
      try {
        const payload = _validatePayload(
          JSON.parse(Buffer.from(event.data, 'base64').toString())
        );
        const options = {filter: `labels.${payload.label}`};
        compute.getVMs(options).then(vms => {
          vms[0].forEach(instance => {
            if (payload.zone === instance.zone.id) {
              compute
                .zone(payload.zone)
                .vm(instance.name)
                .start()
                .then(data => {
                  // Operation pending.
                  const operation = data[0];
                  return operation.promise();
                })
                .then(() => {
                  // Operation complete. Instance successfully started.
                  const message = 'Successfully started instance ' + instance.name;
                  console.log(message);
                  callback(null, message);
                })
                .catch(err => {
                  console.log(err);
                  callback(err);
                });
            }
          });
        });
      } catch (err) {
        console.log(err);
        callback(err);
      }
    };
    
    /**
     * Validates that a request payload contains the expected fields.
     *
     * @param {!object} payload the request payload to validate.
     * @return {!object} the payload object.
     */
    function _validatePayload(payload) {
      if (!payload.zone) {
        throw new Error(`Attribute 'zone' missing from payload`);
      } else if (!payload.label) {
        throw new Error(`Attribute 'label' missing from payload`);
      }
      return payload;
    }
  11. Above the code text block, select the package.json tab.

  12. Replace the starter code with the following code:

    {
      "name": "cloud-functions-schedule-instance",
      "version": "0.1.0",
      "private": true,
      "license": "Apache-2.0",
      "author": "Google Inc.",
      "repository": {
        "type": "git",
        "url": "https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git"
      },
      "engines": {
        "node": ">=8.0.0"
      },
      "scripts": {
        "test": "mocha test/*.test.js --timeout=20000"
      },
      "devDependencies": {
        "@google-cloud/nodejs-repo-tools": "^3.3.0",
        "mocha": "^6.0.0",
        "proxyquire": "^2.0.0",
        "sinon": "^7.0.0"
      },
      "dependencies": {
        "@google-cloud/compute": "^1.0.0",
        "safe-buffer": "^5.1.2"
      }
    }
    
  13. For Function to execute, enter startInstancePubSub.

  14. Click Create.

Create the stop function.

  1. You should be on the Cloud Functions page in the GCP Console.
  2. Click Create Function.
  3. Set the Name to stopInstancePubSub.
  4. Leave Memory allocated at its default value.
  5. For Trigger, select Cloud Pub/Sub.
  6. For Topic, select Create new topic....
  7. A New pub/sub topic dialog box should appear.
    1. Under Name, enter stop-instance-event.
    2. Click Create to finish the dialog box.
  8. For Runtime, select Node.js 8.
  9. Above the code text block, select the index.js tab.
  10. Replace the starter code with the following code:

    const {Buffer} = require('safe-buffer');
    const Compute = require('@google-cloud/compute');
    const compute = new Compute();
    
    /**
     * Stops a Compute Engine instance.
     *
     * Expects a PubSub message with JSON-formatted event data containing the
     * following attributes:
     *  zone - the GCP zone the instances are located in.
     *  instance - the name of a single instance.
     *  label - the label of instances to start.
     *
     * Exactly one of instance or label must be specified.
     *
     * @param {!object} event Cloud Function PubSub message event.
     * @param {!object} callback Cloud Function PubSub callback indicating completion.
     */
    exports.stopInstancePubSub = (event, context, callback) => {
      try {
        const payload = _validatePayload(
          JSON.parse(Buffer.from(event.data, 'base64').toString())
        );
        const options = {filter: `labels.${payload.label}`};
        compute.getVMs(options).then(vms => {
          vms[0].forEach(instance => {
            if (payload.zone === instance.zone.id) {
              compute
                .zone(payload.zone)
                .vm(instance.name)
                .stop()
                .then(data => {
                  // Operation pending.
                  const operation = data[0];
                  return operation.promise();
                })
                .then(() => {
                  // Operation complete. Instance successfully stopped.
                  const message = 'Successfully stopped instance ' + instance.name;
                  console.log(message);
                  callback(null, message);
                })
                .catch(err => {
                  console.log(err);
                  callback(err);
                });
            }
          });
        });
      } catch (err) {
        console.log(err);
        callback(err);
      }
    };
    
    /**
     * Validates that a request payload contains the expected fields.
     *
     * @param {!object} payload the request payload to validate.
     * @return {!object} the payload object.
     */
    function _validatePayload(payload) {
      if (!payload.zone) {
        throw new Error(`Attribute 'zone' missing from payload`);
      } else if (!payload.label) {
        throw new Error(`Attribute 'label' missing from payload`);
      }
      return payload;
    }
  11. Above the code text block, select the package.json tab.

  12. Replace the starter code with the following code:

    {
      "name": "cloud-functions-schedule-instance",
      "version": "0.1.0",
      "private": true,
      "license": "Apache-2.0",
      "author": "Google Inc.",
      "repository": {
        "type": "git",
        "url": "https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git"
      },
      "engines": {
        "node": ">=8.0.0"
      },
      "scripts": {
        "test": "mocha test/*.test.js --timeout=20000"
      },
      "devDependencies": {
        "@google-cloud/nodejs-repo-tools": "^3.3.0",
        "mocha": "^6.0.0",
        "proxyquire": "^2.0.0",
        "sinon": "^7.0.0"
      },
      "dependencies": {
        "@google-cloud/compute": "^1.0.0",
        "safe-buffer": "^5.1.2"
      }
    }
    
  13. For Function to execute, enter stopInstancePubSub.

  14. Click Create.

gcloud

Create the Cloud Pub/Sub topics.

gcloud pubsub topics create start-instance-event
gcloud pubsub topics create stop-instance-event

Get the code

  1. Download the code.

    git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git

    Alternatively, you can download the sample as a zip file and extract it.

  2. Go to the correct directory.

    cd nodejs-docs-samples/functions/scheduleinstance/
    

Create the start and stop functions.

You should be in the nodejs-docs-samples/functions/scheduleinstance/ directory.

gcloud functions deploy startInstancePubSub \
    --trigger-topic start-instance-event \
    --runtime nodejs8
gcloud functions deploy stopInstancePubSub \
    --trigger-topic stop-instance-event \
    --runtime nodejs8

(Optional) Verify the functions work

Console

Stop the instance

  1. Go to the Cloud Functions page in the GCP Console.
    Go to the Cloud Functions page.
  2. Click on the function named stopInstancePubSub.
  3. You should see a number of tabs: General, Trigger, Source, and Testing. Click on the Testing tab.
  4. For Triggering event enter the following:

    {"data":"eyJ6b25lIjoidXMtd2VzdDEtYiIsICJsYWJlbCI6ImVudj1kZXYifQo="}
    

    • This is simply the base64-encoded string for {"zone":"us-west1-b", "label":"env=dev"}

    • If you want to encode your own string, feel free to use any online base64 encoding tool.

  5. Click the Test the function button.

  6. When it is done running, you should see Successfully stopped instance dev-instance printed under Output. It may take up to 60 seconds to finish running.

    • If instead you see error: 'Error: function failed to load.', just wait 10 seconds or so for the function to finish deploying and try again.

    • If instead you see error: 'Error: function execution attempt timed out.', just move on to the next step to see if the instance is just taking a long time to shut down.

    • If instead it finishes running, but shows nothing, it probably also just timed out. Just move on to the next step to see if the instance is just taking a long time to shut down.

  7. Go to the VM instances page in the GCP Console.
    Go to the VM instances page.

  8. Verify that the instance named dev-instance has a grey square next to its name, indicating that it has stopped. It may take up to 30 seconds to finish shutting down.

    • If it doesn't seem to be finishing, try clicking Refresh at the top of the page.

Start the instance

  1. Go to the Cloud Functions page in the GCP Console.
    Go to the Cloud Functions page.
  2. Click on the function named startInstancePubSub.
  3. You should see a number of tabs: General, Trigger, Source, and Testing. Click on the Testing tab.
  4. For Triggering event enter the following:

    {"data":"eyJ6b25lIjoidXMtd2VzdDEtYiIsICJsYWJlbCI6ImVudj1kZXYifQo="}
    

    • Again, this is simply the base64-encoded string for {"zone":"us-west1-b", "label":"env=dev"}
  5. Click the Test the function button.

  6. When it is done running, you should see Successfully started instance dev-instance printed under Output.

  7. Go to the VM instances page in the GCP Console.
    Go to the VM instances page.

  8. Verify that the instance named dev-instance has a green checkmark next to its name, indicating that it is running. It may take up to 30 seconds to finish starting up.

gcloud

Stop the instance

  1. Call the function to stop the instance.

    gcloud functions call stopInstancePubSub \
        --data '{"data":"eyJ6b25lIjoidXMtd2VzdDEtYiIsICJsYWJlbCI6ImVudj1kZXYifQo="}'
    
    • This is simply the base64-encoded string for {"zone":"us-west1-b", "label":"env=dev"}

    • If you want to encode your own string, feel free to use any tool. Here's an example using the base64 command line tool:

      echo '{"zone":"us-west1-b", "label":"env=dev"}' | base64
      
      eyJ6b25lIjoidXMtd2VzdDEtYiIsICJsYWJlbCI6ImVudj1kZXYifQo=
      

    When the function has finished you should see the following:

    result: Successfully stopped instance dev-instance
    

    It may take up to 60 seconds to finish running.

    • If instead you get the error:

      error: 'Error: function failed to load.`
      

      Just wait 10 seconds or so for the function to finish deploying and try again.

    • If instead you get the error:

      error: `Error: function execution attempt timed out.`
      

      Move on to the next step to see if the instance is just taking a long time to shut down.

    • If instead you get no result, the function probably just timed out. Move on to the next step to see if the instance is just taking a long time to shut down.

  2. Check that the instance has a status of TERMINATED. It may take up to 30 seconds to finish shutting down.

    gcloud compute instances describe dev-instance \
        --zone us-west1-b \
        | grep status
    
    status: TERMINATED
    

Start the instance

  1. Call the function to start the instance.

    gcloud functions call startInstancePubSub \
        --data '{"data":"eyJ6b25lIjoidXMtd2VzdDEtYiIsICJsYWJlbCI6ImVudj1kZXYifQo="}'
    
    • Again, this is simply the base64-encoded string for {"zone":"us-west1-b", "label":"env=dev"}

    When the function has finished you should see the following:

    result: Successfully started instance dev-instance
    
  2. Check that the instance has a status of RUNNING. It may take up to 30 seconds to finish starting up.

    gcloud compute instances describe dev-instance \
        --zone us-west1-b \
        | grep status
    
    status: RUNNING
    

Set up the Cloud Scheduler jobs to call Cloud Pub/Sub

Create the jobs

Console

Create the start job.

  1. Go to the Cloud Scheduler page in the GCP Console.
    Go to the Cloud Scheduler page.
  2. Click Create Job.
  3. Set the Name to startup-dev-instances.
  4. For Frequency, enter 0 9 * * 1-5.
  5. For Timezone, select your desired country and timezone. This example will use United States and Los Angeles.
  6. For Target, select Pub/Sub.
  7. For Topic, enter start-instance-event.
  8. For Payload, enter the following:
    {"zone":"us-west1-b","label":"env=dev"}
    
  9. Click Create.

Create the stop job.

  1. You should be on the Cloud Functions page in the GCP Console.
  2. Click Create Job.
  3. Set the Name to shutdown-dev-instances.
  4. For Frequency, enter 0 17 * * 1-5.
  5. For Timezone, select your desired country and timezone. This example will use United States and Los Angeles.
  6. For Target, select Pub/Sub.
  7. For Topic, enter stop-instance-event.
  8. For Payload, enter the following:
    {"zone":"us-west1-b","label":"env=dev"}
    
  9. Click Create.

gcloud

Create the start job.

gcloud beta scheduler jobs create pubsub startup-dev-instances \
    --schedule '0 9 * * 1-5' \
    --topic start-instance-event \
    --message-body '{"zone":"us-west1-b", "label":"env=dev"}' \
    --time-zone 'America/Los_Angeles'

Create the stop job.

gcloud beta scheduler jobs create pubsub shutdown-dev-instances \
    --schedule '0 17 * * 1-5' \
    --topic stop-instance-event \
    --message-body '{"zone":"us-west1-b", "label":"env=dev"}' \
    --time-zone 'America/Los_Angeles'

(Optional) Verify the jobs work

Console

Stop the instance

  1. Go to the Cloud Scheduler page in the GCP Console.
    Go to the Cloud Scheduler page.
  2. For the job named shutdown-dev-instances, click the Run now button on the far right side of the page.
  3. Go to the VM instances page in the GCP Console.
    Go to the VM instances page.
  4. Verify that the instance named dev-instance has a grey square next to its name, indicating that it has stopped. It may take up to 30 seconds for it to finish shutting down.

Start the instance

  1. Go to the Cloud Scheduler page in the GCP Console.
    Go to the Cloud Scheduler page.
  2. For the job named startup-dev-instances, click the Run now button on the far right side of the page.
  3. Go to the VM instances page in the GCP Console.
    Go to the VM instances page.
  4. Verify that the instance named dev-instance has a green checkmark next to its name, indicating that it is running. It may take up to 30 seconds for it to finish starting up.

gcloud

Stop the instance

  1. Run the scheduler job to stop the instance.

    gcloud beta scheduler jobs run shutdown-dev-instances
    
  2. Check that the instance has a status of TERMINATED. It may take up to 30 seconds for it to finish shutting down.

    gcloud compute instances describe dev-instance \
        --zone us-west1-b \
        | grep status
    
    status: TERMINATED
    

Start the instance

  1. Run the scheduler job to start the instance.

    gcloud beta scheduler jobs run startup-dev-instances
    
  2. Check that the instance has a status of RUNNING. It may take up to 30 seconds to finish starting up.

    gcloud compute instances describe dev-instance \
        --zone us-west1-b \
        | grep status
    
    status: RUNNING
    

Cleaning up

After you've finished the instance scheduling tutorial, you can clean up the resources that you created on GCP so they won't take up quota and you won't be billed for them in the future. The following sections describe how to delete or turn off these resources.

Delete the Cloud Scheduler jobs

  1. Go to the Cloud Scheduler page in the GCP Console.

    Go to the Cloud Scheduler page.

  2. Click the checkboxes next to your jobs.

  3. Click the Delete Job button at the top of the page and confirm your delete.

Delete the Cloud Pub/Sub topics

  1. Go to the Cloud Pub/Sub page in the GCP Console.

    Go to the Cloud Pub/Sub page

  2. Click the checkboxes next to your topics.

  3. Click Delete at the top of the page and confirm your delete.

Delete the Cloud Functions functions

  1. Go to the Cloud Functions page in the GCP Console.

    Go to the Cloud Functions page.

  2. Click the checkboxes next to your functions.

  3. Click the Delete button at the top of the page and confirm your delete.

Delete the Compute Engine instance

To delete a Compute Engine instance:

  1. In the GCP Console, go to the VM Instances page.

    Go to the VM Instances page

  2. Click the checkbox for the instance you want to delete.
  3. Click Delete to delete the instance.

Delete the project

The easiest way to eliminate billing is to delete the project that you created for the tutorial.

To delete the project:

  1. In the GCP Console, go to the Projects page.

    Go to the Projects page

  2. In the project list, select the project you want to delete and click Delete .
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

What's next

  • Try out other Google Cloud Platform features for yourself. Have a look at our tutorials.
Var denne siden nyttig? Si fra hva du synes:

Send tilbakemelding om ...