Quickstart for monitoring a Compute Engine instance

This page shows you how to monitor a Compute Engine virtual machine (VM) instance with Cloud Monitoring. If you want to monitor an Amazon EC2 VM instance, see Quickstart for Amazon EC2.

In this quickstart, you do the following:

  1. Create a Compute Engine VM instance.
  2. Install Apache HTTP Server.
  3. Install the Cloud Monitoring and Logging agents.
  4. Create an uptime check with an alerting policy.
  5. Create a custom dashboard and chart.
  6. View your logs.
  7. Clean up.

Before you begin

To use Cloud Monitoring, you need to create a Google Cloud project, enable billing for your project, and associate your project with a Workspace.

To create a project and enable billing, do the following:

  1. In the Cloud Console, go to New Project:

    Create a New Project

  2. In the Project Name field, enter Quickstart and then click Create.

  3. Go to Billing:

    Go to Billing

  4. Select your Quickstart project if it isn't already selected at the top of the page.

  5. You are prompted to choose an existing payments profile or to create a new one.

Create a Compute Engine instance

  1. In the Cloud Console, go to Compute and then select Compute Engine:

    Go to Compute Engine

  2. To create a VM instance, click Create.

  3. Fill in the fields for your instance as follows:

    • In the Name field, enter lamp-1-vm.
    • In the Machine type field, select Small.
    • Ensure the Boot disk is configured for Debian GNU/Linux.
    • In the Firewall field, select both Allow HTTP traffic and Allow HTTPS traffic.

    Leave the rest of the fields at their default values.

  4. Click Create. Wait a couple of minutes for your instance to launch on the VM Instances page.

  5. To open a terminal to your instance, in the Connect column, click SSH.

  6. Update the package lists on your instance.

    sudo apt-get update
    
  7. Set up the Apache2 HTTP Server.

    sudo apt-get install apache2 php7.0
    
  8. Open your browser and connect to your Apache2 HTTP server by using the URL http://[External IP]. Replace [External IP] with the external IP address of your Compute Engine instance. You see the Apache2 default page:

    Display the Apache2 default page.

Install agents

The instructions in this section are specific to Debian. See Installing the Cloud Monitoring agent and Installing the Cloud Logging agent for installation instructions for other operating systems, links to troubleshooting guides, and steps to verify agent installation.

The Cloud Monitoring and Logging agents pass logs and metrics from your VM instance to Monitoring and Logging:

  1. Switch to the terminal connected to your VM instance or create a new one.

  2. Install and start the Cloud Monitoring agent:

    • Add the package repository and update the package list:

      curl -sSO https://dl.google.com/cloudagents/add-monitoring-agent-repo.sh
      sudo bash add-monitoring-agent-repo.sh
      sudo apt-get update
      
    • Install the agent:

      sudo apt-get install stackdriver-agent
      
    • Start the agent:

      sudo service stackdriver-agent start
      
  3. Install, configure, and start the Cloud Logging agent:

    • Add the package repository and update the package list:

      curl -sSO https://dl.google.com/cloudagents/add-logging-agent-repo.sh
      sudo bash add-logging-agent-repo.sh
      sudo apt-get update
      
    • Install the agent:

      sudo apt-get install google-fluentd
      
    • Install the default agent configuration for ingesting structured data to Cloud Logging:

      sudo apt-get install google-fluentd-catch-all-config-structured
      

      The Cloud Logging agent must be configured to ingest structured data or unstructured data. These are exclusive options. If you configure the agent for unstructured data, then all log entries contain a textPayload field. However, if you configure the agent for structured data, as is done in this quickstart, then certain types of log data contain a jsonPayload field. For more information, see Structured logging operations.

    • Start the agent

      sudo service google-fluentd start
      

Create an uptime check and an alerting policy

There are two different user flows for creating an uptime check. You are assigned to one of the flows at random. This section contains instructions for both flows.

New flow

  1. In the Cloud Console, select Monitoring:

    Go to Monitoring

    The first time you access Monitoring for a Google Cloud project, Monitoring creates a Workspace and associates it with your project. This process is automatic unless you have a multi-project Workspace. In this case, a dialog appears that asks you to select between creating a Workspace and adding the project to an existing Workspace. Select the option to create a Workspace.

  2. Click Uptime checks.

  3. Click Create Uptime check.

  4. If the Create uptime check window looks like the following screenshot, then continue with these instructions; otherwise, click the Classic flow tab on this table and continue with those instructions:

    Create an uptime check dialog.

  5. For the title, enter My Uptime Check and then click Next.

  6. Target:

    1. Select HTTP as the protocol.
    2. Select Instance as the Resource Type.
    3. In the Applies To field, select Single and for the instance name select lamp-1-vm.
    4. Leave all other fields at their default values and click Next.
  7. Response Validation: Leave these fields at their default values and click Next

  8. Alert & Notification:

    • Ensure that the toggle's label is Alerting is enabled.
    • Leave the name and duration fields at their default values.
    • To add a notification channel to the alerting policy, in the text box labeled Notification channels, click Menu . Select the channels to add and click OK. The notifications are grouped alphabetically for each channel type.

      To add an entry to the checkbox list, click Manage notification channels and follow the instructions. When you return to this dialog, click Refresh .

  9. To verify your uptime check configuration, click Test. If you see a "Connection error - refused" message, you might have not installed the Apache HTTP Server or you might have specified the HTTPS check type rather than HTTP. For other errors, see Verify your uptime check.

  10. Click Create. When the create action is successful, a banner displays Check and alert created. The Uptime checks window lists the new check and contains a link to the uptime checks' dashboard. The Alerting window lists the new alerting policy and contains a link to the policies dashboard. If required data is missing, the create action fails and a list of fields that require data is displayed next to the dialog buttons.

Classic flow

To create an uptime check, do the following:

  1. Go to Monitoring:

    Go to Monitoring

    The first time you access Monitoring for a Google Cloud project, Monitoring creates a Workspace and associates it with your project. This process is automatic unless you have a multi-project Workspace. In this case, a dialog appears that asks you to select between creating a Workspace and adding the project to an existing Workspace. Select the option to create a Workspace.

  2. Click Uptime checks.

  3. Click Create Uptime check.

  4. If the New uptime check window looks like the following screenshot, then continue with these instructions; otherwise, click the New flow tab on this table and continue with those instructions:

    Display of new uptime check dialog with default fields.

  5. Fill in the following fields for the new uptime check:

    • In the Title field, enter My Uptime Check.
    • In the Check type menu, select HTTP.
    • In the Resource Type menu, select Instance.
    • In the Applies To field, select Single and for the instance name select lamp-1-vm.
    • Leave the other fields with their default values.
  6. To verify that your uptime check is working, click Test. If you see a "Connection error - refused" message, you might have not installed the Apache HTTP Server or you might have specified the HTTPS check type rather than HTTP. For other errors, see Verify your uptime check.

  7. Click Save. The confirmation dialog displays:

    Display of the Uptime Check Created dialog.

Create an alerting policy

  1. In the Uptime Check Created pane, click Create Alerting Policy.

  2. In the Untitled Condition field, enter a title for the alert policy condition. All other fields are in the conditions pane are automatically populated from the uptime check you created.

    Display of the create condition dialog with default settings.

  3. Click Save.

  4. Enter My Uptime Check Policy as the Name for the alerting policy.

    Display of the create new alerting policy dialog with default settings.

  5. (Optional) To configure an email notification, click Add notification channel, select Email from the menu, enter your email address, and then click Add.

  6. Click Save. You see a summary of the policy.

Create a dashboard and chart

To display the metrics collected by Monitoring, create a chart and a dashboard:

  1. Go to Monitoring

    Go to Monitoring

  2. Select Dashboards and then select Create dashboard.

  3. Enter Quickstart dashboard as the name for the dashboard and click Confirm.

  4. Click Add Chart.

  5. Ensure the Metric tab is selected.

    Display the add chart dialog with default settings.

  6. Under Find resource type and metric heading, click instance, cpu, usage, etc.:

    • Select G​C​E VM Instance for the resource type.
    • Enter CPU, and then select CPU load(1m) for the metric.
  7. Click Save.

    Display of the configured chart.

  8. To create a second chart, click Add Chart.

  9. Ensure the Metric tab is selected.

  10. Under Find resource type and metric heading, click instance, cpu, usage, etc.:

    • Select G​C​E VM Instance for the resource type.
    • Select Received bytes for the metric.
  11. Click Save.

Test the check and alert

This procedure can take up to 40 minutes. A sample timeline is included for your reference. In the timeline, the current time is 12:00.

12:00 Wait

Go to Monitoring. In the Uptime checks tile, wait until the icon for the My Uptime Check changes to a green circle with a check mark. The maximum wait time for this stage is 5 minutes:

Display the uptime check pass.

12:15 The My Uptime Check icon is green. Test the check and alert.

Go to the VM Instances page, select your instance, and click Stop.

12:25 Alert notification received.

Correct the "problem" by restarting the VM. Return to the VM Instances page, select your instance, and click Start.

12:40 Incident Resolved

Received second alert notification.

View your logs

Monitoring and Logging are closely integrated.

To view the logs for the resources displayed in a chart, do the following:

  1. Go to the dashboard that displays the chart of interest.

  2. In the chart, click More , and then click View logs:

    Display selecting View logs from chart more menu.

Alternatively, you can go to Logging, and then specify the filter parameters:

  1. In the Cloud Console, go to Logging:

    Go to Logging

  2. Change the Logs Viewer settings to see the logs you want:

    • Click menu and select Clear filters and return to basic mode.
    • In the first drop-down list, select G​C​E VM Instance, lamp-1-vm.
    • In the second drop-down list, select syslog, and click OK.
    • Leave the other fields with their default values. The logs from your VM instance display.

      Display a view view of logs from console.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used in this quickstart, follow these steps.

Delete the project

If you created a new project for this quickstart, then delete the project. Otherwise, skip this section.

  1. In the Cloud Console, go to the Manage resources page.

    Go to the Manage resources page

  2. In the project list, select the project that you want to delete and then click Delete .
  3. In the dialog, type the project ID and then click Shut down to delete the project.

Delete quickstart resources

If you used an existing project for this quickstart, delete your alerting policy, uptime checks, dashboard, and VM instance:

  1. Go to Monitoring:

    Go to Monitoring

  2. Delete the alerting policy:

    1. Select Alerting.
    2. Select My Uptime Check Policy and then click Delete .
  3. Delete the uptime check:

    1. Select Uptime Checks.
    2. Select My Uptime check and then click Delete .
  4. Delete the dashboard:

    1. Select Dashboards.
    2. For the dashboard named Quickstart dashboard, click Delete . Confirm the deletion operation.
  5. Delete the VM instance you created:

    1. In the Cloud Console, go to the VM Instances page.

      Go to the VM Instances page

    2. Click the checkbox for the instance you want to delete.
    3. Click Delete to delete the instance.

What's next