Installing the Monitoring Agent

This guide explains how to install the monitoring agent for Stackdriver Monitoring on Google Compute Engine and Amazon Elastic Compute Cloud (EC2) virtual machine (VM) instances. For more information, see Supported VM instances.

Quick links:

Using the monitoring agent is optional but recommended. Stackdriver Monitoring can access some metrics without the agent, including CPU utilization, some disk traffic metrics, network traffic, and uptime. On instances running Microsoft Windows, the agent records CPU utilization and memory, pagefile, and volume usage. If you are running IIS or SQL server, the agent collects metrics from those services by default.

Before you begin

You must have a VM instance that is associated with a GCP project and is being monitored in a Stackdriver account. The monitoring agent's metrics are only available with the Stackdriver Premium service tier, but the agent can be installed on VM instances in the Basic tier. You can try out this and other Stackdriver premium features free for 30 days, when you create a new Stackdriver account. For more information, see one of the following Quickstart or "how-to" guides:

Authorizing the agent

Before installing the monitoring agent, take a minute to be sure your VM instance has the credentials that the agent needs. The agent must have permission to send monitoring information to Stackdriver Monitoring. Permission is given by using service account credentials that are stored on your VM instance and serve as Application Default Credentials for the agent.

  • If you are installing on a Google Compute Engine VM instance, then the default service account on your instance should have the credentials needed by the agent. However, very old instances or instances created without the default credentials will need private-key credentials. To verify your credentials, see the following section, Verifying Compute Engine credentials.

  • If you are installing on an Amazon EC2 VM instance, then there is no default service account. Instead, you must manually obtain private-key credentials from a service account of the GCP project connected to your AWS account. If you think your instance already has private-key credentials, then see Verifying private-key credentials to check them. To add private-key credentials, skip ahead to Adding credentials.

Verifying Compute Engine credentials

Use the Compute Engine > VM instances page of the Cloud Platform Console to verify that your Compute Engine VM instance has adequate credentials for the Monitoring agent. The credentials are typically added in the default service account of all new Compute Engine VM instances, but it is possible to override those defaults when creating an instance.

Open the Compute Engine Instances page

  1. If necessary, change the current GCP project to be the one associated with your Compute Engine VM instance. For example, if you are prompted to Enable billing, it means the current project does not have any Compute Engine VM instances in it.
  2. In the VM Instances page, click the name of your VM instance. The detail page for your instance appears.
  3. Look under the Cloud API access scopes heading:
    1. If you see "This instance has full API access to all Google Cloud Services," then you have adequate credentials.
    2. If you see next to Cloud Monitoring API that you have Write Only or Full permission, then you have adequate credentials.
    3. Otherwise, your instance's default service account does not have the credentials needed by the agent. To use the agent on your instance, you must add private-key service account credentials. For instructions, see Adding credentials.

If you have the correct default credentials, skip ahead to Installing on Linux or Installing on Microsoft Windows.

Verifying private-key credentials

To verify that valid private-key credentials are installed on your VM instance, first verify that the credentials file exists in its expected location, and then verify that the information in the credentials file is valid. Previously-valid credentials can be revoked using the IAM & Admin > Service accounts section of the Cloud Platform Console. If valid credentials aren't present, see Adding credentials to replace the existing credentials or to add new ones.

Are the credentials present?

To see if private-key service account credentials are on your instance, run the following Linux commands on your instance:

sudo cat $GOOGLE_APPLICATION_CREDENTIALS
sudo cat /etc/google/auth/application_default_credentials.json

If either command displays a file like the one shown below, then your instance might have valid private-key credentials. If both commands display a file, then the file denoted by GOOGLE_APPLICATION_CREDENTIALS is used.

{
  "type": "service_account",
  "project_id": "{your-project-id}",
  "private_key_id": "{your-private-key-id}",
  "private_key": "{your-private-key}",
  "client_email": "{your-project-number}-{your-key@developer.gserviceaccount.com",
  "client_id": "{your-client-id}",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://accounts.google.com/o/oauth2/token",
  "auth_provider_x509_cert_url": "{x509-cert-url}",
  "client_x509_cert_url": "{client-x509-cert-url}"
}

If there are no credential files present, then see Adding credentials.

Are the credentials valid?

In the credentials file, project_id is your GCP project, client_email identifies the service account in the project, and private_key_id identifies the private key in the service account. Match this information with what is shown in the IAM & Admin > Service accounts section of the Cloud Platform Console. The credentials file is not valid if any of the following are true:

  • You are checking a Compute Engine instance, but the GCP project in the credentials file is not the project that contains your instance.
  • You are checking an Amazon EC2 instance, but the GCP project in the credentials file is not the connector project (named AWS Link...) for your AWS account.
  • The listed service account doesn't exist. It could have been deleted.
  • The listed service account doesn't have the right roles enabled: Project > Editor for the Monitoring agent and Logging > Logs writer for the Logging agent.
  • The private key doesn't exist. It could have been revoked.

If the service account is all right but the private key has been revoked, then you can create a new private key and copy it to your instance. Otherwise, you must create a new service account as described in the following section, Adding credentials.

Adding credentials

If you are installing the agent on a Compute Engine VM instance and you created your instance with the default credentials, you can skip this section and go right to Installing on Linux or Installing on Windows. If you are not sure whether you have the right credentials, see Verifying Compute Engine credentials.

If your Compute Engine instance does not have the correct credentials, or if you are installing the agent on an Amazon EC2 instance, then the following sections explain how to:

  1. Create a service account with the required privileges and private-key credentials.
  2. Copy the private-key credentials to your VM instance, where they serve as Application Default Credentials for software running on your instance.

Creating a service account

Use the IAM & Admin > Service accounts page of the Cloud Platform Console to create a service account and private key for the GCP project associated with your VM instance.

Open The IAM Service Account page

  1. Click Select a project and choose the GCP project in which to create the service account:

    • For Compute Engine instances, choose the project in which you created the instance. If you created your instance in the Stackdriver account hosting project, then choose the Stackdriver account.

    • For Amazon EC2 instances, choose the AWS connector project created when you connected Stackdriver Monitoring your AWS account. The connector project's name typically begins with AWS Link. Do not create your service account in the Stackdriver account project.

    Click Open. You see the following Service Accounts page:

    Service Accounts

  2. In the Service Accounts page, click Create service account.

  3. In the Create Service Account panel, fill in the following information:

    1. Enter a service account name. For example, Agent service account.
    2. In the Role drop-down menu, select each of the following roles:
      • Project > Editor. This authorizes the Stackdriver Monitoring agent.
      • Logging > Logs writer. This authorizes the Stackdriver Logging agent. Adding this role lets you use this service account to run both Stackdriver agents.
    3. Check Furnish a new private key.
    4. Choose JSON as the Key type. The completed panel is shown below:

    Create service account

    Click Create.

  4. The Cloud Platform Console writes the private key file to your workstation's download directory. It typically has a name such as the following:

    ~/Downloads/{project_name}_{key_id}.json

    For your convenience in the following instructions, set the variable CREDS to point to the credentials file on your workstation:

    CREDS="~/Downloads/{project_name}-{key_id}.json"

Copying the private key to your instance

For the added service account credentials to be recognized, you must copy the private-key file to one of the following locations on your VM instance, using whatever file-copy tool you wish:

  • Linux only: /etc/google/auth/application_default_credentials.json
  • Windows only: C:\ProgramData\Google\Auth\application_default_credentials.json
  • Any location you store in the variable, GOOGLE_APPLICATION_CREDENTIALS. The variable must be visible to the agent's process.

The following file-copy instructions assume you have a Linux environment on both your workstation and your instance. If you are using a different configuration, consult the documentation from your cloud provider for how to copy the private-key file. In the previous step, Creating a service account, your private-key credentials should have been stored on your workstation at a location you saved in the variable CREDS:

Compute Engine

On your workstation, use the gcloud command-line tool:

REMOTE_USER="$USER"
INSTANCE="{your-instance-id}"
ZONE="{your-instance-zone}"
gcloud compute copy-files "$CREDS" "$REMOTE_USER@$INSTANCE:~/temp.json" --zone "$ZONE"

On your Compute Engine instance, run these commands:

APPLICATION_DEFAULT_CREDS="/etc/google/auth/application_default_credentials.json"
sudo mkdir -p /etc/google/auth
sudo mv "$HOME/temp.json" "$APPLICATION_DEFAULT_CREDS"
sudo chown root:root "$APPLICATION_DEFAULT_CREDS"
sudo chmod 0400 "$APPLICATION_DEFAULT_CREDS"

Amazon EC2

On your workstation, use scp:

KEY="{your-ssh-key-pair-file}"
INSTANCE="ec2-{your-instance's-public-ip}.{your-zone}.compute.amazonaws.com"
# The remote user depends on the installed OS: ec2-user, ubuntu, root, etc.
REMOTE_USER="ec2-user"
scp -i "$KEY" "$CREDS" "$REMOTE_USER@$INSTANCE:~/temp.json"

On your EC2 instance, run these commands:

APPLICATION_DEFAULT_CREDS="/etc/google/auth/application_default_credentials.json"
sudo mkdir -p /etc/google/auth
sudo mv "$HOME/temp.json" "$APPLICATION_DEFAULT_CREDS"
sudo chown root:root "$APPLICATION_DEFAULT_CREDS"
sudo chmod 0400 "$APPLICATION_DEFAULT_CREDS"

Next steps

Your VM instance now has the credentials that the agent needs.

  • If you would like to double-check the credentials, see Verifying private-key credentials on this page.

  • If you have not yet installed the monitoring agent, then go to Installing on Linux or Installing on Windows.

  • If you have already installed the agent, then restart the agent so that it uses your new credentials. For example, use the following Linux command on your VM instance:

    sudo service stackdriver-agent restart
    

Installing on Linux

This step assumes you have a VM instance running Linux that is being monitored by a Stackdriver account, and that your instance has the proper credentials for the agent. For more information, see Adding credentials. These instructions work for both Google Compute Engine instances and Amazon EC2 instances:

  1. Run the following commands on your VM instance to install the monitoring agent:

    curl -O "https://repo.stackdriver.com/stack-install.sh"
    sudo bash stack-install.sh --write-gcm
    

    At the end of the installation, you should see something like the following message.

    Restarting services
    [ ok ] Restarting stackdriver-agent (via systemctl): stackdriver-agent.service.
    
  2. If you use an HTTP proxy, do the following:

    1. Edit the monitoring agent's system-defaults file to set PROXY_URL to your HTTP proxy. The name of the configuration file depends on your version of Linux:

      • For Debian and Ubuntu, edit /etc/default/stackdriver-agent
      • For Amazon Linux, Red Hat, and CentOS, edit /etc/sysconfig/stackdriver
    2. Restart the monitoring agent by running the following command on your VM instance:

      sudo service stackdriver-agent restart
      

You have finished installing the agent. If you have problems, see Troubleshooting.

Installing on Windows

To install the agent on a VM instance running Windows, perform the following steps after you have established an RDP or similar connection to your instance and have logged into Windows:

  1. If you use an HTTP proxy, run the following command from an administrator command prompt to set the http_proxy environment variable so that the agent can send data to Stackdriver Monitoring using outbound HTTPS:

    setx http_proxy http://YOUR-PROXY /m
    
  2. Browse to the following URL. Download and run the agent's installer:

    https://repo.stackdriver.com/windows/StackdriverInstaller-GCM-42.exe

    This can also be done via the following PowerShell commands:

    cd C:\Users\[USERNAME]
    invoke-webrequest https://repo.stackdriver.com/windows/StackdriverInstaller-GCM-42.exe -OutFile StackdriverInstaller-GCM-42.exe;
    .\StackdriverInstaller-GCM-42.exe
    

Your agent installation is now complete.

Automated installation

Installation scripts for popular configuration managers, including Ansible, Chef, and Puppet, are provided by the vendors and other community members. We will post links to new scripts as they become available.

Determining the agent version

To determine the version of the monitoring agent on your system, run the following commands on your VM instance:

Debian & Ubuntu

dpkg -l stackdriver-agent

Amazon Linux

sudo yum list stackdriver-agent

Red Hat & CentOS

sudo yum list stackdriver-agent

Microsoft Windows

There is presently no way to determine the version of the agent running on Windows.

Updating the agent

Use the commands from the following table to update your agent:

Debian & Ubuntu

Run the following commands on Debian or Ubuntu Linux:

sudo apt-get update
sudo apt-get install stackdriver-agent

Amazon Linux

Run the following command on Amazon Linux:

sudo yum update stackdriver-agent

Red Hat & CentOS

Run the following command on Red Hat or CentOS Linux:

sudo yum update stackdriver-agent

Microsoft Windows

To upgrade the Microsoft Windows agent, install the new agent version as described in Installing on Microsoft Windows. It will remove any older agent you have installed.

Removing the agent

Use the commands from the following table to remove your agent:

Debian & Ubuntu

Run the following commands on Debian or Ubuntu Linux:

sudo apt-get remove stackdriver-agent

Amazon Linux

Run the following command on Amazon Linux:

sudo yum remove stackdriver-agent

Red Hat & CentOS

Run the following command on Red Hat or CentOS Linux:

sudo yum remove stackdriver-agent

Microsoft Windows

In the Windows Control Panel, choose Uninstall a program. You should see the Stackdriver Monitoring agent in the list of programs you can uninstall.

Troubleshooting

See the Troubleshooting page.

Send feedback about...

Stackdriver Monitoring