Authorize the Monitoring agent

This guide explains how to install private-key service account credentials on a VM instance to authorize the Monitoring agent. Before installing the agent, check that your VM instance has the credentials that the agent needs. The agent must have permission to send information to Google Cloud's operations suite. Permission is given by using service account credentials that are stored on your VM instance and serve as Application Default Credentials for the agent.

Authorization overview

Authorization refers to the process of determining what permissions an authenticated client has for a set of resources. Google Cloud authorizes the Monitoring agent on a Compute Engine VM instance by using application default credentials (ADC).

The Monitoring agent supports ADC that authenticate either a VM's attached service account, or a private key from a service account.

  • An attached service account refers to a service account that's specific to a given resource, such as a VM. The service account has its own unique credentials. ADC uses the VM's metadata server to obtain credentials for a service.

  • A private key from a service account refers to a private key used to authorize the key pair on a service account in a project, which lets you create an access token. You use the token to provide an identity so that you can interact with Google Cloud APIs on behalf of the service account.

    We recommend that you configure the ADC to authenticate an attached service account whenever possible, as the private key requires local storage, and that storage can be compromised. For more information about service account keys, see Best practices for managing service account keys.

    For AWS EC2 VM instances, the Monitoring agent supports authentication only from a service account with a private key.

    Before you begin

    Read this guide if any of the following applies to you:

    • If you're running very old Compute Engine instances or if you have modified the access scopes or service account settings for your Compute Engine instances, then you must complete the steps in this guide before installing the agent. These VMs might not have the required private-key credentials. For information about how to verify the access scopes and service account settings of your instances, see Verify Compute Engine credentials.

      On newly created Compute Engine VM instances, the access scopes and service account settings are sufficient to run the agents.

    • If you're running Amazon EC2 (AWS VC2) VM instances, then you must complete the steps in this guide before installing the agent. Because Amazon EC2 VM instances don't have the required Google Cloud credentials, you must manually obtain private-key credentials from a service account of the AWS connector project.

    To verify your access scopes, do the following:

    1. Query the access scopes by running the following command on your Compute Engine instance: 
      curl --silent --connect-timeout 1 -f -H "Metadata-Flavor: Google" http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/scopes

    2. In the command output, if the access scope https://www.googleapis.com/auth/cloud-platform is listed, then you have sufficient authorization.

      If https://www.googleapis.com/auth/cloud-platform isn't listed, then you require two access scopes, one from each of the following "logging" and "monitoring" pairs:

      • https://www.googleapis.com/auth/logging.write or
        https://www.googleapis.com/auth/logging.admin
      • https://www.googleapis.com/auth/monitoring.write or
        https://www.googleapis.com/auth/monitoring.admin

    To modify your access scopes, do the following:

    1. In the navigation panel of the Google Cloud console, select Compute Engine, and then select VM instances:

      Go to VM instances

    2. If necessary, click the drop-down list of Google Cloud projects and select the name of your project.
    3. Select VM instances from the navigation menu, and then select the Instances tab if necessary.
    4. In the list of VM instances, click on the name of your VM to view the Details page for the VM.
    5. Shut down the VM by clicking  Stop.
    6. After the VM stops, click  Edit.
    7. Locate the Access scopes in the Identity and API access section of the page.
    8. Select Set access for each API, then locate the entries for Stackdriver Logging API and Stackdriver Monitoring API.
    9. Select the Write Only value for the two APIs.
    10. Click Save.
    11. Restart the VM by clicking  Start/Resume.

    Use a service account

    Authentication refers to the process of determining a client's identity. For authentication, we recommend using a service account, a Google account that is associated with your Google Cloud project, rather than with a specific user. You can use service accounts for authentication regardless of where your code runs: on Compute Engine, App Engine, or on-premise. For more information, see Authentication at Google.

    This section describes how to create a new service account and grant it the necessary roles, and how to update an existing service account if it doesn't have the necessary roles.

    Create a service account

    To create a service account, complete the Creating a service account procedures with the following information:

    • Select the Google Cloud project in which to create the service account.

      • For Compute Engine instances, choose the project in which you created the instance.

      • For Amazon EC2 instances, choose the AWS connector project created when you connected your AWS account to Google Cloud services.

    • In the Role drop-down menu, select the following roles:

      • Monitoring > Monitoring Metric Writer. This authorizes the Monitoring agent.

      If you will also install the Logging agent, then add the following role for that agent:

      • Logging > Logs Writer. This authorizes the Logging agent.

    • If you plan to use private key authentication, then select JSON as the Key type and click Create.

      When you click Create, a file that contains a service account key is downloaded to your local system. For more information, see Create and delete service account keys.

    Next, configure your service account and settings based on whether you authorize by using attached service accounts or by using service account private keys.

    Verify and modify roles of an existing service account

    You can use the Google Cloud console to determine which roles an existing service account has, and to add any necessary roles that are missing:

    1. In the navigation panel of the Google Cloud console, select IAM:

      Go to IAM

    2. If necessary, click the drop-down list of Google Cloud projects and select the name of your project.

    3. If you don't see a list of IAM principals (users and service accounts), then select the Permissions tab.

    4. In the View by Principals list, locate the entry for the service account. The Role column lists the roles granted to the service account.

    5. If your service account does not have the necessary roles for the Monitoring agent, then use the following steps to add the roles described in Create a service account:

      1. Click  Edit in the entry for the service account.
      2. Click Add Another Role to add any missing roles.
      3. Click Save.

    Authorize with an attached service account

    To authorize the Monitoring agent installed on a VM instance that has an attached service account, do the following:

    1. Grant your service account the least privileged IAM roles possible.

    2. Attach the service account to the resource where your code is running.

    3. Install or restart the agent.

    Authorize a service account with a private key

    To authorize the Monitoring agent by using service account private keys, do the following:

    1. Copy the private-key file to one of the following locations on your VM instance so that the agent can recognize the credentials. You can use any file-copy tool you wish.

      • Linux only: /etc/google/auth/application_default_credentials.json

      • Windows only: C:\ProgramData\Google\Auth\application_default_credentials.json

      • For both Linux and Windows: Any location you store in the variable, GOOGLE_APPLICATION_CREDENTIALS. The variable must be visible to the agent's process.

    2. Create an environment variable to point to the credentials file on your workstation. The following example creates a variable called CREDS:

      CREDS="~/Downloads/PROJECT-NAME-KEY-ID.json"

    3. Complete the steps shown in the following table:

      Compute Engine

      On your local system, use the gcloud command-line tool.

      1. In the navigation panel of the Google Cloud console, select Compute Engine, and then select VM instances:

        Go to VM instances

        Identify the INSTANCE_NAME and INSTANCE_ZONE for your VM.

      2. Set the following environment variables:

        REMOTE_USER="$USER"
INSTANCE="INSTANCE_NAME"
ZONE="INSTANCE_ZONE"
gcloud compute scp "$CREDS" "$REMOTE_USER@$INSTANCE:~/temp.json" --zone "$ZONE"

      3. On your Compute Engine instance, run these commands:

      GOOGLE_APPLICATION_CREDENTIALS="/etc/google/auth/application_default_credentials.json"
sudo mkdir -p /etc/google/auth
sudo mv "$HOME/temp.json" "$GOOGLE_APPLICATION_CREDENTIALS"
sudo chown root:root "$GOOGLE_APPLICATION_CREDENTIALS"
sudo chmod 0400 "$GOOGLE_APPLICATION_CREDENTIALS"

      Amazon EC2

      On your local system, use scp:

      KEY="YOUR-SSH-KEY-PAIR-FILE"
INSTANCE="ec2-YOUR-INSTANCE'S-PUBLIC-ID.ZONE.compute.amazonaws.com"
# The remote user depends on the installed OS: ec2-user, ubuntu, root, etc.
REMOTE_USER="ec2-user"
scp -i "$KEY" "$CREDS" "$REMOTE_USER@$INSTANCE:~/temp.json"

      On your EC2 instance, run these commands:

      GOOGLE_APPLICATION_CREDENTIALS="/etc/google/auth/application_default_credentials.json"
sudo mkdir -p /etc/google/auth
sudo mv "$HOME/temp.json" "$GOOGLE_APPLICATION_CREDENTIALS"
sudo chown root:root "$GOOGLE_APPLICATION_CREDENTIALS"
sudo chmod 0400 "$GOOGLE_APPLICATION_CREDENTIALS"

      If your credential file is not in the previously listed default location, then in addition to the commands in the preceding examples, ensure that GOOGLE_APPLICATION_CREDENTIALS is defined and visible to the agent process.

    4. Install or restart the agent.

    Next steps

    Your VM instance now has the credentials that the agent needs.