Authorizing the Agent

This guide explains how to install private-key service account credentials on a VM instance in order to authorize the Stackdriver Logging and Monitoring agents.

Read this guide if either of the following applies to you:

  • You know that you need to install these credentials. For example, you are using an Amazon EC2 VM instance.

  • You suspect that the existing credentials on your VM instance are not valid for some reason, and you want to replace them with something that will work.

Skip this guide and return to the agent installation guide if you are using a Compute Engine VM instance and you have not yet attempted to install the agent. Compute Engine instances should be pre-authorized to run the agent. If there is a problem with agent installation, you might be directed back here.

Adding credentials

Installing private-key service account credentials on a VM instance involves the following steps:

  1. Creating a service account with the required privileges and private-key credentials in the GCP project associated with your VM instance. For Amazon EC2 VM instances, this is the AWS LINK project Stackdriver creates when you connect your AWS account.

  2. Copying the private-key credentials to your VM instance, where they serve as Application Default Credentials for software running on your instance.

  3. Installing or restarting the agent.

Creating a service account

Use the IAM & Admin > Service accounts page of the Cloud Platform Console to create a service account and private key for the GCP project associated with your VM instance:

Open The IAM Service Account page

  1. Click Select a project and choose the GCP project associated with your VM instance. Click Open. If there is no existing service account, a dialog will prompt you to create one. Otherwise, you see the following Service Accounts page:

    Service Accounts

  2. Click Create service account.

  3. In the panel that appears, fill in the following information:

    1. Enter a service account name. For example, Agent service account.
    2. In the Role drop-down menu, select both of the following roles:
      • Logging > Logs Writer. This authorizes the Stackdriver Logging agent.
      • Monitoring > Monitoring Metric Writer. This authorizes the Stackdriver Monitoring agent. Adding this role lets you use this service account to run both Stackdriver agents.
    3. Check Furnish a new private key.
    4. Choose JSON as the Key type. The completed panel is shown below:

    Create service account

    Click Create.

  4. The Cloud Platform Console writes the private key file to your workstation's download directory. It typically has a name such as the following:

    ~/Downloads/{project_name}_{key_id}.json

    For your convenience in the following instructions, set the variable CREDS to point to the credentials file on your workstation. For example,

    CREDS="~/Downloads/{project_name}-{key_id}.json"

Copying the private key to your instance

For the added service account credentials to be recognized, you must copy the private-key file to one of the following locations on your VM instance, using whatever file-copy tool you wish:

  • Linux only: /etc/google/auth/application_default_credentials.json
  • Windows only: C:\ProgramData\Google\Auth\application_default_credentials.json
  • Linux or Windows: Any location that you store in the variable GOOGLE_APPLICATION_CREDENTIALS. The variable must be visible to the agent process.

The following file-copy instructions assume you have a Linux environment on both your workstation and your instance. If you are using a different configuration, consult the documentation from your cloud provider for how to copy the private-key file. In the previous step, Creating a service account, your private-key credentials should have been stored on your workstation at a location you saved in the variable CREDS:

Compute Engine

On your workstation, use the gcloud command-line tool:

REMOTE_USER="$USER"
INSTANCE="{your-instance-id}"
ZONE="{your-instance-zone}"
gcloud compute scp "$CREDS" "$REMOTE_USER@$INSTANCE:~/temp.json" --zone "$ZONE"

On your Compute Engine instance, run these commands:

GOOGLE_APPLICATION_CREDENTIALS="/etc/google/auth/application_default_credentials.json"
sudo mkdir -p /etc/google/auth
sudo mv "$HOME/temp.json" "$GOOGLE_APPLICATION_CREDENTIALS"
sudo chown root:root "$GOOGLE_APPLICATION_CREDENTIALS"
sudo chmod 0400 "$GOOGLE_APPLICATION_CREDENTIALS"

Amazon EC2

On your workstation, use scp:

KEY="{your-ssh-key-pair-file}"
INSTANCE="ec2-{your-instance's-public-ip}.{your-zone}.compute.amazonaws.com"
# The remote user depends on the installed OS: ec2-user, ubuntu, root, etc.
REMOTE_USER="ec2-user"
scp -i "$KEY" "$CREDS" "$REMOTE_USER@$INSTANCE:~/temp.json"

On your EC2 instance, run these commands:

GOOGLE_APPLICATION_CREDENTIALS="/etc/google/auth/application_default_credentials.json"
sudo mkdir -p /etc/google/auth
sudo mv "$HOME/temp.json" "$GOOGLE_APPLICATION_CREDENTIALS"
sudo chown root:root "$GOOGLE_APPLICATION_CREDENTIALS"
sudo chmod 0400 "$GOOGLE_APPLICATION_CREDENTIALS"

Next steps

Your VM instance now has the credentials that the agent needs. Return to the agent installation page and install the agent. If you have already installed the agent, then restart it to use the new credentials.

If you would like to double-check the credentials, see Verifying private-key credentials on this page.

Verifying Compute Engine credentials

This section helps you diagnose credentials problems on Google Compute Engine VM instances. Unless you have a specific interest, you can probably skip this section.

Open the Compute Engine Instances page

  1. Select the GCP project associated with your Compute Engine VM instance. If a project is selected that has no Compute Engine VM instances in it, you might be prompted to "Enable billing." Change to the correct project.

  2. In the VM Instances page, click the name of your VM instance. The detail page for your instance appears.

  3. Look under the Cloud API access scopes heading:

    1. If you see "This instance has full API access to all Google Cloud Services," then you have adequate credentials.
    2. If you see next to Cloud Monitoring API that you have Write Only or Full permission, then you have adequate credentials.
    3. Otherwise, your instance's default service account does not have the credentials needed by the agent. To use the agent on your instance, you must add private-key service account credentials. For instructions, see Adding credentials.

If you have the correct default credentials, you are done with authorization. You can now install your software.

Verifying private-key credentials

This section helps you diagnose private-key credentials problems. Unless you have a specific interest, you can probably skip this section.

To verify that valid private-key credentials are installed on your VM instance, first verify that the credentials file exists in its expected location, and then verify that the information in the credentials file is valid. Previously-valid credentials can be revoked using the IAM & Admin > Service accounts section of the Cloud Platform Console. If valid credentials aren't present, see Adding credentials to replace the existing credentials or to add new ones.

Are the credentials present?

To see if private-key service account credentials are on your instance, run the following Linux commands on your instance:

sudo cat $GOOGLE_APPLICATION_CREDENTIALS
sudo cat /etc/google/auth/application_default_credentials.json

If either command displays a file like the one shown below, then your instance might have valid private-key credentials. If both commands display a file, then the file denoted by GOOGLE_APPLICATION_CREDENTIALS is used.

{
  "type": "service_account",
  "project_id": "{your-project-id}",
  "private_key_id": "{your-private-key-id}",
  "private_key": "{your-private-key}",
  "client_email": "{your-project-number}-{your-key@developer.gserviceaccount.com",
  "client_id": "{your-client-id}",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://accounts.google.com/o/oauth2/token",
  "auth_provider_x509_cert_url": "{x509-cert-url}",
  "client_x509_cert_url": "{client-x509-cert-url}"
}

If there are no credential files present, then see Adding credentials.

Are the credentials valid?

In the credentials file, project_id is your GCP project, client_email identifies the service account in the project, and private_key_id identifies the private key in the service account. Match this information with what is shown in the IAM & Admin > Service accounts section of the Cloud Platform Console. The credentials file is not valid if any of the following are true:

  • You are checking a Compute Engine instance, but the GCP project in the credentials file is not the project that contains your instance.
  • You are checking an Amazon EC2 instance, but the GCP project in the credentials file is not the connector project (named AWS Link...) for your AWS account.
  • The listed service account doesn't exist. It could have been deleted.
  • The listed service account doesn't have the right roles enabled: Logging > Logs Writer for the Logging agent and Monitoring > Monitoring Metric Writer for the Monitoring agent.
  • The private key doesn't exist. It could have been revoked.

If the service account is all right but the private key has been revoked, then you can create a new private key and copy it to your instance. Otherwise, you must create a new service account as described in the section, Adding credentials.

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Stackdriver Logging