Quickstart for AWS

This Quickstart shows you how to connect Stackdriver Monitoring to your Amazon Web Services (AWS) account, and how to install the Stackdriver Monitoring and Logging agents on your EC2 instances.

September 20, 2017: This Quickstart has been simplified. It no longer goes through the process of creating an EC2 instance in your AWS account.

Before you begin

  • You must have an AWS account that is not currently monitored by Stackdriver. You cannot monitor an AWS account from more than one Stackdriver account.

    To disconnect an AWS account from a Stackdriver account, go to the Account settings > Monitored accounts page of the Stackdriver Monitoring Console. Choose a connected AWS account and click Remove from account from its menu.

  • Only Stackdriver accounts in the Premium service tier can connect to AWS accounts. This Quickstart begins by creating a new Stackdriver account that has a 30-day free trial of the Premium Tier, so you do not incur any charges. For more information, see Stackdriver pricing.

Connecting to an AWS account

To use your AWS account with Stackdriver, you must connect your AWS account to a Stackdriver account, which is a Google Cloud Platform project belonging to you that contains special Stackdriver monitoring and billing information.

To let Stackdriver monitor resources in your AWS account, you must grant your Stackdriver account read-only access to your AWS account. You do this by creating a role in AWS IAM and by passing the role to Stackdriver. You can revoke the role at any time.

To let applications on AWS send metrics, logs, and service requests to Stackdriver, you must grant those applications access to your Stackdriver account. You do this by creating a service account in Google Cloud IAM and by passing the service account to the AWS applications or to their VM instances. You can revoke the service account at any time.

Creating an AWS role

To create the AWS role to authorize Stackdriver, do the following:

  1. Log in to your Amazon IAM console and click Roles in the left-side menu.
  2. Click Create New Role and do the following:

    • Select the role type Another AWS account.
    • For Account ID, enter 314658760392. This is Stackdriver's account.
    • Check the box, Require external ID.
    • For External ID, enter sd1172213.
    • Do not check Require MFA.
    • Click Next: Permissions.
  3. From the Policy name list, select ReadOnlyAccess, which is near the bottom of the long list:

    ReadOnlyAccess policy

  4. Click Next: Review and fill in the information:

    • For the Role name, enter a name such as GoogleStackdriver.
    • For the Role description, enter anything you wish.
    • The Trusted entities value should be the Stackdriver account number, and the Policies value should be ReadOnlyAccess.
  5. Click Create Role in the AWS IAM page.

  6. On the Summary page for your AWS role, copy the Role ARN so you can give it to Stackdriver. If you do not see the summary, click the name of your role (for example, GoogleStackdriver) in the list of AWS roles.

    You are now finished with AWS.

Connecting the Stackdriver account

This section shows you how to create a new Stackdriver account and connect your AWS account to it. Go to the Stackdriver Monitoring Console:

Go to the Stackdriver Monitoring Console

  1. If you are not asked to create a Stackdriver account immediately, then select Create Stackdriver account from the drop-down list of Stackdriver accounts at the top of the page. You see the Create your free Stackdriver account page:

    Create Stackdriver account

  2. In the text box, Google Cloud Platform Project, select New Project and enter a project name, such as stackdriver-aws-quickstart.

    Alternatively, you can select an existing Cloud Platform project that is not already a Stackdriver account.

  3. Click Create account. There is a pause while Stackdriver creates the new Cloud Platform project, which is also your Stackdriver account.

  4. When you see the page Add Google Cloud Platform projects to monitor, click Continue because you are not adding any Cloud Platform projects.

  5. You see the page Monitor AWS accounts. You see instructions for creating the AWS role for Stackdriver, but you already did that in the previous section, Creating an AWS role. Do the following in the form at the bottom of the page:

    1. For Role ARN, enter the value you copied from your AWS role.
    2. For Description of account, enter a description of your AWS account.
    3. Click Add AWS account. After a moment, Stackdriver confirms the connection.

    When you add your AWS account, Stackdriver creates a second Cloud Platform project, with a name that begins with AWS Link and an ID that begins aws-. This is the AWS connector project for your AWS account, and it is this project that is added to your Stackdriver account and represents the resources in your AWS account. For more information, see AWS Connector projects.

  6. Click Continue at the bottom of the page to finish building your Stackdriver account.

  7. You can skip through the following pages until you see Gathering information... and then Finished initial collection!. Your Stackdriver account is now created and your AWS account is connected to it.

To open your new Stackdriver account, click Launch monitoring.

Provisioning AWS VM instances

You can skip this section if you do not intend to run any applications that call Stackdriver or other Cloud Platform APIs, which includes the Stackdriver Monitoring and Logging agents.

Creating a GCP service account

GCP service accounts authorize your Amazon EC2 instances to run the Stackdriver Monitoring and Logging agents, and to request other GCP services.

You create service accounts for AWS in AWS connector projects in GCP, not in your Stackdriver account projects. You can find the name of your AWS connector project by going to your Stackdriver account's Account settings > Monitored projects page in the Stackdriver Monitoring Console.

A single service account can authorize many VM instances in the same AWS account.

To create the service account, go to the IAM & Admin > Service accounts page for your connector project in the Cloud Platform Console:

Go to the Service Accounts page

  1. Click Select a project. Choose the AWS Connector project (AWS Link...) created in the previous section. Click Open:

    Service Accounts

  2. In Service Accounts, click Create service account and enter the following information:

    • Service account name: Stackdriver agent authorization
    • Role: Add both Project > Editor and Logging > Logs Writer
    • Furnish a new private key: (checked)
    • Key type: JSON
    • Enable G Suite Domain-wide Delegation: (leave unchecked)

    Create service account

  3. Click Create. The service account's private-key file is downloaded to your workstation, with a name like Downloads/{project_name}-{key_id}.json. Save the location of the credentials file in variable CREDS on your workstation:

    CREDS="Downloads/{project_name}-{key_id}.json"
    

Adding a service account to a VM instance

After you add a GCP service account to a VM instance, you can run the Stackdriver agents or other software that uses Stackdriver or GCP services on that VM instance.

Copy the Stackdriver credentials file to /etc/google/auth/application_default_credentials.json on your EC2 instance:

  1. From your workstation, copy the credentials file to a temporary file:

    scp -i "$KEY" "$CREDS" "ec2-user@$NAME:temp.json"
    
  2. On your EC2 instance, move temp.json, to its final location:

    PRIVATE_KEY_FILE="/etc/google/auth/application_default_credentials.json"
    sudo mkdir -p /etc/google/auth
    sudo mv "$HOME/temp.json" "$PRIVATE_KEY_FILE"
    sudo chown root:root "$PRIVATE_KEY_FILE"
    sudo chmod 0400 "$PRIVATE_KEY_FILE"
    

(Optional) To verify your credentials, see Verifying private-key credentials.

Installing the agents

(Optional) Install the Stackdriver Monitoring and Logging agents by running the following commands on your EC2 instance:

curl -O https://repo.stackdriver.com/stack-install.sh
sudo bash stack-install.sh --write-gcm

curl -sSO https://dl.google.com/cloudagents/install-logging-agent.sh
sudo bash install-logging-agent.sh

To verify that the agents are running, use the following two commands:

ps ax | grep fluentd
ps ax | grep collectd

Expected output:

{process} ?    Sl   0:00 /opt/google-fluentd/embedded/bin/ruby /usr/sbin/google-fluentd ...
{process} ?    Ssl  0:00 /opt/stackdriver/collectd/sbin/stackdriver-collectd ...

Using Stackdriver services with AWS

This section shows you how to use Stackdriver services with your AWS account.

Creating uptime checks and alerting policies

Uptime checks verify that your web server is always accessible. The alerting policy controls who is notified if the uptime checks should fail:

  1. Go back to the Stackdriver Monitoring Console.

  2. If you see the invitation Create an Uptime Check on the dashboard, then click it. Otherwise, select Uptime Checks > Uptime Check Overview from the left menu and then click Add Uptime Check or Create an Uptime Check. You see the New Uptime Check panel:

    Create an uptime check

  3. Fill in the following fields for the uptime check:

    • Resource Type: Choose from the menu of available resources
    • Depending on the resource type, you might have other options.
  4. Click Test to verify your uptime check is working.

  5. Click Save. You see the following panel:

    Do you want an alerting policy

  6. Click Create Alerting Policy in the preceding panel.

  7. The Conditions section is already set up with your uptime check. You don't have to change it.

  8. In the Notifications section, click Add Notification and fill in your email address.

  9. In the Documentation section, click Add Documentation and enter: Stackdriver AWS Quickstart example.

  10. In the Name this policy section, you can accept the default Uptime Check Policy.

  11. Click Save Policy.

Creating dashboards

Display the metrics collected by Stackdriver Monitoring in your own charts and dashboards:

  1. In the top menu of Stackdriver Monitoring Console, select Dashboards > Create....

    Go to the Create Dashboard page

  2. Click Add Chart. In the Metric Type menu, select an AWS metric.

  3. Click Save.

  4. In the new dashboard, change Untitled Dashboard to Stackdriver AWS Quickstart dashboard.

Viewing your logs

Stackdriver Monitoring and Stackdriver Logging are closely integrated. In the Stackdriver Monitoring Console left-side menu, choose Logging > AWS Link.... You see the Logs Viewer for your AWS connector project, which also holds your AWS logs. Change the Logs Viewer focus to see the logs you want:

  • Select Google Project > All project_id in the first drop-down menu. You should see at least one audit log from setting up your AWS connector project:

    AWS Logs Viewer

  • If you installed the Stackdriver Monitoring agent on your supported AWS VM instances, you might see other log options.

Clean up

To avoid incurring charges to your Google Cloud Platform account for the resources used in this quickstart:

  1. Remove your Stackdriver charts and alerts, so that you won't get errors when you shut down your VM instance. In the Stackdriver Monitoring Console:

    1. Delete your alerting policy from Alerting > Policy Overview.
    2. Delete your uptime check from Alerting > Uptime Checks.
    3. Delete your charts from Dashboards > Stackdriver AWS Quickstart example
  2. In the Stackdriver Monitoring Console, go to the Account Settings page for your Stackdriver account, stackdriver-aws-quickstart. In the Monitored projects section, remove your AWS account.

  3. In your Amazon account, delete the AWS IAM role that you created for the Quickstart.

  4. In the Google Cloud Platform Console, delete your AWS connector project, AWS Link..., and your Stackdriver account project, stackdriver-aws-quickstart. You delete a project by selecting the project, going to the IAM & Admin > Settings page, and clicking Delete Project at the top of the page.

What's next

  • See Supported Metrics for a list of all the built-in metrics. There are over 500 metrics for Amazon AWS. If you want to create your own Stackdriver Monitoring metrics, see Custom metrics.

  • To use the Stackdriver Monitoring API, see the API reference.

  • For more information on logging and its relation to monitoring, see Stackdriver Logging.

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Stackdriver Monitoring