Google Cloud Platform for AWS Professionals: Monitoring

Updated October 31, 2018

This article describes the monitoring services that Amazon and Google provide in their respective cloud environments.

Monitoring services

Amazon Web Services (AWS) and Google Cloud Platform (GCP) both provide integrated monitoring services for their platforms. Amazon CloudWatch provides both logging and monitoring for AWS, while Stackdriver Logging and Stackdriver Monitoring provide logging and monitoring services, respectively, for GCP. Both services provide automatic metrics for cloud services and can be extended through agents and custom metrics.

Service model comparison

The following table aps Amazon CloudWatch terminology and concepts to those of Stackdriver Monitoring.

Feature Amazon CloudWatch Stackdriver Monitoring
Account structure AWS account Workspace that can monitor 1–1000 GCP projects
Types of metrics monitored AWS
On-premises metrics
Custom metrics
GCP
AWS
Custom metrics
Monitoring scope Separate for each region Workspace that can monitor 1–1000 GCP projects
Metrics collection Automatic for cloud services
CloudWatch agents
Custom metrics
CloudWatch Logs metrics
Automatic for cloud services
Monitoring agent
Monitoring agent for Prometheus
Custom metrics
Logs-based metrics
Metric data types Double value
Statistic sets
Gauge
Delta
Cumulative
Metrics aggregation Aggregated by account, AWS region, metric name, namespace, and dimensions Aggregated by workspace, group, project, resource type, metric name, and label
Retention Specific to metric data type 6 weeks
Dashboards Yes Yes
Alerts Alarms Alerting policies
Service availability checks Route 53 health check Uptime checks
Service access Console
AWS command-line interface (CLI)
CloudWatch API
AWS SDKs
Google Cloud Platform Console
Cloud SDK
Stackdriver Monitoring API
GCP client libraries

Account structure

Amazon CloudWatch collects metrics for services used in an AWS account. In Stackdriver, you can add GCP projects to a Workspace, which then collects metrics for those GCP projects. Metrics from all of the added GCP projects are available in Stackdriver Monitoring under that Workspace.

Types of metrics monitored

Amazon CloudWatch automatically monitors AWS cloud services such as Amazon EC2, Amazon S3, and Amazon Route 53. Stackdriver Monitoring automatically monitors GCP services such as Compute Engine, Google Kubernetes Engine (GKE), and Cloud Storage. Both services offer the ability to collect additional metrics through agents and to write custom metrics. You can configure both the Amazon CloudWatch agent and the Stackdriver Monitoring agent to ingest metrics from third-party sources such as StatsD.

Monitoring scope

Amazon CloudWatch designates metrics by AWS account and region. Metrics are separated by AWS account and then by region. Metrics are aggregated within the combination of AWS account and AWS region.

Diagram that illustrates the aggregation of AWS metrics by region and account

Stackdriver Monitoring includes metrics from all GCP projects that are added to a Workspace. Metrics are separated by the combination of the Workspace and GCP project. You can aggregate metrics within the Stackdriver Monitoring account.

Diagram that illustrates the aggregation of Stackdriver metrics by region and account

Metric collection

Amazon CloudWatch automatically collects metrics for AWS cloud services such as Amazon EC2, Amazon ECS, and AWS Lambda. Additionally, CloudWatch Agent provides the ability to collect more system-level metrics from EC2 instances and from on-premises servers. You can publish custom metrics to CloudWatch by using the CloudWatch agent, the AWS CLI, or an API.

Stackdriver Monitoring automatically collects metrics for GCP services such as GKE, Cloud Functions, and Cloud Pub/Sub. You can use the Stackdriver Monitoring agent to collect system and application metrics from virtual machine instances. Additionally, the Stackdriver Prometheus agent collects metrics that are exposed through Prometheus, a logging service. You can publish custom metrics to Stackdriver Monitoring by using the monitoring agent, the gcloud CLI, or the Stackdriver Monitoring API.

Metric data types

The Amazon CloudWatch and Stackdriver Monitoring metric data types are similar in structure. The biggest difference in metric structures is that Stackdriver Monitoring metrics provide several metric and value types, and Amazon CloudWatch accepts statistic sets.

The Amazon CloudWatch API accepts metrics that have a value type of Double. The API also accepts statistic sets that are aggregated metrics.

Stackdriver Monitoring metrics accepts three types of metrics—Gauge, Delta, and Cumulative—with six value types: Boolean, Integer, Floating-point, String, Money, and Distribution.

Metric aggregation

Amazon CloudWatch metrics are uniquely defined within a CloudWatch account and AWS region by a metric name, a namespace, and zero or more dimensions. You can perform filtering and aggregation by metric name, namespace, and dimensions.

Diagram that illustrates how aggregated metrics can be filtered by AWS

Stackdriver Monitoring metrics are uniquely defined within a Workspace by GCP project, resource type, metric, and label. You can perform filtering and aggregation by GCP project, resource type, metric, and label.

Stackdriver Monitoring also includes the concept of a group. Stackdriver groups can be based on names, tags, regions, applications, and other criteria and are used to logically arrange applications, infrastructure, or environments for filtering and aggregation.

Diagram that illustrates how aggregated metrics can be filtered by Stackdriver

Retention

Amazon CloudWatch

The length of time that Amazon CloudWatch retains each data point depends on the monitoring period that the data point represents.

Period Retention time
Less than 60 seconds (high-resolution custom metrics) 3 hours
60 seconds (1 minute) 15 days
300 seconds (5 minutes) 63 days
3600 seconds (1 hour) 455 days (15 months)

Stackdriver Monitoring

In Stackdriver Monitoring, all metric data is retained for six weeks.

Dashboards

Amazon CloudWatch and Stackdriver Monitoring dashboards both provide the ability to add custom metric charts to a dashboard and then filter the charts by specific time windows.

An Amazon CloudWatch dashboard is composed of one or more charts. Amazon CloudWatch dashboards can include separate graphs for services that are offered in separate regions. To create Amazon CloudWatch charts, you first select a region and then select metrics from those that are available within that region. You can add multiple metrics to a single chart, and edit and update the dashboard by using the UI or editing the source of the dashboard.

Stackdriver Monitoring dashboards can include charts for any set or for all GCP projects within a Workspace. A Stackdriver Monitoring dashboard is composed of one or more charts. To create Stackdriver Monitoring charts, select a set of metrics along with the metric filters, grouping, and aggregation. You can add multiple metrics to a single chart. After you have added charts to a dashboard, you can use the dashboard itself to filter by resource or metric label.

Alerting

Both Amazon CloudWatch and Stackdriver Monitoring provide the ability to take actions based on metric values. In Amazon CloudWatch, these thresholds are called alarms, and in Stackdriver Monitoring they are called alerting policies.

Amazon CloudWatch alarms

Amazon CloudWatch alarms specify two things:

  • Metric conditions
  • Actions

Amazon CloudWatch metric conditions specify the metric and threshold for generating an alarm. The alarms can be in three different states: OK, ALARM, or INSUFFICIENT_DATA. Actions specify what to do when the alarm changes state.

You can use the following services with Amazon CloudWatch:

  • Amazon Simple Notification Service (SNS), which extends to first- and third-party integrations like PagerDuty and Slack
  • Amazon EC2 Auto Scaling
  • Amazon EC2

Stackdriver Monitoring alerting policies

Stackdriver Monitoring alerting policies specify three things:

  • Metric conditions
  • Optional notifications
  • Optional documentation

Stackdriver Monitoring metric conditions specify the metric threshold for generating an alert. When events trigger conditions in one of your alerting policies, Stackdriver Monitoring creates and displays an incident in the Stackdriver Monitoring console. Stackdriver Monitoring can also send notifications through email or to third-party notification services. The notifications include any documentation that has been configured when creating the alert. Users who respond to an incident and alert can acknowledge receipt of the notification, although the incident remains open until resources are no longer in an unhealthy state.

In Stackdriver Monitoring, the following notifications are available:

  • Email
  • Webhooks, which extends to first- and third-party integrations
  • PagerDuty (third party)
  • Slack (third party)

Service availability checks

AWS and Stackdriver both provide tools to verify the availability of a service by accessing it from locations around the world. AWS uses a Route 53 health check, while Stackdriver Monitoring uses uptime checks. Both checks surface metrics that can be used in alarms, alerts, and dashboards.

Service access

The following table lists the methods of accessing logging services in Amazon CloudWatch and Stackdriver Monitoring.

Access method Amazon CloudWatch Stackdriver Monitoring
Web console Amazon CloudWatch console GCP Console
Command line AWS CLI gcloud command-line tool
API CloudWatch API Monitoring API
Client libraries AWS SDKs GCP client libraries

Costs

The following table compares the ways that Amazon CloudWatch and Stackdriver Monitoring charge for services. Both services offer usage-based pricing after a monthly free tier or free allotment is exhausted.

Product component Amazon CloudWatch pricing model Stackdriver Monitoring pricing model
Cloud services metrics Free Free
Custom metrics Count of created metrics Metric volume in mebibytes (MiB)
Agent-based metrics Count of created metrics Metric volume in MiB, which includes metrics from AWS resources
Dashboards Number of dashboards Free
Alarms and alerting Number of alarms Free
API calls Number of API calls Number of read API calls—write API calls are free

Amazon CloudWatch pricing details

Most AWS services report metrics automatically for free. Custom metrics, agent metrics, dashboards, alarms, and API calls are priced based on the number of metrics that are reported per month in excess of the monthly free tier.

Stackdriver Monitoring pricing details

All GCP service metrics, such as metrics reported from GKE, Compute Engine, and Cloud Pub/Sub, are free. Custom metrics, AWS metrics, and agent metrics are charged based on the volume of metric data that exceeds your free monthly allotment. The use of the API for reading metric data is charged per API call after the free monthly allotment. Writing metric data through APIs is free. For more details, see Stackdriver Monitoring pricing.

What's next?

Check out the other GCP for AWS Professionals articles:

Czy ta strona była pomocna? Podziel się z nami swoją opinią:

Wyślij opinię na temat...

Google Cloud Platform for AWS Professionals