Google Cloud for AWS Professionals: Monitoring

Updated October 31, 2018

This article describes the monitoring services that Amazon and Google provide in their respective cloud environments.

Monitoring services

Amazon Web Services (AWS) and Google Cloud both provide integrated monitoring services for their platforms. Amazon CloudWatch provides both logging and monitoring for AWS, while Cloud Logging and Cloud Monitoring provide logging and monitoring services, respectively, for Google Cloud. Both services provide automatic metrics for cloud services and can be extended through agents and custom metrics.

Service model comparison

The following table aps Amazon CloudWatch terminology and concepts to those of Cloud Monitoring.

Feature Amazon CloudWatch Cloud Monitoring
Account structure AWS account Workspace that can monitor 1–1000 Google Cloud projects
Types of metrics monitored AWS
On-premises metrics
Custom metrics
Google Cloud
AWS
Custom metrics
Monitoring scope Separate for each region Workspace that can monitor 1–1000 Google Cloud projects
Metrics collection Automatic for cloud services
CloudWatch agents
Custom metrics
CloudWatch Logs metrics
Automatic for cloud services
Monitoring agent
Monitoring agent for Prometheus
Custom metrics
Logs-based metrics
Metric data types Double value
Statistic sets
Gauge
Delta
Cumulative
Metrics aggregation Aggregated by account, AWS region, metric name, namespace, and dimensions Aggregated by workspace, group, project, resource type, metric name, and label
Retention Specific to metric data type 6 weeks
Dashboards Yes Yes
Alerts Alarms Alerting policies
Service availability checks Route 53 health check Uptime checks
Service access Console
AWS command-line interface (CLI)
CloudWatch API
AWS SDKs
Google Cloud Console
Cloud SDK
Cloud Monitoring API
Google Cloud client libraries

Account structure

Amazon CloudWatch collects metrics for services used in an AWS account. In Google Cloud's operations suite, you can add Google Cloud projects to a Workspace, which then collects metrics for those Google Cloud projects. Metrics from all of the added Google Cloud projects are available in Cloud Monitoring under that Workspace.

Types of metrics monitored

Amazon CloudWatch automatically monitors AWS cloud services such as Amazon EC2, Amazon S3, and Amazon Route 53. Cloud Monitoring automatically monitors Google Cloud services such as Compute Engine, Google Kubernetes Engine (GKE), and Cloud Storage. Both services offer the ability to collect additional metrics through agents and to write custom metrics. You can configure both the Amazon CloudWatch agent and the Cloud Monitoring agent to ingest metrics from third-party sources such as StatsD.

Monitoring scope

Amazon CloudWatch designates metrics by AWS account and region. Metrics are separated by AWS account and then by region. Metrics are aggregated within the combination of AWS account and AWS region.

Diagram that illustrates the aggregation of AWS metrics by region and account

Cloud Monitoring includes metrics from all Google Cloud projects that are added to a Workspace. Metrics are separated by the combination of the Workspace and Google Cloud project. You can aggregate metrics within the Cloud Monitoring account.

Diagram that illustrates the aggregation of Google Cloud's operations suite metrics by region and account

Metric collection

Amazon CloudWatch automatically collects metrics for AWS cloud services such as Amazon EC2, Amazon ECS, and AWS Lambda. Additionally, CloudWatch Agent provides the ability to collect more system-level metrics from EC2 instances and from on-premises servers. You can publish custom metrics to CloudWatch by using the CloudWatch agent, the AWS CLI, or an API.

Cloud Monitoring automatically collects metrics for Google Cloud services such as GKE, Cloud Functions, and Pub/Sub. You can use the Cloud Monitoring agent to collect system and application metrics from virtual machine instances. Additionally, the Google Cloud's operations suite Prometheus agent collects metrics that are exposed through Prometheus, a logging service. You can publish custom metrics to Cloud Monitoring by using the monitoring agent, the gcloud CLI, or the Cloud Monitoring API.

Metric data types

The Amazon CloudWatch and Cloud Monitoring metric data types are similar in structure. The biggest difference in metric structures is that Cloud Monitoring metrics provide several metric and value types, and Amazon CloudWatch accepts statistic sets.

The Amazon CloudWatch API accepts metrics that have a value type of Double. The API also accepts statistic sets that are aggregated metrics.

Cloud Monitoring metrics accepts three types of metrics—Gauge, Delta, and Cumulative—with six value types: Boolean, Integer, Floating-point, String, Money, and Distribution.

Metric aggregation

Amazon CloudWatch metrics are uniquely defined within a CloudWatch account and AWS region by a metric name, a namespace, and zero or more dimensions. You can perform filtering and aggregation by metric name, namespace, and dimensions.

Diagram that illustrates how aggregated metrics can be filtered by AWS

Cloud Monitoring metrics are uniquely defined within a Workspace by Google Cloud project, resource type, metric, and label. You can perform filtering and aggregation by Google Cloud project, resource type, metric, and label.

Cloud Monitoring also includes the concept of a group. Google Cloud's operations suite groups can be based on names, tags, regions, applications, and other criteria and are used to logically arrange applications, infrastructure, or environments for filtering and aggregation.

Diagram that illustrates how aggregated metrics can be filtered by Google Cloud's operations suite

Retention

Amazon CloudWatch

The length of time that Amazon CloudWatch retains each data point depends on the monitoring period that the data point represents.

Period Retention time
Less than 60 seconds (high-resolution custom metrics) 3 hours
60 seconds (1 minute) 15 days
300 seconds (5 minutes) 63 days
3600 seconds (1 hour) 455 days (15 months)

Cloud Monitoring

In Cloud Monitoring, all metric data is retained for six weeks.

Dashboards

Amazon CloudWatch and Cloud Monitoring dashboards both provide the ability to add custom metric charts to a dashboard and then filter the charts by specific time windows.

An Amazon CloudWatch dashboard is composed of one or more charts. Amazon CloudWatch dashboards can include separate graphs for services that are offered in separate regions. To create Amazon CloudWatch charts, you first select a region and then select metrics from those that are available within that region. You can add multiple metrics to a single chart, and edit and update the dashboard by using the UI or editing the source of the dashboard.

Cloud Monitoring dashboards can include charts for any set or for all Google Cloud projects within a Workspace. A Cloud Monitoring dashboard is composed of one or more charts. To create Cloud Monitoring charts, select a set of metrics along with the metric filters, grouping, and aggregation. You can add multiple metrics to a single chart. After you have added charts to a dashboard, you can use the dashboard itself to filter by resource or metric label.

Alerting

Both Amazon CloudWatch and Cloud Monitoring provide the ability to take actions based on metric values. In Amazon CloudWatch, these thresholds are called alarms, and in Cloud Monitoring they are called alerting policies.

Amazon CloudWatch alarms

Amazon CloudWatch alarms specify two things:

  • Metric conditions
  • Actions

Amazon CloudWatch metric conditions specify the metric and threshold for generating an alarm. The alarms can be in three different states: OK, ALARM, or INSUFFICIENT_DATA. Actions specify what to do when the alarm changes state.

You can use the following services with Amazon CloudWatch:

  • Amazon Simple Notification Service (SNS), which extends to first- and third-party integrations like PagerDuty and Slack
  • Amazon EC2 Auto Scaling
  • Amazon EC2

Cloud Monitoring alerting policies

Cloud Monitoring alerting policies specify three things:

  • Metric conditions
  • Optional notifications
  • Optional documentation

Cloud Monitoring metric conditions specify the metric threshold for generating an alert. When events trigger conditions in one of your alerting policies, Cloud Monitoring creates and displays an incident in the Cloud Monitoring console. Cloud Monitoring can also send notifications through email or to third-party notification services. The notifications include any documentation that has been configured when creating the alert. Users who respond to an incident and alert can acknowledge receipt of the notification, although the incident remains open until resources are no longer in an unhealthy state.

In Cloud Monitoring, the following notifications are available:

  • Email
  • Webhooks, which extends to first- and third-party integrations
  • PagerDuty (third party)
  • Slack (third party)

Service availability checks

AWS and Google Cloud's operations suite both provide tools to verify the availability of a service by accessing it from locations around the world. AWS uses a Route 53 health check, while Cloud Monitoring uses uptime checks. Both checks surface metrics that can be used in alarms, alerts, and dashboards.

Service access

The following table lists the methods of accessing logging services in Amazon CloudWatch and Cloud Monitoring.

Access method Amazon CloudWatch Cloud Monitoring
Web console Amazon CloudWatch console Cloud Console
Command line AWS CLI gcloud command-line tool
API CloudWatch API Monitoring API
Client libraries AWS SDKs Google Cloud client libraries

Costs

The following table compares the ways that Amazon CloudWatch and Cloud Monitoring charge for services. Both services offer usage-based pricing after a monthly free tier or free allotment is exhausted.

Product component Amazon CloudWatch pricing model Cloud Monitoring pricing model
Cloud services metrics Free Free
Custom metrics Count of created metrics Metric volume in mebibytes (MiB)
Agent-based metrics Count of created metrics Metric volume in MiB, which includes metrics from AWS resources
Dashboards Number of dashboards Free
Alarms and alerting Number of alarms Free
API calls Number of API calls Number of read API calls—write API calls are free

Amazon CloudWatch pricing details

Most AWS services report metrics automatically for free. Custom metrics, agent metrics, dashboards, alarms, and API calls are priced based on the number of metrics that are reported per month in excess of the monthly free tier.

Cloud Monitoring pricing details

All Google Cloud service metrics, such as metrics reported from GKE, Compute Engine, and Pub/Sub, are free. Custom metrics, AWS metrics, and agent metrics are charged based on the volume of metric data that exceeds your free monthly allotment. The use of the API for reading metric data is charged per API call after the free monthly allotment. Writing metric data through APIs is free. For more details, see Cloud Monitoring pricing.

What's next?

Check out the other Google Cloud for AWS Professionals articles: