Viewing autoscaler logs

When you enable autoscaling, the autoscaler makes scaling decisions based on the options that you specify. Each scaling decision is logged by Stackdriver Logging. Read these logs in the Logs Viewer to better understand the scaling decisions that an autoscaler has made.

Using the Logs Viewer, you can see events related to:

  • Resizing an instance group. If the autoscaler resizes an instance group, the resize is described in the autoscaler logs.
  • Changes in autoscaler status. For example, if the autoscaler was working as expected but then encountered an issue, the change in status would be described in the autoscaler logs. Some of these common status messages are described in the section for Commonly returned status messages.

Before you begin

Viewing autoscaler logs

To view autoscaler logs, use the Google Cloud Console, the gcloud command-line tool, or the Stackdriver Logging API.

Console

  1. Go to the Logs Viewer page.

    Go to the Logs Viewer page

  2. Pull down the resource dropdown menu and select GCE Autoscaler.
  3. Next, expand the All logs dropdown. Choose autoscaler to see logs for autoscaling.

gcloud

Using the gcloud tool, run the beta logging read command to look up all logs related to autoscaling (including activity and audit logs). For example:

gcloud beta logging read "resource.type=gce_autoscaler" --limit 10 \
    --format json

To restrict the entries to just logs related to autoscaler resize actions, specify the logName parameter and filter by the field jsonPayload.newSize. Set the logName to:

`projects/[PROJECT_ID]/logs/compute.googleapis.com%2Fautoscaler`

For example:

gcloud beta logging read 'logName="projects/[PROJECT_ID]/logs/compute.googleapis.com%2Fautoscaler" AND jsonPayload.newSize:*' \
    --limit 10 --format json

For autoscaler status changes, use the logName parameter and filter by the field jsonPayload.new_status. Set the logName to:

`projects/[PROJECT_ID]/logs/compute.googleapis.com%2Fautoscaler`

For example:

gcloud beta logging read 'logName="projects/[PROJECT_ID]/logs/compute.googleapis.com%2Fautoscaler" AND jsonPayload.new_status:*' \
    --limit 10 --format json

API

Make a request to the Logging V2 API. The request body must contain the filter parameter to use when searching for logs and the project you want to view logs for. For example, to make a request to get a list of resize actions:

POST https://logging.googleapis.com/v2/entries:list

{
  "filter": "logName=\"projects\/[PROJECT_ID]\/logs\/compute.googleapis.com%2Fautoscaler\" AND jsonPayload.new_size:*",
  "pageSize": 10,
  "resourceNames": [
    "projects\/[PROJECT_ID]"
  ]
}

To make a request to get a list of status changes:

POST https://logging.googleapis.com/v2/entries:list

{
  "filter": "logName=\"projects\/[PROJECT_ID]\/logs\/compute.googleapis.com%2Fautoscaler\" AND jsonPayload.new_status:*",
  "pageSize": 10,
  "resourceNames": [
    "projects\/[PROJECT_ID]"
  ]
}

Resize logs

Logs related to resize actions describe information about the previous size and the new size of the instance group. These log entries are identifiable with the existence of the following fields in the JSON payload:

jsonPayload.new_size
jsonPayload.old_size

For example, in the following entry, the log describes changes from the earlier instance group of size 8 to a new instance group of size 10.

{
"insertId": "1l68z7sg4jw7kzo",
"jsonPayload": {
  "autoscaler_name": "example-autoscaler",
  "instance_group_name": "example-igm",
  "new_size": 10,
  "old_size": 8,
  "project_id": "myproject",
  "zone_name": "europe-west1-b"
},
"logName": "projects/myproject/logs/compute.googleapis.com%2Fautoscaler",
"receiveTimestamp": "2017-10-24T16:12:08.294439822Z",
"resource": {
  "labels": {
    "autoscaler_id": "1234567890123456789",
    "location": "europe-west1-b",
    "project_id": "myproject"
  },
  "type": "gce_autoscaler"
},
"severity": "INFO",
"timestamp": "2017-10-24T16:12:07.715125478Z"

}

Here is an explanation of each field in the log:

Property Value
insertId A unique identifier for the log entry.
jsonPayload The log entry payload, represented as a structure that is expressed as a JSON object.
new_size The new size that this group was autoscaled to. If the group was scaled up, the new size is bigger than the old size. If it was scaled down, the new size is smaller than the old size.
old_size The old size of this group before it was changed.
instance_group_name The name of the managed instance group this log entry applies to.
project_id The project ID of the instance group. This should always be the ID of the current project.
autoscaler_name The name of the autoscaler responsible for this action.
zone_name The zone of the autoscaler and instance group.
resource Details about the autoscaler.
type The type of resource this is. For autoscalers, this field is always gce_autoscaler.
labels Metadata about the autoscaler.
location The zone of the autoscaler.
autoscaler_id A server-generated numeric autoscaler ID.
project_id The project ID the autoscaler belongs to.
timestamp The time that the event described by the log entry occurred. This is a timestamp in RFC 3339 UTC "Zulu" format, accurate to nanoseconds.
severity The severity of this log. For autoscaler logs, this is always "INFO".
logName The resource name of the log to which this log entry belongs. For autoscaler resize logs, this is always projects/[PROJECT_ID]/logs/compute.googleapis.com%2Fautoscaler.
receiveTimestamp The time the log entry was received by Stackdriver Logging. This is a timestamp in RFC 3339 UTC "Zulu" format, accurate to nanoseconds.

Status change logs

When the status of the autoscaler changes, Compute Engine creates a log entry capturing the status change. These log entries are identifiable by the existence of the following fields in the JSON payload:

jsonPayload.old_status
jsonPayload.new_status

For example, the following entry describes a status change from OK to a new status:

The autoscaler is configured to scale based on a load balancing signal but
the instance group has not received any utilization data from the load
balancer or the utilization is constantly 0. Check that the load balancing
configuration is working

You can use these status messages to find out why your autoscaler is not behaving as you expect or to debug past issues you noticed. In this case, you might notice the new status message and check that your load balancing configuration is set up correctly.

{
"insertId": "ivho6kg4icqfio",
"jsonPayload": {
  "autoscaler_name": "example-autoscaler",
  "instance_group_name": "example-igm",
  "new_status": {
    "details": "The autoscaler is configured to scale based on a load
        balancing signal but the instance group has not received any
        utilization data from the load balancer or the utilization is
        constantly 0. Check that the load balancing configuration is working."
  },
  "old_status": {
    "details": "OK"
  },
  "project_id": "myproject",
  "zone_name": "us-east1-b"
},
"logName": "projects/myproject/logs/compute.googleapis.com%2Fautoscaler",
"receiveTimestamp": "2017-10-24T15:11:58.076196762Z",
"resource": {
  "labels": {
    "autoscaler_id": "1234567890123456789",
    "location": "us-east1-b",
    "project_id": "myproject"
  },
  "type": "gce_autoscaler"
},
"severity": "INFO",
"timestamp": "2017-10-24T15:11:57.619997893Z"

}

Here is an explanation of each field in the log:

Property Value
insertId A unique identifier for the log entry.
jsonPayload The log entry payload, represented as a structure that is expressed as a JSON object.
new_status The new autoscaler status. See some commonly returned status messages.
old_status The old autoscaler status before it was changed.
instance_group_name The name of the managed instance group this log entry applies to.
project_id The project ID of the instance group.
autoscaler_name The name of the autoscaler responsible for this action.
zone_name The zone of the autoscaler and instance group.
resource Details about the autoscaler.
type The type of resource this is. For autoscalers, this field is always gce_autoscaler.
labels Metadata about the autoscaler.
location The zone of the autoscaler.
autoscaler_id A server-generated numeric autoscaler ID.
project_id The project ID the autoscaler belongs to. This is always the current project.
timestamp The time the event described by the log entry occurred. This is a timestamp in RFC 3339 UTC "Zulu" format, accurate to nanoseconds.
severity The severity of this log. For autoscaler logs, this is always "INFO".
logName The resource name of the log to which this log entry belongs. For status change logs, this is always projects/[PROJECT_ID]/logs/compute.googleapis.com%2Fautoscaler.
receiveTimestamp The time the log entry was received by Stackdriver Logging. This is a timestamp in RFC 3339 UTC "Zulu" format, accurate to nanoseconds.

What's next