When you enable autoscaling, the autoscaler makes scaling decisions based on the options that you specify. Each scaling decision is logged by Cloud Logging. Read these logs in the Logs Explorer to better understand the autoscaler's scaling decisions.
Using the Logs Explorer, you can see events related to:
- Autoscaler recommendation for resizing a managed instance group (MIG). If the autoscaler calculates a new recommended size for a MIG, the change in the recommended size is described in the autoscaler logs. See Resize log entry.
- Changes in autoscaler status. For example, if the autoscaler was working as expected but then encountered an issue, the change in status would be described in the autoscaler logs. See Status change log entry.
Before you begin
- Read the Autoscaling documentation.
- Have an active autoscaler for which you want to view logs.
- Review how to use the Logs Explorer.
-
If you haven't already, then set up authentication.
Authentication is
the process by which your identity is verified for access to Google Cloud services and APIs.
To run code or samples from a local development environment, you can authenticate to
Compute Engine by selecting one of the following options:
Select the tab for how you plan to use the samples on this page:
Console
When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.
gcloud
-
Install the Google Cloud CLI, then initialize it by running the following command:
gcloud init
- Set a default region and zone.
REST
To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.
Install the Google Cloud CLI, then initialize it by running the following command:
gcloud init
For more information, see Authenticate for using REST in the Google Cloud authentication documentation.
-
View autoscaler logs
To view autoscaler logs, use the Google Cloud console, the gcloud CLI, or REST.
Console
If you've turned on autoscaling, then you can see autoscaling logs in the Google Cloud console.
- Go to the Logs Explorer page.
- Pull down the Resource drop-down menu and select Autoscaler.
You can also navigate to the Logs Explorer page from your MIG as follows:
In the Google Cloud console, go to the Instance groups page.
Click the name of the MIG for which you want to view the logs.
Click Monitoring tab.
In the Monitoring tab, the Logs panel is at the bottom of the page. Open the panel by clicking the icon at the end of this panel.
To open the Logs explorer page, click the View in logs explorer icon in this panel. The log explorer opens with a default query to fetch all autoscaler logs of the MIG.
gcloud
To look up all logs related to autoscaling, use the
logging read command.
For example:
gcloud logging read "resource.type=autoscaler" --limit 10 \
--format json
To view the autoscaler logs of a specific MIG, specify the
instance_group_manager_name as follows:
gcloud logging read "resource.type=autoscaler AND \
resource.labels.instance_group_manager_name=example-igm" \
--limit 10 --format json
To view the log entries related to autoscaler
resize actions, specify the compute.autoscalers.resize method
name:
gcloud logging read "resource.type=autoscaler AND \
resource.labels.instance_group_manager_name=example-igm AND \
protoPayload.methodName=compute.autoscalers.resize" \
--limit 10 --format json
To view the log entries related to autoscaler
status changes, specify the compute.autoscalers.changeStatus
method name:
gcloud logging read "resource.type=autoscaler AND \
resource.labels.instance_group_manager_name=example-igm AND \
protoPayload.methodName=compute.autoscalers.changeStatus" \
--limit 10 --format json
REST
Make a request to the
Logging V2 API.
The request body must contain the filter parameter to use when searching
for logs and the project you want to view logs for. For example, to make
a request to get a list of resize actions of a specific MIG:
POST https://logging.googleapis.com/v2/entries:list
{
"filter": "resource.type=autoscaler AND
resource.labels.instance_group_manager_name=example-igm AND
protoPayload.methodName=compute.autoscalers.resize",
"pageSize": 10,
"resourceNames": [
"projects/example-project"
]
}
To make a request to get a list of status changes of a specific MIG:
POST https://logging.googleapis.com/v2/entries:list
{
"filter": "resource.type=autoscaler AND
resource.labels.instance_group_manager_name=example-igm AND
protoPayload.methodName=compute.autoscalers.changeStatus",
"pageSize": 10,
"resourceNames": [
"projects/example-project"
]
}
Resize log entry
When the autoscaler calculates a recommended size for your MIG, Compute Engine creates a log entry capturing the new and old recommended sizes. For zonal MIGs, the log entry also captures the reason for the change in recommended size.
The following is an example of a log entry for a zonal MIG. In this example,
based on predictive autoscaling the autoscaler calculated 3 VMs to achieve the
CPU utilization target of 35%. But, the maximum number of instances
configured in the autoscaling policy is only 2. So, the autoscaler limits the
recommended size to 2.
{
insertId: "1l68z7sg4jw7kzo"
logName: "projects/example-project/logs/cloudaudit.googleapis.com%2Fsystem_event"
protoPayload: {
@type: "type.googleapis.com/google.cloud.audit.AuditLog"
metadata: {
@type: "type.googleapis.com/autoscaler.AutoscalerSizeChangeExplanation"
autoscalingMode: "ON"
autoscalingReason: {
scalingLimit: {
limitPolicy: {
maxNumReplicas: 2
}
limitSize: 2
name: "MAX_INSTANCES"
}
scalingSignal: {
calculatedSize: 3
calculationDetails: {
servingSize: 1
signalTarget: 0.35
signalValue: 1.03533
}
name: "PREDICTED_CPU_UTILIZATION"
signalPolicy: {
coolDownPeriodSec: 15
cpuUtilization: {
predictiveMethod: "OPTIMIZE_AVAILABILITY"
utilizationTarget: 0.35
}
}
}
summary: "The autoscaler's recommended size changed from 1 to 2
because the autoscaler predicted that in 15 seconds the average CPU
utilization across 1 serving instance will be 103.533%, which is above
the utilization target of 35%. The calculated size was 3 to achieve
the target CPU utilization but the size was limited by the maximum
number of instances set in the autoscaling policy."
}
newSize: 2
oldSize: 1
}
methodName: "compute.autoscalers.resize"
resourceName: "projects/example-project/zones/us-east1-d/autoscalers/example-autoscaler"
serviceName: "compute.googleapis.com"
}
receiveTimestamp: "2022-06-13T22:34:43.045973046Z"
resource: {
labels: {
autoscaler_id: "1234567890123456789"
autoscaler_name: "example-autoscaler"
instance_group_manager_id: "1357908642148074125"
instance_group_manager_name: "example-igm"
location: "us-east1-d"
project_id: "example-project"
}
type: "autoscaler"
}
severity: "INFO"
timestamp: "2022-06-13T22:34:42.810216614Z"
}
The following table describes the fields from the log entry:
| Property | Value |
|---|---|
autoscalingMode |
The autoscaling mode
at the time when the entry was logged. Regardless of the mode, the
autoscaler calculates the
recommended size,
which is represented by newSize in the log. Depending on
the autoscaling mode, the autoscaler resizes the MIG as follows:
|
autoscalingReason |
Details about the reason for autoscaling. This field appears only for zonal MIGs. |
autoscalingReason.scalingLimit
|
Details about the autoscaling feature that limited the
calculatedSize, which is the size that autoscaler calculated
based on the autoscaling signal. The autoscaling features that can limit
the calculatedSize are maximum or minimum number of
instances, the scale-in controls,
or the stabilization period.
If no limit applies to calculatedSize, then the
scalingLimit field does not appear in the logs.
|
autoscalingReason.scalingLimit.limitPolicy
|
The configuration of the autoscaling
feature that limited the calculatedSize. The
limitPolicy field is not set when stabilization limits the
calculatedSize because stabilization is a
default feature of autoscaling.
|
autoscalingReason.scalingLimit.limitSize
|
The number of VMs based on the autoscaling
feature that limited the calculatedSize.
|
autoscalingReason.scalingLimit.name
|
The name of the autoscaling feature that limited the
calculatedSize. |
autoscalingReason.scalingSignal
|
Details about the autoscaling signal that is used to calculate the number of VMs the MIG should have. If the autoscaling policy has multiple signals, then the autoscaler considers the signal that requires the largest number of VMs. |
autoscalingReason.scalingSignal.calculatedSize
|
The number of VMs that the MIG should have based on the autoscaling
signal. If scalingLimit applies, then the autoscaler limits
calculatedSize to scalingLimit.limitSize.
|
autoscalingReason.scalingSignal.calculationDetails
|
Details used to determine the calculatedSize.
|
autoscalingReason.scalingSignal.name
|
The name of the autoscaling signal based on which the autoscaler
calculated the number of VMs, represented by calculatedSize
in the log.
|
autoscalingReason.scalingSignal.signalPolicy
|
The configuration of the autoscaling signal based on which the
autoscaler determined the calculatedSize.
|
autoscalingReason.summary
|
A description of the reason for autoscaling. |
newSize
|
The current recommended size of the MIG. Depending on the
autoscalingMode, the autoscaler resizes the MIG to the
newSize as follows:
|
oldSize
|
The previous recommended size of the MIG. |
For information about the other fields in the log entry, see Format of audit log entries.
Status change log entry
When the status of the autoscaler changes, Compute Engine creates a log entry capturing the old and new status messages. You can use these status messages to find out why your autoscaler is not behaving as you expect or to debug past issues you noticed. For some of the common status messages, see Commonly returned status messages.
The following fields in the
protoPayload describe the old and new statuses:
metadata.newStatus.details: The new status of the autoscaler.metadata.oldStatus.details: The previous status of the autoscaler.
The following is an example of a log entry that describes a status change. In this example, you might notice the new status message indicating problems with your load balancing configuration.
{
insertId: "ivho6kg4icqfio"
logName: "projects/example-project/logs/cloudaudit.googleapis.com%2Fsystem_event"
protoPayload: {
metadata: {
@type: "type.googleapis.com/autoscaler.AutoscalerStatusChange"
newStatus: {
details: "The autoscaler is configured to scale based on a load
balancing signal but the instance group has not received any
utilization data from the load balancer or the utilization is
constantly 0. Check that the load balancing configuration is
working"
}
oldStatus: {
details: "OK"
},
methodName: "compute.autoscalers.changeStatus"
resourceName: "projects/example-project/zones/us-east1-b/autoscalers/example-autoscaler"
serviceName: "compute.googleapis.com"
}
receiveTimestamp: "2022-06-08T03:54:12.332329320Z"
resource: {
labels: {
autoscaler_id: "1234567890123456789"
autoscaler_name: "example-autoscaler"
instance_group_manager_id: "1357908642148074125"
instance_group_manager_name: "example-igm"
location: "us-east1-b"
project_id: "example-project"
}
type: "autoscaler"
}
severity: "INFO"
timestamp: "2022-06-08T03:54:10.675416812Z"
}
}
For information about the other fields in the log entry, see Format of audit log entries.
What's next
- Update your autoscaler using information that you received from the logs.
- Learn how autoscaler makes decisions.
- Review autoscaling options.