This page describes how to monitor and get alerts from elevated latency in Google App Engine.
High latency for App Engine apps can occur for various reasons. For example, the app does not have optimized code, there aren't enough resources, there are busy machines, and so forth. This guide focuses on the latency generated by App Engine regardless of the reason.
It is important to understand that high latency can often lead to 5xx errors. Therefore, rapid detection of high latency can help you act in time and thus resolve application errors.
Setting up Monitoring and Alerting
Monitoring
If you want to create a customized graph to monitor your app's response latency, use Cloud Monitoring > Metrics Explorer to build your own graph.
In the Resource Type, select GAE Application and select Response latency in the Metric filter. If you have a specific service to check, you can also use Filters to filter by the specific module to see the response latency chart.
Now that you've set up your customized graph, let's take a look at the next section and learn how to set up notifications for latencies over a certain amount of seconds.
Alerting
To create an alerting policy, do the following:
- Set up notifications, create an alerting policy in Cloud Monitoring > Alerting and click on + CREATE POLICY.
Add a condition to your alert by clicking ADD CONDITION.
Select GAE Application as a target:
For example, if you wanted to get alerts for response latency, such as how long your App Engine app is taking to respond, select response latency.
You can also select the module ID if it's appropriate, and add a threshold and a notification channel. See Managing notification channels for instructions.
After you have finished creating the policy, you will see the policy in a list in the Alerting page. Click on the created policy to see the details, as shown in the following image:
Congratulations, you just created your first App Engine latency policy! After you create your policy, the app will send an alert to the selected channel if it detects an incident.
Debugging latency
Checking the App Engine latency dashboard
In general, when you want to see the latency of App Engine applications, go to App Engine > Dashboard and select the Latency graph.
In the latency graph, you can drill into the data by filtering the graphs by service, version, and interval.
In the previous image, notice how the application latency varies from a few milliseconds to almost 25s.
Understanding app latency
To understand now how long a particular request has taken, go to:
- Cloud Logging to view request logs and application logs.
- Cloud Trace to track how requests propagate.
Cloud Logging
In Cloud Logging you can find two categories of logs, request logs and application logs. Request logs are automatically written by App Engine for each request to your application and contains the HTTP method, response code, and so forth. Application logs are those that you write for your application.
To view request logs or application logs, see Viewing logs.
Cloud Trace
Cloud Trace is useful for understanding how requests propagate through your application. You can inspect detailed latency information for a single request or view aggregate latency for your entire application.
To view trace details in Cloud Trace, you can follow Find and explore traces. In the Trace explorer, as shown in the following image, you can use the filters to filter by your specific App Engine service and version.
