Reading and Writing Application Logs

Overview

When a request is sent to your app, a request log is automatically written by App Engine. During the handling of the request, your app can also write application logs. In this page, you'll learn how to write application logs from your application, how to view logging in the Google Cloud console, and how to understand the request log data that App Engine writes during the request.

For information about downloading log data, see Overview of logs exports.

Request logs vs application logs

There are two categories of log data: request logs and application logs. A request log is automatically written by App Engine for each request handled by your app, and contains information such as the project ID, HTTP version, and so forth. For a complete list of available properties for request logs, see RequestLogs. See also the request log table for descriptions of the request log fields.

Each request log contains a list of application logs (AppLogLine) associated with that request, returned in the RequestLogs.getAppLogLines() method. Each app log contains the time the log was written, the log message, and the log level.

Note: A request log can only hold 1000 app logs. If you add more than 1000, the older logs will not be retained.

Writing application logs

Important: Individual log entries from an application have a 16KB limit. Nothing beyond this limit will be written to the app log.

The App Engine Java SDK allows a developer to log the following levels of severity:

• SEVERE
• WARNING
• INFO
• CONFIG
• FINE
• FINER
• FINEST

The log levels, INFO, WARNING, and SEVERE, appear in the logs with no extra configuration. In order to add other levels of logging to your Java app, you need to add the appropriate system properties to your project's appengine-web.xml file, and you might also need to modify the logging.properties file to set the desired log level. Both of these files are created when you create a new App Engine Java project using Maven. You can find these files at the following locations:

To add log levels other than INFO, WARNING, or SEVERE, edit the appengine-web.xml file to add the following inside the <appengine-web-app> tags:

    <system-properties>
       <property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>
    </system-properties>

The default log level in logging.properties is INFO. To change the default log level for all classes in your app, edit the logging.properties file to change the level. For example you could change .level = INFO to .level = WARNING.

In your application code, write log messages using the java.util.logging.Logger API. The example below writes an Info log message:

import java.util.logging.Logger;
//...

public class MyClass {

  private static final Logger log = Logger.getLogger(MyClass.class.getName());
  log.info("Your information log message.");
  //....

When your app runs, App Engine records the messages and makes them available in the Logs Explorer.

Log URL format in the Google Cloud console

See the following sample URL for an example of the log URL format in the Google Cloud console:

https://console.cloud.google.com/logs?filters=request_id:000000db00ff00ff827e493472570001737e73686966746361727331000168656164000100

Note: The URL no longer includes the filter_type.

Reading logs in the console

Note: the logs described here are shown as visible in the Log Viewer.

To view logs written by apps running in the standard environment, use the Logs Explorer.

A typical App Engine log contains data in the Apache combined log format, along with some special App Engine fields, as shown in the following sample log:

192.0.2.0 - test [27/Jun/2014:09:11:47 -0700] "GET / HTTP/1.1" 200 414
"http://www.example.com/index.html"
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
"1-dot-calm-sylph-602.appspot.com" ms=195 cpu_ms=42 cpm_usd=0.000046
loading_request=1 instance=00c61b117cfeb66f973d7df1b7f4ae1f064d app_engine_release=

Understanding request log fields

The following table lists the fields in order of occurrence along with a description:

Field Order	Field Name	Always Present?	Description
1	Client address	Yes	Client IP address. Example: 192.0.2.0
2	RFC 1413 identity	No	RFC 1413 identity of the client. This is nearly always the character -
3	User	No	Present only if the app uses the Users API and the user is logged in. This value is the "nickname" portion of the Google Account, for example, if the Google Account is [e-mail], the nickname that is logged in this field is test.
4	Timestamp	Yes	Request timestamp. Example: [27/Jun/2014:09:11:47 -0700]
5	Request querystring	Yes	First line of the request, containing method, path, and HTTP version. Example: GET / HTTP/1.1
6	HTTP Status Code	Yes	Returned HTTP status code. Example: 200
7	Response size	Yes	Response size in bytes. Example: 414
8	Referrer path	No	If there is no referrer, the log contains no path, but only -. Example referrer path: "http://www.example.com/index.html".
9	User-agent	Yes	Identifies the browser and operating system to the web server. Example: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36
10	Hostname	Yes	The hostname used by the client to connect to the App Engine application. Example : (1-dot-calm-sylph-602.appspot.com)
11	Wallclock time	Yes	Total clock time in milliseconds spent by App Engine on the request. This time duration does not include time spent between the client and the server running the instance of your application. Example: ms=195.
12	CPU milliseconds	Yes	CPU milliseconds required to fulfill the request. This is the number of milliseconds spent by the CPU actually executing your application code, expressed in terms of a baseline 1.2 GHz Intel x86 CPU. If the CPU actually used is faster than the baseline, the CPU milliseconds can be larger than the actual clock time defined above. Example: cpu_ms=42
13	Exit code	No	Only present if the instance shut down after getting the request. In the format exit_code=XXX where XXX is a 3 digit number corresponding to the reason the instance shut down. The exit codes are not documented since they are primarily intended to help Google spot and fix issues.
14	Estimated cost	Yes	DEPRECATED. Estimated cost of 1000 requests just like this one, in USD. Example: cpm_usd=0.000046
15	Queue name	No	The name of the task queue used. Only present if request used a task queue. Example: queue_name=default
16	Task name	No	The name of the task executed in the task queue for this request. Only present if the request resulted in the queuing of a task. Example: task_name=7287390692361099748
17	Pending queue	No	Only present if a request spent some time in a pending queue. If there are many of these in your logs and/or the values are high, it might be an indication that you need more instances to serve your traffic. Example: pending_ms=195
18	Loading request	No	Only present if the request is a loading request. This means an instance had to be started up. Ideally, your instances should be up and healthy for as long as possible, serving large numbers of requests before being recycled and needing to be started again. Which means you shouldn't see too many of these in your logs. Example: loading_request=1.
19	Instance	Yes	Unique identifier for the instance that handles the request. Example: instance=00c61b117cfeb66f973d7df1b7f4ae1f064d
20	Version	Yes	The current App Engine release version used in production App Engine:

Quotas and limits

Your application is affected by the following logs-related quotas:

• Logs data retrieved via the Logs API.
• Log ingestion allotment and retention.

Quota for data retrieved

The first 100 megabytes of logs data retrieved per day via the Logs API calls are free. Data in excess of 100 megabytes results in charges of $0.12/GB.

Logs ingestion allotment

Logging for App Engine apps is provided by Google Cloud Observability. See Google Cloud Observability Pricing for more information on logging costs and limits. For long-term storage of logs, you can export logs from Google Cloud Observability to Cloud Storage, BigQuery, and Pub/Sub.