Cloud SQL logging

This page describes how to find and use Cloud Logging to view and query logs for your Cloud SQL instance.

Cloud SQL uses Cloud Logging. See the cloud logging documentation for complete information and review the Cloud SQL sample queries.

Viewing logs

You can view SQL queries executed on the database instance, including SQL queries executed directly by end users, in the general log file by enabling the general_log flag and setting the log_output flag to FILE.

To view logs for your Cloud SQL instance log entries:

Console

  1. In the Google Cloud Console, go to the Cloud Logging page.

    Go to Cloud Logging

  2. Select an existing Cloud SQL project at the top of the page.
  3. In the Query builder, add the following:
    • Resource: select Cloud SQL Database. In the dialog, select a Cloud SQL instance.
    • Log names: scroll to the Cloud SQL section and select appropriate log files for your instance. For example:
      • cloudsql.googleapis.com/mysql-general.log
      • cloudsql.googleapis.com/mysql.err
    • Severity: select a log level.
    • Time range: select a preset or create a custom range.

gcloud

Use the gcloud logging command to view log entries. In the example below, replace PROJECT_ID. The limit flag is an optional parameter that indicates the maximum number of entries to return.

gcloud logging read "resource.type=cloudsql_database" \
--project=PROJECT-ID \
--limit=10 \
--format=json

Viewing instance operations log

You can view the logs for an instance in the Operations pane. The Operations pane logs every operation performed on the instance with the following information:

  • The time the operation completed, reported in your local time zone.
  • The type of operation.
  • The status of the operation.
  • A message describing the outcome the operation.

If the operation fails, you can use the message to troubleshoot the problem.

To view an instance operations log:

  1. In the Google Cloud Console, go to the Cloud SQL Instances page.

    Go to Cloud SQL Instances

  2. Click the instance name to open its Overview page.
  3. Click Operations to change to the pane showing the operation log.
Note: The operations log does not include operations performed using external management tools, such as the mysql client. Only user management and password change operations performed using the Google Cloud Console, gcloud command-line tool, or the Cloud SQL Admin API appear in the operations log.

Viewing application logs

Applications that connect to Cloud SQL store their logs in different locations.

App Engine (flexible environment)

In Compute > App Engine > Services:

  • In the list of services, find your service.
  • Click on the Tools dropdown.
  • Select logs

In the Operations > Logging > Logs explorer section of Google Cloud Console, use the following query:

resource.type="gae_app"
resource.labels.module_id="default"

Cloud Run

View the logs in the Cloud Run Logs Explorer section of the Google Cloud Console. Note that Cloud Run reports only error messages from the Cloud SQL Auth proxy. Use a query like the following:

resource.type="cloud_run_revision"
resource.labels.service_name="$SERVICE_NAME"
resource.labels.revision_name="$REVISION_NAME"

Cloud SQL Auth proxy

In Operations > Logging > Logs explorer, use the following query:

log_id("appengine.googleapis.com/cloud_sql_proxy")

Troubleshooting

Issue Troubleshooting
Logging is using a lot of CPU and memory on your Cloud SQL instance. Logging needs to be tuned.

The log_statement flag can be set to none and the logging_collector flag can be set to off. If logging is still occurring, there may be other log-related flags that can be tuned. You can edit the instance to modify these flags.
Audit logs are not found. Data-Access logs are only written if the operation is an authenticated user-driven API call that creates, modifies, or reads user-created data, or if the operation accesses configuration files or metadata of resources.
Operations information not found in logs. You want to find more information about an operation.

For example, a user was deleted but you can't find out who did it. The logs show the operation started but don't provide any more information. You must enable audit logging for detailed and personal identifying information (PII) like this to be logged.
Logging is using a lot of disk space. There are three kinds of log files that use disk space: redo logs, general logs and binary logs.

Connect to the database and run these commands for details on each type:

SHOW VARIABLES LIKE 'innodb_log_file%';

SELECT ROUND(SUM(LENGTH(argument)/POW(1024,2),2)
AS GB from mysql.general_log;

SHOW BINARY LOGS;
Log files are hard to read. You'd rather view the logs as json or text.You can use the gcloud logging read command along with linux post-processing commands to download the logs.

To download the logs as JSON:

gcloud logging read \
"resource.type=cloudsql_database \
AND logName=projects/PROJECT_ID \
/logs/cloudsql.googleapis.com%2FLOG_NAME" \
--format json \
--project=PROJECT_ID \
--freshness="1d" \
> downloaded-log.json

To download the logs as TEXT:

gcloud logging read \
"resource.type=cloudsql_database \
AND logName=projects/PROJECT_ID \
/logs/cloudsql.googleapis.com%2FLOG_NAME" \
--format text \
--project=PROJECT_ID \
--freshness="1d"| jq -rnc --stream 'fromstream(1|truncate_stream(inputs)) \
| .textPayload' \
> downloaded-log.txt