Cloud SQL logging

This page describes how to find and use Cloud Logging to view and query logs for your Cloud SQL instance.

Cloud SQL uses Cloud Logging. See the cloud logging documentation for complete information and review the Cloud SQL sample queries.

Viewing logs

You can view SQL queries executed on the database instance, including SQL queries executed directly by end users, in the general log file by enabling the general_log flag and setting the log_output flag to FILE.

To view logs for your Cloud SQL instance log entries:

Console

  1. In the Google Cloud Console, go to the Cloud Logging page.

    Go to Cloud Logging

  2. Select an existing Cloud SQL project at the top of the page.
  3. In the Query builder, add the following:
    • Resource: select Cloud SQL Database. In the dialog, select a Cloud SQL instance.
    • Log names: scroll to the Cloud SQL section and select appropriate log files for your instance. For example:
      • cloudsql.googleapis.com/mysql-general.log
      • cloudsql.googleapis.com/mysql.err
    • Severity: select a log level.
    • Time range: select a preset or create a custom range.

gcloud

Use the gcloud logging command to view log entries. In the example below, replace PROJECT_ID. The limit flag is an optional parameter that indicates the maximum number of entries to return.

gcloud logging read "resource.type=cloudsql_database" \
--project=PROJECT-ID \
--limit=10 \
--format=json

Viewing instance operations log

You can view the logs for an instance in the Operations pane. The Operations pane logs every operation performed on the instance with the following information:

  • The time the operation completed, reported in your local time zone.
  • The type of operation.
  • The status of the operation.
  • A message describing the outcome the operation.

If the operation fails, you can use the message to troubleshoot the problem.

To view an instance operations log:

  1. In the Google Cloud Console, go to the Cloud SQL Instances page.

    Go to Cloud SQL Instances

  2. Click the instance name to open its Overview page.
  3. Click Operations to change to the pane showing the operation log.
Note: The operations log does not include operations performed using external management tools, such as the mysql client. Only user management and password change operations performed using the Google Cloud Console, gcloud command-line tool, or the Cloud SQL Admin API appear in the operations log.

Viewing application logs

Applications that connect to Cloud SQL store their logs in different locations.

App Engine (flexible environment)

In Compute > App Engine > Services:

  • In the list of services, find your service.
  • Click on the Tools dropdown.
  • Select logs

In the Operations > Logging > Logs explorer section of Google Cloud Console, use the following query:

resource.type="gae_app"
resource.labels.module_id="default"

Cloud Run

View the logs in the Cloud Run Logs Explorer section of the Google Cloud Console. Note that Cloud Run reports only error messages from the Cloud SQL Auth proxy. Use a query like the following:

resource.type="cloud_run_revision"
resource.labels.service_name="$SERVICE_NAME"
resource.labels.revision_name="$REVISION_NAME"

Cloud SQL Auth proxy

In Operations > Logging > Logs explorer, use the following query:

log_id("appengine.googleapis.com/cloud_sql_proxy")

Troubleshooting

Click the links in the table for details:

For this problem... The issue might be... Try this...
Logging is using a lot of CPU and memory. Logging needs to be tuned. Try tuning logging resource usage.
Audit logs are not found. User authentication. Check user roles and permissions.
Operations information not found in logs. Audit logs are not enabled. Enable audit logging.
Logging is using a lot of disk space. Redo logs, binary logs, and general logs use disk space. Run these commands to get disk usage details.
Log files are hard to read. You'd rather view the logs as json or text. Use gcloud logging commands.

Logging is using a lot of CPU and memory

Logging is using a lot of CPU and memory.

The issue might be

Logging usage needs to be tuned.

Things to try

The log_statement flag can be set to none and the logging_collector flag can be set to off. If logging is still occurring, there may be other log-related flags that can be tuned. You can edit the instance to modify these flags.

Audit logging

You turned on audit logging for Cloud SQL but are unable to find any audit logs in Cloud Logging

The issue might be

Data-Access logs are only written if the operation is an authenticated user-driven API call that creates, modifies, or reads user-created data, or if the operation accesses configuration files or metadata of resources.

Things to try

Check the roles and permissions of the user performing the operations.

Operation information not found in logs

You want to find more information about an operation. For example, a user was deleted but you can't find out who did it. The logs show the operation started but do not provide any more information.

The issue might be

You must enable audit logging for detailed and personal identifying information (PII) like this to be logged.

Things to try

Enable audit logging in your project.

Logging is using a lot of disk space

You want to find out how much disk space MySQL log files are using.

The issue might be

There are three kinds of log files that use disk space: redo logs, general logs and binary logs.

Things to try:

Run these commands for details on each type of log file:

SHOW VARIABLES LIKE 'innodb_log_file%';

SELECT ROUND(SUM(LENGTH(argument)/POW(1024,2),2) AS GB from mysql.general_log;

SHOW BINARY LOGS;

Log files are hard to read

You find it hard to read the logs in the Logs Explorer.

The issue might be

You'd rather download the logs locally in either JSON or text format.

Things to try:

You can use the gcloud logging read command along with linux post-processing commands to download the logs.

To download as JSON: 

gcloud logging read "resource.type=cloudsql_database AND logName=projects/PROJECT-ID/logs/cloudsql.googleapis.com%2FLOGFILE-NAME" --format json --project=PROJECT-ID--freshness="1d" > downloaded-log.json

To download as TEXT: 

gcloud logging read "resource.type=cloudsql_database AND logName=projects/PROJECT-ID/logs/cloudsql.googleapis.com%2FLOGFILE-NAME" --format json --project=PROJECT-ID--freshness="1d"| jq -rnc --stream 'fromstream(1|truncate_stream(inputs)) | .textPayload' > downloaded-log.txt