Diagnose issues

This page contains a list of the most frequent issues you might run into when working with Cloud SQL instances and steps you can take to address them. Also review the Known issues, Troubleshooting, and Support page pages.

View logs

To see information about recent operations, you can view the Cloud SQL instance operation logs or the PostgreSQL error logs.

Connection issues

See the Debugging connection issues page or the Connectivity section in the troubleshooting page for help with connection problems.

Instance issues

Backups

For the best performance for backups, keep the number of tables to a reasonable number.

For other backups issues, see the Backups section in the troubleshooting page.

Import and export

Imports into Cloud SQL and exports out of Cloud SQL can take a long time to complete, depending on the size of the data being processed. This can have the following impacts:

  • You can't stop a long-running Cloud SQL instance operation.
  • You can perform only one import or export operation at a time for each instance, and a long-running import or export blocks other operations, such as daily automated backups. Serverless exports allow you to run other operations, including editing instances, import, failover, and unblocking daily automated backups.

You can decrease the amount of time it takes to complete each operation by using the Cloud SQL import or export functionality with smaller batches of data.

For exports, you can perform the export from a read replica or use serverless export to minimize the impact on database performance and allow other operations to run on your instance while an export is running.

For other import and export issues, see the Import and export section in the troubleshooting page.

Disk space

If your instance reaches the maximum storage amount allowed, writes to the database fail. If you delete data, for example, by dropping a table, the space is freed, but it is not reflected in the reported Storage Used of the instance. You can run the VACUUM FULL command to recover unused space; note that write operations are blocked while the vacuum command is running. Learn more.

Suspended state

There are various reasons why Cloud SQL may suspend an instance, including:

  • Billing issues

    For example, if the credit card for the project's billing account has expired, the instance may be suspended. You can check the billing information for a project by going to the Google Cloud console billing page, selecting the project, and viewing the billing account information used for the project. After you resolve the billing issue, the instance returns to runnable status within a few hours.

  • Key issues with Cloud Key Management Service

    For example, if the key version of the Cloud KMS that's used to encrypt the user data in the Cloud SQL instance isn't present, access to the key is revoked, or if the key is deactivated or deleted. For more information, see Using customer-managed encryption keys (CMEK).

  • Legal issues

    For example, a violation of the Google Cloud Acceptable Use Policy may cause the instance to be suspended. For more information, see "Suspensions and Removals" in the Google Cloud Terms of Service.

  • Operational issues

    For example, if an instance is stuck in a crash loop (it crashes while starting or just after starting), Cloud SQL may suspend it.

While an instance is suspended, you can continue to view information about it or you can delete it, if billing issues triggered the suspension.

Cloud SQL users with Platinum, Gold, or Silver support packages can contact our support team directly about suspended instances. All users can use the earlier guidance along with the google-cloud-sql forum.

Performance

Overview

Cloud SQL supports performance-intensive workloads with up to 60,000 IOPS and no extra cost for I/O. IOPS and throughput performance depends on disk size, instance vCPU count, and I/O block size, among other factors.

Your instance's performance also depends on your choice of storage type and workload.

Learn more about:

Keep a reasonable number of database tables

Database tables consume system resources. A large number can affect instance performance and availability, and cause the instance to lose its SLA coverage. Learn more.

Enable query logs

You can log slow queries for Cloud SQL for PostgreSQL by setting log_min_duration_statement flag. The queries that ran for at least the specified amount of time will be logged. If this value is specified without units, it is taken as milliseconds. Navigate to Operations Logging to view the logs.

General performance tips

Make sure that your instance is not constrained on memory or CPU. For performance-intensive workloads, ensure your instance has at least 60 GB of memory . For slow database inserts, updates, or deletes, check the locations of the writer and database; sending data a long distance introduces latency.

Improve query performance by using Query Insights.

For slow database selects, consider the following:

  • Caching is important for read performance. Check the various blks_hit / (blks_hit + blks_read) ratios from the PostgreSQL Statistics Collector. Ideally, the ratio is above 99%. If not, consider increasing the size of your instance's RAM.
  • If your workload consists of CPU intensive queries (sorting, regular expressions, other complex functions), your instance might be throttled; add vCPUs.
  • Check the location of the reader and database - latency affects read performance even more than write performance.
  • Investigate non-Cloud SQL specific performance improvements, such as adding appropriate indexing, reducing data scanned, and avoiding extra round trips.

If you observe poor performance executing queries, use EXPLAIN to identify where to add indexes to tables to improve query performance. For example, make sure every field that you use as a JOIN key has an index on both tables.

Troubleshoot

For other Cloud SQL issues, see the troubleshooting page.

Error messages

For specific API error messages, see the Error messages reference page.

Troubleshoot customer-managed encryption keys (CMEK)

Cloud SQL administrator operations, such as create, clone, or update, might fail due to Cloud KMS errors, and missing roles or permissions. Common reasons for failure include a missing Cloud KMS key version, a disabled or destroyed Cloud KMS key version, insufficient IAM permissions to access the Cloud KMS key version, or the Cloud KMS key version is in a different region than the Cloud SQL instance. Use the following troubleshooting table to diagnose and resolve common problems.

Customer-managed encryption keys troubleshooting table

For this error... The issue might be... Try this...
Per-product, per-project service account not found The service account name is incorrect. Make sure you created a service account for the correct user project.

GO TO THE SERVICE ACCOUNTS PAGE.

Cannot grant access to the service account The user account does not have permission to grant access to this key version. Add the Organization Administrator role to your user or service account.

GO TO THE IAM ACCOUNTS PAGE

Cloud KMS key version is destroyed The key version is destroyed. If the key version is destroyed, you cannot use it to encrypt or decrypt data.
Cloud KMS key version is disabled The key version is disabled. Re-enable the Cloud KMS key version.

GO TO THE CRYPTO KEYS PAGE

Insufficient permission to use the Cloud KMS key The cloudkms.cryptoKeyEncrypterDecrypter role is missing on the user or service account you are using to run operations on Cloud SQL instances, or the Cloud KMS key version doesn't exist. In the Google Cloud project that hosts the key, add the cloudkms.cryptoKeyEncrypterDecrypter role to your user or service account.

GO TO THE IAM ACCOUNTS PAGE


If the role is already granted to your account, see Creating a key to learn how to create a new key version. See note.
Cloud KMS key is not found The key version does not exist. Create a new key version. See Creating a key. See note.
Cloud SQL instance and Cloud KMS key version are in different regions The Cloud KMS key version and Cloud SQL instance must be in the same region. It does not work if the Cloud KMS key version is in a global region or multi-region. Create a key version in the same region where you want to create instances. See Creating a key. See note.
Cloud KMS key version is restored but instance is still suspended The key version is disabled or doesn't grant proper permissions. Re-enable the key version, and grant the cloudkms.cryptoKeyEncrypterDecrypter role to your user or service account in the Google Cloud project that hosts the key.

Re-encryption troubleshooting table

For this error... The issue might be... Try this...
CMEK resource re-encryption failed because the Cloud KMS key is inaccessible. Please ensure that the primary key version is enabled and the permission is granted properly. The key version is disabled or doesn't grant proper permissions.

Re-enable the Cloud KMS key version:

GO TO THE CRYPTO KEYS PAGE

In the Google Cloud project that hosts the key, confirm the cloudkms.cryptoKeyEncrypterDecrypter role is granted to your user or service account:

GO TO THE IAM ACCOUNTS PAGE
CMEK resource re-encryption failed due to server internal error. Please retry later There is a server internal error. Retry re-encryption. For more information, see Re-encrypt an existing CMEK-enabled instance or replica