Configuring private IP

This page describes how to configure a Cloud SQL instance to use private IP.

For information about how private IP works, as well as environment and management requirements, see Private IP.

Before you begin

API and IAM requirements

  • You must enable the Service Networking API for your project.
  • If you are using a Shared VPC network, you also need to enable this API for the host project.

  • In order to manage a private services access connection, the user should have the following IAM permissions. If you don't have the required permissions you can get insufficient-permissions errors.
    • compute.networks.list
    • compute.addresses.create
    • compute.addresses.list
    • servicenetworking.services.addPeering

    If you are using a Shared VPC network, you also need to add your user to the host project and assign the same permissions to the user on the host project.

Private services access

When you create a new VPC network in your project, you need to configure private services access to allocate an IP address range and create a private service connection. This allows resources in the VPC network to connect to Cloud SQL instances. The console provides a wizard to help you set up this configuration.

Configuring an instance to use private IP

You can configure a Cloud SQL instance to use private IP when you create the instance, or for an existing instance.

Configuring private IP for a new instance

To configure a Cloud SQL instance to use private IP when creating an instance:

Console

  1. Go to the Cloud SQL Instances page in the Google Cloud Console.

    Go to the Cloud SQL Instances page

  2. Click CREATE INSTANCE.
  3. In the Creation wizard, under Configuration Options, expand the Connectivity section.
  4. Select the Private IP checkbox.

    A drop-down list shows the available VPC networks in your project. If your project is the service project of a Shared VPC, VPC networks from the host project are also shown.

  5. Select the VPC network you want to use:
  6. If you see Private service connection required:

    1. Click SET UP CONNECTION.
    2. Under Allocate an IP range, choose one of the following options:
      • Select one or more existing IP ranges or create a new one from the dropdown. The dropdown includes previously allocated ranges, if there are any, or you can select ALLOCATE A NEW IP RANGE and enter a new range and name.
      • Use an automatically allocated IP range in your network.
    3. Click CONTINUE.
    4. Click CREATE CONNECTION.
    5. Verify that you see the Private service connection for network VPN_NAME has been successfully created status.
  7. Click Save.

gcloud

If you have not previously done so, follow the instructions below to configure private services access for Cloud SQL. Create your Cloud SQL instance, using the --network parameter to specify the name of your chosen VPC network, and the --no-assign-ip flag to disable public IP. The --network parameter value is in the format projects/PROJECT_ID/global/networks/VPC_NETWORK_NAME. The PROJECT_ID is the project ID of the VPC network. If the VPC network is a Shared VPC, it should be the id of the Shared VPC host project.
gcloud --project=[PROJECT_ID] beta sql instances create [INSTANCE_ID]
       --network=[VPC_NETWORK_NAME]
       --no-assign-ip

Configuring private IP for an existing instance

Configuring an existing Cloud SQL instance to use private IP causes the instance to restart, resulting in downtime.

To configure an existing instance to use private IP:

Console

  1. Go to the Cloud SQL Instances page in the Google Cloud Console.
    Go to the Cloud SQL Instances page
  2. Click the instance name to open its Overview page.
  3. Select the Connections tab.
  4. Select the Private IP checkbox.

    A drop-down list shows the available networks in your project. If your project is the service project of a Shared VPC, VPC networks from the host project are also shown.

  5. Select the VPC network you want to use:
  6. If you see Private service connection required:

    1. Click SET UP CONNECTION.
    2. Under Allocate an IP range, choose one of the following options:
      • Select one or more existing IP ranges or create a new one from the dropdown. The dropdown includes previously allocated ranges, if there are any, or you can select ALLOCATE A NEW IP RANGE and enter a new range and name.
      • Use an automatically allocated IP range in your network.
    3. Click CONTINUE.
    4. Click CREATE CONNECTION.
    5. Verify that you see the Private service connection for network VPN_NAME has been successfully created status.
  7. Click Save.

gcloud

If you have not previously done so, follow the instructions below to configure private services access for Cloud SQL. Update your Cloud SQL instance, using the --network parameter to specify the name of your chosen VPC network.

VPC_NETWORK_NAME is the name of your chosen VPC network, for example: my-vpc-network. The --network parameter value is in the format: https://www.googleapis.com/compute/alpha/projects/[PROJECT_ID]/global/networks/[VPC_NETWORK_NAME]

gcloud --project=[PROJECT_ID] beta sql instances patch [INSTANCE_ID]
       --network=[VPC_NETWORK_NAME]
       --no-assign-ip

Connecting to an instance using its Private IP

You use private services access to connect to Cloud SQL instances from Compute Engine or Google Kubernetes Engine instances in the same VPC network (defined here as internal sources) or from outside of that network (an external source).

Connecting from an internal source

To connect from a source in the same Google Cloud project as your Cloud SQL instance, such as the Cloud SQL Auth proxy running on a Compute Engine resource, that resource must be in the same VPC network as the Cloud SQL instance.

To connect from a serverless source, such as App Engine standard environment, App Engine flexible environment, Cloud Run, or Cloud Functions, your application or function connects directly to your instance through Serverless VPC Access without the Cloud SQL Auth proxy.

Connecting from an external source

You can connect from a client in an external network (on-premises network or VPC network) if the external network is connected to the VPC network to which your Cloud SQL instance is connected. To permit connections from an external network, do the following:

  1. Ensure your VPC network is connected to the external network using a Cloud VPN tunnel or a VLAN attachment for Dedicated Interconnect or Partner Interconnect.
  2. Ensure the BGP sessions on the Cloud Routers managing your Cloud VPN tunnels and Cloud Interconnect attachments (VLANs) have received specific prefixes (destinations) from your on-premises network. Default routes (destination 0.0.0.0/0) cannot be imported into the Cloud SQL VPC network because that network has its own local default route. Local routes for a destination are always used, even though the Cloud SQL peering is configured to import custom routes from your VPC network.
  3. Identify the peering connections produced by the private services connection:
    • cloudsql-mysql-googleapis-com
    • cloudsql-postgres-googleapis-com
    • servicenetworking-googleapis-com
  4. Update all of the peering connections to enable Export custom routes.
  5. Identify the allocated range used by the private services connection.
  6. Create a Cloud Router custom route advertisement for the allocated range on the Cloud Routers managing BGP sessions for your Cloud VPN tunnels or Cloud Interconnect attachments (VLANs).

Connecting from Cloud Shell

Cloud Shell doesn't currently support connecting to a Cloud SQL instance that has only a private IP address.

Connecting from non-RFC 1918 addresses

RFC 1918 specifies IP addresses that are assigned to be used internally (that is, within an organization) and will not route on the Internet. Specifically, these are:

  • 10.0.0.0/8
  • 172.16.0.0/12
  • 192.168.0.0/16

Connections to a Cloud SQL instance using a private IP address are automatically authorized for RFC 1918 address ranges. This way, all private clients can access the database without going through the proxy.

Non-RFC 1918 address ranges (addresses not within the RFC 1918 address space) must be configured as authorized networks.

To connect from a non-RFC 1918 address, you must set per-instance IP authorization to allow traffic from non-RFC 1918 address ranges.

For example, use a gcloud command like the following:

gcloud sql instances patch INSTANCE_NAME --authorized-networks 192.88.99.0/24,11.0.0.0/24

Cloud SQL doesn't learn Non-RFC 1918 subnet routes from your VPC by default. You need to update the network peering to Cloud SQL to export any Non-RFC 1918 routes.

gcloud compute networks peerings update cloudsql-mysql-googleapis-com --network=NETWORK --export-subnet-routes-with-public-ip --project=PROJECT
  • cloudsql-mysql-googleapis-com is a Private Service Connection name from your VPC Network page.

    Select your network, then look for the Private Service Connection section.

  • NETWORK is the name of your VPC network.

Troubleshooting

Click the links in the table for details:

For this problem... The issue might be... Try this...
Unauthorized to connect errors. There can be many root causes. See these things to try.
Network association failed. Service Networking API is not enabled in the project. Enable the Service Networking API in the project.
Remaining connection slots are reserved. The maximum number of connections has been reached. Increase the max_connections flag.
Set Service Networking service account as servicenetworking.serviceAgent role on consumer project. The Service Networking service account is not bound to the servicenetworking.serviceAgent role. Bind the Service Networking service account to the servicenetworking.serviceAgent role.
error x509: certificate is not valid for any names, but wanted to match project-name:db-name. Known issue: The Cloud SQL Proxy Dialer is not compatible with Go 1.15 at this time. Until fixed, see this discussion on GitHub, which includes a workaround.
Cannot parse certificates in some operating systems. Clients using x509 libraries from mac OS 11.0 (Big Sur) may fail to parse some certificates of sqlserver instances. This may be surfaced to the client as a generic error, like "cancelled". The workaround is to rotate the server certificate and recreate client certificates.
Cannot modify allocated ranges in CreateConnection. Please use UpdateConnection. VPC peerings were not updated after an allocated range was modified or removed. See Things to try for VPC peering update details.
Allocated IP range not found in network. VPC peerings were not updated after an allocated range was modified or removed. See Things to try for VPC peering update details.
ERROR: (gcloud.sql.connect) It seems your client does not have ipv6 connectivity and the database instance does not have an ipv4 address. Please request an ipv4 address for this database instance.. You're trying to connect to your private IP instance using Cloud Shell. Connecting from Cloud Shell to an instance with only a private IP address is not currently supported.

Aborted connection

You see the error message Got an error reading communication packets , or Aborted connection xxx to db: DB_NAME.

The issue might be

  • Networking instability.
  • No response to TCP keep-alive commands (either the client or the server is not responsive, possibly overloaded).
  • The database engine connection lifetime was exceeded and the server ends the connection.

Things to try

Applications must tolerate network failures and follow best practices such as connection pooling and retrying. Most connection poolers catch these errors where possible. Otherwise the application must either retry or fail gracefully.

For connection retry, we recommend the following methods:

  1. Exponential backoff. Increase the time interval between each retry, exponentially.
  2. Add randomized backoff also.
Combining these methods helps reduce throttling.


Unauthorized to connect

You see the error message Unauthorized to connect.

The issue might be

There can be many causes, as authorization occurs at many levels.

  • At the database level, the database user must exist and its password must match.
  • At the project level, the user may lack the correct IAM permissions.
  • At the Cloud SQL level, the root cause can depend on how you connect to your instance. If you are connecting directly to an instance through the public IP, the connection's source IP must be in the authorized network of the instance.

    Private IP connectivity is allowed by default, except when you are connecting from a non-RFC 1918 address. Non-RFC 1918 client addresses must be configured as authorized networks.

    Cloud SQL doesn't learn Non-RFC 1918 subnet routes from your VPC by default. You need to update the network peering to Cloud SQL to export any Non-RFC 1918 routes. For example:

    gcloud compute networks peerings update cloudsql-mysql-googleapis-com --network=NETWORK --export-subnet-routes-with-public-ip --project=PROJECT
    

    If you are connecting through the Cloud SQL Auth proxy, ensure that the IAM permissions are set up correctly.

  • At the network level, if the Cloud SQL instance is using public IP, the connection's source IP must be in an authorized network.

Things to try

  • Check the username and password.
  • Check the user's IAM roles and permissions.
  • If using public IP, make sure the source is in the authorized networks.

Network association failed

You see the error message Error: Network association failed due to the following error: set Service Networking service account as servicenetworking.serviceAgent role on consumer project.

The issue might be

The Service Networking API is not enabled in the project.

Things to try

Enable the Service Networking API in your project. If you see this error when you are trying to assign a private IP address to a Cloud SQL instance, and you are using a shared VPC, you also need to enable the Service Networking API for the host project.


Remaining connection slots are reserved

You see the error message FATAL: remaining connection slots are reserved for non-replication superuser connections.

The issue might be

The maximum number of connections has been reached.

Things to try

Edit the max_connections flag value.


Set Service Networking service account as servicenetworking.serviceAgent role on consumer project

You see the error message set Service Networking service account as servicenetworking.serviceAgent role on consumer project..

The issue might be

Service Networking service account is not bound to the servicenetworking.serviceAgent role.

Things to try

To mitigate this issue, try using these gcloud commands to bind the Service Networking service account to the servicenetworking.serviceAgent role.

gcloud beta services identity create --service=servicenetworking.googleapis.com --project=PROJECT_ID
gcloud projects add-iam-policy-binding PROJECT_ID --member="serviceAccount:service-PROJECT_NUMBER@service-networking.iam.gserviceaccount.com" --role="roles/servicenetworking.serviceAgent"

Error x509: certificate is not valid for any names

You see the error message error x509: certificate is not valid for any names, but wanted to match project-name:db-name

The issue might be ...

Known issue: The Cloud SQL Proxy Dialer is not compatible with Go 1.15 at this time.

Things to try

Until the bug is fixed, see this discussion on GitHub, which includes a workaround.


Cannot parse certificates in some operating systems

When you use x509 libraries from mac OS 11.0 (Big Sur), you may fail to parse the certificates of sqlserver instances. This may be surfaced as a generic error, like "cancelled."

Things to try

The bug is fixed and the new instances won't hit this problem. For old instances hitting this problem, rotate the server certificate and recreate client certificates.


Cannot modify allocated ranges in CreateConnection. Please use UpdateConnection

You see the error message Cannot modify allocated ranges in CreateConnection. Please use UpdateConnection, or The operation "operations/1234" resulted in a failure "Allocated IP range 'xyz' not found in network.

The issue might be ...

You get the first error when you attempt to make a connection again using a different reserved range.

You see the second error when the allocated range was modified but vpc-peerings was not updated.

Things to try

You need to modify the private connection. Use the following command, and make sure to use the --force argument:

gcloud services vpc-peerings update --network=VPC_NETWORK --ranges=ALLOCATED_RANGES --service=servicenetworking.googleapis.com --force

What's next