Connect using Cloud SQL Language Connectors

MySQL   |  PostgreSQL   |  SQL Server

The Cloud SQL Connectors are libraries that provide encryption and Identity and Access Management (IAM)-based authorization when connecting to a Cloud SQL instance. They can't provide a network path to a Cloud SQL instance if one is not already present.

Other ways to connect to a Cloud SQL instance include using a database client or the Cloud SQL Auth Proxy. See the About connection options page for more information about connecting to a Cloud SQL instance.

This page discusses the following Cloud SQL Connectors:

• The Cloud SQL Java Connector
• The Cloud SQL Python Connector (Open in Colab)
• The Cloud SQL Go Connector
• The Cloud SQL Node.js Connector

Benefits

Using a Cloud SQL connector provides the following benefits:

• IAM authorization: Uses IAM permissions to control who or what can connect to your Cloud SQL instances.
• Convenience: Removes the requirement to manage SSL certificates, configure firewall rules, or enable authorized networks.
• IAM database authentication: Provides support for Cloud SQL's automatic IAM database authentication feature.

Before you begin

• Enable the Cloud SQL Admin API.

  Enable the API

• Create a Cloud SQL instance, including configuring the default user.
  

  For more information about creating instances, see Create instances.

  For more information about configuring the default user, see Set the password for the default user account.

• Configure the roles and permissions required to connect to a Cloud SQL instance.
  Note: If the authorizing IAM principal belongs to a different project than the one that contains the Cloud SQL instance, then you must do the following:

  • Enable the Cloud SQL Admin API in both the projects.
  • Add the IAM permissions for the IAM principal in the project that contains the Cloud SQL instance.

Setup

Java

The Cloud SQL Java Connector is a library that provides IAM-based authorization and encryption when connecting to a Cloud SQL instance. It can not provide a network path to a Cloud SQL instance if one is not already present.

Install

For instructions on building and using the drivers for JDBC and R2DBC with the Cloud SQL Java Connector, see the following links:

• JDBC: Connecting to Cloud SQL using JDBC.
• R2DBC: Connecting to Cloud SQL using R2DBC.

For examples of this library being used in the context of an application, check out these sample applications.

Authenticate

This library uses Application Default Credentials to authenticate the connection to the Cloud SQL server.

To activate credentials locally, use the following gcloud command:

    gcloud auth application-default login
    

Connect with Intellij

In order to connect IntelliJ to your Cloud SQL instance, you will need to add the library as a jar with dependencies in the Additional Files section on the driver settings page. For example, prebuilt fat jars can be found on the Cloud SQL Java Connector Releases page for this purpose.

Python

The Cloud SQL Python Connector is a library that can be used alongside a database driver to allow users with sufficient permissions to connect to a Cloud SQL database without having to manually allowlist IPs or manage SSL certificates.

For interactive examples of using the Cloud SQL Python Connector, open the Cloud SQL Python Connector notebook.

The drivers that PostgreSQL supports are pg8000 and asyncpg.

Install

To install the latest release of the Cloud SQL Python Connector, use the pip install command and specify either the pg8000 or asyncpg driver for your database:

    pip install "cloud-sql-python-connector[pg8000]"
    

or

    pip install "cloud-sql-python-connector[asyncpg]"
    

Authenticate

This library uses Application Default Credentials to authenticate the connection to the Cloud SQL server.

To activate credentials locally, use the following gcloud command:

    gcloud auth application-default login
    

Go

The Cloud SQL Go connector is a Cloud SQL connector designed for use with the Go language. For improved security, this connector uses robust, manually authenticated TLS 1.3 encryption between the client connector and the server-side proxy, independent of the database protocol.

Install

You can install this repo with go get:

    go get cloud.google.com/go/cloudsqlconn
    

Node.js

The Node.js Connector is a library designed for use with the Node.js runtime that allows you to connect securely to your Cloud SQL instance.

Install

You can install the library with npm install:

    npm install @google-cloud/cloud-sql-connector
    

Use

Java

To see this snippet in the context of a web application, view the README on GitHub.

import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;
import javax.sql.DataSource;

public class ConnectorConnectionPoolFactory extends ConnectionPoolFactory {

  // Note: Saving credentials in environment variables is convenient, but not
  // secure - consider a more secure solution such as
  // Cloud Secret Manager (https://cloud.google.com/secret-manager) to help
  // keep secrets safe.
  private static final String INSTANCE_CONNECTION_NAME =
      System.getenv("INSTANCE_CONNECTION_NAME");
  private static final String INSTANCE_UNIX_SOCKET = System.getenv("INSTANCE_UNIX_SOCKET");
  private static final String DB_USER = System.getenv("DB_USER");
  private static final String DB_PASS = System.getenv("DB_PASS");
  private static final String DB_NAME = System.getenv("DB_NAME");

  public static DataSource createConnectionPool() {
    // The configuration object specifies behaviors for the connection pool.
    HikariConfig config = new HikariConfig();

    // The following URL is equivalent to setting the config options below:
    // jdbc:postgresql:///<DB_NAME>?cloudSqlInstance=<INSTANCE_CONNECTION_NAME>&
    // socketFactory=com.google.cloud.sql.postgres.SocketFactory&user=<DB_USER>&password=<DB_PASS>
    // See the link below for more info on building a JDBC URL for the Cloud SQL JDBC Socket Factory
    // https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket-factory#creating-the-jdbc-url

    // Configure which instance and what database user to connect with.
    config.setJdbcUrl(String.format("jdbc:postgresql:///%s", DB_NAME));
    config.setUsername(DB_USER); // e.g. "root", _postgres"
    config.setPassword(DB_PASS); // e.g. "my-password"

    config.addDataSourceProperty("socketFactory", "com.google.cloud.sql.postgres.SocketFactory");
    config.addDataSourceProperty("cloudSqlInstance", INSTANCE_CONNECTION_NAME);


    // The ipTypes argument can be used to specify a comma delimited list of preferred IP types
    // for connecting to a Cloud SQL instance. The argument ipTypes=PRIVATE will force the
    // SocketFactory to connect with an instance's associated private IP.
    config.addDataSourceProperty("ipTypes", "PUBLIC,PRIVATE");


    // ... Specify additional connection properties here.
    // ...

    // Initialize the connection pool using the configuration object.
    return new HikariDataSource(config);
  }
}

Python

See How to use this Connector for detailed instructions on using the library. View example connection test code on GitHub.

import os

from google.cloud.sql.connector import Connector, IPTypes
import pg8000

import sqlalchemy


def connect_with_connector() -> sqlalchemy.engine.base.Engine:
    """
    Initializes a connection pool for a Cloud SQL instance of Postgres.

    Uses the Cloud SQL Python Connector package.
    """
    # Note: Saving credentials in environment variables is convenient, but not
    # secure - consider a more secure solution such as
    # Cloud Secret Manager (https://cloud.google.com/secret-manager) to help
    # keep secrets safe.

    instance_connection_name = os.environ[
        "INSTANCE_CONNECTION_NAME"
    ]  # e.g. 'project:region:instance'
    db_user = os.environ["DB_USER"]  # e.g. 'my-db-user'
    db_pass = os.environ["DB_PASS"]  # e.g. 'my-db-password'
    db_name = os.environ["DB_NAME"]  # e.g. 'my-database'

    ip_type = IPTypes.PRIVATE if os.environ.get("PRIVATE_IP") else IPTypes.PUBLIC

    # initialize Cloud SQL Python Connector object
    connector = Connector()

    def getconn() -> pg8000.dbapi.Connection:
        conn: pg8000.dbapi.Connection = connector.connect(
            instance_connection_name,
            "pg8000",
            user=db_user,
            password=db_pass,
            db=db_name,
            ip_type=ip_type,
        )
        return conn

    # The Cloud SQL Python Connector can be used with SQLAlchemy
    # using the 'creator' argument to 'create_engine'
    pool = sqlalchemy.create_engine(
        "postgresql+pg8000://",
        creator=getconn,
        # ...
    )
    return pool

Go

See Usage for detailed instructions on using the library. View example connection test code on GitHub.

package cloudsql

import (
	"context"
	"database/sql"
	"fmt"
	"log"
	"net"
	"os"

	"cloud.google.com/go/cloudsqlconn"
	"github.com/jackc/pgx/v5"
	"github.com/jackc/pgx/v5/stdlib"
)

func connectWithConnector() (*sql.DB, error) {
	mustGetenv := func(k string) string {
		v := os.Getenv(k)
		if v == "" {
			log.Fatalf("Fatal Error in connect_connector.go: %s environment variable not set.\n", k)
		}
		return v
	}
	// Note: Saving credentials in environment variables is convenient, but not
	// secure - consider a more secure solution such as
	// Cloud Secret Manager (https://cloud.google.com/secret-manager) to help
	// keep passwords and other secrets safe.
	var (
		dbUser                 = mustGetenv("DB_USER")                  // e.g. 'my-db-user'
		dbPwd                  = mustGetenv("DB_PASS")                  // e.g. 'my-db-password'
		dbName                 = mustGetenv("DB_NAME")                  // e.g. 'my-database'
		instanceConnectionName = mustGetenv("INSTANCE_CONNECTION_NAME") // e.g. 'project:region:instance'
		usePrivate             = os.Getenv("PRIVATE_IP")
	)

	dsn := fmt.Sprintf("user=%s password=%s database=%s", dbUser, dbPwd, dbName)
	config, err := pgx.ParseConfig(dsn)
	if err != nil {
		return nil, err
	}
	var opts []cloudsqlconn.Option
	if usePrivate != "" {
		opts = append(opts, cloudsqlconn.WithDefaultDialOptions(cloudsqlconn.WithPrivateIP()))
	}
	d, err := cloudsqlconn.NewDialer(context.Background(), opts...)
	if err != nil {
		return nil, err
	}
	// Use the Cloud SQL connector to handle connecting to the instance.
	// This approach does *NOT* require the Cloud SQL proxy.
	config.DialFunc = func(ctx context.Context, network, instance string) (net.Conn, error) {
		return d.Dial(ctx, instanceConnectionName)
	}
	dbURI := stdlib.RegisterConnConfig(config)
	dbPool, err := sql.Open("pgx", dbURI)
	if err != nil {
		return nil, fmt.Errorf("sql.Open: %w", err)
	}
	return dbPool, nil
}

Node.js

For detailed instructions on using the library, see Usage.

const Knex = require('knex');
const {Connector} = require('@google-cloud/cloud-sql-connector');

// In case the PRIVATE_IP environment variable is defined then we set
// the ipType=PRIVATE for the new connector instance, otherwise defaults
// to public ip type.
const getIpType = () =>
  process.env.PRIVATE_IP === '1' || process.env.PRIVATE_IP === 'true'
    ? 'PRIVATE'
    : 'PUBLIC';

// connectWithConnector initializes connection pool for a Cloud SQL instance
// of Postgres using the Cloud SQL Node.js Connector.
const connectWithConnector = async config => {
  // Note: Saving credentials in environment variables is convenient, but not
  // secure - consider a more secure solution such as
  // Cloud Secret Manager (https://cloud.google.com/secret-manager) to help
  // keep secrets safe.
  const connector = new Connector();
  const clientOpts = await connector.getOptions({
    instanceConnectionName: process.env.INSTANCE_CONNECTION_NAME,
    ipType: getIpType(),
  });
  const dbConfig = {
    client: 'pg',
    connection: {
      ...clientOpts,
      user: process.env.DB_USER, // e.g. 'my-user'
      password: process.env.DB_PASS, // e.g. 'my-user-password'
      database: process.env.DB_NAME, // e.g. 'my-database'
    },
    // ... Specify additional properties here.
    ...config,
  };
  // Establish a connection to the database.
  return Knex(dbConfig);
};

Enforce

By using connector enforcement, you can enforce using only the Cloud SQL Auth Proxy or Cloud SQL Language Connectors to connect to Cloud SQL instances. With connector enforcement, Cloud SQL rejects direct connections to the database.

If you're using a Private Service Connect-enabled instance, then there's a limitation. If the instance has connector enforcement enabled, then you can't create read replicas for the instance. Similarly, if the instance has read replicas, then you can't enable connector enforcement for the instance.

gcloud

To enforce using only the Cloud SQL Auth Proxy or Cloud SQL Language Connectors to connect to an instance, use the gcloud sql instances patch command:

gcloud sql instances patch INSTANCE_NAME \
--connector-enforcement=REQUIRED

Replace INSTANCE_NAME with the name of your Cloud SQL instance.

Note: By setting the value of the connector-enforcement parameter to REQUIRED, all connections must use the Cloud SQL Language Connectors, including the Cloud SQL Auth Proxy and the Java, Python, Go, and Node.js connectors.

REST

Before using any of the request data, make the following replacements:

• PROJECT_ID: the ID or project number of the Google Cloud project that contains the instance
• INSTANCE_NAME: the name of your Cloud SQL instance

Note: By setting the value of the connectorEnforcement parameter to REQUIRED, all connections must use the Cloud SQL Language Connectors, including the Cloud SQL Auth Proxy and the Java, Python, Go, and Node.js connectors.

HTTP method and URL:

PATCH https://sqladmin.googleapis.com/v1/projects/PROJECT_ID/instances/INSTANCE_NAME

Request JSON body:

{
  "kind": "sql#instance",
  "name": INSTANCE_NAME,
  "project": PROJECT_ID,
  "settings": {
  "connectorEnforcement": "REQUIRED",
  "kind": "sql#settings"
  }
}

To send your request, expand one of these options:

curl (Linux, macOS, or Cloud Shell)

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login , or by using Cloud Shell, which automatically logs you into the gcloud CLI . You can check the currently active account by running gcloud auth list.

Save the request body in a file named request.json, and execute the following command:

curl -X PATCH \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://sqladmin.googleapis.com/v1/projects/PROJECT_ID/instances/INSTANCE_NAME"

PowerShell (Windows)

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login . You can check the currently active account by running gcloud auth list.

Save the request body in a file named request.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method PATCH `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://sqladmin.googleapis.com/v1/projects/PROJECT_ID/instances/INSTANCE_NAME" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "kind": "sql#operation",
  "targetLink": "https://sqladmin.googleapis.com/v1/projects/PROJECT_ID/instances/INSTANCE_NAME",
  "status": "PENDING",
  "user": "[e-mail]",
  "insertTime": "2020-01-16T02:32:12.281Z",
  "operationType": "UPDATE",
  "name": "OPERATION_ID",
  "targetId": "INSTANCE_NAME",
  "selfLink": "https://sqladmin.googleapis.com/v1/projects/PROJECT_ID/operations/OPERATION_ID",
  "targetProject": "PROJECT_ID"
}

Troubleshoot

Driver versions

Make sure you are using the latest version of the Cloud SQL Connectors and your database driver to avoid incompatibilities. Some older versions of drivers are not supported.

Connection paths

The Cloud SQL Connectors provide authorization for connections, but they don't provide new paths to connectivity. For example, in order to connect to a Cloud SQL instance using a Private IP address, your application must already have VPC access.

Debug connection issues

For additional help with connection issues, see the Troubleshoot and Debug connection issues pages.

What's next

• See the Cloud SQL Java Connector home page on GitHub.
• See the Cloud SQL Python Connector home page on GitHub.
• Learn more about the Cloud SQL Auth Proxy.
• Learn more about IAM.
• Learn about options for support.