Connecting from App Engine standard environment to Cloud SQL

This page contains information and examples for connecting to a Cloud SQL instance from a service running in App Engine standard environment.

Cloud SQL is a fully-managed database service that helps you set up, maintain, manage, and administer your relational databases in the cloud.

App Engine is a fully managed, serverless platform for developing and hosting web applications at scale. You can choose from several popular languages, libraries, and frameworks to develop your apps, then let App Engine take care of provisioning servers and scaling your app instances based on demand.

Setting up a Cloud SQL instance

  1. Enable the Cloud SQL Admin API in the project you are connecting from, if you haven't already done so:

    Enable the API

  2. Create a Cloud SQL for PostgreSQL instance.

    By default, Cloud SQL assigns a public IP address to a new instance. You also have the option to assign a private IP address. For more information about the connectivity options for both, see the Connecting Overview page.

Configuring App Engine

The steps to configure App Engine standard environment depend on the type of IP address you assigned to your Cloud SQL instance.

Public IP (default)

To configure App Engine standard environment to enable connections to a Cloud SQL instance using public IP:

  • Make sure that the instance created above has a public IP address. You can verify this on the Overview page for your instance in the Google Cloud Console. If you need to add one, see the Configuring public IP page for instructions.
  • Get the INSTANCE_CONNECTION_NAME for your instance. This can be found on the Overview page for your instance in the Google Cloud Console. or by running the following command: gcloud sql instances describe [INSTANCE_NAME].
  • Ensure that the service account your app is using to authenticate calls to Cloud SQL has the appropriate Cloud SQL role and permissions.
    • The service account for your service needs one of the following IAM roles:
      • Cloud SQL Client (preferred)
      • Cloud SQL Editor
      • Cloud SQL Admin
      Or, you can manually assign the following IAM permissions:
      • cloudsql.instances.connect
      • cloudsql.instances.get
      For detailed instructions on adding IAM roles to a service account, see Granting Roles to Service Accounts.

    By default, your app will authorize your connections using an App Engine service account. The service account identity is in the format PROJECT_ID@appspot.gserviceaccount.com.

    If the authorizing service account belongs to a different project than the Cloud SQL instance, the Cloud SQL Admin API and IAM permissions will need to be added for both projects.

Private IP

A Serverless VPC Access connector handles communication to your VPC network. To connect directly with private IP, you need to:

  1. Make sure that the Cloud SQL instance created above has a private IP address. If you need to add one, see the Configuring private IP page for instructions.
  2. Create a Serverless VPC Access connector in the same VPC network as your Cloud SQL instance.
  3. Unless you're using Shared VPC, a connector must be in the same project and region as the resource that uses it, but the connector can send traffic to resources in different regions.

    Serverless VPC Access supports communication to VPC networks connected via Cloud VPN and VPC Network Peering.

    Serverless VPC Access does not support legacy networks.

  4. Configure App Engine to use the connector.
  5. Connect using your instance's private IP and port 5432.

Connecting to Cloud SQL

After you configure App Engine, you can connect to your Cloud SQL instance.

Public IP (default)

For public IP paths, App Engine provides connections with encryption/authorization with the Cloud SQL Proxy

Connecting with Unix sockets

Once correctly configured, you can connect your service to your Cloud SQL instance's Unix domain socket accessed on the environment's filesystem at the following path: /cloudsql/INSTANCE_CONNECTION_NAME. This can be found on the Overview page for your instance in the Google Cloud Console or by running the following command: gcloud sql instances describe [INSTANCE_NAME].

These connections are automatically encrypted without any additional configuration. The code samples shown below are extracts from more complete examples on the GitHub site. Click View on GitHub to see more.

Python

To see this snippet in the context of a web application, view the README on GitHub.

# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
# secrets secret.
db_user = os.environ["DB_USER"]
db_pass = os.environ["DB_PASS"]
db_name = os.environ["DB_NAME"]
db_socket_dir = os.environ.get("DB_SOCKET_DIR", "/cloudsql")
cloud_sql_connection_name = os.environ["CLOUD_SQL_CONNECTION_NAME"]

pool = sqlalchemy.create_engine(

    # Equivalent URL:
    # postgres+pg8000://<db_user>:<db_pass>@/<db_name>
    #                         ?unix_sock=<socket_path>/<cloud_sql_instance_name>/.s.PGSQL.5432
    sqlalchemy.engine.url.URL(
        drivername="postgresql+pg8000",
        username=db_user,  # e.g. "my-database-user"
        password=db_pass,  # e.g. "my-database-password"
        database=db_name,  # e.g. "my-database-name"
        query={
            "unix_sock": "{}/{}/.s.PGSQL.5432".format(
                db_socket_dir,  # e.g. "/cloudsql"
                cloud_sql_connection_name)  # i.e "<PROJECT-NAME>:<INSTANCE-REGION>:<INSTANCE-NAME>"
        }
    ),
    **db_config
)

Java

To see this snippet in the context of a web application, view the README on GitHub.

// Note: For Java users, the Cloud SQL JDBC Socket Factory can provide authenticated connections
// which is preferred to using the Cloud SQL Proxy with Unix sockets.
// See https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket-factory for details.

// The configuration object specifies behaviors for the connection pool.
HikariConfig config = new HikariConfig();

// The following URL is equivalent to setting the config options below:
// jdbc:postgresql:///<DB_NAME>?cloudSqlInstance=<CLOUD_SQL_CONNECTION_NAME>&
// socketFactory=com.google.cloud.sql.postgres.SocketFactory&user=<DB_USER>&password=<DB_PASS>
// See the link below for more info on building a JDBC URL for the Cloud SQL JDBC Socket Factory
// https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket-factory#creating-the-jdbc-url

// Configure which instance and what database user to connect with.
config.setJdbcUrl(String.format("jdbc:postgresql:///%s", DB_NAME));
config.setUsername(DB_USER); // e.g. "root", "postgres"
config.setPassword(DB_PASS); // e.g. "my-password"

config.addDataSourceProperty("socketFactory", "com.google.cloud.sql.postgres.SocketFactory");
config.addDataSourceProperty("cloudSqlInstance", CLOUD_SQL_CONNECTION_NAME);


// The ipTypes argument can be used to specify a comma delimited list of preferred IP types 
// for connecting to a Cloud SQL instance. The argument ipTypes=PRIVATE will force the 
// SocketFactory to connect with an instance's associated private IP. 
config.addDataSourceProperty("ipTypes", "PUBLIC,PRIVATE");

// ... Specify additional connection properties here.
// ...

// Initialize the connection pool using the configuration object.
DataSource pool = new HikariDataSource(config);

Node.js

To see this snippet in the context of a web application, view the README on GitHub.

const createUnixSocketPool = config => {
  const dbSocketPath = process.env.DB_SOCKET_PATH || '/cloudsql';

  // Establish a connection to the database
  return Knex({
    client: 'pg',
    connection: {
      user: process.env.DB_USER, // e.g. 'my-user'
      password: process.env.DB_PASS, // e.g. 'my-user-password'
      database: process.env.DB_NAME, // e.g. 'my-database'
      host: `${dbSocketPath}/${process.env.CLOUD_SQL_CONNECTION_NAME}`,
    },
    // ... Specify additional properties here.
    ...config,
  });
};

C#

To see this snippet in the context of a web application, view the README on GitHub.

// Equivalent connection string:
// "Server=<dbSocketDir>/<INSTANCE_CONNECTION_NAME>;Uid=<DB_USER>;Pwd=<DB_PASS>;Database=<DB_NAME>"
String dbSocketDir = Environment.GetEnvironmentVariable("DB_SOCKET_PATH") ?? "/cloudsql";
String instanceConnectionName = Environment.GetEnvironmentVariable("INSTANCE_CONNECTION_NAME");
var connectionString = new NpgsqlConnectionStringBuilder()
{
    // The Cloud SQL proxy provides encryption between the proxy and instance.
    SslMode = SslMode.Disable,
    // Remember - storing secrets in plain text is potentially unsafe. Consider using
    // something like https://cloud.google.com/secret-manager/docs/overview to help keep
    // secrets secret.
    Host = String.Format("{0}/{1}", dbSocketDir, instanceConnectionName),
    Username = Environment.GetEnvironmentVariable("DB_USER"), // e.g. 'my-db-user
    Password = Environment.GetEnvironmentVariable("DB_PASS"), // e.g. 'my-db-password'
    Database = Environment.GetEnvironmentVariable("DB_NAME"), // e.g. 'my-database'
};
connectionString.Pooling = true;
// Specify additional properties here.
return connectionString;

Go

To see this snippet in the context of a web application, view the README on GitHub.

var (
	dbUser                 = mustGetenv("DB_USER")                  // e.g. 'my-db-user'
	dbPwd                  = mustGetenv("DB_PASS")                  // e.g. 'my-db-password'
	instanceConnectionName = mustGetenv("INSTANCE_CONNECTION_NAME") // e.g. 'project:region:instance'
	dbName                 = mustGetenv("DB_NAME")                  // e.g. 'my-database'
)

socketDir, isSet := os.LookupEnv("DB_SOCKET_DIR")
if !isSet {
	socketDir = "/cloudsql"
}

var dbURI string
dbURI = fmt.Sprintf("user=%s password=%s database=%s host=%s/%s", dbUser, dbPwd, dbName, socketDir, instanceConnectionName)

// dbPool is the pool of database connections.
dbPool, err := sql.Open("pgx", dbURI)
if err != nil {
	return nil, fmt.Errorf("sql.Open: %v", err)
}

// ...

return dbPool, nil

Ruby

To see this snippet in the context of a web application, view the README on GitHub.

production:
  adapter: postgresql
  # Configure additional properties here.
  username: <%= ENV["DB_USER"] %>  # e.g. "my-database-user"
  password: <%= ENV["DB_PASS"] %> # e.g. "my-database-password"
  database: <%= ENV.fetch("DB_NAME") { "vote_production" } %>
  # If connecting via unix domain socket, specify the socket path as host
  host:  "<%= ENV.fetch("DB_SOCKET_DIR") { '/cloudsql' } %>/<%= ENV["INSTANCE_CONNECTION_NAME"] %>"

PHP

To see this snippet in the context of a web application, view the README on GitHub.

// $username = 'your_db_user';
// $password = 'yoursupersecretpassword';
// $dbName = 'your_db_name';
// $connectionName = getenv("CLOUD_SQL_CONNECTION_NAME");
// $socketDir = getenv('DB_SOCKET_DIR') ?: '/cloudsql';

// Connect using UNIX sockets
$dsn = sprintf(
    'pgsql:dbname=%s;host=%s/%s',
    $dbName,
    $socketDir,
    $connectionName
);

// Connect to the database.
$conn = new PDO($dsn, $username, $password, $connConfig);

Private IP

For private IP paths, your application will connect directly to your instance through Serverless VPC Access.

Connecting with TCP

Connect directly using the private IP address and port 5432 for your instance.

Python

To see this snippet in the context of a web application, view the README on GitHub.

# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
# secrets secret.
db_user = os.environ["DB_USER"]
db_pass = os.environ["DB_PASS"]
db_name = os.environ["DB_NAME"]
db_host = os.environ["DB_HOST"]

# Extract host and port from db_host
host_args = db_host.split(":")
db_hostname, db_port = host_args[0], int(host_args[1])

pool = sqlalchemy.create_engine(
    # Equivalent URL:
    # postgres+pg8000://<db_user>:<db_pass>@<db_host>:<db_port>/<db_name>
    sqlalchemy.engine.url.URL(
        drivername="postgresql+pg8000",
        username=db_user,  # e.g. "my-database-user"
        password=db_pass,  # e.g. "my-database-password"
        host=db_hostname,  # e.g. "127.0.0.1"
        port=db_port,  # e.g. 5432
        database=db_name  # e.g. "my-database-name"
    ),
    **db_config
)

Java

To see this snippet in the context of a web application, view the README on GitHub.

// Note: For Java users, the Cloud SQL JDBC Socket Factory can provide authenticated connections
// which is preferred to using the Cloud SQL Proxy with Unix sockets.
// See https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket-factory for details.

// The configuration object specifies behaviors for the connection pool.
HikariConfig config = new HikariConfig();

// The following URL is equivalent to setting the config options below:
// jdbc:postgresql:///<DB_NAME>?cloudSqlInstance=<CLOUD_SQL_CONNECTION_NAME>&
// socketFactory=com.google.cloud.sql.postgres.SocketFactory&user=<DB_USER>&password=<DB_PASS>
// See the link below for more info on building a JDBC URL for the Cloud SQL JDBC Socket Factory
// https://github.com/GoogleCloudPlatform/cloud-sql-jdbc-socket-factory#creating-the-jdbc-url

// Configure which instance and what database user to connect with.
config.setJdbcUrl(String.format("jdbc:postgresql:///%s", DB_NAME));
config.setUsername(DB_USER); // e.g. "root", "postgres"
config.setPassword(DB_PASS); // e.g. "my-password"

config.addDataSourceProperty("socketFactory", "com.google.cloud.sql.postgres.SocketFactory");
config.addDataSourceProperty("cloudSqlInstance", CLOUD_SQL_CONNECTION_NAME);


// The ipTypes argument can be used to specify a comma delimited list of preferred IP types 
// for connecting to a Cloud SQL instance. The argument ipTypes=PRIVATE will force the 
// SocketFactory to connect with an instance's associated private IP. 
config.addDataSourceProperty("ipTypes", "PUBLIC,PRIVATE");

// ... Specify additional connection properties here.
// ...

// Initialize the connection pool using the configuration object.
DataSource pool = new HikariDataSource(config);

Node.js

To see this snippet in the context of a web application, view the README on GitHub.

const createTcpPool = config => {
  // Extract host and port from socket address
  const dbSocketAddr = process.env.DB_HOST.split(':'); // e.g. '127.0.0.1:5432'

  // Establish a connection to the database
  return Knex({
    client: 'pg',
    connection: {
      user: process.env.DB_USER, // e.g. 'my-user'
      password: process.env.DB_PASS, // e.g. 'my-user-password'
      database: process.env.DB_NAME, // e.g. 'my-database'
      host: dbSocketAddr[0], // e.g. '127.0.0.1'
      port: dbSocketAddr[1], // e.g. '5432'
    },
    // ... Specify additional properties here.
    ...config,
  });
};

Go

To see this snippet in the context of a web application, view the README on GitHub.

var (
	dbUser    = mustGetenv("DB_USER") // e.g. 'my-db-user'
	dbPwd     = mustGetenv("DB_PASS") // e.g. 'my-db-password'
	dbTCPHost = mustGetenv("DB_HOST") // e.g. '127.0.0.1' ('172.17.0.1' if deployed to GAE Flex)
	dbPort    = mustGetenv("DB_PORT") // e.g. '5432'
	dbName    = mustGetenv("DB_NAME") // e.g. 'my-database'
)

var dbURI string
dbURI = fmt.Sprintf("host=%s user=%s password=%s port=%s database=%s", dbTCPHost, dbUser, dbPwd, dbPort, dbName)

// dbPool is the pool of database connections.
dbPool, err := sql.Open("pgx", dbURI)
if err != nil {
	return nil, fmt.Errorf("sql.Open: %v", err)
}

// ...

return dbPool, nil

C#

To see this snippet in the context of a web application, view the README on GitHub.

            // Equivalent connection string:
            // "Uid=<DB_USER>;Pwd=<DB_PASS>;Host=<DB_HOST>;Database=<DB_NAME>;"
            var connectionString = new NpgsqlConnectionStringBuilder()
            {
                // The Cloud SQL proxy provides encryption between the proxy and instance.
                SslMode = SslMode.Disable,

                // Remember - storing secrets in plain text is potentially unsafe. Consider using
                // something like https://cloud.google.com/secret-manager/docs/overview to help keep
                // secrets secret.
                Host = Environment.GetEnvironmentVariable("DB_HOST"),     // e.g. '127.0.0.1'
                // Set Host to 'cloudsql' when deploying to App Engine Flexible environment
                Username = Environment.GetEnvironmentVariable("DB_USER"), // e.g. 'my-db-user'
                Password = Environment.GetEnvironmentVariable("DB_PASS"), // e.g. 'my-db-password'
                Database = Environment.GetEnvironmentVariable("DB_NAME"), // e.g. 'my-database'
            };
            connectionString.Pooling = true;
            // Specify additional properties here.
            return connectionString;

Ruby

To see this snippet in the context of a web application, view the README on GitHub.

development:
  adapter: postgresql
  # Configure additional properties here.
  username: <%= ENV["DB_USER"] %>  # e.g. "my-database-user"
  password: <%= ENV["DB_PASS"] %> # e.g. "my-database-password"
  database: <%= ENV.fetch("DB_NAME") { "vote_development" } %>
  host: <%= ENV.fetch("DB_HOST") { "127.0.0.1" }%> # '172.17.0.1' if deployed to GAE Flex
  port: <%= ENV.fetch("DB_PORT")  { 5432 }%> 

PHP

To see this snippet in the context of a web application, view the README on GitHub.

// $username = 'your_db_user';
// $password = 'yoursupersecretpassword';
// $dbName = 'your_db_name';
// $dbHost = "127.0.0.1";

// Connect using TCP
$dsn = sprintf('pgsql:dbname=%s;host=%s', $dbName, $dbHost);

// Connect to the database
$conn = new PDO($dsn, $username, $password, $connConfig);

Best practices and other information

You can use the Cloud SQL Proxy when testing your application locally. See the quickstart for using the Cloud SQL Proxy for detailed instructions.

Connection Pools

Connections to underlying databases may be dropped, either by the database server itself, or by the underlying infrastructure. To mitigate this, we recommend that you use a client library that supports connection pools and automatic reconnection.

Connection Limits

Each App Engine instance running in a standard environment cannot have more than 100 concurrent connections to an instance. For PHP 5.5 apps, the limit is 60 concurrent connections. This limit applies per application instance. This means that each instance of the App Engine application can have that many connections to the database, and as it scales the total number of connections per deployment can grow. For more information, see Scaling elements.

You can limit the maximum number of connections used per instance by using a connection pool. For more detailed examples on how to limit the number of connections, see the Managing database connections page.

App Engine applications are subject to request time limits depending on usage and environment. For more information, see how instances are managed in App Engine standard and flexible environments.

App Engine applications are also subject to additional App Engine quotas and limits as discussed on the App Engine Quotas page.

API Quota Limits

App Engine provides a mechanism that connects using the Cloud SQL Proxy, which uses the Cloud SQL Admin API. API quota limits apply to the Cloud SQL Proxy.