Connecting from App Engine flexible environment to Cloud SQL

This page contains information and examples for connecting to a Cloud SQL instance from a service running in App Engine flexible environment.

Cloud SQL is a fully-managed database service that helps you set up, maintain, manage, and administer your relational databases in the cloud.

App Engine is a fully managed, serverless platform for developing and hosting web applications at scale. You can choose from several popular languages, libraries, and frameworks to develop your apps, then let App Engine flexible environment take care of provisioning servers and scaling your app instances based on demand.

Setting up a Cloud SQL instance

  1. Enable the Cloud SQL Admin API in the project you are connecting from, if you haven't already done so:

    Enable the API

  2. Create a Cloud SQL for MySQL instance.

    By default, Cloud SQL assigns a public IP address to a new instance. You also have the option to assign a private IP address. For more information about the connectivity options for both, see the Connecting Overview page.

Configuring App Engine flexible environment

The steps to configure App Engine flexible environment depend on the type of IP address you assigned to your Cloud SQL instance.

Public IP (default)

To configure App Engine flexible environment to enable connections to a Cloud SQL instance:

  1. Make sure that the instance created above has a public IP address. You can verify this on the Overview page for your instance in the Google Cloud Console. If you need to add one, see the Configuring public IP page for instructions.
  2. Get the INSTANCE_CONNECTION_NAME for your instance. This can be found on the Overview page for your instance in the Google Cloud Console. or by running the following command: gcloud sql instances describe [INSTANCE_NAME].
  3. Ensure that the service account your app is using to authenticate calls to Cloud SQL has the appropriate Cloud SQL roles and permissions.
    • The service account for your service needs one of the following IAM roles:
      • Cloud SQL Client (preferred)
      • Cloud SQL Editor
      • Cloud SQL Admin
      Or, you can manually assign both of the following IAM permissions:
      • cloudsql.instances.connect
      • cloudsql.instances.get
      For detailed instructions on adding IAM roles to a service account, see Granting Roles to Service Accounts.

    By default, your app will authorize your connections using the App Engine flexible environment service account. The service account is in the format service-PROJECT_NUMBER@gae-api-prod.google.com.iam.gserviceaccount.com.

    If the authorizing service account belongs to a different project than the Cloud SQL instance, the Cloud SQL Admin API and IAM permissions will need to be added for both projects.

  4. Update your project's app.yaml file with the option that works best. You can use a comma-separated list of instances to specify multiple options at once.

    Enabling a Unix domain socket

    To enable a Unix domain socket, add one of the following to your project's app.yaml file, depending on whether you are connecting to one or multiple instances:

    beta_settings:
      cloud_sql_instances: INSTANCE_CONNECTION_NAME
    

    beta_settings:
      cloud_sql_instances: INSTANCE_CONNECTION_NAME,INSTANCE_CONNECTION_NAME_2,...
    

    Enabling a TCP Port

    To enable a local TCP port, add one of the following to your project's app.yaml file, depending on whether you are connecting to one or multiple instances:

    beta_settings:
      cloud_sql_instances: INSTANCE_CONNECTION_NAME=tcp:PORT
    

    beta_settings:
      cloud_sql_instances: INSTANCE_CONNECTION_NAME_1=tcp:PORT_1,INSTANCE_CONNECTION_NAME_2=tcp:PORT_2,...
    

Private IP

In order to connect to your Cloud SQL instance over private IP, your App Engine flexible environment deployment must be in the same VPC network as your Cloud SQL instance. See the configuration documentation on Network Settings for instructions on how to specify a VPC network for your deployment.

Once deployed, your application will be able to connect directly using your instance's private IP address and port 3306.

Connecting to Cloud SQL

After you configure App Engine flexible environment, you can connect to your Cloud SQL instance. App Engine flexible environment provides a mechanism that connects using the Cloud SQL Proxy.

Public IP (default)

Private IP

Connecting with TCP

Connect directly using the private IP address and port 3306 for your instance.

Python

To see this snippet in the context of a web application, view the README on GitHub.

# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
# secrets secret.
db_user = os.environ["DB_USER"]
db_pass = os.environ["DB_PASS"]
db_name = os.environ["DB_NAME"]
db_host = os.environ["DB_HOST"]

# Extract host and port from db_host
host_args = db_host.split(":")
db_hostname, db_port = host_args[0], int(host_args[1])

pool = sqlalchemy.create_engine(
    # Equivalent URL:
    # mysql+pymysql://<db_user>:<db_pass>@<db_host>:<db_port>/<db_name>
    sqlalchemy.engine.url.URL(
        drivername="mysql+pymysql",
        username=db_user,  # e.g. "my-database-user"
        password=db_pass,  # e.g. "my-database-password"
        host=db_hostname,  # e.g. "127.0.0.1"
        port=db_port,  # e.g. 3306
        database=db_name,  # e.g. "my-database-name"
    ),
    **db_config
)

Node.js

To see this snippet in the context of a web application, view the README on GitHub.

const createTcpPool = async config => {
  // Extract host and port from socket address
  const dbSocketAddr = process.env.DB_HOST.split(':');

  // Establish a connection to the database
  return await mysql.createPool({
    user: process.env.DB_USER, // e.g. 'my-db-user'
    password: process.env.DB_PASS, // e.g. 'my-db-password'
    database: process.env.DB_NAME, // e.g. 'my-database'
    host: dbSocketAddr[0], // e.g. '127.0.0.1'
    port: dbSocketAddr[1], // e.g. '3306'
    // ... Specify additional properties here.
    ...config,
  });
};

Go

To see this snippet in the context of a web application, view the README on GitHub.

var (
	dbUser    = mustGetenv("DB_USER")     // e.g. 'my-db-user'
	dbPwd     = mustGetenv("DB_PASS")     // e.g. 'my-db-password'
	dbTcpHost = mustGetenv("DB_TCP_HOST") // e.g. '127.0.0.1' ('172.17.0.1' if deployed to GAE Flex)
	dbPort    = mustGetenv("DB_PORT")     // e.g. '3306'
	dbName    = mustGetenv("DB_NAME")     // e.g. 'my-database'
)

var dbURI string
dbURI = fmt.Sprintf("%s:%s@tcp(%s:%s)/%s?parseTime=true", dbUser, dbPwd, dbTcpHost, dbPort, dbName)

// dbPool is the pool of database connections.
dbPool, err := sql.Open("mysql", dbURI)
if err != nil {
	return nil, fmt.Errorf("sql.Open: %v", err)
}

// ...

return dbPool, nil

C#

To see this snippet in the context of a web application, view the README on GitHub.

            // Equivalent connection string:
            // "Uid=<DB_USER>;Pwd=<DB_PASS>;Host=<DB_HOST>;Database=<DB_NAME>;"
            var connectionString = new MySqlConnectionStringBuilder()
            {
                // The Cloud SQL proxy provides encryption between the proxy and instance.
                SslMode = MySqlSslMode.None,

                // Remember - storing secrets in plaintext is potentially unsafe. Consider using
                // something like https://cloud.google.com/secret-manager/docs/overview to help keep
                // secrets secret.
                Server = Environment.GetEnvironmentVariable("DB_HOST"),   // e.g. '127.0.0.1'
                // Set Host to 'cloudsql' when deploying to App Engine Flexible environment
                UserID = Environment.GetEnvironmentVariable("DB_USER"),   // e.g. 'my-db-user'
                Password = Environment.GetEnvironmentVariable("DB_PASS"), // e.g. 'my-db-password'
                Database = Environment.GetEnvironmentVariable("DB_NAME"), // e.g. 'my-database'
            };
            connectionString.Pooling = true;
            // Specify additional properties here.
            // ...
            DbConnection connection =
                new MySqlConnection(connectionString.ConnectionString);

Ruby

To see this snippet in the context of a web application, view the README on GitHub.

development:
  adapter: mysql2
  # Configure additional properties here
  username: <%= ENV["DB_USER"] %>  # e.g. "my-database-user"
  password: <%= ENV["DB_PASS"] %> # e.g. "my-database-password"
  database: <%= ENV.fetch("DB_NAME") { "vote_development" } %> 
  host: <%= ENV.fetch("DB_HOST") { "127.0.0.1" }%> # '172.17.0.1' if deployed to GAE Flex
  port: <%= ENV.fetch("DB_PORT") { 3306 }%> 

PHP

To see this snippet in the context of a web application, view the README on GitHub.

// // $username = 'your_db_user';
// // $password = 'yoursupersecretpassword';
// // $db_name = 'your_db_name';
// // $host = "127.0.0.1";

// Connect using TCP
$dsn = sprintf('mysql:dbname=%s;host=%s', $db_name, $host);

// Connect to the database.
$conn = new PDO($dsn, $username, $password, $conn_config);

Best practices and other information

You can use the Cloud SQL proxy when testing your application locally. See the quickstart for using the proxy for local testing for detailed instructions.

Connection Pools

Connections to underlying databases may be dropped, either by the database server itself, or by the underlying infrastructure. To mitigate this, we recommend that you use a client library that supports connection pools and automatic reconnection.

For more detailed examples on how to use connection pools, see Managing database connections.

Connection Limits

Cloud SQL imposes a maximum limit on concurrent connections, and these limits may vary depending on the database engine chosen (see Cloud SQL Quotas and Limits).

App Engine has the ability to automatically create more instances as load increases, which may cause you to exceed these limits. To avoid this issue, limit the maximum number of App Engine instances. For more information, see Scaling elements.

Each App Engine instance running in a standard environment cannot have more than 100 concurrent connections to an instance. For PHP 5.5 apps, the limit is 60 concurrent connections.

App Engine applications are subject to request time limits depending on usage and environment. For more information, see how instances are managed in App Engine standard and flexible environments.

App Engine applications are also subject to additional App Engine quotas and limits as discussed on the App Engine Quotas page.

API Quota Limits

App Engine provides a mechanism that connects using the Cloud SQL Proxy, which uses the Cloud SQL Admin API. API quota limits apply to the Cloud SQL Proxy.