Managing database connections

This page provides best practices and language-specific code samples to help you create applications that use Cloud SQL database connections effectively.

These samples are excerpts from a complete App Engine application available to you on GitHub. Learn more.

Connection pools

A connection pool is a cache of database connections that are shared and reused to improve connection latency and performance. When your application needs a database connection, it borrows one from its pool temporarily; when the application is finished with the connection, it returns the connection to the pool, where it can be reused the next time the application needs a database connection.

Opening and closing connections

When you use a connection pool, you must open and close connections properly, so that your connections are always returned to the pool when you are done with them. Unreturned or "leaked" connections are not reused, which wastes resources and can cause performance bottlenecks for your application.

Python

cloud-sql/mysql/sqlalchemy/main.py 
# Preparing a statement before hand can help protect against injections.
stmt = sqlalchemy.text(
    "INSERT INTO votes (time_cast, candidate)" " VALUES (:time_cast, :candidate)"
)
try:
    # Using a with statement ensures that the connection is always released
    # back into the pool at the end of statement (even if an error occurs)
    with db.connect() as conn:
        conn.execute(stmt, time_cast=time_cast, candidate=team)
except Exception as e:
    # If something goes wrong, handle the error in this section. This might
    # involve retrying or adjusting parameters depending on the situation.
    # ...

Java

cloud-sql/mysql/servlet/src/main/java/com/example/cloudsql/IndexServlet.java 
// Using a try-with-resources statement ensures that the connection is always released back
// into the pool at the end of the statement (even if an error occurs)
try (Connection conn = pool.getConnection()) {

  // PreparedStatements can be more efficient and project against injections.
  String stmt = "INSERT INTO votes (time_cast, candidate) VALUES (?, ?);";
  try (PreparedStatement voteStmt = conn.prepareStatement(stmt);) {
    voteStmt.setTimestamp(1, now);
    voteStmt.setString(2, team);

    // Finally, execute the statement. If it fails, an error will be thrown.
    voteStmt.execute();
  }
} catch (SQLException ex) {
  // If something goes wrong, handle the error in this section. This might involve retrying or
  // adjusting parameters depending on the situation.
  // ...
}

Node.js

cloud-sql/mysql/mysql/server.js 
try {
  const stmt = 'INSERT INTO votes (time_cast, candidate) VALUES (?, ?)';
  // Pool.query automatically checks out, uses, and releases a connection
  // back into the pool, ensuring it is always returned successfully.
  await pool.query(stmt, [timestamp, team]);
} catch (err) {
  // If something goes wrong, handle the error in this section. This might
  // involve retrying or adjusting parameters depending on the situation.
  // ...
}

C#

cloud-sql/mysql/Controllers/HomeController.cs 
insertTimestamp = DateTime.UtcNow;
try
{
    using(var connection = new MySqlConnection(_connectionString.ConnectionString))
    { 
        connection.OpenWithRetry();
        using (var insertVoteCommand = connection.CreateCommand())
        {
            insertVoteCommand.CommandText =
                @"INSERT INTO votes (candidate, time_cast) VALUES (@candidate, @time_cast)";
            var candidate = insertVoteCommand.CreateParameter();
            candidate.ParameterName = "@candidate";
            candidate.DbType = DbType.String;
            candidate.Value = team;
            insertVoteCommand.Parameters.Add(candidate);
            var timeCast = insertVoteCommand.CreateParameter();
            timeCast.ParameterName = "@time_cast";
            timeCast.DbType = DbType.DateTime;
            timeCast.Value = insertTimestamp;
            insertVoteCommand.Parameters.Add(timeCast);
            await insertVoteCommand.ExecuteNonQueryAsync();
        }
    }
    return Content($"Vote successfully cast for '{team}' at time {insertTimestamp}!");
}
catch (Exception ex)
{
    // If something goes wrong, handle the error in this
    // section. This might involve retrying or adjusting
    // parameters depending on the situation.
    return StatusCode((int)HttpStatusCode.InternalServerError, ex);
}

Go

cloudsql/mysql/database-sql/cloudsql.go 
sqlInsert := "INSERT INTO votes(candidate, created_at, updated_at) VALUES(?, NOW(), NOW())"
if team == "TABS" || team == "SPACES" {
	if _, err := app.db.Exec(sqlInsert, team); err != nil {
		fmt.Fprintf(w, "unable to save vote: %s", err)
		return fmt.Errorf("DB.Exec: %v", err)
	}
	fmt.Fprintf(w, "Vote successfully cast for %s!\n", team)
}
return nil

Ruby

cloud-sql/mysql/activerecord/app/controllers/votes_controller.rb 
@vote = Vote.new candidate: candidate

# ActiveRecord creates and executes your SQL and automatically
# handles the opening and closing of the database connection.
if @vote.save
  render json: "Vote successfully cast for \"#{@vote.candidate}\" at #{@vote.time_cast} PST!"
else
  render json: @vote.errors, status: :unprocessable_entity
end

PHP

cloud_sql/mysql/pdo/src/Votes.php 
// Use prepared statements to guard against SQL injection.
$sql = "INSERT INTO votes (time_cast, candidate) VALUES (NOW(), :voteValue)";

try {
    $statement = $conn->prepare($sql);
    $statement->bindParam('voteValue', $value);

    $res = $statement->execute();
} catch (PDOException $e) {
    throw new RuntimeException(
        "Could not insert vote into database. The PDO exception was " .
        $e->getMessage(),
        $e->getCode(),
        $e
    );
}

Connection count

Every database connection uses client and server-side resources. In addition, Cloud SQL imposes overall connection limits that cannot be exceeded. Creating and using fewer connections reduces overhead and helps you stay under the connection limit.

Python

cloud-sql/mysql/sqlalchemy/main.py 
# Pool size is the maximum number of permanent connections to keep.
"pool_size": 5,
# Temporarily exceeds the set pool_size if no connections are available.
"max_overflow": 2,
# The total number of concurrent connections for your application will be
# a total of pool_size and max_overflow.

Java

cloud-sql/mysql/servlet/src/main/java/com/example/cloudsql/ConnectionPoolContextListener.java 
// maximumPoolSize limits the total number of concurrent connections this pool will keep. Ideal
// values for this setting are highly variable on app design, infrastructure, and database.
config.setMaximumPoolSize(5);
// minimumIdle is the minimum number of idle connections Hikari maintains in the pool.
// Additional connections will be established to meet this value unless the pool is full.
config.setMinimumIdle(5);

Node.js

cloud-sql/mysql/mysql/server.js 
// 'connectionLimit' is the maximum number of connections the pool is allowed
// to keep at once.
connectionLimit: 5,

C#

cloud-sql/mysql/Startup.cs 
// MaximumPoolSize sets maximum number of connections allowed in the pool.
connectionString.MaximumPoolSize = 5;
// MinimumPoolSize sets the minimum number of connections in the pool.
connectionString.MinimumPoolSize = 0;

Go

cloudsql/mysql/database-sql/cloudsql.go 

// Set maximum number of connections in idle connection pool.
dbPool.SetMaxIdleConns(5)

// Set maximum number of open connections to the database.
dbPool.SetMaxOpenConns(7)

Ruby

cloud-sql/mysql/activerecord/config/database.yml 
# 'pool' is the maximum number of permanent connections to keep.
pool: 5

PHP

PDO currently doesn't offer any functionality to configure connection limits.

Exponential backoff

If your application attempts to connect to the database and does not succeed, the database could be temporarily unavailable. In this case, sending too many simultaneous connection requests might waste additional database resources and increase the time needed to recover. Using exponential backoff prevents your application from sending an unresponsive number of connection requests when it can't connect to the database.

This retry only makes sense when first connecting, or when first grabbing a connection from the pool. If errors happen in the middle of a transaction, the application must do the retrying, and it must retry from the beginning of a transaction. So even if your pool is configured properly, the application might still see errors if connections are lost.

Python

cloud-sql/mysql/sqlalchemy/main.py 
# SQLAlchemy automatically uses delays between failed connection attempts,
# but provides no arguments for configuration.

Java

cloud-sql/mysql/servlet/src/main/java/com/example/cloudsql/ConnectionPoolContextListener.java 
// Hikari automatically delays between failed connection attempts, eventually reaching a
// maximum delay of `connectionTimeout / 2` between attempts.

Node.js

cloud-sql/mysql/mysql/server.js 
// The mysql module automatically uses exponential delays between failed
// connection attempts.

C#

cloud-sql/mysql/Startup.cs 
Policy
    .Handle<MySqlException>()
    .WaitAndRetry(new[]
    {
        TimeSpan.FromSeconds(1),
        TimeSpan.FromSeconds(2),
        TimeSpan.FromSeconds(5)
    })
    .Execute(() => connection.Open());

Go

The database/sql package currently doesn't offer any functionality to configure exponential backoff.

Ruby

cloud-sql/mysql/activerecord/config/database.yml 
# ActiveRecord automatically uses delays between failed connection attempts,
# but provides no arguments for configuration.

PHP

PDO currently doesn't offer any functionality to configure exponential backoff.

Connection timeout

There are many reasons why a connection attempt might not succeed. Network communication is never guaranteed, and the database might be temporarily unable to respond. Make sure your application handles broken or unsuccessful connections gracefully.

Python

cloud-sql/mysql/sqlalchemy/main.py 
# 'pool_timeout' is the maximum number of seconds to wait when retrieving a
# new connection from the pool. After the specified amount of time, an
# exception will be thrown.
"pool_timeout": 30,  # 30 seconds

Java

cloud-sql/mysql/servlet/src/main/java/com/example/cloudsql/ConnectionPoolContextListener.java 
// setConnectionTimeout is the maximum number of milliseconds to wait for a connection checkout.
// Any attempt to retrieve a connection from this pool that exceeds the set limit will throw an
// SQLException.
config.setConnectionTimeout(10000); // 10 seconds
// idleTimeout is the maximum amount of time a connection can sit in the pool. Connections that
// sit idle for this many milliseconds are retried if minimumIdle is exceeded.
config.setIdleTimeout(600000); // 10 minutes

Node.js

cloud-sql/mysql/mysql/server.js 
// 'connectTimeout' is the maximum number of milliseconds before a timeout
// occurs during the initial connection to the database.
connectTimeout: 10000, // 10 seconds
// 'acquireTimeout' is the maximum number of milliseconds to wait when
// checking out a connection from the pool before a timeout error occurs.
acquireTimeout: 10000, // 10 seconds
// 'waitForConnections' determines the pool's action when no connections are
// free. If true, the request will queued and a connection will be presented
// when ready. If false, the pool will call back with an error.
waitForConnections: true, // Default: true
// 'queueLimit' is the maximum number of requests for connections the pool
// will queue at once before returning an error. If 0, there is no limit.
queueLimit: 0, // Default: 0

C#

cloud-sql/mysql/Startup.cs 
// ConnectionTimeout sets the time to wait (in seconds) while
// trying to establish a connection before terminating the attempt.
connectionString.ConnectionTimeout = 15;

Go

The database/sql package currently doesn't offer any functionality to configure connection timeout. Timeout is configured at the driver level.

Ruby

cloud-sql/mysql/activerecord/config/database.yml 
# 'timeout' is the maximum number of seconds to wait when retrieving a
# new connection from the pool. After the specified amount of time, an
# ActiveRecord::ConnectionTimeoutError will be raised.
timeout: 5000

PHP

cloud_sql/mysql/pdo/src/app.php 
// Here we set the connection timeout to five seconds and ask PDO to
// throw an exception if any errors occur.
$connConfig = [
    PDO::ATTR_TIMEOUT => 5,
    PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
];

Ending a connection

A mysql user with the PROCESS privilege in Cloud SQL is able to execute a KILL statement on a connection of any other mysql user (except Cloud SQL administrative users).

You can list the connections to an instance using the mysql client and executing the SHOW PROCESSLIST command. Use the Id to end the connection. For example:

mysql> SHOW PROCESSLIST;
mysql> KILL 6;

Connection duration

Limiting a connection's lifetime can help prevent abandoned connections from accumulating. You can use the connection pool to limit your connection lifetimes.

Python

cloud-sql/mysql/sqlalchemy/main.py 
# 'pool_recycle' is the maximum number of seconds a connection can persist.
# Connections that live longer than the specified amount of time will be
# reestablished
"pool_recycle": 1800,  # 30 minutes

Java

cloud-sql/mysql/servlet/src/main/java/com/example/cloudsql/ConnectionPoolContextListener.java 
// maxLifetime is the maximum possible lifetime of a connection in the pool. Connections that
// live longer than this many milliseconds will be closed and reestablished between uses. This
// value should be several minutes shorter than the database's timeout value to avoid unexpected
// terminations.
config.setMaxLifetime(1800000); // 30 minutes

Node.js

The 'mysql' Node.js library currently doesn't offer any functionality to control the duration of a connection.

C#

cloud-sql/mysql/Startup.cs 
// ConnectionLifeTime sets the lifetime of a pooled connection
// (in seconds) that a connection lives before it is destroyed
// and recreated. Connections that are returned to the pool are
// destroyed if it's been more than the number of seconds
// specified by ConnectionLifeTime since the connection was
// created. The default value is zero (0) which means the
// connection always returns to pool.
connectionString.ConnectionLifeTime = 1800; // 30 minutes

Go

cloudsql/mysql/database-sql/cloudsql.go 

// Set Maximum time (in seconds) that a connection can remain open.
dbPool.SetConnMaxLifetime(1800)

Ruby

ActiveRecord currently doesn't offer any functionality to control the duration of a connection.

PHP

PDO currently doesn't offer any functionality to control the duration of a connection.

To see the complete application, click the link below.

Python

View the complete application for the Python programming language.

Java

View the complete application for the Java programming language.

Node.js

View the complete application for the Node.js programming language.

C#

View the complete application for the C# programming language.

Go

View the complete application for the Go programming language.

Ruby

View the complete application for the Ruby programming language.

PHP

View the complete application for the PHP programming language.

What's next