管理查询优化器

Cloud Spanner 通过查询优化器版本控制提供优化器控制。本指南介绍如何管理查询所用的查询优化器版本。

随着 Cloud Spanner 的查询优化器更新的发布,我们的目标是改进查询执行计划并改善查询性能。默认情况下,Cloud Spanner 会对每个数据库使用最新的查询优化器版本。借助查询优化器版本控制,您可以使用旧版优化器运行查询,从而获得可预测的性能。

列出支持的优化器版本

查询优化器版本是一个整数值,每次更新时递增 1。查询优化器的最新版本为 2

执行以下 SQL 语句以返回所有受支持的优化器版本的列表及其相应的发布日期。返回的最大版本号是支持的优化器最新版本。

SELECT * FROM SPANNER_SYS.SUPPORTED_OPTIMIZER_VERSIONS

Cloud Spanner 默认使用最新版本的优化器,当前为 2。使用本指南中描述的方法之一,针对您的情况替换此默认行为。

如需详细了解每个版本,请参阅查询优化器版本版本记录

版本替换优先顺序

Cloud Spanner 提供了多种更改优化器版本的方法。例如,您可以为特定查询设置版本,或在进程或查询级层配置客户端库中的版本。在您通过多种方式设置版本后,系统会应用以下优先顺序。(选择链接以转到本文档中的相应部分)。

Cloud Spanner 默认 ← 数据库选项客户端应用环境变量客户端查询语句提示

上述优先顺序的含义是:创建数据库时,数据库将使用 Cloud Spanner 默认优化器版本,该版本始终为最新版本。使用上面列出的方法之一设置优化器版本的优先级高于其左侧的任何内容。例如,使用环境变量为应用设置优化器的优先级高于您使用数据库选项为数据库设置的任何值。对于一个给定的查询,通过语句提示设置优化器版本的优先级最高,优先于使用任何其他方法设置的值。

现在,让我们来更详细地了解每种方法。

使用 ALTER DATABASE 为数据库设置优化器版本

您可以使用以下 ALTER DATABASE DDL 命令在数据库上设置默认优化器版本。

ALTER DATABASE MyDatabase
SET OPTIONS (optimizer_version = 2);

您可以使用 gcloud spanner databases ddl update 命令在 gcloud spanner 中运行 ALTER DATABASE,如下所示。

gcloud spanner databases ddl update MyDatabase --instance=test-instance \
    --ddl='ALTER DATABASE MyDatabase SET OPTIONS ( optimizer_version = 2 )'

将数据库选项设置为 NULL 以清除该选项,并将数据库默认设置为使用最新版本。这是新数据库的默认设置。

要查看数据库此选项的当前值,请从 INFORMATION_SCHEMA.DATABASE_OPTIONS 视图中读取此值。

为客户端应用设置优化器版本

通过客户端库以编程方式与 Cloud Spanner 进行交互时,可以通过多种方式更改客户端应用的优化器版本。

应用可以通过配置查询选项属性,在客户端库上设置全局查询选项,如以下代码段所示。优化器版本设置存储在客户端实例中,并应用于在客户端生命周期内运行的所有查询。即使选项在后端的数据库级层应用,但当选项在客户端级层设置时,它们会应用于通过该客户端连接的所有数据库。

C++

namespace spanner = ::google::cloud::spanner;
spanner::Client client(
    spanner::MakeConnection(db),
    spanner::ClientOptions().set_query_options(
        spanner::QueryOptions().set_optimizer_version("1")));

C#

var builder = new SpannerConnectionStringBuilder
{
    DataSource = $"projects/{projectId}/instances/{instanceId}/databases/{databaseId}"
};
// Create connection to Cloud Spanner.
using (var connection = new SpannerConnection(builder))
{
    // Set query options on the connection.
    connection.QueryOptions = QueryOptions.Empty.WithOptimizerVersion("1");
    var cmd = connection.CreateSelectCommand(
        "SELECT SingerId, AlbumId, AlbumTitle FROM Albums");
    using (var reader = await cmd.ExecuteReaderAsync())
    {
        while (await reader.ReadAsync())
        {
            Console.WriteLine("SingerId : "
            + reader.GetFieldValue<string>("SingerId")
            + " AlbumId : "
            + reader.GetFieldValue<string>("AlbumId")
            + " AlbumTitle : "
            + reader.GetFieldValue<string>("AlbumTitle"));
        }
    }
}

Go


import (
	"context"
	"fmt"
	"io"
	"time"

	"cloud.google.com/go/spanner"
	"google.golang.org/api/iterator"
	sppb "google.golang.org/genproto/googleapis/spanner/v1"
)

func createClientWithQueryOptions(w io.Writer, database string) error {
	ctx := context.Background()
	queryOptions := spanner.QueryOptions{
		Options: &sppb.ExecuteSqlRequest_QueryOptions{OptimizerVersion: "1"},
	}
	client, err := spanner.NewClientWithConfig(
		ctx, database, spanner.ClientConfig{QueryOptions: queryOptions},
	)
	if err != nil {
		return err
	}
	defer client.Close()

	stmt := spanner.Statement{SQL: `SELECT VenueId, VenueName, LastUpdateTime FROM Venues`}
	iter := client.Single().Query(ctx, stmt)
	defer iter.Stop()
	for {
		row, err := iter.Next()
		if err == iterator.Done {
			return nil
		}
		if err != nil {
			return err
		}
		var venueID int64
		var venueName string
		var lastUpdateTime time.Time
		if err := row.Columns(&venueID, &venueName, &lastUpdateTime); err != nil {
			return err
		}
		fmt.Fprintf(w, "%d %s %s\n", venueID, venueName, lastUpdateTime)
	}
}

Java

static void clientWithQueryOptions(DatabaseId db) {
  SpannerOptions options =
      SpannerOptions.newBuilder()
          .setDefaultQueryOptions(
              db, QueryOptions.newBuilder().setOptimizerVersion("1").build())
          .build();
  Spanner spanner = options.getService();
  DatabaseClient dbClient = spanner.getDatabaseClient(db);
  try (ResultSet resultSet =
      dbClient
          .singleUse()
          .executeQuery(Statement.of("SELECT SingerId, AlbumId, AlbumTitle FROM Albums"))) {
    while (resultSet.next()) {
      System.out.printf(
          "%d %d %s\n", resultSet.getLong(0), resultSet.getLong(1), resultSet.getString(2));
    }
  }
}

Node.js

// Imports the Google Cloud client library
const {Spanner} = require('@google-cloud/spanner');

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// const projectId = 'my-project-id';
// const instanceId = 'my-instance';
// const databaseId = 'my-database';

// Creates a client
const spanner = new Spanner({
  projectId: projectId,
});

// Gets a reference to a Cloud Spanner instance and database
const instance = spanner.instance(instanceId);
const database = instance.database(databaseId, {}, {optimizerVersion: '1'});

const query = {
  sql: `SELECT AlbumId, AlbumTitle, MarketingBudget
        FROM Albums
        ORDER BY AlbumTitle`,
};

// Queries rows from the Albums table
try {
  const [rows] = await database.run(query);

  rows.forEach(row => {
    const json = row.toJSON();
    const marketingBudget = json.MarketingBudget
      ? json.MarketingBudget
      : null; // This value is nullable
    console.log(
      `AlbumId: ${json.AlbumId}, AlbumTitle: ${json.AlbumTitle}, MarketingBudget: ${marketingBudget}`
    );
  });
} catch (err) {
  console.error('ERROR:', err);
} finally {
  // Close the database when finished.
  database.close();
}

PHP

use Google\Cloud\Spanner\SpannerClient;
use Google\Cloud\Spanner\Database;

/**
 * Create a client with query options.
 * Example:
 * ```
 * create_client_with_query_options($instanceId, $databaseId);
 * ```
 *
 * @param string $instanceId The Spanner instance ID.
 * @param string $databaseId The Spanner database ID.
 */
function create_client_with_query_options($instanceId, $databaseId)
{
    $spanner = new SpannerClient([
        'queryOptions' => [
            'optimizerVersion' => "1"
        ]
    ]);
    $instance = $spanner->instance($instanceId);
    $database = $instance->database($databaseId);

    $results = $database->execute(
        'SELECT VenueId, VenueName, LastUpdateTime FROM Venues'
    );

    foreach ($results as $row) {
        printf('VenueId: %s, VenueName: %s, LastUpdateTime: %s' . PHP_EOL,
            $row['VenueId'], $row['VenueName'], $row['LastUpdateTime']);
    }
}

Python

# instance_id = "your-spanner-instance"
# database_id = "your-spanner-db-id"
spanner_client = spanner.Client(query_options={"optimizer_version": "1"})
instance = spanner_client.instance(instance_id)
database = instance.database(database_id)

with database.snapshot() as snapshot:
    results = snapshot.execute_sql(
        "SELECT VenueId, VenueName, LastUpdateTime FROM Venues"
    )

    for row in results:
        print(u"VenueId: {}, VenueName: {}, LastUpdateTime: {}".format(*row))

Ruby

# project_id  = "Your Google Cloud project ID"
# instance_id = "Your Spanner instance ID"
# database_id = "Your Spanner database ID"

require "google/cloud/spanner"

query_options = { optimizer_version: "1" }

spanner = Google::Cloud::Spanner.new project: project_id
client  = spanner.client instance_id, database_id, query_options: query_options

sql_query = "SELECT VenueId, VenueName, LastUpdateTime FROM Venues"

client.execute(sql_query).rows.each do |row|
  puts "#{row[:VenueId]} #{row[:VenueName]} #{row[:LastUpdateTime]}"
end

使用环境变量为客户端应用设置优化器版本

如需更轻松地尝试不同的优化工具版本,而无需重新编译应用,您可以设置 SPANNER_OPTIMIZER_VERSION 环境变量并运行应用,如以下代码段所示。

Linux/macOS

export SPANNER_OPTIMIZER_VERSION="2"

Windows

set SPANNER_OPTIMIZER_VERSION="2"

在客户端初始化时,系统会读取指定的查询优化器版本值并将其存储在客户端实例中,并应用于在客户端生命周期内运行的所有查询。

为客户端查询设置优化器版本

通过在构建查询时指定查询选项属性,您可以在客户端应用的查询级层指定优化器版本的值。每种受支持语言的代码段如下所示。

C++

void QueryWithQueryOptions(google::cloud::spanner::Client client) {
  namespace spanner = ::google::cloud::spanner;
  auto sql = spanner::SqlStatement("SELECT SingerId, FirstName FROM Singers");
  auto opts = spanner::QueryOptions().set_optimizer_version("1");
  auto rows = client.ExecuteQuery(std::move(sql), std::move(opts));

  using RowType = std::tuple<std::int64_t, std::string>;
  for (auto const& row : spanner::StreamOf<RowType>(rows)) {
    if (!row) throw std::runtime_error(row.status().message());
    std::cout << "SingerId: " << std::get<0>(*row) << "\t";
    std::cout << "FirstName: " << std::get<1>(*row) << "\n";
  }
  std::cout << "Read completed for [spanner_query_with_query_options]\n";
}

C#

var builder = new SpannerConnectionStringBuilder
{
    DataSource = $"projects/{projectId}/instances/{instanceId}/databases/{databaseId}"
};
// Create connection to Cloud Spanner.
using (var connection = new SpannerConnection(builder))
{
    var cmd = connection.CreateSelectCommand(
        "SELECT SingerId, AlbumId, AlbumTitle FROM Albums");
    // Set query options just for this command.
    cmd.QueryOptions = QueryOptions.Empty.WithOptimizerVersion("1");
    using (var reader = await cmd.ExecuteReaderAsync())
    {
        while (await reader.ReadAsync())
        {
            Console.WriteLine("SingerId : "
            + reader.GetFieldValue<string>("SingerId")
            + " AlbumId : "
            + reader.GetFieldValue<string>("AlbumId")
            + " AlbumTitle : "
            + reader.GetFieldValue<string>("AlbumTitle"));
        }
    }
}

Go


import (
	"context"
	"fmt"
	"io"
	"time"

	"cloud.google.com/go/spanner"
	"google.golang.org/api/iterator"
	sppb "google.golang.org/genproto/googleapis/spanner/v1"
)

func queryWithQueryOptions(w io.Writer, db string) error {
	ctx := context.Background()
	client, err := spanner.NewClient(ctx, db)
	if err != nil {
		return err
	}
	defer client.Close()

	stmt := spanner.Statement{SQL: `SELECT VenueId, VenueName, LastUpdateTime FROM Venues`}
	queryOptions := spanner.QueryOptions{
		Options: &sppb.ExecuteSqlRequest_QueryOptions{OptimizerVersion: "1"},
	}
	iter := client.Single().QueryWithOptions(ctx, stmt, queryOptions)
	defer iter.Stop()
	for {
		row, err := iter.Next()
		if err == iterator.Done {
			return nil
		}
		if err != nil {
			return err
		}
		var venueID int64
		var venueName string
		var lastUpdateTime time.Time
		if err := row.Columns(&venueID, &venueName, &lastUpdateTime); err != nil {
			return err
		}
		fmt.Fprintf(w, "%d %s %s\n", venueID, venueName, lastUpdateTime)
	}
}

Java

static void queryWithQueryOptions(DatabaseClient dbClient) {
  try (ResultSet resultSet =
      dbClient
          .singleUse()
          .executeQuery(
              Statement
                  .newBuilder("SELECT SingerId, AlbumId, AlbumTitle FROM Albums")
                  .withQueryOptions(QueryOptions.newBuilder().setOptimizerVersion("1").build())
                  .build())) {
    while (resultSet.next()) {
      System.out.printf(
          "%d %d %s\n", resultSet.getLong(0), resultSet.getLong(1), resultSet.getString(2));
    }
  }
}

Node.js

// Imports the Google Cloud client library
const {Spanner} = require('@google-cloud/spanner');

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
// const projectId = 'my-project-id';
// const instanceId = 'my-instance';
// const databaseId = 'my-database';

// Creates a client
const spanner = new Spanner({
  projectId: projectId,
});

// Gets a reference to a Cloud Spanner instance and database
const instance = spanner.instance(instanceId);
const database = instance.database(databaseId);

const query = {
  sql: `SELECT AlbumId, AlbumTitle, MarketingBudget
        FROM Albums
        ORDER BY AlbumTitle`,
  queryOptions: {
    optimizerVersion: 'latest',
  },
};

// Queries rows from the Albums table
try {
  const [rows] = await database.run(query);

  rows.forEach(row => {
    const json = row.toJSON();
    const marketingBudget = json.MarketingBudget
      ? json.MarketingBudget
      : null; // This value is nullable
    console.log(
      `AlbumId: ${json.AlbumId}, AlbumTitle: ${json.AlbumTitle}, MarketingBudget: ${marketingBudget}`
    );
  });
} catch (err) {
  console.error('ERROR:', err);
} finally {
  // Close the database when finished.
  database.close();
}

PHP

use Google\Cloud\Spanner\SpannerClient;
use Google\Cloud\Spanner\Database;

/**
 * Queries sample data using SQL with query options.
 * Example:
 * ```
 * query_data_with_query_options($instanceId, $databaseId);
 * ```
 *
 * @param string $instanceId The Spanner instance ID.
 * @param string $databaseId The Spanner database ID.
 */
function query_data_with_query_options($instanceId, $databaseId)
{
    $spanner = new SpannerClient();
    $instance = $spanner->instance($instanceId);
    $database = $instance->database($databaseId);

    $results = $database->execute(
        'SELECT VenueId, VenueName, LastUpdateTime FROM Venues',
        [
            'queryOptions' => [
                'optimizerVersion' => "1"
            ]
        ]
    );

    foreach ($results as $row) {
        printf('VenueId: %s, VenueName: %s, LastUpdateTime: %s' . PHP_EOL,
            $row['VenueId'], $row['VenueName'], $row['LastUpdateTime']);
    }
}

Python

# instance_id = "your-spanner-instance"
# database_id = "your-spanner-db-id"
spanner_client = spanner.Client()
instance = spanner_client.instance(instance_id)
database = instance.database(database_id)

with database.snapshot() as snapshot:
    results = snapshot.execute_sql(
        "SELECT VenueId, VenueName, LastUpdateTime FROM Venues",
        query_options={"optimizer_version": "1"},
    )

    for row in results:
        print(u"VenueId: {}, VenueName: {}, LastUpdateTime: {}".format(*row))

Ruby

# project_id  = "Your Google Cloud project ID"
# instance_id = "Your Spanner instance ID"
# database_id = "Your Spanner database ID"

require "google/cloud/spanner"

spanner = Google::Cloud::Spanner.new project: project_id
client  = spanner.client instance_id, database_id

sql_query = "SELECT VenueId, VenueName, LastUpdateTime FROM Venues"
query_options = { optimizer_version: "1" }

client.execute(sql_query, query_options: query_options).rows.each do |row|
  puts "#{row[:VenueId]} #{row[:VenueName]} #{row[:LastUpdateTime]}"
end

使用语句提示为查询设置优化器版本

语句提示是对查询语句的提示,会更改查询执行的默认行为。在语句上设置 OPTIMIZER_VERSION 提示会强制使用指定的查询优化器版本运行该查询。

OPTIMIZER_VERSION 提示具有最高的优化器版本优先级。如果指定了语句提示,则将使用该提示,而不考虑其他所有优化器版本设置。

@{OPTIMIZER_VERSION=2} SELECT * FROM MyTable

您还可以使用 latest 字面量将查询的优化器版本设置为最新版本,如此处所示。

@{OPTIMIZER_VERSION=latest} SELECT * FROM MyTable

使用 JDBC 驱动程序时设置优化器版本

如以下示例所示,您可以通过在 JDBC 连接字符串中指定版本来替换优化器版本的默认值。

import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;

class ConnectionWithQueryOptionsExample {

  static void connectionWithQueryOptions() throws SQLException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "my-project";
    String instanceId = "my-instance";
    String databaseId = "my-database";
    connectionWithQueryOptions(projectId, instanceId, databaseId);
  }

  @SuppressFBWarnings(
      value = "OBL_UNSATISFIED_OBLIGATION",
      justification = "https://github.com/spotbugs/spotbugs/issues/293")
  static void connectionWithQueryOptions(String projectId, String instanceId, String databaseId)
      throws SQLException {
    String optimizerVersion = "1";
    String connectionUrl =
        String.format(
            "jdbc:cloudspanner:/projects/%s/instances/%s/databases/%s?optimizerVersion=%s",
            projectId, instanceId, databaseId, optimizerVersion);
    try (Connection connection = DriverManager.getConnection(connectionUrl);
        Statement statement = connection.createStatement()) {
      // Execute a query using the optimizer version '1'.
      try (ResultSet rs =
          statement.executeQuery(
              "SELECT SingerId, FirstName, LastName FROM Singers ORDER BY LastName")) {
        while (rs.next()) {
          System.out.printf("%d %s %s%n", rs.getLong(1), rs.getString(2), rs.getString(3));
        }
      }
      try (ResultSet rs = statement.executeQuery("SHOW VARIABLE OPTIMIZER_VERSION")) {
        while (rs.next()) {
          System.out.printf("Optimizer version: %s%n", rs.getString(1));
        }
      }
    }
  }
}

您还可以使用 SET OPTIMIZER_VERSION 语句设置查询优化器版本,如以下示例所示。

import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;

class SetQueryOptionsExample {

  static void setQueryOptions() throws SQLException {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "my-project";
    String instanceId = "my-instance";
    String databaseId = "my-database";
    setQueryOptions(projectId, instanceId, databaseId);
  }

  @SuppressFBWarnings(
      value = "OBL_UNSATISFIED_OBLIGATION",
      justification = "https://github.com/spotbugs/spotbugs/issues/293")
  static void setQueryOptions(String projectId, String instanceId, String databaseId)
      throws SQLException {
    String connectionUrl =
        String.format(
            "jdbc:cloudspanner:/projects/%s/instances/%s/databases/%s",
            projectId, instanceId, databaseId);
    try (Connection connection = DriverManager.getConnection(connectionUrl);
        Statement statement = connection.createStatement()) {
      // Instruct the JDBC connection to use version '1' of the query optimizer.
      statement.execute("SET OPTIMIZER_VERSION='1'");
      // Execute a query using the latest optimizer version.
      try (ResultSet rs =
          statement.executeQuery(
              "SELECT SingerId, FirstName, LastName FROM Singers ORDER BY LastName")) {
        while (rs.next()) {
          System.out.printf("%d %s %s%n", rs.getLong(1), rs.getString(2), rs.getString(3));
        }
      }
      try (ResultSet rs = statement.executeQuery("SHOW VARIABLE OPTIMIZER_VERSION")) {
        while (rs.next()) {
          System.out.printf("Optimizer version: %s%n", rs.getString(1));
        }
      }
    }
  }
}

如需详细了解如何使用开源驱动程序,请参阅使用开源 JDBC 驱动程序

无效优化器版本的处理方式

Cloud Spanner 支持一系列范围的优化器版本。当查询优化器更新时,此范围会随之发生变化。如果您使用本指南中所述方法之一指定的版本超出范围,则 Cloud Spanner 查询会失败。例如,如果您尝试运行优化器版本为 100 的查询,并假设其超出了当前最大值,则会收到以下错误。

Query optimizer version: 100 is not supported

确定用于运行查询的查询优化器版本

您可以通过 gcloud spanner 和 Cloud Console 查看用于查询的优化器版本。

gcloud spanner

要查看在 gcloud spanner 中运行查询时使用的版本,请按照以下代码段所示将 --query-mode 标志设置为 PROFILE

gcloud spanner databases execute-sql MyDatabase --instance=test-instance \
    --query-mode=PROFILE --sql='SELECT * FROM MyTable'

Cloud Console

要查看用于查询的优化器版本,请在 Cloud Console 的查询数据库视图中运行查询,然后选择说明标签页。您应该会看到如下所示的消息:

此查询是使用优化器版本 2 运行的。

在 Metrics Explorer 中直观呈现查询优化器版本

Cloud Monitoring 会收计量结果,以帮助您了解应用和系统服务的性能。为 Cloud Spanner 收集的指标之一是查询计数,用于衡量实例中的查询数量(按时间进行采样)。此指标可帮助我们查看按错误代码分组的查询,此外,我们还可以使用它来查看运行每个查询的优化器版本。

您可以使用 Cloud Console 中的 Metrics Explorer 来直观呈现数据库实例的查询计数图 1 显示了三个数据库的查询计数。您可以查看每个数据库中使用的优化器版本。

此图中图表下方的表格显示 my-db-1 尝试使用无效的优化器版本运行查询,返回 Bad usage 状态,并导致查询计数为 0。其他数据库分别使用优化器版本 1 和版本 2 运行查询。

Metrics Explorer 中按查询优化器版本分组的查询计数

图 1:Metrics Explorer 中显示的按优化器版本分组的查询的查询计数

要为您的实例设置类似的图表,请执行以下操作:

  1. 在 Cloud Console 中导航到 Metrics Explorer
  2. 资源类型字段中,选择 Cloud Spanner Instance
  3. 指标字段中,选择 Count of queries
  4. 分组依据字段中,选择 databaseoptimizer_versionstatus

本示例中未显示同一数据库的不同查询使用不同优化器版本的情况。在这种情况下,图表会针对数据库和优化器的每个组合显示条形图。

如需了解如何使用 Cloud Monitoring 监控 Cloud Spanner 实例,请参阅使用 Cloud Monitoring 进行监控