Stay organized with collections Save and categorize content based on your preferences.

Monitor BI Engine with Cloud Monitoring

BigQuery BI Engine integrates with Cloud Monitoring so you can monitor BI Engine metrics and configure alerts.

For information on using Monitoring to create charts for your BI Engine metrics, see Creating charts in the Monitoring documentation.

The metrics provided by BI Engine are listed in the following table.

Metrics

The following metrics are provided by BI Engine:

Resource Metric Details
Project Reservation Total Bytes Total capacity allocated to one Google Cloud project
Project Reservation Used Bytes Total capacity used in one Google Cloud project
Project BI Engine Top Tables Cached Bytes (Preview) Cache usage per table. This metric displays the top N tables per region report usage.

The following metrics are also provided by BI Engine when using the Looker Studio interface:

Resource Metric Details
BI Engine model Inflight Requests Number of concurrent requests to a model (max within sample period)
BI Engine model Request Count Total number of requests that were issued to a model
BI Engine model Request Execution Times Median query execution latency

For a complete list of available Google Cloud metrics, see Google Cloud metrics.

Monitoring and diagnostics

This section explains how to find statistics about BI Engine, and how BI Engine integrates with Cloud Monitoring.

Acceleration statistics

Detailed statistics on BI Engine are available through the job statistics API. You can use the bq command-line tool to fetch statistics associated with BI Engine accelerated queries.

With BI Engine acceleration enabled, you can run a query in any of these three modes:

DISABLED
BI Engine disabled the acceleration. biEngineReasons specifies a more detailed reason. The query was run using the BigQuery execution engine.
PARTIAL
Part of the query was accelerated using BI Engine. As described in Query optimization and acceleration, a query plan is generally broken down into multiple input stages. BI Engine supports the common types of subquery patterns that are typically used in dashboarding. If the query consists of multiple input stages, only a few of which fall under the supported use cases, then BI Engine runs the unsupported stages using the normal BigQuery engine without acceleration. In this situation, BI Engine returns a PARTIAL acceleration code, and uses biEngineReasons to populate the reason for not accelerating other input stages.
 FULL
 
All input stages of the query were accelerated using BI Engine.

To fetch the statistics associated with BI Engine accelerated queries, run the following bq command-line tool command:

bq show --format=prettyjson -j job_id

If the project is enabled for BI Engine acceleration, then the output produces a new field, biEngineStatistics. Here is a sample job report:

 "statistics": {
    "creationTime": "1602175128902",
    "endTime": "1602175130700",
    "query": {
      "biEngineStatistics": {
        "biEngineMode": "DISABLED",
        "biEngineReasons": [
          {
            "code": "UNSUPPORTED_SQL_TEXT",
            "message": "Detected unsupported join type"
          }
        ]
      },

For more information about the BiEngineStatistics field, see the Job reference.

Acceleration statistics in INFORMATION_SCHEMA

BI Engine acceleration statistics can be queried as part of INFORMATION_SCHEMA.JOBS_BY_* views via the column bi_engine_statistics. For example, this query returns the bi_engine_statistics, for all of your projects' jobs for the last 24 hours:

select creation_time, job_id, bi_engine_statistics
from `region-us`.INFORMATION_SCHEMA.JOBS_BY_PROJECT
where creation_time BETWEEN TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 DAY) AND CURRENT_TIMESTAMP()
  and job_type = "QUERY"

Use the following format to specify regionality for the project-id, region, and views in the INFORMATION_SCHEMA view:

`PROJECT_ID`.`region-REGION_NAME`.INFORMATION_SCHEMA.VIEW

Logging information

You can view log information about BI Engine by viewing it on the Cloud Logging page of the Google Cloud console. How you view log information for BI Engine depends upon the tool that is querying the data:

  • For queries with Looker Studio: View log information on the Cloud Logging page with a payload of protoPayload.serviceName="bigquerybiengine.googleapis.com".
  • For queries with other tools: View log information on the Cloud Logging page with a payload of protoPayload.serviceName="bigquery.googleapis.com".

What's next