Introduction to BigQuery Admin Resource Charts

Introduction

BigQuery Admin Resource Charts let BigQuery administrators observe how their organization, folder, or reservation uses BigQuery slots and how their queries perform. Primary use cases for Admin Resource Charts include root-cause analysis for capacity planning based on historical resource utilization and diagnosing performance changes. Data in the Admin Resource Charts is updated in real-time, and the history remains available in the Cloud Console for up to 14 days.

BigQuery Admin Resource Charts overview.

Enabling Admin Resource Charts

The Admin Resource Charts Preview is only available for BigQuery Reservations customers. Admin Resource Charts monitor BigQuery Reservations usage. If you use on-demand pricing for bytes processed, you can't use this Preview.

Permissions

Admin Resource Charts require the following permissions:

  • To see all data in the Admin Resource Charts, you must have permission to view INFORMATION_SCHEMA.JOBS_BY_ORGANIZATION tables. JOBS_BY_ORGANIZATION requires bigquery.jobs.listAll for the organization and is available to the following:

    • Organization Owner
    • Organization Admin
    • Organization-level BigQuery Admin
    • Organization-level BigQuery Resource Admin
    • Custom Identity and Access Management (IAM) roles
  • To see data at the folder level, you must have permission to view INFORMATION_SCHEMA.JOBS_BY_FOLDER tables. Access to JOBS_BY_FOLDER requires bigquery.jobs.listAll for the parent folder and is available to the Folder Admin and BigQuery Admin for the folder.

  • To see partial data at the reservation level, you must have bigquery.reservations.list permission to the reservation admin project. Reservation usage data that belongs to the admin project is available, but folder, project, user, and job options are disabled.

  • Viewing capacity data on Admin Resource Charts requires the same permission to view the INFORMATION_SCHEMA.CAPACITY_COMMITMENT_CHANGES_BY_PROJECT table. CAPACITY_COMMITMENT_CHANGES_BY_PROJECT requires bigquery.capacityCommitments.list permission to the admin project and is available to the following roles:

    • BigQuery User
    • BigQuery Resource Admin
    • BigQuery Admin roles

Using Admin Resource Charts

  1. To start using Admin Resource Charts, in the Cloud Console, open the BigQuery page.

    Go to BigQuery

  2. In the project picker, select the admin project that you used to purchase slots and create reservations.

  3. In the navigation panel, select the Reservations section.

  4. Open the Overview tab.

Overview tab.

Admin Resource Charts have three primary interactive components:

  • The event timeline chart displays an overview of data across 14 days.

  • The main chart displays the selected metrics (for example, slot usage, job concurrency, or job performance) over time for the organization, folder, or reservation.

  • The table legend that correlates with data in the chart shows lower-level insights aggregated over the time period.

Main chart view

Metrics type options

You can select different metrics types to display in the charts: Slot Usage, Job Concurrency, or Job Performance.

Metrics type.

  • Slot Usage: Average slot usage that's consumed in the selected time period.

  • Job Concurrency: Maximum number of jobs that ran simultaneously in the selected time period.

  • Job Performance: Average job run time, calculated from job creation time to job end time.

Group by options

You can group data in the chart view by several dimensions. For example, the slot usage chart can display the following:

Break down dimensions.

  • The Overall view is a high-level overview of the selected metric over time across the entire organization. For slot usage, capacity and usage are shown on the same chart to give an idea of overall slot utilization over time.

  • The Project view shows the selected metric grouped by project. If an organization has more than 10 projects, then only the top 10 projects with the highest slot usage, job concurrency, or job duration in the selected time period are shown. For slot usage, all other projects are grouped into the Other category at the bottom of the chart for easier comparison to overall capacity.

  • The Reservation, Folder, User, and Job views are similar to the project view, with data grouped by Reservation, Folder, User, and Job respectively.

You can modify the time period in the following ways:

  • Select and drag the time period into the event timeline chart.

  • Select and drag the time period into the main chart.

To narrow down the chart data, apply filters in the filter panel.

Filters.

You can filter views based on Projects, Reservations, Folders, Users, and Jobs. All filters except Jobs are populated with resources that have consumed slots in the past 14 days. If a project hasn't been used in the last 14 days, it does not appear in the project filter list.

The chart refreshes after you apply filters to show data within the selected parameters.

Table component

The table displays metrics that are relevant to the time period and dimension that you selected in the chart:

  • With the Admin Resource Chart grouped by reservation, the table component shows the top reservations in the organization.

  • With the Admin Resource Chart grouped by job, the table component displays the jobs that used the most slots in the organization for the selected time period.

For the Slot Usage chart, the table component displays the following data:

  • Average slot usage

    • Jobs view: The total slot usage of the job over its duration. Only usage and time within the selected time period are included in the calculation.

    • All other views: The sum of the slot usage across all jobs over the selected time period. Jobs that didn't finish within the selected time period include only slots used within the time period.

  • Slots: The number of slots that were used at a specific time.

For the Job Performance chart, the table component displays the following data:

  • Number of jobs: The number of completed jobs ran or partially ran within the selected time period.

  • Average job duration: The average job duration for completed jobs that at least partially ran within the selected time period.

  • Median job duration: The median job duration for completed jobs that at least partially ran within the selected time period.

  • Min job duration: The minimum job duration for completed jobs that at least partially ran within the selected time period. This column is hidden by default, but you can enable it in Column display options.

  • Max job duration: The maximum job duration for completed jobs that at least partially ran within the selected time period. This column is hidden by default, but you can enable it in Column display options.

Billing

During the Preview phase, you can view data in the Admin Resource Charts at no charge. These queries are not charged and do not use slots in user-owned reservations.