Interacting with BigQuery
-
BigQuery sandbox
Get started with BigQuery without creating a billing account for your project.
-
Using the Google Cloud console
BigQuery exposes a graphical interface in the Google Cloud console that you can use to load and export data, run queries, and perform other user and management tasks in your browser.
-
Using the
bqcommand-line toolThe
bqcommand-line tool is a python-based tool that accesses BigQuery from the command line.
Try it for yourself
If you're new to Google Cloud, create an account to evaluate how BigQuery performs in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
Try BigQuery freeRunning and managing jobs
Working with datasets
-
Introduction to datasets
An introduction to BigQuery datasets.
-
Dataset locations
An overview of locations for storing BigQuery data.
-
Creating datasets
How to create datasets in BigQuery.
-
Copying datasets
How to copy datasets, including copying datasets across regions.
-
Controlling access to datasets
How to control access to datasets in BigQuery.
-
Listing datasets
How to list datasets in BigQuery projects.
-
Getting information about datasets
How to get dataset information or metadata.
-
Updating datasets
How to update dataset properties including updating descriptions, default expiration times, and access controls.
-
Managing datasets
Manage datasets including renaming datasets, deleting datasets, and copying datasets.
Working with table schemas
-
Specifying a schema
How to create and use table schemas.
-
Specifying nested and repeated columns
How to specify nested and repeated columns in table schemas.
-
Auto-detecting schemas
Use schema auto-detection when you load data or query external data.
-
Modifying table schemas
Modify table schema definitions including adding columns, renaming columns, deleting columns, and changing column names, data types, and modes.
Working with tables
-
Introduction to tables
An introduction to BigQuery tables.
-
Creating and using tables
Create and use tables including listing tables, getting information about tables, and controlling access to table data.
-
Managing tables
Manage tables including updating table properties, renaming tables, deleting tables, and copying tables.
-
Managing table data
An overview of working with table data including querying tables, browsing table data, modifying table data, exporting data, and appending or overwriting data.
-
Exporting table data
How to export table data to Cloud Storage.
-
Using data manipulation language (DML)
Perform bulk modifications to table data using data manipulation language (DML) in GoogleSQL.
Working with partitioned tables
-
Introduction to partitioned tables
An introduction to partitioned tables in BigQuery.
-
Creating partitioned tables
How to create partitioned tables, including time-unit column-partitioned tables, ingestion-time partitioned tables, and integer-range partitioned tables.
-
Managing partitioned tables
How to manage partitioned tables including updating properties, copying partitions, and deleting partitions.
-
Managing partitioned table data
How to manage partitioned table data including browsing data, modifying data, exporting data, and appending or overwriting data.
-
Querying partitioned tables
How to query partitioned tables.
-
Updating partitioned table data using DML statements
How to use DML statements to insert, delete, or update data in partitioned tables.
Working with clustered tables
Working with table snapshots
-
Introduction to table snapshots
An introduction to table snapshots in BigQuery.
-
Creating table snapshots
How to create a snapshot of a BigQuery table.
-
Restoring table snapshots
How to restore a BigQuery table from a table snapshot.
-
Listing the table snapshots in a dataset
How to list the table snapshots in a BigQuery dataset.
-
Getting information about a table snapshot
How to view the metadata for a BigQuery table snapshot.
-
Updating table snapshot metadata
How to update the description, expiration date, or access policy for a BigQuery table snapshot.
-
Copying table snapshots
How to make a copy of a BigQuery table snapshot.
-
Deleting table snapshots
How to delete a BigQuery table snapshot.
-
Creating table snapshots with a scheduled query
How to create monthly snapshots of a BigQuery table.
Working with views
-
Introduction to views
An introduction to BigQuery views.
-
Creating views
How to create views.
-
Creating authorized views
How to create a view that allows you to share query results with users and groups without giving them access to the underlying tables.
-
Getting information about views
How to get information or metadata about views.
-
Managing views
Manage views including copying views, renaming views, and deleting views.
Working with materialized views
-
Introduction to materialized views
An introduction to BigQuery materialized views.
-
Create materialized views
Create materialized views.
-
Use materialized views
Query materialized views, including details on partition alignment and smart tuning.
-
Manage materialized views
Alter, delete, refresh, and monitor materialized views.
Working with sessions
-
Introduction to sessions
An introduction to sessions in BigQuery.
-
Creating sessions
How to create a session to capture a group of your SQL activities.
-
Running queries in sessions
How to run queries in a BigQuery session.
-
Writing queries in sessions
How to write queries in a BigQuery session.
-
Terminating sessions
How to terminate a BigQuery session.
-
Viewing query history in sessions
How to view the history of a BigQuery session.
-
Finding sessions
How to get a session ID in BigQuery.
Using Reservations for workload management
Getting metadata using INFORMATION_SCHEMA
Labeling BigQuery resources
-
Introduction to labels
An introduction to labeling resources in BigQuery.
-
Adding labels
Add labels to group resources by purpose, environment, department, and so on.
-
Viewing labels
Viewing labels on BigQuery resources.
-
Updating labels
Updating labels on BigQuery resources.
-
Filtering resources using labels
Filtering resources using labels.
-
Deleting labels
Deleting labels on BigQuery resources.
Loading data into BigQuery
-
Introduction to loading data
Load your data into BigQuery from a variety of source formats, including CSV, JSON, Avro, Parquet, ORC and Datastore backups.
-
Loading data from Cloud Storage
Load data from Cloud Storage.
-
Loading Avro data from Cloud Storage
How to load Avro data from Cloud Storage.
-
Loading Parquet data from Cloud Storage
How to load Parquet data from Cloud Storage.
-
Loading ORC data from Cloud Storage
How to load ORC data from Cloud Storage.
-
Loading CSV data from Cloud Storage
How to load CSV data from Cloud Storage.
-
Loading JSON data from Cloud Storage
How to load newline delimited JSON data from Cloud Storage.
-
Loading Datastore exports from Cloud Storage
How to load data from a Datastore export.
-
Loading Firestore exports from Cloud Storage
How to load data from a Firestore export.
-
Loading data from a local data source
How to load data from a local data source.
-
Streaming data into BigQuery
Stream your data into BigQuery using the BigQuery API.
Querying BigQuery data
-
Introduction to querying data
An introduction to running queries in BigQuery.
-
Running interactive and batch queries
How to run interactive and batch queries.
-
Performing a query dry run
How to perform a dry run for a query job to estimate the amount of data processed.
-
Writing query results
Write and page through query results.
-
Using cached results
How to use cached query results.
-
Running parameterized queries
How to use parameterized queries.
-
Querying data using a wildcard table
Query several tables concisely using a wildcard table.
-
Work with saved queries
Create, view, share, update, and delete saved queries.
-
Scheduling queries
Schedule recurring queries in BigQuery.
-
Using the query plan explanation
BigQuery provides diagnostic information about a completed query's execution plan.
-
Using the BigQuery connector for Excel
BigQuery offers a connector that allows you to query BigQuery data to from Excel.
Querying external data sources
-
Introduction to external data sources
Query data directly from external data sources such as Cloud Storage, Cloud Bigtable, and Google Drive.
-
Creating a table definition file
Create a table definition file for an external data source.
-
Federated queries with Cloud SQL data
Query data in BigQuery and Cloud SQL with a federated query.
-
Querying Cloud Bigtable data
Use BigQuery to query data stored in Cloud Bigtable.
-
Querying Cloud Storage data
Use BigQuery to query data stored in Cloud Storage.
-
Querying Google Drive data
Use BigQuery to query data stored in Google Drive.
Controlling BigQuery costs
-
Introduction to controlling BigQuery costs
An introduction to cost control measures for BigQuery.
-
Estimating storage and query costs
How to estimate storage and query costs before loading and querying data.
-
Custom cost controls
Manage costs by setting custom quotas on query processing.
-
Best practices for controlling costs
Best practices for controlling costs in BigQuery.
Securing BigQuery resources
-
Access control with IAM
Information on BigQuery's predefined IAM roles and permissions.
-
Introduction to Column-level security (beta)
Learn about BigQuery Column-level security.
-
Restricting access with Column-level security (beta)
How to restrict access with BigQuery Column-level security.
-
Encryption at rest
Learn how BigQuery encrypts your data at rest.
-
Using Cloud Data Loss Prevention to scan BigQuery data
Use Cloud Data Loss Prevention to identify and protect sensitive BigQuery data.
-
Protecting data with Cloud KMS keys
Use customer-managed encryption keys (CMEK) for BigQuery.
Monitoring and logging
BigQuery API basics
-
Introduction to authentication
Perform authentication in various application scenarios.
-
Getting started with authentication
Get started with application default credentials.
-
Authenticating with a service account key file
Manually create and obtain service account credentials.
-
Authenticating with a user account for installed applications
Perform application authentication using user accounts.
-
Authorizing API requests
Authorize API requests with access tokens.
-
Batch requests
Reduce the number of HTTP connections your client has to make by batching API calls.
-
Paging through query results
Page through query results using the BigQuery API.
-
API performance tips
Techniques you can use to improve the performance of your application.