Introduction to tables

This document helps you understand tables in Dataform.

In Dataform, a table is one of the types of objects that make up a SQL workflow. You can create tables that reference data from the data sources declared for your SQL workflow, or other tables in your SQL workflow. Dataform compiles your table definitions into SQL in real time. When you trigger execution, Dataform executes the SQL code and creates your defined tables in BigQuery.

You can create the following table types in a type: "table" SQLX file:

  • table: a regular table.
  • incremental: an incremental table.
  • view: a table view. For more information on views in BigQuery, see Introduction to views.

You can also define table partitions and clusters.

To keep a record of the purpose of a table or its relation to other tables in your SQL workflow, you can add documentation to the table or its selected columns.

To validate data in a table against specific conditions, you can create data quality test queries called assertions. Dataform runs assertions every time it updates your SQL workflow and alerts you if any assertions fail.

To override default table settings, such as database or schema, and disable table creation, or execute a SQL statement before or after table creation, you can configure additional table settings.

To organize your tables in BigQuery after you execute them, you can add BigQuery labels. To learn more about BigQuery labels, see Introduction to labels.

To restrict data access at the table column level, you can add BigQuery policy tags. To learn more about BigQuery policy tags, see Introduction to column-level access control.

In addition to defining tables in a type: "table" SQLX file, you can create empty tables by defining a custom SQL query in a type: "operations" SQLX file. You might want to create an empty table so that a different service can populate it with data.

What's next