This document helps you understand tables in Dataform.
In Dataform, a table is one of the types of objects that make up a SQL workflow. You can create tables that reference data from the data sources declared for your SQL workflow, or other tables in your SQL workflow. Dataform compiles your table definitions into SQL in real time. When you trigger execution, Dataform executes the SQL code and creates your defined tables in BigQuery.
You can create the following table types in a type: "table"
SQLX file:
table
: a regular table.incremental
: an incremental table.view
: a table view. For more information on views in BigQuery, see Introduction to views.materialized
: a materialized table view. For more information on materialized views, see Introduction to materialized views.
You can also define table partitions and clusters.
To keep a record of the purpose of a table or its relation to other tables in your SQL workflow, you can add documentation to the table or its selected columns.
To test data in a table against specific conditions, you can create data quality test queries called assertions. Dataform runs assertions every time it updates your SQL workflow and alerts you if any assertions fail.
To override default table settings, such as database
or schema
,
and disable table creation, or execute a SQL statement before or after
table creation, you can
configure additional table settings.
To organize your tables in BigQuery after you execute them, you can add BigQuery labels. To learn more about BigQuery labels, see Introduction to labels.
To restrict data access at the table column level, you can add BigQuery policy tags. To learn more about BigQuery policy tags, see Introduction to column-level access control.
In addition to defining tables in a type: "table"
SQLX file, you can
create empty tables
by defining a custom SQL query in a type: "operations"
SQLX file.
You might want to create an empty table so that a different service
can populate it with data.
What's next
- To learn how to create a table, see Create a table.
- To learn how to configure additional table settings, see Configure additional table settings.
- To learn how to partition a table, see Create table partitions and clusters.
- To learn how to configure an incremental table, see Configure an incremental table.