Introduction to developing in a workspace

This document helps you understand the concept of workspaces in Dataform and their capabilities.

In Dataform, you interact with your SQL workflow code in a development workspace. A development workspace is your own editable copy of a repository. In a workspace, you create, edit, or delete the contents of the repository without affecting others who work on the same repository, and then commit and push your changes to the repository.

You can develop your SQL workflow with Dataform core or JavaScript. Dataform compiles your code into SQL queries in real time.

In a development workspace, you can perform the following tasks:

Components of a development workspace

A Dataform development workspace consists of the following tabs:

Code
Contains a development environment where you can explore files, develop queries, version control your changes, and trigger execution to BigQuery.
Compiled graph
Displays the compiled SQL workflow defined in your workspace as an interactive directed acyclic graph (DAG). If there are compilation errors in your SQL workflow, the tab displays an error message instead of the graph.
Executions
Displays logs of all executions triggered in the workspace. You can also view actions and their details for a selected execution. For more information, see Inspect workspace execution logs.

Components of the Code tab

The Code tab of a development workspace contains the following components that let you develop SQL workflows:

Files pane
Displays the file structure of your workspace. Autosaves your edits to the file structure. Selected files open in tabs in the development pane. In the Type to search field, you can search for files or directories in your workspace by their path.
Version control pane
Displays the version control status of your workspace and the version control options.
Development pane
Displays open files in tabs. Autosaves your changes to edited files.
Run button
Runs a preview of the compiled SQL query.
Format button
Formats your Dataform core or JavaScript code.
Compilation status
Displays the compilation status of the code in your workspace.
Metadata pane
Displays the metadata of the currently edited file.
Compiled queries pane
Compiles the queries in the currently edited file to SQL. If query compilation fails, displays compilation errors. If query compilation is successful, displays the compiled queries and validates them through dry runs against BigQuery. If query validation fails, displays validation errors. If query validation is successful, displays how many bytes the execution of the queries to BigQuery would use.
Start execution drop-down
Displays options for triggering execution of the entire workflow in your workspace or its selected elements.

Components of the Compiled graph tab

The Compiled graph tab displays an interactive graph of your compiled SQL workflow. The graph shows all actions in your SQL workflow and relationships between them.

You can zoom in and out, and use drag to navigate the graph.

You can filter the graph by the following properties:

  • Name
  • Tag
  • Type
    • Assertion
    • Declaration
    • Incremental Table
    • Materialized view
    • Operations
    • Table
    • Unknown
    • View

You can select multiple filters at once. Dataform will apply them with the OR condition.

You can click an action in the graph to view its details in the following panes:

Metadata pane
Displays the full name, type, dependencies, and dependents of the selected action.
Query pane
Displays the compiled SQL query of the selected action.

Version control in a workspace

In a workspace, you interact with the repository through Git-based version control. Edits you make to files in your workspace start off as uncommitted local changes that you can commit, and then push to your repository so they become available to all repository users.

If your repository is connected to a remote Git provider, you push changes from your workspace to your main development branch. You select the main branch when you connect your repository to the remote Git provider. In a non-connected Dataform repository, you push changes to the default branch.

You can also pull any changes from the Dataform repository or the remote repository and merge them with local changes in your development workspace. For more information about version control in Dataform, see Version control your code.

What's next