Dataform

Dataform Core enables data engineers and data analysts to centrally create table definitions, configure dependencies, add column descriptions, and configure data quality assertions in a single repository using just SQL.

Dataform Core functions can be adopted incrementally and additively, without modifying existing code.

Dataform Core is open source and can be used locally, giving users freedom from lock-in, and flexibility for more advanced use cases. 

Dataform handles the operational infrastructure to update your tables following the dependencies between your tables and using the latest version of your code. Lineage and data information can be tracked seamlessly with Dataform integrations. Trigger SQL workflows manually, or schedule via Cloud Composer, Workflows, or third-party services.

Define tables, fix issues with real-time error messages, visualize dependencies, commit the changes to Git, and schedule pipelines in minutes, from a single interface, without leaving your web browser. Connect your repository with third-party providers such as GitHub and GitLab. Commit changes and push or open pull requests from the IDE.