To create a package that you can reuse in multiple Dataform repositories, you need to create a Dataform repository dedicated to the package and connect it to a third-party Git repository to make it available to other Dataform repositories.
Then, you need to create a top-level index.js
file and add your exportable
package contents, such as functions and constants, to the file. For an example
of a package created in Dataform, see
dataform-package-base
on GitHub.
Once you create the package, you can install the package to in different Dataform repository and use the exportable contents on the package, such as constants and functions, to develop SQL workflows.
As an alternative to creating a package, you can reuse JavaScript functions and constants across a single Dataform repository with includes. For more information, see Reuse variables and functions with includes in Dataform.
Before you begin
In the Google Cloud console, go to the Dataform page.
Create a Dataform repository dedicated to your package. Match the repository name to the name of your package.
Connect the repository to a third-party Git repository that will host your package.
Create and initialize a workspace in the Dataform repository.
Required roles
To get the permissions that you need to create a package,
ask your administrator to grant you the
Dataform Admin (roles/dataform.admin
) IAM role on repositories.
For more information about granting roles, see Manage access to projects, folders, and organizations.
You might also be able to get the required permissions through custom roles or other predefined roles.
Create a package in Dataform
To create your own package with JavaScript code that you can reuse in Dataform, follow these steps in your workspace:
In the Files pane, click the
More menu.Click Create file.
In the Create new file pane, do the following:
In the Add a file path field, enter
index.js
.Click Create file.
In the
index.js
file, enter the JavaScript code that you want your package to export.Create constants in the following format:
const CONSTANT_NAME = CONSTANT_VALUE; module.exports = { CONSTANT_NAME };
Replace the following:
CONSTANT_NAME
: the name of your constantCONSTANT_VALUE
: the value of your constant
Create functions in the following format:
function FUNCTION_NAME(PARAMETERS) { FUNCTION_BODY } module.exports = { FUNCTION_NAME }
Replace the following:
FUNCTION_NAME
: the name of your functionPARAMETERS
: the parameters of your functionFUNCTION_BODY
: the code that you want the function to execute
Optional: Click Format.
Optional: In the
definitions
directory, add code of your package that will not be exported.
The following package code sample shows the index.js
file of the
postoffice
package that exports the getDomain
function:
// filename index.js
// package name postoffice
const GENERIC_DOMAINS = "('samplemail.com','samplemail.co.uk','examplemailbox.com'";
function getDomain(email) {
let cleanEmail = `trim(${email})`
const domain = `substr(${cleanEmail}, strpos(${cleanEmail}, '@') + 1)`;
return `case
when ${domain} in ${common.GENERIC_DOMAINS} then ${cleanEmail}
when ${domain} = "othermailbox.com" then "other.com"
when ${domain} = "mailbox.com" then "mailbox.global"
when ${domain} = "support.postman.com" then "postman.com"
else ${domain}
end`;
}
module.exports = { getDomain }
What's next
- To learn how to install your package in a repository and import its contents in JavaScript or SQLX files, see Install a package to Dataform.
- To learn more about packages in Dataform, see Reuse code across multiple repositories with packages.
- To learn how to write JavaScript variables and functions that you can reuse in Dataform, see Reuse variables and functions with includes in Dataform.