Data Transformer functions

This page describes the additional set of predefined Jsonnet functions for Data Transformer Script task available in Application Integration.

To use the following predefined functions, you must import the functions library in your script. Importing the functions library lets you use both the standard Jsonnet functions and the predefined Data Transformer functions.

Application Integration supports Jsonnet functions library v0.20.0. For information about the Jsonnet standard functions, see Jsonnet Standard Library.

Array functions

Average

Syntax 
        
avg(arr)
Description Calculates the numerical average of all the values in an array.
Input parameter arr: The input array.
Return type DOUBLE
Output Returns the average of all the values in the array.

Contains

Syntax 
        
contains(arr, elem)
Description Checks the occurrence of the specified element in an array.
Input parameter arr: The input array.

elem: The array element to search.
Return type BOOLEAN
Output Returns TRUE if a match is found and FALSE otherwise.

Max Array

Syntax 
        
maxArray(arr)
Description Finds the highest value in an array.
Input parameter arr: The input array.
Return type The data type of the input array.
Output Returns the highest value found in the array.

Min Array

Syntax 
        
minArray(arr)
Description Finds the lowest value in an array.
Input parameter arr: The input array.
Return type The data type of the input array.
Output/th> Returns the lowest value found in the array.

Remove

Syntax 
        
remove(arr, elem)
Description Removes the specified element from an array.
Input parameter arr: The input array.

elem: The array element to remove.
Return type The data type of the input array.
Output Returns the updated array after removing the specified element.

Remove At

Syntax 
        
removeAt(arr, index)
Description Removes an element from an array at the specified index.
Input parameter arr: The input array.

index: The array index of the element to remove.
Return type The data type of the input array.
Output Returns the updated array after removing the specified element.

Sum

Syntax 
        
sum(arr)
Description Adds all the values in an array.
Input parameter arr: The input array.
Return type The data type of the input array.
Output Returns the sum of all the elements in the array.

GroupBy

Syntax 
        
groupBy(array, function)
Description Creates an object composed of keys generated from the results of running each element of array through the iteratee function. For example, f.groupBy([1,1.3,1.8],std.floor) would generate {"1": [1,1.3,1.8]}.
Input parameter
  • array: The collection that the method iterates over.

    function: The function that is invoked for every element in the array.
Return type JSON
Output Returns the composed aggregate object.

Zip

Syntax 
        
zip([arrays])
Description It create an array of grouped elements, the first of which contains the first elements of the given arrays, the second of which contains the second element of the given arrays, and so on. For example, f.zip([[1, "one", "I"],[2, "two", "II"],[3, "three", "III"]]) would generate [[1,2,3],["one", "two", "three"],["I", "II", "III"]].
Input parameter [arrays]: This parameter holds the arrays to process.
Return type array
Output Returns the new array of regrouped elements.

Unzip

Syntax 
        
unzip(array)
Description It creates an array of grouped elements. Accepts an array of grouped elements and also creates an array regrouping the elements to their pre-zip configuration. For example, f.unzip([[1, "one", "I"],[2, "two", "II"],[3, "three", "III"]]) would generate [[1,2,3],["one", "two", "three"],["I", "II", "III"]].
Input parameter array: This parameter holds the array of grouped elements to process.
Return type array
Output Returns the new array of regrouped elements.

Aggregate functions

XNOR

Syntax 
        
xnor(a, b)
Description Performs a logical XNOR operation on the specified boolean values.
Input parameter a: A boolean value.

b: A boolean value.
Return type BOOLEAN
Output Returns the XNOR of the two given booleans.

Xor

Syntax 
        
xor(a, b)
Description Performs a logical XOR operation on the specified boolean values.
Input parameter a: A boolean value.

b: A boolean value.
Return type BOOLEAN
Output Returns the XOR of the two given booleans.

Hash functions

SHA1

Syntax 
        
sha1(str)
Description Computes the cryptographic hash of the input STRING using the Secure Hash Algorithm 1 (SHA-1) algorithm.
Input parameter str: The input string.
Return type BYTES
Output Returns a 160 bit (20-byte) hash value.

SHA256

Syntax 
        
sha256(str)
Description Computes the cryptographic hash of the input STRING using the Secure Hash Algorithm 256 (SHA-256) algorithm.
Input parameter str: The input string.
Return type BYTES
Output Returns a 256 bit (32-byte) hash value.

SHA512

Syntax 
        
sha512(str)
Description Computes the cryptographic hash of the input STRING using the Secure Hash Algorithm 512 (SHA-512) algorithm.
Input parameter str: The input string.
Return type BYTES
Output Returns a 512 bit (64-byte) hash value.

SHA3

Syntax 
        
sha3(str)
Description Computes the cryptographic hash of the input STRING using the Secure Hash Algorithm 3 (SHA-3) algorithm.
Input parameter str: The input string.
Return type BYTES
Output Returns a 512 bit (64-byte) hash value.

JSON functions

Manifest XML

Syntax 
        
manifestXml(json, format = "badgerfish", indent_output = false)
Description Converts the specified input JSON object into a XML string.
Input parameter json: The input JSON object.

format: The input JSON representation format.

indent_output: Boolean input to indicate if the output XML string should be indented. Default value is False.
Return type STRING
Output Returns the XML string of the provided JSON object representation.

Object Remove Key

Syntax 
        
objectRemoveKey(obj, key)
Description Removes a property from a JSON object.
Input parameter obj: The input JSON object.

key: The JSON property to remove.
Return type JSON
Output Updated JSON object after removing the specified property and its corresponding value.

Parse XML

Syntax 
        
parseXml(xml, format = "badgerfish")
Description Parses the specified input XML string into a JSON object.
Input parameter xml: The input XML string.

format: The output JSON representation format.

Return type JSON
Output Returns the JSON object representation of the provided XML string.

Parse CSV With Header

Syntax 
        
parseCsvWithHeader(input, delimiter = ",", overwrite_duplicate_headers = true)
Description Parse given input csv string as json. The first row would be considered as header. For example, f.parseCsvWithHeader("index,product,company\n1,Application Integration,Google\n2,Pubsub,Google") would generate [{"index": "1", "product": "Application Integration", "company": "Google"},{"index": "2", "product": "Pubsub", "company": "Google"}].
Input parameter input: The CSV string to be parsed.

delimiter: Delimiter string to be used. Default value is ','.

overwrite_duplicate_headers: Option to overwrite values for the duplicate headers. If set to false, duplicate header would be renamed. Default value is true.
Return type JSON
Output Returns the json representation of given csv string.

Manifest CSV

Syntax 
        
manifestCsv(json, headers = null)
Description Convert given json into csv string. For example, f.manifestCsv([{"index": "1", "product": "Application Integration", "company": "Google"},{"index": "2", "product": "Pubsub", "company": "Google"}]) would generate index,product,company\n1,Application Integration,Google\n2,Pubsub,Google.
Input parameter json: JSON to be converted into csv.

headers: Headers list for csv output. If no value is provided, it would use all the headers.

Return type STRING
Output Returns the csv string from json in given format.

Mathematical functions

Is Decimal

Syntax 
        
isDecimal(num)
Description Checks if the given number is a decimal.
Input parameter num: The input number.
Return type BOOLEAN
Output Returns TRUE if the number is a decimal and FALSE otherwise.

Is Even

Syntax 
        
isEven(num)
Description Checks if the given number is even.
Input parameter num: The input number.
Return type BOOLEAN
Output Returns TRUE if the number is even and FALSE otherwise.

Is Integer

Syntax 
        
isInteger(num)
Description Checks if the given number is an integer.
Input parameter num: The input number.
Return type BOOLEAN
Output Returns TRUE if the number is an integer and FALSE otherwise.

Is Odd

Syntax 
        
isOdd(num)
Description Checks if the given number is odd.
Input parameter num: The input number.
Return type BOOLEAN
Output Returns TRUE if the number is odd and FALSE otherwise.

Random Number

Syntax 
        
randomNumber(lowerBound, upperBound)
Description Generates a random real number between a specified range.
Input parameter lowerBound: Lower range value (inclusive). Default value is 0.

upperBound: Higher range value (exclusive). Default value is 1.
Return type A real number.
Output Returns a random real number between lowerBound and upperBound.

Round

Syntax 
        
round(input)
Description Rounds off a number to the nearest integer.
Input parameter input: A value to round off.
Return type INTEGER
Return value Returns the nearest rounded off integer.

String functions

Equals Ignore Case

Syntax 
        
equalsIgnoreCase(str1, str2)
Description Compare the two given string values irrespective of the string case (lower or upper).
Input parameter str1: An input string.

str2: An input string.
Return type BOOLEAN
Output Returns TRUE if the specified value matches the existing value (ignoring case) and FALSE otherwise.

Is Empty

Syntax 
        
isEmpty(str)
Description Checks if the length of the given string is 0.
Input parameter str: The input string.
Return type BOOLEAN
Output Returns TRUE if the string is empty and FALSE otherwise.

Match

Syntax 
        
match(str, regexp)
Description Retrieves the result of matching a string against a regular expression.
Input parameter str: Input string.

regexp: The re2 regular expression.
Return type ARRAY
Output Returns an array of all substrings of str that match the regular expression, regexp. Returns an empty array if there is no match.

Trim

Syntax 
        
trim(str)
Description Removes all the leading and trailing spaces of a given string.
Input parameter str: The input string.
Return type STRING
Output Returns a copy of the string, with leading and trailing whitespace removed.

Timestamp functions

Date Format

Syntax 
        
dateFormat(timestamp, format_string, timezone, current_format)
Description Formats a timestamp according to the specified format_string.
Input parameter timestamp: The input timestamp.

format_string: The new timestamp format. See Supported timestamp format elements for a list of format elements that can be used to form the new timestamp format string.

timezone: The timezone of the provided input timestamp. Default value is UTC.

current_format: The format which would be used to parse the provided input timestamp. Default value is E (epoch).
Return type STRING
Output Returns a new string after converting the input timestamp based on the specified format_string.

Supported timestamp format elements

The following table lists the supported format elements for the dateFormat() function:

Format element Description Example
YYYY 4-digit year. 2023
YY Last 2 digits of the year. 23
MMMM Name of the month in titlecase. March
MMM Abbreviated name of the month in titlecase. Mar
MM Month from 01 to 12. 03
M Month from 1 to 12. 3
DDDD Name of the day in titlecase. Monday
DDD Abbreviated name of the day. Mon
DD Day from 01 to 31. 06
D Day from 1 to 31. 6
HH Hour in 24 hour format from 00 to 23. 15
hh Hour in 12 hour format from 01 to 12. 03
h Hour in 12 hour format from 1 to 12. 3
mm Minute from 00 to 59. 04
m Minute from 0 to 59. 4
ss Seconds from 00 to 59. 08
s Seconds from 0 to 59. 8
FFFFFFFFF Fractional seconds with 9 digits, including zeros, on the right of the specified digits. 00.123456789
FFFFFFFF Fractional seconds with 8 digits, including zeros, on the right of the specified digits. 00.12345678
FFFFFFF Fractional seconds with 7 digits, including zeros, on the right of the specified digits. 00.1234567
FFFFFF Fractional seconds with 6 digits, including zeros, on the right of the specified digits. 00.123456
FFFFF Fractional seconds with 5 digits, including zeros, on the right of the specified digits. 00.12345
FFFF Fractional seconds with 4 digits, including zeros, on the right of the specified digits. 00.1234
FFF Fractional seconds with 3 digits, including zeros, on the right of the specified digits. 00.123
FF Fractional seconds with 2 digits, including zeros, on the right of the specified digits. 00.12
F Fractional seconds with 1 digits, including zeros, on the right of the specified digits. 00.1
fffffffff Fractional seconds with 9 digits. 987654321
ffffffff Fractional seconds with 8 digits. 98765432
fffffff Fractional seconds with 7 digits. 9876543
ffffff Fractional seconds with 6 digits. 987654
fffff Fractional seconds with 5 digits. 98765
ffff Fractional seconds with 4 digits. 9876
fff Fractional seconds with 3 digits. 987
ff Fractional seconds with 2 digits. 98
f Fractional seconds with 1 digits. 9
tt Meridian indicator (morning or evening). When formatting, this is either AM or PM. PM
t Short meridian indicator (morning or evening). Maps to the lowercase am or pm. pm
K Time zone name. PST
kk Time zone in ±xx:xx format. -8:00:00
k Timezone in ±xxxx format. -8000

Is Leap Year

Syntax 
        
isLeapYear(year)
Description Checks if the given year is a leap year.
Input parameter year: The input year (real number).
Return type BOOLEAN
Output Returns TRUE if the year is a leap year and FALSE otherwise.

Now In Millis

Syntax 
        
nowInMillis()
Description Returns the current Unix epoch time of an integration in milliseconds.
Input parameter N/A
Return type INT64
Output Returns the current Unix epoch time in milliseconds.

Utility functions

UUID

Syntax 
        
uuid()
Description Generates a random universally unique identifier (UUID) as a STRING. The returned STRING consists of 32 hexadecimal digits in five groups separated by hyphens in the form 8-4-4-4-12. The hexadecimal digits represent 122 random bits and 6 fixed bits, in compliance with RFC 4122 section 4.4. The returned STRING is lowercase.
Input parameter N/A
Return type STRING
Output Returns the generated UUID in string format.

Integration name

Syntax 
        
getIntegrationName()
Description Get the name of the current integration.
Input parameter N/A
Return type STRING
Output Returns the name of the current integration.

Integration region

Syntax 
        
getIntegrationRegion()
Description Get the region of the current integration.
Input parameter N/A
Return type STRING
Output Returns the region of the current integration.

Integration version ID

Syntax 
        
getIntegrationVersionId()
Description Get the integration version ID of the current integration.
Input parameter N/A
Return type STRING
Output Returns the integration version ID of the current integration.

Integration version number

Syntax 
        
getIntegrationVersionNumber()
Description Get the version number of the current integration.
Input parameter N/A
Return type NUMBER
Output Returns the version number of the current integration.

Execution ID

Syntax 
        
getExecutionId()
Description Get execution ID of the current integration execution.
Input parameter N/A
Return type STRING
Output Returns the execution ID of the current integration execution.

Project ID

Syntax 
        
getProjectId()
Description Get the Google Cloud project ID of the current integration.
Input parameter N/A
Return type STRING
Output Returns the Google Cloud project ID of the current integration.

Recommendation