Operators in Dataflow SQL

Operators are represented by special characters or keywords; they do not use function call syntax. An operator manipulates any number of data inputs, also called operands, and returns a result.

Common conventions:

  • Unless otherwise specified, all operators return NULL when one of the operands is NULL.

The following table lists all Cloud Dataflow SQL operators from highest to lowest precedence, i.e. the order in which they will be evaluated within a statement.

Order of Precedence Operator Input Data Types Name Operator Arity
1 . STRUCT
Member field access operator Binary
  [ ] ARRAY Not supported. To access elements in an ARRAY, use the UNNEST function ( see unnest ARRAY ). Binary
2 - All numeric types Unary minus Unary
3 * All numeric types Multiplication Binary
  / All numeric types Division Binary
4 + All numeric types Addition Binary
  - All numeric types Subtraction Binary
5 (Comparison Operators) = Any comparable type. See Data Types for a complete list. Equal Binary
  < Any comparable type. See Data Types for a complete list. Less than Binary
  > Any comparable type. See Data Types for a complete list. Greater than Binary
  <= Any comparable type. See Data Types for a complete list. Less than or equal to Binary
  >= Any comparable type. See Data Types for a complete list. Greater than or equal to Binary
  !=, <> Any comparable type. See Data Types for a complete list. Not equal Binary
  [NOT] LIKE STRING and byte Value does [not] match the pattern specified Binary
  [NOT] BETWEEN Any comparable types. See Data Types for list. Value is [not] within the range specified Binary
  [NOT] IN Any comparable types. See Data Types for list. Value is [not] in the set of values specified Binary
  IS [NOT] NULL All Value is [not] NULL Unary
  IS [NOT] TRUE BOOL Value is [not] TRUE. Unary
  IS [NOT] FALSE BOOL Value is [not] FALSE. Unary
6 NOT BOOL Logical NOT Unary
7 AND BOOL Logical AND Binary
8 OR BOOL Logical OR Binary

Operators with the same precedence are left associative. This means that those operators are grouped together starting from the left and moving right. For example, the expression:

x AND y AND z

is interpreted as

( ( x AND y ) AND z )

The expression:

x * y / z

is interpreted as:

( ( x * y ) / z )

All comparison operators have the same priority and are grouped using left associativity. However, comparison operators are not associative. As a result, it is recommended that you use parentheses to improve readability and ensure expressions are resolved as desired. For example:

(x < y) IS FALSE

is recommended over:

x < y IS FALSE

Element access operators

Operator Syntax Input Data Types Result Data Type Description
. expression.fieldname1... STRUCT
Type T stored in fieldname1 Dot operator. Can be used to access nested fields, e.g.expression.fieldname1.fieldname2...
[ ] array_expression [position_keyword (int_expression ) ] ARRAY Type T stored in ARRAY Not supported. To access elements in an ARRAY, use the UNNEST function ( see unnest ARRAY ).

Arithmetic operators

All arithmetic operators accept input of numeric type T, and the result type has type T unless otherwise indicated in the description below:

Name Syntax
Addition X + Y
Subtraction X - Y
Multiplication X * Y
Division X / Y
Unary Minus - X

Result types for Addition and Multiplication:

 INT64FLOAT64
INT64INT64FLOAT64
FLOAT64FLOAT64FLOAT64

Result types for Subtraction:

 INT64FLOAT64
INT64INT64FLOAT64
FLOAT64FLOAT64FLOAT64

Result types for Division:

 INT64FLOAT64
INT64FLOAT64FLOAT64
FLOAT64FLOAT64FLOAT64

Result types for Unary Minus:

Input Data Type Result Data Type
INT64 INT64
FLOAT64 FLOAT64

Logical operators

All logical operators allow only BOOL input.

Name Syntax Description
Logical NOT NOT X Returns FALSE if input is TRUE. Returns TRUE if input is FALSE. Returns NULL otherwise.
Logical AND X AND Y Returns FALSE if at least one input is FALSE. Returns TRUE if both X and Y are TRUE. Returns NULL otherwise.
Logical OR X OR Y Returns FALSE if both X and Y are FALSE. Returns TRUE if at least one input is TRUE. Returns NULL otherwise.

Comparison operators

Comparisons always return BOOL. Comparisons generally require both operands to be of the same type. If operands are of different types, and if Cloud Dataflow SQL can convert the values of those types to a common type without loss of precision, Cloud Dataflow SQL will generally coerce them to that common type for the comparison; Cloud Dataflow SQL will generally coerce literals to the type of non-literals, where present. Comparable data types are defined in Data Types.

STRUCTs support only 4 comparison operators: equal (=), not equal (!= and <>), and IN.

The following rules apply when comparing these data types:

  • FLOAT64 : All comparisons with NaN return FALSE, except for != and <>, which return TRUE.
  • BOOL: FALSE is less than TRUE.
  • STRING: Strings are compared codepoint-by-codepoint, which means that canonically equivalent strings are only guaranteed to compare as equal if they have been normalized first.
  • NULL: The convention holds here: any operation with a NULL input returns NULL.
Name Syntax Description
Less Than X < Y Returns TRUE if X is less than Y.
Less Than or Equal To X <= Y Returns TRUE if X is less than or equal to Y.
Greater Than X > Y Returns TRUE if X is greater than Y.
Greater Than or Equal To X >= Y Returns TRUE if X is greater than or equal to Y.
Equal X = Y Returns TRUE if X is equal to Y.
Not Equal X != Y
X <> Y
Returns TRUE if X is not equal to Y.
BETWEEN X [NOT] BETWEEN Y AND Z Returns TRUE if X is [not] within the range specified. The result of "X BETWEEN Y AND Z" is equivalent to "Y <= X AND X <= Z" but X is evaluated only once in the former.
LIKE X [NOT] LIKE Y Checks if the STRING in the first operand X matches a pattern specified by the second operand Y. Expressions can contain these characters:
  • A percent sign "%" matches any number of characters or bytes
  • An underscore "_" matches a single character or byte
  • You can escape "\", "_", or "%" using two backslashes. For example, "\\%". If you are using raw strings, only a single backslash is required. For example, r"\%".
IN Multiple - see below Returns FALSE if the right operand is empty. Returns NULL if the left operand is NULL. Returns TRUE or NULL, never FALSE, if the right operand contains NULL. Arguments on either side of IN are general expressions. Neither operand is required to be a literal, although using a literal on the right is most common. X is evaluated only once.

When testing values that have a STRUCT data type for equality, it's possible that one or more fields are NULL. In such cases:

  • If all non-NULL field values are equal, the comparison returns NULL.
  • If any non-NULL field values are not equal, the comparison returns false.

The following table demonstrates how STRUCT data types are compared when they have fields that are NULL valued.

Struct1 Struct2 Struct1 = Struct2
STRUCT(1, NULL) STRUCT(1, NULL) NULL
STRUCT(1, NULL) STRUCT(2, NULL) FALSE
STRUCT(1,2) STRUCT(1, NULL) NULL

IS operators

IS operators return TRUE or FALSE for the condition they are testing. They never return NULL, even for NULL inputs. If NOT is present, the output BOOL value is inverted.

Function Syntax Input Data Type Result Data Type Description
X IS [NOT] NULL
Any value type BOOL Returns TRUE if the operand X evaluates to NULL, and returns FALSE otherwise.
X IS [NOT] TRUE
BOOL BOOL Returns TRUE if the BOOL operand evaluates to TRUE. Returns FALSE otherwise.
X IS [NOT] FALSE
BOOL BOOL Returns TRUE if the BOOL operand evaluates to FALSE. Returns FALSE otherwise.