Functions & Operators

This page explains BigQuery expressions, including functions and operators.

Function call rules

The following rules apply to all functions unless explicitly indicated otherwise in the function description:

  • For functions that accept numeric types, if one operand is a floating point operand and the other operand is another numeric type, both operands are converted to FLOAT64 before the function is evaluated.

  • If an operand is NULL, the result is NULL, with the exception of the IS operator.

  • For functions that are time zone sensitive (as indicated in the function description), the default time zone, UTC, is used if a time zone is not specified.

Conversion rules

"Conversion" includes, but is not limited to, casting and coercion.

  • Casting is explicit conversion and uses the CAST() function.
  • Coercion is implicit conversion, which BigQuery performs automatically under the conditions described below.
  • There is a third group of conversion functions that have their own function names, such as UNIX_DATE().

The table below summarizes all possible CAST and coercion possibilities for BigQuery data types. "Coercion To" applies to all expressions of a given data type (e.g. a column) , but literals and parameters can also be coerced. See Literal Coercion and Parameter Coercion for details.

From Type CAST to Coercion To
INT64 BOOL
FLOAT64
INT64
STRING
 FLOAT64
FLOAT64 FLOAT64
INT64
STRING
  
BOOL BOOL
INT64
STRING
  
STRING BOOL
BYTES
DATE
DATETIME
FLOAT64
INT64
STRING
TIME
TIMESTAMP
  
BYTES BYTES
STRING
  
DATE DATE
DATETIME
STRING
TIMESTAMP
  
DATETIME DATE
DATETIME
STRING
TIME
TIMESTAMP
  
TIME STRING
TIME
  
TIMESTAMP DATE
DATETIME
STRING
TIME
TIMESTAMP
  
ARRAY ARRAY  
STRUCT STRUCT  

Casting

Syntax:

CAST(expr AS typename)

Cast syntax is used in a query to indicate that the result type of an expression should be converted to some other type.

Example:

CAST(x=1 AS STRING)

This results in "true" if x is 1, "false" for any other non-NULL value, and NULL if x is NULL.

Casts between supported types that do not successfully map from the original value to the target domain produce runtime errors. For example, casting BYTES to STRING where the byte sequence is not valid UTF-8 results in a runtime error.

When casting an expression x of the following types, these rules apply:

From To Rule(s) when casting x
INT64 FLOAT64 Returns a close but potentially not exact FLOAT64 value.
INT64 BOOL Returns FALSE if x is 0, TRUE otherwise.
FLOAT64 INT64 Returns the closest INT64 value.
Halfway cases such as 1.5 or -0.5 round away from zero.
FLOAT64 STRING Returns an approximate string representation.
BOOL INT64 Returns 1 if x is TRUE, 0 otherwise.
BOOL STRING Returns "true" if x is TRUE, "false" otherwise.
STRING FLOAT64 Returns x as a FLOAT64 value, interpreting it as having the same form as a valid FLOAT64 literal.
Also supports casts from "inf", "+inf", "-inf", and "nan".
Conversions are case-insensitive.
STRING BOOL Returns TRUE if x is "true" and FALSE if x is "false"
All other values of x are invalid and throw an error instead of casting to BOOL.
STRINGs are case-insensitive when converting to BOOL.
STRING BYTES STRINGs are cast to BYTES using UTF-8 encoding. For example, the STRING "©", when cast to BYTES, would become a 2-byte sequence with the hex values C2 and A9.
BYTES STRING Returns x interpreted as a UTF-8 STRING.
For example, the BYTES literal b'\xc2\xa9', when cast to STRING, is interpreted as UTF-8 and becomes the unicode character "©".
An error occurs if x is not valid UTF-8.
ARRAY ARRAY Must be the exact same ARRAY type.
STRUCT STRUCT Allowed if the following conditions are met:
  1. The two STRUCTs have the same number of fields.
  2. The original STRUCT field types can be explicitly cast to the corresponding target STRUCT field types (as defined by field order, not field name).

Safe casting

When using CAST, a query can fail if BigQuery is unable to perform the cast. For example, the following query generates an error:

SELECT CAST("apple" AS INT64) AS not_a_number;

If you want to protect your queries from these types of errors, you can use SAFE_CAST. SAFE_CAST is identical to CAST, except it returns NULL instead of raising an error.

SELECT SAFE_CAST("apple" AS INT64) AS not_a_number;

+--------------+
| not_a_number |
+--------------+
| NULL         |
+--------------+

If you are casting from bytes to strings, you can also use the function, SAFE_CONVERT_BYTES_TO_STRING. Any invalid UTF-8 characters are replaced with the unicode replacement character, U+FFFD. See SAFE_CONVERT_BYTES_TO_STRING for more information.

Casting hex strings to integers

If you are working with hex strings (0x123), you can cast those strings as integers:

SELECT '0x123' as hex_value, CAST('0x123' as INT64) as hex_to_int;

+-----------+------------+
| hex_value | hex_to_int |
+-----------+------------+
| 0x123     | 291        |
+-----------+------------+

SELECT '0x123' as hex_value, CAST('-0x123' as INT64) as hex_to_int;

+-----------+------------+
| hex_value | hex_to_int |
+-----------+------------+
| 0x123     | -291       |
+-----------+------------+

Casting date types

BigQuery supports casting date types to/from strings as follows:

CAST(date_expression AS STRING)
CAST(string_expression AS DATE)

Casting from a date type to a string is independent of time zone and is of the form YYYY-MM-DD. When casting from string to date, the string must conform to the supported date literal format, and is independent of time zone. If the string expression is invalid or represents a date that is outside of the supported min/max range, then an error is produced.

Casting timestamp types

BigQuery supports casting timestamp types to/from strings as follows:

CAST(timestamp_expression AS STRING)
CAST(string_expression AS TIMESTAMP)

When casting from timestamp types to string, the timestamp is interpreted using the default time zone, UTC. The number of subsecond digits produced depends on the number of trailing zeroes in the subsecond part: the CAST function will truncate zero, three, or six digits.

When casting from string to a timestamp, string_expression must conform to the supported timestamp literal formats, or else a runtime error occurs. The string_expression may itself contain a time_zone—see time zones. If there is a time zone in the string_expression, that time zone is used for conversion, otherwise the default time zone, UTC, is used. If the string has fewer than six digits, then it is implicitly widened.

An error is produced if the string_expression is invalid, has more than six subsecond digits (i.e. precision greater than microseconds), or represents a time outside of the supported timestamp range.

Casting between date and timestamp types

BigQuery supports casting between date and timestamp types as follows:

CAST(date_expression AS TIMESTAMP)
CAST(timestamp_expression AS DATE)

Casting from a date to a timestamp interprets date_expression as of midnight (start of the day) in the default time zone, UTC. Casting from a timestamp to date effectively truncates the timestamp as of the default time zone.

Coercion

BigQuery coerces the result type of an expression to another type if needed to match function signatures. For example, if function func() is defined to take a single argument of type INT64 and an expression is used as an argument that has a result type of FLOAT64, then the result of the expression will be coerced to INT64 type before func() is computed.

Literal coercion

BigQuery supports the following literal coercions:

Input Data Type Result Data Type Notes
STRING literal DATE
TIMESTAMP

Literal coercion is needed when the actual literal type is different from the type expected by the function in question. For example, if function func() takes a DATE argument, then the expression func("2014-09-27") is valid because the STRING literal "2014-09-27" is coerced to DATE.

Literal conversion is evaluated at analysis time, and gives an error if the input literal cannot be converted successfully to the target type.

Note: String literals do not coerce to numeric types.

Parameter coercion

BigQuery supports the following parameter coercions:

Input Data Type Result Data Type
STRING parameter

If the parameter value cannot be coerced successfully to the target type, an error is provided.

Additional conversion functions

BigQuery provides the following additional conversion functions:

Aggregate functions

An aggregate function is a function that performs a calculation on a set of values. COUNT, MIN and MAX are examples of aggregate functions.

SELECT COUNT(*) as total_count, COUNT(fruit) as non_null_count,
       MIN(fruit) as min, MAX(fruit) as max
FROM UNNEST([NULL, "apple", "pear", "orange"]) as fruit;
+-------------+----------------+-------+------+
| total_count | non_null_count | min   | max  |
+-------------+----------------+-------+------+
| 4           | 3              | apple | pear |
+-------------+----------------+-------+------+

The following sections describe the aggregate functions that BigQuery supports.

ANY_VALUE

ANY_VALUE(expression)  [OVER (...)]

Description

Returns any value from the input or NULL if there are zero input rows. The value returned is non-deterministic, which means you might receive a different result each time you use this function.

Supported Argument Types

Any

Optional Clause

OVER: Specifies a window. See Analytic Functions.

Returned Data Types

Matches the input data type.

Examples

SELECT ANY_VALUE(fruit) as any_value
FROM UNNEST(["apple", "banana", "pear"]) as fruit;

+-----------+
| any_value |
+-----------+
| apple     |
+-----------+

SELECT
  fruit,
  ANY_VALUE(fruit) OVER (ORDER BY LENGTH(fruit) ROWS BETWEEN 1 PRECEDING AND CURRENT ROW) AS any_value
FROM UNNEST(["apple", "banana", "pear"]) as fruit;

+--------+-----------+
| fruit  | any_value |
+--------+-----------+
| pear   | pear      |
| apple  | pear      |
| banana | apple     |
+--------+-----------+

ARRAY_AGG

ARRAY_AGG([DISTINCT]  expression  [{IGNORE|RESPECT} NULLS]
          [ORDER BY key [{ASC|DESC}] [, ... ]]  [LIMIT n])
[OVER (...)]

Description

Returns an ARRAY of expression values.

Supported Argument Types

All data types except ARRAY.

Optional Clauses

The clauses are applied in the following order:

  1. OVER: Specifies a window. See Analytic Functions. This clause is currently incompatible with all other clauses within ARRAY_AGG().

  2. DISTINCT: Each distinct value of expression is aggregated only once into the result.

  3. IGNORE NULLS or RESPECT NULLS: If IGNORE NULLS is specified, the NULL values are excluded from the result. If RESPECT NULLS is specified or if neither is specified, the NULL values are included in the result (an error is raised, however, if an array in the final query result contains a NULL element).

  4. ORDER BY: Specifies the order of the values.

    • For each sort key, the default sort direction is ASC.
    • NULLs: In the context of the ORDER BY clause, NULLs are the minimum possible value; that is, NULLs appear first in ASC sorts and last in DESC sorts.
    • Floating point data types: see Floating Point Semantics on ordering and grouping.
    • If DISTINCT is also specified, then the sort key must be the same as expression.
    • If ORDER BY is not specified, the order of the elements in the output array is non-deterministic, which means you might receive a different result each time you use this function.

  5. LIMIT: Specifies the maximum number of expression inputs in the result. The limit n must be a constant INT64.

Returned Data Types

ARRAY

If there are zero input rows, this function returns NULL.

Examples

SELECT FORMAT("%T", ARRAY_AGG(x)) AS array_agg
FROM UNNEST([NULL, 1, -2, 3, -2, 1, NULL]) AS x;

+-------------------------------+
| array_agg                     |
+-------------------------------+
| [NULL, 1, -2, 3, -2, 1, NULL] |
+-------------------------------+

SELECT FORMAT("%T", ARRAY_AGG(DISTINCT x)) AS array_agg
FROM UNNEST([NULL, 1, -2, 3, -2, 1, NULL]) AS x;

+------------------+
| array_agg        |
+------------------+
| [1, 3, -2, NULL] |
+------------------+

SELECT FORMAT("%T", ARRAY_AGG(x IGNORE NULLS)) AS array_agg
FROM UNNEST([NULL, 1, -2, 3, -2, 1, NULL]) AS x;

+-------------------+
| array_agg         |
+-------------------+
| [1, -2, 3, -2, 1] |
+-------------------+

SELECT FORMAT("%T", ARRAY_AGG(x ORDER BY ABS(x))) AS array_agg
FROM UNNEST([NULL, 1, -2, 3, -2, 1, NULL]) AS x;
+-------------------------------+
| array_agg                     |
+-------------------------------+
| [NULL, NULL, 1, 1, -2, -2, 3] |
+-------------------------------+

SELECT FORMAT("%T", ARRAY_AGG(x LIMIT 5)) AS array_agg
FROM UNNEST([NULL, 1, -2, 3, -2, 1, NULL]) AS x;

+----------------------+
| array_agg            |
+----------------------+
| [NULL, 1, -2, 3, -2] |
+----------------------+

SELECT FORMAT("%T", ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY x LIMIT 2)) AS array_agg
FROM UNNEST([NULL, 1, -2, 3, -2, 1, NULL]) AS x;

+-----------+
| array_agg |
+-----------+
| [-2, 1]   |
+-----------+

SELECT
  x,
  FORMAT("%T", ARRAY_AGG(x) OVER (ORDER BY ABS(x))) AS array_agg
FROM UNNEST([NULL, 1, -2, 3, -2, 1, NULL]) AS x;

+------+-------------------------------+
| x    | array_agg                     |
+------+-------------------------------+
| NULL | [NULL, NULL]                  |
| NULL | [NULL, NULL]                  |
| 1    | [NULL, NULL, 1, 1]            |
| 1    | [NULL, NULL, 1, 1]            |
| -2   | [NULL, NULL, 1, 1, -2, -2]    |
| -2   | [NULL, NULL, 1, 1, -2, -2]    |
| 3    | [NULL, NULL, 1, 1, -2, -2, 3] |
+------+-------------------------------+

ARRAY_CONCAT_AGG

ARRAY_CONCAT_AGG(expression  [ORDER BY key [{ASC|DESC}] [, ... ]]  [LIMIT n])

Description

Concatenates elements from expression of type ARRAY, returning a single ARRAY as a result. This function ignores NULL input arrays, but respects the NULL elements in non-NULL input arrays (an error is raised, however, if an array in the final query result contains a NULL element).

Supported Argument Types

ARRAY

Optional Clauses

The clauses are applied in the following order:

  1. ORDER BY: Specifies the order of the values.

    • For each sort key, the default sort direction is ASC.
    • Array ordering is not supported, and thus the sort key cannot be the same as expression.
    • NULLs: In the context of the ORDER BY clause, NULLs are the minimum possible value; that is, NULLs appear first in ASC sorts and last in DESC sorts.
    • Floating point data types: see Floating Point Semantics on ordering and grouping.
    • If ORDER BY is not specified, the order of the elements in the output array is non-deterministic, which means you might receive a different result each time you use this function.

  2. LIMIT: Specifies the maximum number of expression inputs in the result. The limit applies to the number of input arrays, not the number of elements in the arrays. An empty array counts as 1. A NULL array is not counted. The limit n must be a constant INT64.

Returned Data Types

ARRAY

Returns NULL if there are zero input rows or expression evaluates to NULL for all rows.

Examples

SELECT FORMAT("%T", ARRAY_CONCAT_AGG(x)) AS array_concat_agg FROM (
  SELECT [NULL, 1, 2, 3, 4] AS x
  UNION ALL SELECT NULL
  UNION ALL SELECT [5, 6]
  UNION ALL SELECT [7, 8, 9]
);

+-----------------------------------+
| array_concat_agg                  |
+-----------------------------------+
| [NULL, 1, 2, 3, 4, 5, 6, 7, 8, 9] |
+-----------------------------------+

SELECT FORMAT("%T", ARRAY_CONCAT_AGG(x ORDER BY ARRAY_LENGTH(x))) AS array_concat_agg FROM (
  SELECT [NULL, 1, 2, 3, 4] AS x
  UNION ALL SELECT NULL
  UNION ALL SELECT [5, 6]
  UNION ALL SELECT [7, 8, 9]
);

+-----------------------------------+
| array_concat_agg                  |
+-----------------------------------+
| [5, 6, 7, 8, 9, NULL, 1, 2, 3, 4] |
+-----------------------------------+

SELECT FORMAT("%T", ARRAY_CONCAT_AGG(x LIMIT 2)) AS array_concat_agg FROM (
  SELECT [NULL, 1, 2, 3, 4] AS x
  UNION ALL SELECT NULL
  UNION ALL SELECT [5, 6]
  UNION ALL SELECT [7, 8, 9]
);

+--------------------------+
| array_concat_agg         |
+--------------------------+
| [NULL, 1, 2, 3, 4, 5, 6] |
+--------------------------+

SELECT FORMAT("%T", ARRAY_CONCAT_AGG(x ORDER BY ARRAY_LENGTH(x) LIMIT 2)) AS array_concat_agg FROM (
  SELECT [NULL, 1, 2, 3, 4] AS x
  UNION ALL SELECT NULL
  UNION ALL SELECT [5, 6]
  UNION ALL SELECT [7, 8, 9]
);

+------------------+
| array_concat_agg |
+------------------+
| [5, 6, 7, 8, 9]  |
+------------------+

AVG

AVG([DISTINCT]  expression)  [OVER (...)]

Description

Returns the average of non-NULL input values, or NaN if the input contains a NaN.

Supported Argument Types

Any numeric input type, such as INT64. Note that, for floating point input types, the return result is non-deterministic, which means you might receive a different result each time you use this function.

Optional Clauses

The clauses are applied in the following order:

  1. OVER: Specifies a window. See Analytic Functions. This clause is currently incompatible with all other clauses within AVG().

  2. DISTINCT: Each distinct value of expression is aggregated only once into the result.

Returned Data Types

FLOAT64

Examples

SELECT AVG(x) as avg
FROM UNNEST([0, 2, NULL, 4, 4, 5]) as x;

+-----+
| avg |
+-----+
| 3   |
+-----+

SELECT AVG(DISTINCT x) AS avg
FROM UNNEST([0, 2, NULL, 4, 4, 5]) AS x;

+------+
| avg  |
+------+
| 2.75 |
+------+

SELECT
  x,
  AVG(x) OVER (ORDER BY x ROWS BETWEEN 1 PRECEDING AND CURRENT ROW) AS avg
FROM UNNEST([0, 2, NULL, 4, 4, 5]) AS x;

+------+------+
| x    | avg  |
+------+------+
| NULL | NULL |
| 0    | 0    |
| 2    | 1    |
| 4    | 3    |
| 4    | 4    |
| 5    | 4.5  |
+------+------+

BIT_AND

BIT_AND(expression)

Description

Performs a bitwise AND operation on expression and returns the result.

Supported Argument Types

INT64

Returned Data Types

INT64

Examples

SELECT BIT_AND(x) as bit_and FROM UNNEST([0xF001, 0x00A1]) as x;

+---------+
| bit_and |
+---------+
| 1       |
+---------+

BIT_OR

BIT_OR(expression)

Description

Performs a bitwise OR operation on expression and returns the result.

Supported Argument Types

INT64

Returned Data Types

INT64

Examples

SELECT BIT_OR(x) as bit_or FROM UNNEST([0xF001, 0x00A1]) as x;

+--------+
| bit_or |
+--------+
| 61601  |
+--------+

BIT_XOR

BIT_XOR([DISTINCT]  expression)

Description

Performs a bitwise XOR operation on expression and returns the result.

Supported Argument Types

INT64

Optional Clause

DISTINCT: Each distinct value of expression is aggregated only once into the result.

Returned Data Types

INT64

Examples

SELECT BIT_XOR(x) AS bit_xor FROM UNNEST([5678, 1234]) AS x;

+---------+
| bit_xor |
+---------+
| 4860    |
+---------+

SELECT BIT_XOR(x) AS bit_xor FROM UNNEST([1234, 5678, 1234]) AS x;
+---------+
| bit_xor |
+---------+
| 5678    |
+---------+

SELECT BIT_XOR(DISTINCT x) AS bit_xor FROM UNNEST([1234, 5678, 1234]) AS x;

+---------+
| bit_xor |
+---------+
| 4860    |
+---------+

COUNT

1. COUNT(*)  [OVER (...)]
2. COUNT([DISTINCT]  expression)  [OVER (...)]

Description

  1. Returns the number of rows in the input.
  2. Returns the number of rows with expression evaluated to any value other than NULL.

Supported Argument Types

expression can be any data type.

Optional Clauses

The clauses are applied in the following order:

  1. OVER: Specifies a window. See Analytic Functions.

  2. DISTINCT: Each distinct value of expression is aggregated only once into the result.

Return Data Types

INT64

Examples

SELECT
  COUNT(*) AS count_star,
  COUNT(x) AS count_x,
  COUNT(DISTINCT x) AS count_dist_x
FROM UNNEST([1, 4, NULL, 4, 5]) AS x;

+------------+---------+--------------+
| count_star | count_x | count_dist_x |
+------------+---------+--------------+
| 5          | 4       | 3            |
+------------+---------+--------------+

SELECT
  x,
  COUNT(*) OVER (PARTITION BY MOD(x, 3)) AS count_star,
  COUNT(x) OVER (PARTITION BY MOD(x, 3)) AS count_x,
  COUNT(DISTINCT x) OVER (PARTITION BY MOD(x, 3)) AS count_dist_x
FROM UNNEST([1, 4, NULL, 4, 5]) AS x;

+------+------------+---------+--------------+
| x    | count_star | count_x | count_dist_x |
+------+------------+---------+--------------+
| NULL | 1          | 0       | 0            |
| 1    | 3          | 3       | 2            |
| 4    | 3          | 3       | 2            |
| 4    | 3          | 3       | 2            |
| 5    | 1          | 1       | 1            |
+------+------------+---------+--------------+

COUNTIF

COUNTIF(expression)  [OVER (...)]

Description

Returns the count of TRUE values for expression. Returns 0 if there are zero input rows or expression evaluates to FALSE for all rows.

Supported Argument Types

BOOL

Optional Clause

OVER: Specifies a window. See Analytic Functions.

Return Data Types

INT64

Examples

SELECT COUNTIF(x<0) AS num_negative, COUNTIF(x>0) AS num_positive
FROM UNNEST([5, -2, 3, 6, -10, NULL, -7, 4, 0]) AS x;

+--------------+--------------+
| num_negative | num_positive |
+--------------+--------------+
| 3            | 4            |
+--------------+--------------+

SELECT
  x,
  COUNTIF(x<0) OVER (ORDER BY ABS(x) ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING) AS num_negative
FROM UNNEST([5, -2, 3, 6, -10, NULL, -7, 4, 0]) AS x;

+------+--------------+
| x    | num_negative |
+------+--------------+
| NULL | 0            |
| 0    | 1            |
| -2   | 1            |
| 3    | 1            |
| 4    | 0            |
| 5    | 0            |
| 6    | 1            |
| -7   | 2            |
| -10  | 2            |
+------+--------------+

LOGICAL_AND

LOGICAL_AND(expression)

Description

Returns the logical AND of all non-NULL expressions. Returns NULL if there are zero input rows or expression evaluates to NULL for all rows.

Supported Argument Types

BOOL

Return Data Types

BOOL

Examples

SELECT LOGICAL_AND(x) as logical_and FROM UNNEST([true, false, true]) as x;

+-------------+
| logical_and |
+-------------+
| false       |
+-------------+

LOGICAL_OR

LOGICAL_OR(expression)

Description

Returns the logical OR of all non-NULL expressions. Returns NULL if there are zero input rows or expression evaluates to NULL for all rows.

Supported Argument Types

BOOL

Return Data Types

BOOL

Examples

SELECT LOGICAL_OR(x) as logical_or FROM UNNEST([true, false, true]) as x;

+------------+
| logical_or |
+------------+
| true       |
+------------+

MAX

MAX(expression)  [OVER (...)]

Description

Returns the maximum value of non-NULL expressions. Returns NULL if there are zero input rows or expression evaluates to NULL for all rows. Returns NaN if the input contains a NaN.

Supported Argument Types

Any data type except: STRUCT ARRAY

Optional Clause

OVER: Specifies a window. See Analytic Functions.

Return Data Types

Same as the data type used as the input values.

Examples

SELECT MAX(x) AS max
FROM UNNEST([8, NULL, 37, 4, NULL, 55]) AS x;

+-----+
| max |
+-----+
| 55  |
+-----+

SELECT x, MAX(x) OVER (PARTITION BY MOD(x, 2)) AS max
FROM UNNEST([8, NULL, 37, 4, NULL, 55]) AS x;

+------+------+
| x    | max  |
+------+------+
| NULL | NULL |
| NULL | NULL |
| 8    | 8    |
| 4    | 8    |
| 37   | 55   |
| 55   | 55   |
+------+------+

MIN

MIN(expression)  [OVER (...)]

Description

Returns the minimum value of non-NULL expressions. Returns NULL if there are zero input rows or expression evaluates to NULL for all rows. Returns NaN if the input contains a NaN.

Supported Argument Types

Any data type except: STRUCT ARRAY

Optional Clause

OVER: Specifies a window. See Analytic Functions.

Return Data Types

Same as the data type used as the input values.

Examples

SELECT MIN(x) AS min
FROM UNNEST([8, NULL, 37, 4, NULL, 55]) AS x;

+-----+
| min |
+-----+
| 4   |
+-----+

SELECT x, MIN(x) OVER (PARTITION BY MOD(x, 2)) AS min
FROM UNNEST([8, NULL, 37, 4, NULL, 55]) AS x;

+------+------+
| x    | min  |
+------+------+
| NULL | NULL |
| NULL | NULL |
| 8    | 4    |
| 4    | 4    |
| 37   | 37   |
| 55   | 37   |
+------+------+

STRING_AGG

STRING_AGG([DISTINCT]  expression [, delimiter]  [ORDER BY key [{ASC|DESC}] [, ... ]]  [LIMIT n])
[OVER (...)]

Description

Returns a value (either STRING or BYTES) obtained by concatenating non-null values.

If a delimiter is specified, concatenated values are separated by that delimiter; otherwise, a comma is used as a delimiter.

Supported Argument Types

STRING BYTES

Optional Clauses

The clauses are applied in the following order:

  1. OVER: Specifies a window. See Analytic Functions. This clause is currently incompatible with all other clauses within STRING_AGG().

  2. DISTINCT: Each distinct value of expression is aggregated only once into the result.

  3. ORDER BY: Specifies the order of the values.

    • For each sort key, the default sort direction is ASC.
    • NULLs: In the context of the ORDER BY clause, NULLs are the minimum possible value; that is, NULLs appear first in ASC sorts and last in DESC sorts.
    • Floating point data types: see Floating Point Semantics on ordering and grouping.
    • If DISTINCT is also specified, then the sort key must be the same as expression.
    • If ORDER BY is not specified, the order of the elements in the output array is non-deterministic, which means you might receive a different result each time you use this function.

  4. LIMIT: Specifies the maximum number of expression inputs in the result. The limit applies to the number of input strings, not the number of characters or bytes in the inputs. An empty string counts as 1. A NULL string is not counted. The limit n must be a constant INT64.

Return Data Types

STRING BYTES

Examples

SELECT STRING_AGG(fruit) AS string_agg
FROM UNNEST(["apple", NULL, "pear", "banana", "pear"]) AS fruit;

+------------------------+
| string_agg             |
+------------------------+
| apple,pear,banana,pear |
+------------------------+

SELECT STRING_AGG(fruit, " & ") AS string_agg
FROM UNNEST(["apple", NULL, "pear", "banana", "pear"]) AS fruit;

+------------------------------+
| string_agg                   |
+------------------------------+
| apple & pear & banana & pear |
+------------------------------+

SELECT STRING_AGG(DISTINCT fruit, " & ") AS string_agg
FROM UNNEST(["apple", NULL, "pear", "banana", "pear"]) AS fruit;

+-----------------------+
| string_agg            |
+-----------------------+
| apple & banana & pear |
+-----------------------+

SELECT STRING_AGG(fruit, " & " ORDER BY LENGTH(fruit)) AS string_agg
FROM UNNEST(["apple", NULL, "pear", "banana", "pear"]) AS fruit;

+------------------------------+
| string_agg                   |
+------------------------------+
| pear & pear & apple & banana |
+------------------------------+

SELECT STRING_AGG(fruit, " & " LIMIT 2) AS string_agg
FROM UNNEST(["apple", NULL, "pear", "banana", "pear"]) AS fruit;

+--------------+
| string_agg   |
+--------------+
| apple & pear |
+--------------+

SELECT STRING_AGG(DISTINCT fruit, " & " ORDER BY fruit DESC LIMIT 2) AS string_agg
FROM UNNEST(["apple", NULL, "pear", "banana", "pear"]) AS fruit;

+---------------+
| string_agg    |
+---------------+
| pear & banana |
+---------------+

SELECT
  fruit,
  STRING_AGG(fruit, " & ") OVER (ORDER BY LENGTH(fruit)) AS string_agg
FROM UNNEST(["apple", NULL, "pear", "banana", "pear"]) AS fruit;

+--------+------------------------------+
| fruit  | string_agg                   |
+--------+------------------------------+
| NULL   | NULL                         |
| pear   | pear & pear                  |
| pear   | pear & pear                  |
| apple  | pear & pear & apple          |
| banana | pear & pear & apple & banana |
+--------+------------------------------+

SUM

SUM([DISTINCT]  expression)  [OVER (...)]

Description

Returns the sum of non-null values.

If the expression is a floating point value, the sum is non-deterministic, which means you might receive a different result each time you use this function.

Supported Argument Types

Any supported numeric data types.

Optional Clauses

The clauses are applied in the following order:

  1. OVER: Specifies a window. See Analytic Functions.

  2. DISTINCT: Each distinct value of expression is aggregated only once into the result.

Return Data Types

Returns INT64 if the input is an integer.

Returns FLOAT64 if the input is a floating point value.

Returns NULL if the input contains only NULLs.

Returns Inf if the input contains Inf.

Returns -Inf if the input contains -Inf.

Returns NaN if the input contains a NaN.

Returns NaN if the input contains a combination of Inf and -Inf.

Examples

SELECT SUM(x) AS sum
FROM UNNEST([1, 2, 3, 4, 5, 4, 3, 2, 1]) AS x;

+-----+
| sum |
+-----+
| 25  |
+-----+

SELECT SUM(DISTINCT x) AS sum
FROM UNNEST([1, 2, 3, 4, 5, 4, 3, 2, 1]) AS x;

+-----+
| sum |
+-----+
| 15  |
+-----+

SELECT
  x,
  SUM(x) OVER (PARTITION BY MOD(x, 3)) AS sum
FROM UNNEST([1, 2, 3, 4, 5, 4, 3, 2, 1]) AS x;

+---+-----+
| x | sum |
+---+-----+
| 3 | 6   |
| 3 | 6   |
| 1 | 10  |
| 4 | 10  |
| 4 | 10  |
| 1 | 10  |
| 2 | 9   |
| 5 | 9   |
| 2 | 9   |
+---+-----+

SELECT
  x,
  SUM(DISTINCT x) OVER (PARTITION BY MOD(x, 3)) AS sum
FROM UNNEST([1, 2, 3, 4, 5, 4, 3, 2, 1]) AS x;

+---+-----+
| x | sum |
+---+-----+
| 3 | 3   |
| 3 | 3   |
| 1 | 5   |
| 4 | 5   |
| 4 | 5   |
| 1 | 5   |
| 2 | 7   |
| 5 | 7   |
| 2 | 7   |
+---+-----+

Statistical Aggregate Functions

BigQuery supports the following statistical aggregate functions.

CORR

CORR(X1, X2)

Description

Returns the Pearson coefficient of correlation of a set of number pairs. For each number pair, the first number is the dependent variable and the second number is the independent variable. The return result is between -1 and 1. A result of 0 indicates no correlation.

This function ignores any input pairs that contain one or more NULL values. If there are fewer than two input pairs without NULL values, this function returns NULL.

Supported Input Types

FLOAT64

Return Data Type

FLOAT64

COVAR_POP

COVAR_POP(X1, X2)

Description

Returns the population covariance of a set of number pairs. The first number is the dependent variable; the second number is the independent variable. The return result is between -Inf and +Inf.

This function ignores any input pairs that contain one or more NULL values. If there is no input pair without NULL values, this function returns NULL. If there is exactly one input pair without NULL values, this function returns 0.

Supported Input Types

FLOAT64

Return Data Type

FLOAT64

COVAR_SAMP

COVAR_SAMP(X1, X2)

Description

Returns the sample covariance of a set of number pairs. The first number is the dependent variable; the second number is the independent variable. The return result is between -Inf and +Inf.

This function ignores any input pairs that contain one or more NULL values. If there are fewer than two input pairs without NULL values, this function returns NULL.

Supported Input Types

FLOAT64

Return Data Type

FLOAT64

STDDEV_POP

STDDEV_POP([DISTINCT]  expression)  [OVER (...)]

Description

Returns the population (biased) standard deviation of the values. The return result is between 0 and +Inf.

This function ignores any NULL inputs. If all inputs are ignored, this function returns NULL.

If this function receives a single non-NULL input, it returns 0.

Supported Input Types

FLOAT64

Optional Clauses

The clauses are applied in the following order:

  1. OVER: Specifies a window. See Analytic Functions. This clause is currently incompatible with all other clauses within STDDEV_POP().

  2. DISTINCT: Each distinct value of expression is aggregated only once into the result.

Return Data Type

FLOAT64

STDDEV_SAMP

STDDEV_SAMP([DISTINCT]  expression)  [OVER (...)]

Description

Returns the sample (biased) standard deviation of the values. The return result is between 0 and +Inf.

This function ignores any NULL inputs. If there are fewer than two non-NULL inputs, this function returns NULL.

Supported Input Types

FLOAT64

Optional Clauses

The clauses are applied in the following order:

  1. OVER: Specifies a window. See Analytic Functions. This clause is currently incompatible with all other clauses within STDDEV_SAMP().

  2. DISTINCT: Each distinct value of expression is aggregated only once into the result.

Return Data Type

FLOAT64

STDDEV

STDDEV([DISTINCT]  expression)  [OVER (...)]

Description

An alias of STDDEV_SAMP.

VAR_POP

VAR_POP([DISTINCT]  expression)  [OVER (...)]

Description

Returns the population (biased) variance of the values. The return result is between 0 and +Inf.

This function ignores any NULL inputs. If all inputs are ignored, this function returns NULL.

If this function receives a single non-NULL input, it returns 0.

Supported Input Types

FLOAT64

Optional Clauses

The clauses are applied in the following order:

  1. OVER: Specifies a window. See Analytic Functions. This clause is currently incompatible with all other clauses within VAR_POP().

  2. DISTINCT: Each distinct value of expression is aggregated only once into the result.

Return Data Type

FLOAT64

VAR_SAMP

VAR_SAMP([DISTINCT]  expression)  [OVER (...)]

Description

Returns the sample (unbiased) variance of the values. The return result is between 0 and +Inf.

This function ignores any NULL inputs. If there are fewer than two non-NULL inputs, this function returns NULL.

Supported Input Types

FLOAT64

Optional Clauses

The clauses are applied in the following order:

  1. OVER: Specifies a window. See Analytic Functions. This clause is currently incompatible with all other clauses within VAR_SAMP().

  2. DISTINCT: Each distinct value of expression is aggregated only once into the result.

Return Data Type

FLOAT64

VARIANCE

VARIANCE([DISTINCT]  expression)  [OVER (...)]

Description

An alias of VAR_SAMP.

Approximate Aggregate Functions

Approximate aggregate functions are scalable in terms of memory usage and time, but produce approximate results instead of exact results.

APPROX_COUNT_DISTINCT

APPROX_COUNT_DISTINCT(expression)

Description

Returns the approximate result for COUNT(DISTINCT expression). The value returned is a statistical estimate—not necessarily the actual value.

This function is less accurate than COUNT(DISTINCT expression), but performs better on huge input.

Supported Argument Types

All data types except:

  • ARRAY
  • STRUCT
  • FLOAT64

Returned Data Types

INT64

Examples

SELECT APPROX_COUNT_DISTINCT(x) as approx_distinct
FROM UNNEST([0, 1, 1, 2, 3, 5]) as x;

+-----------------+
| approx_distinct |
+-----------------+
| 5               |
+-----------------+

APPROX_QUANTILES

APPROX_QUANTILES([DISTINCT]  expression, number  [{IGNORE|RESPECT} NULLS])

Description

Returns the approximate boundaries for a group of expression values, where number represents the number of quantiles to create. This function returns an array of number + 1 elements, where the first element is the approximate minimum and the last element is the approximate maximum.

Supported Argument Types

expression can be any supported data type except:

  • ARRAY
  • STRUCT

number must be INT64.

Optional Clauses

The clauses are applied in the following order:

  1. DISTINCT: Each distinct value of expression is aggregated only once into the result.

  2. IGNORE NULLS or RESPECT NULLS: If IGNORE NULLS is specified, the NULL values are excluded from the result. If RESPECT NULLS is specified or if neither is specified, the NULL values are included in the result (an error is raised, however, if an array in the final query result contains a NULL element).

Returned Data Types

An ARRAY of the type specified by the expression parameter.

Returns NULL if there are zero input rows or expression evaluates to NULL for all rows.

Examples

SELECT APPROX_QUANTILES(x, 2) AS approx_quantiles
FROM UNNEST([NULL, NULL, 1, 1, 1, 4, 5, 6, 7, 8, 9, 10]) AS x;

+------------------+
| approx_quantiles |
+------------------+
| [1, 5, 10]       |
+------------------+

SELECT APPROX_QUANTILES(DISTINCT x, 2) AS approx_quantiles
FROM UNNEST([NULL, NULL, 1, 1, 1, 4, 5, 6, 7, 8, 9, 10]) AS x;

+------------------+
| approx_quantiles |
+------------------+
| [1, 6, 10]       |
+------------------+

SELECT FORMAT("%T", APPROX_QUANTILES(x, 2 RESPECT NULLS)) AS approx_quantiles
FROM UNNEST([NULL, NULL, 1, 1, 1, 4, 5, 6, 7, 8, 9, 10]) AS x;

+------------------+
| approx_quantiles |
+------------------+
| [NULL, 4, 10]    |
+------------------+

SELECT FORMAT("%T", APPROX_QUANTILES(DISTINCT x, 2 RESPECT NULLS)) AS approx_quantiles
FROM UNNEST([NULL, NULL, 1, 1, 1, 4, 5, 6, 7, 8, 9, 10]) AS x;

+------------------+
| approx_quantiles |
+------------------+
| [NULL, 6, 10]    |
+------------------+

APPROX_TOP_COUNT

APPROX_TOP_COUNT(expression, number)

Description

Returns the approximate top elements of expression. The number parameter specifies the number of elements returned.

Supported Argument Types

expression can be of any data type that the GROUP BY clause supports.

number must be INT64.

Returned Data Types

An ARRAY of type STRUCT. The STRUCT contains two fields. The first field contains an input value. The second field contains an INT64 specifying the number of times the input was returned.

Returns NULL if there are zero input rows.

Examples

SELECT APPROX_TOP_COUNT(x, 2) as approx_top_count
FROM UNNEST(["apple", "apple", "pear", "pear", "pear", "banana"]) as x;

+-------------------------+
| approx_top_count        |
+-------------------------+
| [{pear, 3}, {apple, 2}] |
+-------------------------+

NULL handling

APPROX_TOP_COUNT does not ignore NULLs in the input. For example:

SELECT APPROX_TOP_COUNT(x, 2) as approx_top_count
FROM UNNEST([NULL, "pear", "pear", "pear", "apple", NULL]) as x;

+------------------------+
| approx_top_count       |
+------------------------+
| [{pear, 3}, {NULL, 2}] |
+------------------------+

APPROX_TOP_SUM

APPROX_TOP_SUM(expression, weight, number)

Description

Returns the approximate top elements of expression, based on the sum of an assigned weight. The number parameter specifies the number of elements returned.

If the weight input is negative or NaN, this function returns an error.

Supported Argument Types

expression can be of any data type that the GROUP BY clause supports.

weight must be one of the following:

  • INT64
  • FLOAT64

number must be INT64.

Returned Data Types

An ARRAY of type STRUCT. The STRUCT contains two fields: value and sum. The value field contains the value of the input expression. The sum field is the same type asweight, and is the approximate sum of the input weight associated with the value field.

Returns NULL if there are zero input rows.

Examples

SELECT APPROX_TOP_SUM(x, weight, 2) AS approx_top_sum FROM
UNNEST([
  STRUCT("apple" AS x, 3 AS weight),
  ("pear", 2),
  ("apple", 0),
  ("banana", 5),
  ("pear", 4)
]);

+--------------------------+
| approx_top_sum           |
+--------------------------+
| [{pear, 6}, {banana, 5}] |
+--------------------------+

NULL handling

APPROX_TOP_SUM does not ignore NULL values for the expression and weight parameters.

SELECT APPROX_TOP_SUM(x, weight, 2) AS approx_top_sum FROM
UNNEST([STRUCT("apple" AS x, NULL AS weight), ("pear", 0), ("pear", NULL)]);

+----------------------------+
| approx_top_sum             |
+----------------------------+
| [{pear, 0}, {apple, NULL}] |
+----------------------------+
SELECT APPROX_TOP_SUM(x, weight, 2) AS approx_top_sum FROM
UNNEST([STRUCT("apple" AS x, 0 AS weight), (NULL, 2)]);

+-------------------------+
| approx_top_sum          |
+-------------------------+
| [{NULL, 2}, {apple, 0}] |
+-------------------------+
SELECT APPROX_TOP_SUM(x, weight, 2) AS approx_top_sum FROM
UNNEST([STRUCT("apple" AS x, 0 AS weight), (NULL, NULL)]);

+----------------------------+
| approx_top_sum             |
+----------------------------+
| [{apple, 0}, {NULL, NULL}] |
+----------------------------+

HyperLogLog++ Functions

BigQuery supports several functions that use HyperLogLog++ for estimating the number of unique values in a large dataset.

They are similar to APPROX_COUNT_DISTINCT, but they are more flexible in the following ways:

  • These functions operate on sketches that compress an arbitrary set into a fixed-memory representation. The sketches can be merged to produce a new sketch that represents the union, before a final estimate is extracted from the sketch.
  • The cardinality of the set can be estimated with a probabilistic error.

These sketches are stored as BYTES, but can be merged and converted into numbers using queries.

In some cases, you may want to work with these sketches directly. For example, consider the following table, which contains ice-cream flavors and the number of people who claimed to enjoy that flavor:

Flavor People
Vanilla 3945
Chocolate 1728
Strawberry 2051
Coffee 1895
Mint Chocolate 4688
Cookie Dough 1619
Cookies and Cream 2855
Caramel 307

Some people may have responded positively to more than one flavor. For instance, some people may have indicated they like vanilla or chocolate ice cream. However, to discover this, you cannot just sum across multiple rows. And to perform a standard aggregate function over the raw data can quickly become very resource intensive.

HyperLogLog++ functions can help solve these problems. These functions allow you to reduce the amount of resources required to work with large data sets by working on intermediate aggregates and returning approximate, as opposed to exact, results.

HLL_COUNT.INIT

HLL_COUNT.INIT(input [, precision])

Description

A scalar function that takes one or more input values and aggregates them into a HyperLogLog++ sketch. Each sketch is represented using the BYTES data type. You can then merge these sketches using HLL_COUNT.MERGE or HLL_COUNT.MERGE_PARTIAL, or if no merging is needed, converted to the final unique count using HLL_COUNT.EXTRACT.

An input can be one of the following:

  • INT64
  • STRING
  • BYTES

This function supports an optional parameter, precision. This parameter defines the accuracy of the estimate at the cost of additional memory required to process the sketches or store them on disk. The following table shows the allowed precision values, the maximum sketch size per group, and confidence interval (CI) of typical precisions:

Precision Max. Sketch Size (KiB) 65% CI 95% CI 99% CI
10 1 ±1.63% ±3.25% ±6.50%
11 2 ±1.15% ±2.30% ±4.60%
12 4 ±0.81% ±1.63% ±3.25%
13 8 ±0.57% ±1.15% ±1.72%
14 16 ±0.41% ±0.81% ±1.22%
15 (default) 32 ±0.29% ±0.57% ±0.86%
16 64 ±0.20% ±0.41% ±0.61%
17 128 ±0.14% ±0.29% ±0.43%
18 256 ±0.10% ±0.20% ±0.41%
19 512 ±0.07% ±0.14% ±0.29%
20 1024 ±0.05% ±0.10% ±0.20%
21 2048 ±0.04% ±0.07% ±0.14%
22 4096 ±0.03% ±0.05% ±0.10%
23 8192 ±0.02% ±0.04% ±0.07%
24 16384 ±0.01% ±0.03% ±0.05%

If the input is NULL, this function returns NULL.

For more information, see HyperLogLog in Practice: Algorithmic Engineering of a State of The Art Cardinality Estimation Algorithm.

Supported input type

BYTES

Return type

BYTES

Example

SELECT
  HLL_COUNT.INIT(respondent) AS respondents_hll,
  flavor,
  country
FROM UNNEST([
  STRUCT(1 AS respondent, "Vanilla" AS flavor, "CH" AS country),
  (1, "Chocolate", "CH"),
  (2, "Chocolate", "US"),
  (2, "Strawberry", "US")])
GROUP BY flavor, country;

HLL_COUNT.MERGE

HLL_COUNT.MERGE(sketch)

Description

An aggregate function that returns the cardinality of several HyperLogLog++ set sketches by computing their union.

Each sketch must have the same precision and be initialized on the same type. Attempts to merge sketches with different precisions or for different types results in an error. For example, you cannot merge a sketch initialized from INT64 data with one initialized from STRING data.

This function ignores NULL values when merging sketches. If the merge happens over zero rows or only over NULL values, the function returns 0.

Supported input types

BYTES

Return type

INT64

Example

SELECT HLL_COUNT.MERGE(respondents_hll) AS num_respondents, flavor
FROM (
  SELECT
    HLL_COUNT.INIT(respondent) AS respondents_hll,
    flavor,
    country
  FROM UNNEST([
    STRUCT(1 AS respondent, "Vanilla" AS flavor, "CH" AS country),
    (1, "Chocolate", "CH"),
    (2, "Chocolate", "US"),
    (2, "Strawberry", "US")])
  GROUP BY flavor, country)
GROUP BY flavor;

HLL_COUNT.MERGE_PARTIAL

HLL_COUNT.MERGE_PARTIAL(sketch)

Description

An aggregate function that takes one or more HyperLogLog++ sketch inputs and merges them into a new sketch. This function is useful if you want to reduce a table that already contains sketches, but do not yet want to estimate the cardinality. For example, use this function if you want to create a sketch that you will merge further later.

This function returns NULL if there is no input or all inputs are NULL.

Supported input types

BYTES

Return type

BYTES

Example

SELECT HLL_COUNT.MERGE_PARTIAL(respondents_hll) AS num_respondents, flavor
FROM (
  SELECT
    HLL_COUNT.INIT(respondent) AS respondents_hll,
    flavor,
    country
  FROM UNNEST([
    STRUCT(1 AS respondent, "Vanilla" AS flavor, "CH" AS country),
    (1, "Chocolate", "CH"),
    (2, "Chocolate", "US"),
    (2, "Strawberry", "US")])
  GROUP BY flavor, country)
GROUP BY flavor;

HLL_COUNT.EXTRACT

HLL_COUNT.EXTRACT(sketch)

Description

A scalar function that extracts an cardinality estimate of a single HyperLogLog++ sketch.

If sketch is NULL, this function returns a cardinality estimate of 0.

Supported input types

BYTES

Return type

INT64

Example

SELECT
  flavor,
  country,
  HLL_COUNT.EXTRACT(respondents_hll) AS num_respondents
FROM (
  SELECT
    HLL_COUNT.INIT(respondent) AS respondents_hll,
    flavor,
    country
  FROM UNNEST([
    STRUCT(1 AS respondent, "Vanilla" AS flavor, "CH" AS country),
    (1, "Chocolate", "CH"),
    (2, "Chocolate", "US"),
    (2, "Strawberry", "US")])
  GROUP BY flavor, country);

+------------+---------+-----------------+
| flavor     | country | num_respondents |
+------------+---------+-----------------+
| Vanilla    | CH      | 1               |
| Chocolate  | CH      | 1               |
| Chocolate  | US      | 1               |
| Strawberry | US      | 1               |
+------------+---------+-----------------+

Analytic Functions

In databases, an analytic function is a function that computes aggregate values over a group of rows. Unlike aggregate functions, which return a single aggregate value for a group of rows, analytic functions return a single value for each row by computing the function over a group of input rows.

Analytic functions are a powerful mechanism for succinctly representing complex analytic operations, and they enable efficient evaluations that otherwise would involve expensive self-JOINs or computation outside the SQL query.

Analytic functions are also called "(analytic) window functions" in the SQL standard and some commercial databases. This is because an analytic function is evaluated over a group of rows, referred to as a window or window frame. In some other databases, they may be referred to as Online Analytical Processing (OLAP) functions.

Simplified syntax:

analytic_function_name ( [ argument_list ] )
  OVER (
    [ PARTITION BY partition_expression_list ]
    [ ORDER BY expression [{ ASC | DESC }] [, ...] ]
    [ window_frame_clause ]
  )

An analytic function requires an OVER clause, which defines the window frame that the analytic function is evaluated over. The OVER clause contains the following three optional clauses. BigQuery evaluates the sub-clauses of an OVER clause in the order in which they are written.

  • A PARTITION BY clause divides the input rows into partitions, similar to GROUP BY but without actually combining rows with the same key.
  • An ORDER BY clause specifies the ordering within each partition.
  • A window_frame_clause defines the window frame within the current partition.

The OVER clause can also be empty (OVER()) in which case the window frame includes all input rows.

Analytic functions are evaluated after aggregation (GROUP BY and non-analytic aggregate functions).

Example: Consider a company who wants to create a leaderboard for each department that shows a "seniority ranking" for each employee, i.e. showing which employees have been there the longest. The table Employees contains columns Name, StartDate, and Department.

The following query calculates the rank of each employee within their department:

SELECT firstname, department, startdate,
  RANK() OVER ( PARTITION BY department ORDER BY startdate ) AS rank
FROM Employees;

The conceptual computing process is illustrated in Figure 1.

Markdown image Figure 1: Analytic Function Illustration

BigQuery evaluates the sub-clauses of an OVER clause in the order in which they appear:

  1. PARTITION BY: The table is first split into two partitions by department.
  2. ORDER BY: The employee rows in each partition are ordered by startdate.
  3. Framing: None. The window frame clause is disallowed for RANK(), as it is for all numbering functions.
  4. RANK(): The seniority ranking is computed for each row over the window frame.

Analytic Function Syntax

analytic_function_name ( [ argument_list ] )
  OVER { window_name | ( [ window_specification ] ) }

window_specification:
  [ window_name ]
  [ PARTITION BY partition_expression_list ]
  [ ORDER BY expression [{ ASC | DESC }] [, ...] ]
  [ window_frame_clause ]

window_frame_clause:
{ ROWS | RANGE }
{
  { UNBOUNDED PRECEDING | numeric_expression PRECEDING | CURRENT ROW }
  |
  { BETWEEN window_frame_boundary_start AND window_frame_boundary_end }
}

window_frame_boundary_start:
{ UNBOUNDED PRECEDING | numeric_expression { PRECEDING | FOLLOWING } | CURRENT ROW }

window_frame_boundary_end:
{ UNBOUNDED FOLLOWING | numeric_expression { PRECEDING | FOLLOWING } | CURRENT ROW }

Analytic functions can appear as a scalar expression or a scalar expression operand in only two places in the query:

  • The SELECT list. If the analytic function appears in the SELECT list, its argument_list cannot refer to aliases introduced in the same SELECT list.
  • The ORDER BY clause. If the analytic function appears in the ORDER BY clause of the query, its argument_list can refer to SELECT list aliases.

Additionally, an analytic function cannot refer to another analytic function in its argument_list or its OVER clause, even if indirectly through an alias.

Invalid:

SELECT ROW_NUMBER() OVER () AS alias1
FROM Singers
ORDER BY ROW_NUMBER() OVER(PARTITION BY alias1)

In the query above, the analytic function alias1 resolves to an analytic function: ROW_NUMBER() OVER().

OVER Clause

Syntax:

OVER { window_name | ( [ window_specification ] ) }

window_specification:
  [ window_name ]
  [ PARTITION BY partition_expression_list ]
  [ ORDER BY sort_specification_list ]
  [ window_frame_clause ]

The OVER clause has three possible components:

  • PARTITION BY clause
  • ORDER BY clause
  • A window_frame_clause or a window_name, which refers to a window_specification defined in a WINDOW clause.

If the OVER clause is empty, OVER(), the analytic function is computed over a single partition which contains all input rows, meaning that it will produce the same result for each output row.

PARTITION BY Clause

Syntax:

PARTITION BY expression [, ... ]

The PARTITION BY clause breaks up the input rows into separate partitions, over which the analytic function is independently evaluated. Multiple expressions are allowed in the PARTITION BY clause.

expression must also have a resulting data type that supports partitioning. This means the expression cannot be any of the following data types:

  • Floating point
  • Struct
  • Array

This list is almost identical to the list of data types that GROUP BY does not support, with the additional exclusion of floating point types (see "Groupable" in the Data Type Properties table at the top of BigQuery Data Types).

If no PARTITION BY clause is present, BigQuery treats the entire input as a single partition.

ORDER BY Clause

Syntax:

ORDER BY expression [ ASC | DESC ] [, ... ]

The ORDER BY clause defines an ordering within each partition. If no ORDER BY clause is present, row ordering within a partition is non-deterministic. Some analytic functions require ORDER BY; this is noted in the section for each family of analytic functions. Even if an ORDER BY clause is present, some functions are not sensitive to ordering within a window frame (e.g. COUNT).

The ORDER BY clause within an OVER clause is consistent with the normal ORDER BY clause in that:

  • There can be multiple expressions.
  • expression must have a type that supports ordering.
  • An optional ASC/DESC specification is allowed for each expression.
  • NULL values order as the minimum possible value (first for ASC, last for DESC)

Data type support is identical to the normal ORDER BY clause in that the following types do not support ordering:

  • Array
  • Struct

If the OVER clause contains an ORDER BY clause but no window_frame_clause, then the ORDER BY implicitly defines window_frame_clause as:

RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW

If neither window_frame_clause nor the ORDER BY clause is present, the window frame defaults to the entire partition.

Window Frame Clause

Syntax:

{ ROWS | RANGE }
{
  { UNBOUNDED PRECEDING | numeric_expression PRECEDING | CURRENT ROW }
  |
  { BETWEEN window_frame_boundary_start AND window_frame_boundary_end }
}

window_frame_boundary_start:
{ UNBOUNDED PRECEDING | numeric_expression { PRECEDING | FOLLOWING } | CURRENT ROW }

window_frame_boundary_end:
{ UNBOUNDED FOLLOWING | numeric_expression { PRECEDING | FOLLOWING } | CURRENT ROW }

window_frame_clause defines the window frame, around the current row within a partition, over which the analytic function is evaluated. window_frame_clause allows both physical window frames (defined by ROWS) and logical window frames (defined by RANGE). If the OVER clause contains an ORDER BY clause but no window_frame_clause, then the ORDER BY implicitly defines window_frame_clause as:

RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW

If neither window_frame_clause nor the ORDER BY clause is present, the window frame defaults to the entire partition.

The numeric_expression can only be a constant or a query parameter, both of which must have a non-negative value. Otherwise, BigQuery provides an error.

Examples of window frame clauses:

ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
  • Includes the entire partition.
  • Example use: Compute a grand total over the partition.
ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
  • Includes all the rows in the partition before or including the current row.
  • Example use: Compute a cumulative sum.
ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING
  • Includes all rows between two before and two after the current row.
  • Example use: Compute a moving average.

If window_frame_spec uses the BETWEEN clause:

  • window_frame_boundary_start must specify a boundary that begins no later than that of the window_frame_boundary_end. This has the following consequences:
    1. If window_frame_boundary_start contains CURRENT ROW, window_frame_boundary_end cannot contain PRECEDING.
    2. If window_frame_boundary_start contains FOLLOWING, window_frame_boundary_end cannot contain CURRENT ROW or PRECEDING.
  • window_frame_boundary_start has no default value.

Otherwise, the specified window_frame_spec boundary represents the start of the window frame and the end of the window frame boundary defaults to 'CURRENT ROW'. Thus,

ROWS 10 PRECEDING

is equivalent to

ROWS BETWEEN 10 PRECEDING AND CURRENT ROW
ROWS

ROWS-based window frames compute the window frame based on physical offsets from the current row. For example, the window frame below defines a window frame of size five (at most) around the current row.

ROWS BETWEEN 2 PRECEDING and 2 FOLLOWING

The numeric_expression in window_frame_clause is interpreted as a number of rows from the current row, and must be a constant, non-negative integer. It may also be a query parameter.

If the window frame for a given row extends beyond the beginning or end of the partition, then the window frame will only include rows from within that partition.

Example: Consider the following table with columns z, x, and y.

z x y
1 5 AA
2 2 AA
3 11 AB
4 2 AA
5 8 AC
6 10 AB
7 1 AB

Consider the following analytic function:

SUM(x) OVER (PARTITION BY y ORDER BY z ROWS BETWEEN 1 PRECEDING AND 1
FOLLOWING)

The PARTITION BY clause splits the table into 3 partitions based on their y value, and the ORDER BY orders the rows within each partition by their z value.

Partition 1 of 3:

z x y
1 5 AA
2 2 AA
4 2 AA

Partition 2 of 3:

z x y
3 11 AB
6 10 AB
7 1 AB

Partition 3 of 3:

z x y
5 8 AC

In the tables below, bold indicates the row currently being evaluated, and colored cells indicates all the rows in the window frame for that row.

  • For the first row in the y = AA partition, the window frame includes only 2 rows since there is no preceding row, even though the window_frame_spec indicates a window size of 3. The result of the analytic function is 7 for the first row.
z x y
1 5 AA
2 2 AA
4 2 AA
  • For the second row in the partition, the window frame includes all 3 rows. The result of the analytic function is 9 for the second row.
z x y
1 5 AA
2 2 AA
4 2 AA
  • For the last row in the partition, the window frame includes only 2 rows since there is no following row. The result of the analytic function is 4 for the third row.
z x y
1 5 AA
2 2 AA
4 2 AA
RANGE

RANGE-based window frames compute the window frame based on a logical range of rows around the current row based on the current row's ORDER BY key value. The provided range value is added or subtracted to the current row's key value to define a starting or ending range boundary for the window frame.

The ORDER BY clause must be specified unless the window is:

RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING.

numeric_expression in the window_frame_clause is interpreted as an offset from the current row's value of the ORDER BY key. numeric_expression must have numeric type. DATE and TIMESTAMP are not currently supported. In addition, the numeric_expression must be a constant, non-negative integer or a parameter.

In a RANGE-based window frame, there can be at most one expression in the ORDER BY clause, and expression must have a numeric type.

Example of a RANGE-based window frame where there is a single partition:

SELECT x, COUNT(*) OVER ( ORDER BY x
  RANGE BETWEEN 2 PRECEDING AND 2 FOLLOWING ) AS count_x
FROM T;

In the tables below, bold indicates the row currently being evaluated, and colored cells indicates all the rows in the window frame for that row.

  • For row 1, x = 5 and therefore COUNT(*) will only include rows where 3 <=x <= 7
x count_x
5 1
2
11
2
8
10
1
  • For row 2, x = 2 and therefore COUNT(*) will only include rows where 0 <= x <= 4
x count_x
5 1
2 3
11
2
8
10
1
  • For row 3, x = 11 and therefore COUNT(*) will only include rows where 9 <= x <= 13
x count_x
5 1
2 3
11 2
2
8
10
1

WINDOW Clause

Syntax:

WINDOW window_definition [, ...]
window_definition: window_name AS ( window_specification )

A WINDOW clause defines a list of named windows whose window_name can be referenced in analytic functions in the SELECT list. This is useful when you want to use the same window_frame_clause for multiple analytic functions.

The WINDOW clause can appear only at the end of a SELECT clause, as shown in Query Syntax.

Named Windows

Once you define a WINDOW clause, you can use the named windows in analytic functions, but only in the SELECT list; you cannot use named windows in the ORDER BY clause. Named windows can appear either by themselves or embedded within an OVER clause. Named windows can refer to SELECT list aliases.

Examples:

SELECT SUM(x) OVER window_name FROM ...
SELECT SUM(x) OVER (
  window_name
  PARTITION BY...
  ORDER BY...
  window_frame_clause)
FROM ...

When embedded within an OVER clause, the window_specification associated with window_name must be compatible with the PARTITION BY, ORDER BY, and window_frame_clause that are in the same OVER clause.

The following rules apply to named windows:

  • You can refer to named windows only in the SELECT list; you cannot refer to them in an ORDER BY clause, an outer query, or any subquery.
  • A window W1 (named or unnamed) may reference a named window NW2, with the following rules:
    1. If W1 is a named window, then the referenced named window NW2 must precede W1 in the same WINDOW clause.
    2. W1 cannot contain a PARTITION BY clause
    3. It cannot be true that both W1 and NW2 contain an ORDER BY clause
    4. NW2 cannot contain a window_frame_clause.
  • If a (named or unnamed) window W1 references a named window NW2, then the resulting window specification is defined using:
    1. The PARTITION BY from NW2, if there is one.
    2. The ORDER BY from W1 or NW2, if either is specified; it is not possible for them to both have an ORDER BY clause.
    3. The window_frame_clause from W1, if there is one.

Supported Functions

Numbering Functions

Numbering functions assign integer values to each row based on their position within the specified window.

OVER clause requirements:

  • PARTITION BY: Optional.
  • ORDER BY: Required, except for ROW_NUMBER().
  • window_frame_clause: Disallowed.

If two rows are tied in the ordering, they are peers. The concept of peers is relevant for the functions RANK(), DENSE_RANK(), and CUME_DIST.

Syntax Result Data Type Description
RANK() INT64 Returns the ordinal (1-based) rank of each row within the ordered partition. All peer rows receive the same rank value. The next row or set of peer rows receives a rank value which increments by the number of peers with the previous rank value, instead of a rank value which always increments by 1. See example of RANK() below this table.
DENSE_RANK() INT64 Returns the ordinal (1-based) rank of each row within the window partition. All peer rows receive the same rank value, and the subsequent rank value is incremented by one.
PERCENT_RANK() FLOAT64 Return the percentile rank of a row defined as (RK-1)/(NR-1), where RK is the RANK of the row and NR is the number of rows in the partition. Returns 0 if NR=1.
CUME_DIST() FLOAT64 Return the relative rank of a row defined as NP/NR. NP is defined to be the number of rows that either precede or are peers with the current row. NR is the number of rows in the partition.
NTILE(constant_integer_expression) INT64 This function divides the rows into constant_integer_expression buckets based on row ordering and returns the 1-based bucket number that is assigned to each row. The number of rows in the buckets can differ by at most 1. The remainder values (the remainder of number of rows divided by buckets) are distributed one for each bucket, starting with bucket 1. If constant_integer_expression evaluates to NULL, 0 or negative, an error is provided.
ROW_NUMBER() INT64 Does not require the ORDER BY clause. Returns the sequential row ordinal (1-based) of each row for each ordered partition. If the ORDER BY clause is unspecified then the result is non-deterministic.

Example of RANK(), DENSE_RANK(), and ROW_NUMBER():

SELECT x,
  RANK() OVER (ORDER BY x ASC) AS rank,
  DENSE_RANK() OVER (ORDER BY x ASC) AS dense_rank,
  ROW_NUMBER() OVER (PARTITION BY x ORDER BY y) AS row_num
FROM ...
x rank dense_rank row_num
1 1 1 1
2 2 2 2
2 2 2 3
5 4 3 4
8 5 4 5
10 6 5 6
10 6 5 7
  • RANK():For x=5, rank returns 4, since RANK() increments by the number of peers in the previous window ordering group.
  • DENSE_RANK(): For x=5, dense_rank returns 3, since DENSE_RANK() always increments by 1, never skipping a value.
  • ROW_NUMBER():For x=5, row_num returns 4.

Navigation functions generally compute some value_expression over a different row in the window frame from the current row. The OVER clause syntax varies across navigation functions.

OVER clause requirements:

  • PARTITION BY: Optional.
  • ORDER BY: Required
  • window_frame_clause:
    1. Disallowed for LEAD and LAG
    2. Optional for FIRST_VALUE, LAST_VALUE, and NTH_VALUE.

For all navigation functions, the result data type is the same type as value_expression.

FIRST_VALUE
FIRST_VALUE (value_expression)

Description

Returns the value of the value_expression for the first row in the current window frame.

Supported Argument Types

value_expression can be any data type that an expression can return.

Return Data Type

ANY

Examples

The following example computes the fastest time for each division.

WITH finishers AS
 (SELECT 'Sophia Liu' as name,
  TIMESTAMP '2016-10-18 2:51:45' as finish_time,
  'F30-34' as division
  UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
  UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
  UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
  UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
  UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
  UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
  UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
  UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
  UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
  FORMAT_TIMESTAMP('%X', finish_time) AS finish_time,
  division,
  FORMAT_TIMESTAMP('%X', fastest_time) AS fastest_time,
  TIMESTAMP_DIFF(finish_time, fastest_time, SECOND) AS delta_in_seconds
FROM (
  SELECT name,
  finish_time,
  division,
  FIRST_VALUE(finish_time)
    OVER (PARTITION BY division ORDER BY finish_time ASC
    ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS fastest_time
  FROM finishers);

+-----------------+-------------+----------+--------------+------------------+
| name            | finish_time | division | fastest_time | delta_in_seconds |
+-----------------+-------------+----------+--------------+------------------+
| Carly Forte     | 03:08:58    | F25-29   | 03:08:58     | 0                |
| Sophia Liu      | 02:51:45    | F30-34   | 02:51:45     | 0                |
| Nikki Leith     | 02:59:01    | F30-34   | 02:51:45     | 436              |
| Jen Edwards     | 03:06:36    | F30-34   | 02:51:45     | 891              |
| Meghan Lederer  | 03:07:41    | F30-34   | 02:51:45     | 956              |
| Lauren Reasoner | 03:10:14    | F30-34   | 02:51:45     | 1109             |
| Lisa Stelzner   | 02:54:11    | F35-39   | 02:54:11     | 0                |
| Lauren Matthews | 03:01:17    | F35-39   | 02:54:11     | 426              |
| Desiree Berry   | 03:05:42    | F35-39   | 02:54:11     | 691              |
| Suzy Slane      | 03:06:24    | F35-39   | 02:54:11     | 733              |
+-----------------+-------------+----------+--------------+------------------+
LAST_VALUE
LAST_VALUE (value_expression)

Description

Returns the value of the value_expression for the last row in the current window frame.

Supported Argument Types

value_expression can be any data type that an expression can return.

Return Data Type

ANY

Examples

The following example computes the slowest time for each division.

WITH finishers AS
 (SELECT 'Sophia Liu' as name,
  TIMESTAMP '2016-10-18 2:51:45' as finish_time,
  'F30-34' as division
  UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
  UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
  UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
  UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
  UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
  UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
  UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
  UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
  UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
  FORMAT_TIMESTAMP('%X', finish_time) AS finish_time,
  division,
  FORMAT_TIMESTAMP('%X', slowest_time) AS slowest_time,
  TIMESTAMP_DIFF(slowest_time, finish_time, SECOND) AS delta_in_seconds
FROM (
  SELECT name,
  finish_time,
  division,
  LAST_VALUE(finish_time)
    OVER (PARTITION BY division ORDER BY finish_time ASC
    ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS slowest_time
  FROM finishers);

+-----------------+-------------+----------+--------------+------------------+
| name            | finish_time | division | slowest_time | delta_in_seconds |
+-----------------+-------------+----------+--------------+------------------+
| Carly Forte     | 03:08:58    | F25-29   | 03:08:58     | 0                |
| Sophia Liu      | 02:51:45    | F30-34   | 03:10:14     | 1109             |
| Nikki Leith     | 02:59:01    | F30-34   | 03:10:14     | 673              |
| Jen Edwards     | 03:06:36    | F30-34   | 03:10:14     | 218              |
| Meghan Lederer  | 03:07:41    | F30-34   | 03:10:14     | 153              |
| Lauren Reasoner | 03:10:14    | F30-34   | 03:10:14     | 0                |
| Lisa Stelzner   | 02:54:11    | F35-39   | 03:06:24     | 733              |
| Lauren Matthews | 03:01:17    | F35-39   | 03:06:24     | 307              |
| Desiree Berry   | 03:05:42    | F35-39   | 03:06:24     | 42               |
| Suzy Slane      | 03:06:24    | F35-39   | 03:06:24     | 0                |
+-----------------+-------------+----------+--------------+------------------+

NTH_VALUE

NTH_VALUE (value_expression, constant_integer_expression)

Description

Returns the value of value_expression at the Nth row of the current window frame, where Nth is defined by constant_integer_expression. Returns NULL if there is no such row.

Supported Argument Types

  • value_expression can be any data type that can be returned from an expression.
  • constant_integer_expression can be any constant expression that returns an integer.

Return Data Type

ANY

Examples

WITH finishers AS
 (SELECT 'Sophia Liu' as name,
  TIMESTAMP '2016-10-18 2:51:45' as finish_time,
  'F30-34' as division
  UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
  UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
  UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
  UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
  UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
  UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
  UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
  UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
  UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
  FORMAT_TIMESTAMP('%X', finish_time) AS finish_time,
  division,
  FORMAT_TIMESTAMP('%X', fastest_time) AS fastest_time,
  FORMAT_TIMESTAMP('%X', second_fastest) AS second_fastest
FROM (
  SELECT name,
  finish_time,
  division,finishers,
  FIRST_VALUE(finish_time)
    OVER w1 AS fastest_time,
  NTH_VALUE(finish_time, 2)
    OVER w1 as second_fastest
  FROM finishers
  WINDOW w1 AS (
    PARTITION BY division ORDER BY finish_time ASC
    ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING));

+-----------------+-------------+----------+--------------+----------------+
| name            | finish_time | division | fastest_time | second_fastest |
+-----------------+-------------+----------+--------------+----------------+
| Carly Forte     | 03:08:58    | F25-29   | 03:08:58     | NULL           |
| Sophia Liu      | 02:51:45    | F30-34   | 02:51:45     | 02:59:01       |
| Nikki Leith     | 02:59:01    | F30-34   | 02:51:45     | 02:59:01       |
| Jen Edwards     | 03:06:36    | F30-34   | 02:51:45     | 02:59:01       |
| Meghan Lederer  | 03:07:41    | F30-34   | 02:51:45     | 02:59:01       |
| Lauren Reasoner | 03:10:14    | F30-34   | 02:51:45     | 02:59:01       |
| Lisa Stelzner   | 02:54:11    | F35-39   | 02:54:11     | 03:01:17       |
| Lauren Matthews | 03:01:17    | F35-39   | 02:54:11     | 03:01:17       |
| Desiree Berry   | 03:05:42    | F35-39   | 02:54:11     | 03:01:17       |
| Suzy Slane      | 03:06:24    | F35-39   | 02:54:11     | 03:01:17       |
+-----------------+-------------+----------+--------------+----------------+
LEAD
LEAD (value_expression, [, offset [, default_expression]])

Description

Returns the value of the value_expression on a subsequent row. Changing the offset value changes which subsequent row is returned; the default value is 1, indicating the next row in the window frame. An error occurs if offset is NULL or a negative value.

The optional default_expression is used if there isn't a row in the window frame at the specified offset. This expression must be a constant expression and its type must be implicitly coercible to the type of value_expression. If left unspecified, default_expression defaults to NULL.

Supported Argument Types

  • value_expression can be any data type that can be returned from an expression.
  • offset must be a non-negative integer literal or parameter.
  • default_expression must be compatible with the value expression type.

Return Data Type

ANY

Examples

The following example illustrates a basic use of the LEAD function.

WITH finishers AS
 (SELECT 'Sophia Liu' as name,
  TIMESTAMP '2016-10-18 2:51:45' as finish_time,
  'F30-34' as division
  UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
  UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
  UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
  UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
  UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
  UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
  UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
  UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
  UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
  finish_time,
  division,
  LEAD(name)
    OVER (PARTITION BY division ORDER BY finish_time ASC) AS followed_by
FROM finishers;

+-----------------+-------------+----------+-----------------+
| name            | finish_time | division | followed_by     |
+-----------------+-------------+----------+-----------------+
| Carly Forte     | 03:08:58    | F25-29   | NULL            |
| Sophia Liu      | 02:51:45    | F30-34   | Nikki Leith     |
| Nikki Leith     | 02:59:01    | F30-34   | Jen Edwards     |
| Jen Edwards     | 03:06:36    | F30-34   | Meghan Lederer  |
| Meghan Lederer  | 03:07:41    | F30-34   | Lauren Reasoner |
| Lauren Reasoner | 03:10:14    | F30-34   | NULL            |
| Lisa Stelzner   | 02:54:11    | F35-39   | Lauren Matthews |
| Lauren Matthews | 03:01:17    | F35-39   | Desiree Berry   |
| Desiree Berry   | 03:05:42    | F35-39   | Suzy Slane      |
| Suzy Slane      | 03:06:24    | F35-39   | NULL            |
+-----------------+-------------+----------+-----------------+

This next example uses the optional offset parameter.

WITH finishers AS
 (SELECT 'Sophia Liu' as name,
  TIMESTAMP '2016-10-18 2:51:45' as finish_time,
  'F30-34' as division
  UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
  UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
  UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
  UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
  UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
  UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
  UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
  UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
  UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
  finish_time,
  division,
  LEAD(name, 2)
    OVER (PARTITION BY division ORDER BY finish_time ASC) AS two_runners_back
FROM finishers;

+-----------------+-------------+----------+------------------+
| name            | finish_time | division | two_runners_back |
+-----------------+-------------+----------+------------------+
| Carly Forte     | 03:08:58    | F25-29   | NULL             |
| Sophia Liu      | 02:51:45    | F30-34   | Jen Edwards      |
| Nikki Leith     | 02:59:01    | F30-34   | Meghan Lederer   |
| Jen Edwards     | 03:06:36    | F30-34   | Lauren Reasoner  |
| Meghan Lederer  | 03:07:41    | F30-34   | NULL             |
| Lauren Reasoner | 03:10:14    | F30-34   | NULL             |
| Lisa Stelzner   | 02:54:11    | F35-39   | Desiree Berry    |
| Lauren Matthews | 03:01:17    | F35-39   | Suzy Slane       |
| Desiree Berry   | 03:05:42    | F35-39   | NULL             |
| Suzy Slane      | 03:06:24    | F35-39   | NULL             |
+-----------------+-------------+----------+------------------+

The following example replaces NULL values with a default value.

WITH finishers AS
 (SELECT 'Sophia Liu' as name,
  TIMESTAMP '2016-10-18 2:51:45' as finish_time,
  'F30-34' as division
  UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
  UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
  UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
  UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
  UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
  UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
  UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
  UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
  UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
  finish_time,
  division,
  LEAD(name, 2, 'Nobody')
    OVER (PARTITION BY division ORDER BY finish_time ASC) AS two_runners_back
FROM finishers;

+-----------------+-------------+----------+------------------+
| name            | finish_time | division | two_runners_back |
+-----------------+-------------+----------+------------------+
| Carly Forte     | 03:08:58    | F25-29   | Nobody           |
| Sophia Liu      | 02:51:45    | F30-34   | Jen Edwards      |
| Nikki Leith     | 02:59:01    | F30-34   | Meghan Lederer   |
| Jen Edwards     | 03:06:36    | F30-34   | Lauren Reasoner  |
| Meghan Lederer  | 03:07:41    | F30-34   | Nobody           |
| Lauren Reasoner | 03:10:14    | F30-34   | Nobody           |
| Lisa Stelzner   | 02:54:11    | F35-39   | Desiree Berry    |
| Lauren Matthews | 03:01:17    | F35-39   | Suzy Slane       |
| Desiree Berry   | 03:05:42    | F35-39   | Nobody           |
| Suzy Slane      | 03:06:24    | F35-39   | Nobody           |
+-----------------+-------------+----------+------------------+
LAG
LAG (value_expression, [, offset [, default_expression]])

Description

Returns the value of the value_expression on a preceding row. Changing the offset value changes which preceding row is returned; the default value is 1, indicating the previous row in the window frame. An error occurs if offset is NULL or a negative value.

The optional default_expression is used if there isn't a row in the window frame at the specified offset. This expression must be a constant expression and its type must be implicitly coercible to the type of value_expression. If left unspecified, default_expression defaults to NULL.

Supported Argument Types

  • value_expression can be any data type that can be returned from an expression.
  • offset must be a non-negative integer literal or parameter.
  • default_expression must be compatible with the value expression type.

Return Data Type

ANY

Examples

The following example illustrates a basic use of the LAG function.

WITH finishers AS
 (SELECT 'Sophia Liu' as name,
  TIMESTAMP '2016-10-18 2:51:45' as finish_time,
  'F30-34' as division
  UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
  UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
  UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
  UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
  UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
  UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
  UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
  UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
  UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
  finish_time,
  division,
  LAG(name)
    OVER (PARTITION BY division ORDER BY finish_time ASC) AS preceding_runner
FROM finishers;

+-----------------+-------------+----------+------------------+
| name            | finish_time | division | preceding_runner |
+-----------------+-------------+----------+------------------+
| Carly Forte     | 03:08:58    | F25-29   | NULL             |
| Sophia Liu      | 02:51:45    | F30-34   | NULL             |
| Nikki Leith     | 02:59:01    | F30-34   | Sophia Liu       |
| Jen Edwards     | 03:06:36    | F30-34   | Nikki Leith      |
| Meghan Lederer  | 03:07:41    | F30-34   | Jen Edwards      |
| Lauren Reasoner | 03:10:14    | F30-34   | Meghan Lederer   |
| Lisa Stelzner   | 02:54:11    | F35-39   | NULL             |
| Lauren Matthews | 03:01:17    | F35-39   | Lisa Stelzner    |
| Desiree Berry   | 03:05:42    | F35-39   | Lauren Matthews  |
| Suzy Slane      | 03:06:24    | F35-39   | Desiree Berry    |
+-----------------+-------------+----------+------------------+

This next example uses the optional offset parameter.

WITH finishers AS
 (SELECT 'Sophia Liu' as name,
  TIMESTAMP '2016-10-18 2:51:45' as finish_time,
  'F30-34' as division
  UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
  UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
  UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
  UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
  UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
  UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
  UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
  UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
  UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
  finish_time,
  division,
  LAG(name, 2)
    OVER (PARTITION BY division ORDER BY finish_time ASC) AS two_runners_ahead
FROM finishers;

+-----------------+-------------+----------+-------------------+
| name            | finish_time | division | two_runners_ahead |
+-----------------+-------------+----------+-------------------+
| Carly Forte     | 03:08:58    | F25-29   | NULL              |
| Sophia Liu      | 02:51:45    | F30-34   | NULL              |
| Nikki Leith     | 02:59:01    | F30-34   | NULL              |
| Jen Edwards     | 03:06:36    | F30-34   | Sophia Liu        |
| Meghan Lederer  | 03:07:41    | F30-34   | Nikki Leith       |
| Lauren Reasoner | 03:10:14    | F30-34   | Jen Edwards       |
| Lisa Stelzner   | 02:54:11    | F35-39   | NULL              |
| Lauren Matthews | 03:01:17    | F35-39   | NULL              |
| Desiree Berry   | 03:05:42    | F35-39   | Lisa Stelzner     |
| Suzy Slane      | 03:06:24    | F35-39   | Lauren Matthews   |
+-----------------+-------------+----------+-------------------+

The following example replaces NULL values with a default value.

WITH finishers AS
 (SELECT 'Sophia Liu' as name,
  TIMESTAMP '2016-10-18 2:51:45' as finish_time,
  'F30-34' as division
  UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
  UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
  UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
  UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
  UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
  UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
  UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
  UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
  UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
  finish_time,
  division,
  LAG(name, 2, 'Nobody')
    OVER (PARTITION BY division ORDER BY finish_time ASC) AS two_runners_ahead
FROM finishers;

+-----------------+-------------+----------+-------------------+
| name            | finish_time | division | two_runners_ahead |
+-----------------+-------------+----------+-------------------+
| Carly Forte     | 03:08:58    | F25-29   | Nobody            |
| Sophia Liu      | 02:51:45    | F30-34   | Nobody            |
| Nikki Leith     | 02:59:01    | F30-34   | Nobody            |
| Jen Edwards     | 03:06:36    | F30-34   | Sophia Liu        |
| Meghan Lederer  | 03:07:41    | F30-34   | Nikki Leith       |
| Lauren Reasoner | 03:10:14    | F30-34   | Jen Edwards       |
| Lisa Stelzner   | 02:54:11    | F35-39   | Nobody            |
| Lauren Matthews | 03:01:17    | F35-39   | Nobody            |
| Desiree Berry   | 03:05:42    | F35-39   | Lisa Stelzner     |
| Suzy Slane      | 03:06:24    | F35-39   | Lauren Matthews   |
+-----------------+-------------+----------+-------------------+

Aggregate Analytic Functions

BigQuery supports the following aggregate functions as analytic functions:

With these functions, the OVER clause is simply appended to the aggregate function call; the function call syntax remains otherwise unchanged. Like their aggregate function counterparts, these analytic functions perform aggregations, but specifically over the relevant window frame for each row. The result data types of these analytic functions are the same as their aggregate function counterparts.

OVER clause requirements:

  • PARTITION BY: Optional.
  • ORDER BY: Optional. Disallowed if DISTINCT is present.
  • window_frame_clause: Optional. Disallowed if DISTINCT is present.

Example:

COUNT(*) OVER (ROWS UNBOUNDED PRECEDING)
SUM(DISTINCT x) OVER ()

Bit Functions

BigQuery supports the following bit functions.

BIT_COUNT

BIT_COUNT(expression)

Description

The input, expression, must be an integer or BYTES.

Returns the number of bits that are set in the input expression. For integers, this is the number of bits in two's complement form.

Return Data Type

INT64

Example

SELECT a, BIT_COUNT(a) AS a_bits, FORMAT("%T", b) as b, BIT_COUNT(b) AS b_bits
FROM UNNEST([
  STRUCT(0 AS a, b'' AS b), (0, b'\x00'), (5, b'\x05'), (8, b'\x00\x08'),
  (0xFFFF, b'\xFF\xFF'), (-2, b'\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFE'),
  (-1, b'\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF'),
  (NULL, b'\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF')
]) AS x;

+-------+--------+---------------------------------------------+--------+
| a     | a_bits | b                                           | b_bits |
+-------+--------+---------------------------------------------+--------+
| 0     | 0      | b""                                         | 0      |
| 0     | 0      | b"\x00"                                     | 0      |
| 5     | 2      | b"\x05"                                     | 2      |
| 8     | 1      | b"\x00\x08"                                 | 1      |
| 65535 | 16     | b"\xff\xff"                                 | 16     |
| -2    | 63     | b"\xff\xff\xff\xff\xff\xff\xff\xfe"         | 63     |
| -1    | 64     | b"\xff\xff\xff\xff\xff\xff\xff\xff"         | 64     |
| NULL  | NULL   | b"\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff" | 80     |
+-------+--------+---------------------------------------------+--------+

Mathematical functions

Mathematical functions:

  • Return NULL if any of the input parameters is NULL.
  • Return NaN if any of the arguments is NaN.
Function Description
ABS(X) Computes absolute value. With integer argument, generates an error if the value cannot be represented as the same type (which happens only for the largest negative input value, which has no positive representation). Returns +inf for +/-inf argument.
SIGN(X) Returns -1, 0, or +1 for negative, zero and positive argument respectively. With floating point argument does not distinguish positive and negative zero. Returns NaN for NaN argument.
IS_INF(X) Return TRUE if the value is positive or negative infinity. Returns NULL for NULL inputs.
IS_NAN(X) Return TRUE if the value is a NaN value. Returns NULL for NULL inputs.
IEEE_DIVIDE(X, Y)

Divides X by Y; never fails. Returns FLOAT64. Unlike division operator (/), does not generate errors for division by zero or overflow.

Special cases:

  • If the result overflows, returns +/-inf.
  • If Y=0 and X=0, returns NaN.
  • If Y=0 and X!=0, returns +/-inf.
  • If X = +/-inf and Y = +/-inf, returns NaN.
The behavior of IEEE_DIVIDE is further illustrated in the table below.
RAND() Generates a pseudo-random value of type FLOAT64 in the range of [0, 1), inclusive of 0 and exclusive of 1.
SQRT(X) Computes the square root of X. Generates an error if X less than 0. Returns +inf if X is +inf.
POW(X, Y) Power function: returns the value of X raised to the power of Y. If the result underflows and is not representable then zero value is returned. An error can be generated if one of the following is true:X is a finite value less than 0 and Y is a noninteger,if X is 0, and Y is a finite value less than 0,the result overflows.The behavior of POW() is further illustrated in the table below.
POWER(X, Y) Synonym of POW(). The behavior of POWER() is further illustrated in the table below.
EXP(X) Computes natural exponential function ex. If the result underflows a zero is returned. Generates an error if the result overflows. If X is +/-inf, then +inf (or 0) is returned.
LN(X) Computes the natural logarithm of X. Generates an error if X is less or equal zero. If X is +inf, then +inf is returned.
LOG(X) Synonym of LN(X)
LOG(X, Y) Computes logarithm of X to base Y. Generates an error if:X less or equal zero,Y is 1.0,Y less or equal zero.The behavior of LOG(X, Y) is further illustrated in the table below.
LOG10(X) Similar to LOG(X) but computes logarithm to base 10.
GREATEST(X1,...,XN) Returns NULL if any of the inputs is NULL. Otherwise, returns NaN if any of the inputs is NaN. Otherwise, returns the largest value among X1,...,XN according to the < comparison.
LEAST(X1,...,XN) Returns NULL if any of the inputs is NULL. Otherwise, returns NaN if any of the inputs is NaN. Otherwise, returns the smallest value among X1,...,XN according to the > comparison.
DIV(X, Y) Returns the result of integer division of X by Y. Division by zero returns an error. Division by -1 may overflow. See the table below for possible result types.
SAFE_DIVIDE(X, Y)Equivalent to the division operator (/). Returns NULL if an error occurs, such as division by zero.
MOD(X, Y) Modulo function: returns the remainder of the division of X by Y. Returned value has the same sign as X. An error is generated if Y is 0. See the table below for possible result types.

Special cases for IEEE_DIVIDE(X, Y)

The following table lists special cases for IEEE_DIVIDE(X,Y).

Numerator Data Type (X) Denominator Data Type (Y) Result Value
Anything except 0 0 +/-inf
0 0 NaN
0 NaN NaN
+/-inf +/-inf NaN

Special cases for POW(X, Y) and POWER(X, Y)

The following are special cases for POW(X, Y) and POWER(X, Y).

X Y POW(X, Y) or POWER(X, Y)
1.0 Any value including NaN 1.0
any including NaN 0 1.0
-1.0 +/-inf 1.0
ABS(X) < 1 -inf +inf
ABS(X) > 1 -inf 0
ABS(X) < 1 +inf 0
ABS(X) > 1 +inf +inf
-inf Y < 0 0
-inf Y > 0 -inf if Y is an odd integer, +inf otherwise
+inf Y < 0 0
+inf Y > 0 +inf

Special cases for LOG(X, Y)

X Y LOG(X, Y)
-inf Any value NaN
Any value +inf NaN
+inf 0.0 Y < 1.0 -inf
+inf Y > 1.0 +inf

Rounding functions

Syntax Description
ROUND(X) Rounds X to the nearest integer. Halfway cases are rounded away from zero.
ROUND(X, N) Rounds X to N decimal places after decimal point. N can be negative, which will round off digits to the left of the decimal point. Halfway cases are rounded away from zero. Generates an error if overflow occurs.
TRUNC(X) Rounds X to the nearest integer whose absolute value is not greater than Xs.
TRUNC(X, N) Similar to ROUND(X, N) but always rounds towards zero. Unlike ROUND(X, N) it never overflows.
CEIL(X) Returns the smallest integral value (with FLOAT64 type) that is not less than X.
CEILING(X) Synonym of CEIL(X)
FLOOR(X) Returns the largest integral value (with FLOAT64 type) that is not greater than X.

Example behavior of BigQuery rounding functions:

Input "X" ROUND(X) TRUNC(X) CEIL(X) FLOOR(X)
2.0 2.0 2.0 2.0 2.0
2.3 2.0 2.0 3.0 2.0
2.8 3.0 2.0 3.0 2.0
2.5 3.0 2.0 3.0 2.0
-2.3 -2.0 -2.0 -2.0 -3.0
-2.8 -3.0 -2.0 -2.0 -3.0
-2.5 -3.0 -2.0 -2.0 -3.0
0 0 0 0 0
+/-inf +/-inf +/-inf +/-inf +/-inf
NaN NaN NaN NaN NaN

Trigonometric and hyperbolic functions

Syntax Description
COS(X) Computes cosine of X. Never fails.
COSH(X) Computes the hyperbolic cosine of X. Generates an error if an overflow occurs.
ACOS(X) Computes the principal value of the arc cosine of X. The return value is in the range [0,]. Generates an error if X is a finite value outside of range [-1, 1].
ACOSH(X) Computes the inverse hyperbolic cosine of X. Generates an error if X is a finite value less than 1.
SIN(X) Computes the sine of X. Never fails.
SINH(X) Computes the hyperbolic sine of X. Generates an error if an overflow occurs.
ASIN(X) Computes the principal value of the arc sine of X. The return value is in the range [-π/2,π/2]. Generates an error if X is a finite value outside of range [-1, 1].
ASINH(X) Computes the inverse hyperbolic sine of X. Does not fail.
TAN(X) Computes tangent of X. Generates an error if an overflow occurs.
TANH(X) Computes hyperbolic tangent of X. Does not fail.
ATAN(X) Computes the principal value of the arc tangent of X. The return value is in the range [-π/2,π/2]. Does not fail.
ATANH(X) Computes the inverse hyperbolic tangent of X. Generates an error if the absolute value of X is greater or equal 1.
ATAN2(Y, X) Calculates the principal value of the arc tangent of Y/X using the signs of the two arguments to determine the quadrant. The return value is in the range [-π,π]. The behavior of this function is further illustrated in the table below.

Special cases for ATAN2()

Y X ATAN2(Y, X)
NaN Any value NaN
Any value NaN NaN
0 0 0, π or -π depending on the sign of X and Y
Finite value -inf π or -π depending on the sign of Y
Finite value +inf 0
+/-inf Finite value π/2 or π/2 depending on the sign of Y
+/-inf -inf ¾π or -¾π depending on the sign of Y
+/-inf +inf π/4 or -π/4 depending on the sign of Y

Special cases for trigonometric and hyperbolic rounding functions

X COS(X) COSH(X) ACOS(X) ACOSH(X) SIN(X) SINH(X) ASIN(X) ASINH(X) TAN(X) TANH(X) ATAN(X) ATANH(X)
+/-inf NaN =+inf NaN =+inf NaN =+inf NaN =+inf NaN =+1.0 π/2 NaN
-inf NaN =+inf NaN NaN NaN -inf NaN -inf NaN -1.0 -π/2 NaN
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN

Hash functions

FARM_FINGERPRINT

FARM_FINGERPRINT(value)

Description

Computes the fingerprint of the STRING or BYTES input using the Fingerprint64 function from the open-source FarmHash library. The output of this function for a particular input will never change.

Return type

INT64

Examples

WITH example AS (
  SELECT 1 AS x, "foo" AS y, true AS z UNION ALL
  SELECT 2 AS x, "apple" AS y, false AS z UNION ALL
  SELECT 3 AS x, "" AS y, true AS z
)
SELECT
  *,
  FARM_FINGERPRINT(CONCAT(CAST(x AS STRING), y, CAST(z AS STRING)))
    AS row_fingerprint
FROM example;
+---+-------+-------+----------------------+
| x | y     | z     | row_fingerprint      |
+---+-------+-------+----------------------+
| 1 | foo   | true  | -1541654101129638711 |
| 2 | apple | false | 2794438866806483259  |
| 3 |       | true  | -4880158226897771312 |
+---+-------+-------+----------------------+

MD5

MD5(input)

Description

Computes the hash of the input using the MD5 algorithm. The input can either be STRING or BYTES. The string version treats the input as an array of bytes.

This function returns 16 bytes.

Return type

BYTES

Example

SELECT MD5("Hello World") as md5;

+-------------------------------------------------+
| md5                                             |
+-------------------------------------------------+
| \xb1\n\x8d\xb1d\xe0uA\x05\xb7\xa9\x9b\xe7.?\xe5 |
+-------------------------------------------------+

SHA1

SHA1(input)

Description

Computes the hash of the input using the SHA-1 algorithm. The input can either be STRING or BYTES. The string version treats the input as an array of bytes.

This function returns 20 bytes.

Return type

BYTES

Example

SELECT SHA1("Hello World") as sha1;

+-----------------------------------------------------------+
| sha1                                                      |
+-----------------------------------------------------------+
| \nMU\xa8\xd7x\xe5\x02/\xabp\x19w\xc5\xd8@\xbb\xc4\x86\xd0 |
+-----------------------------------------------------------+

SHA256

SHA256(input)

Description

Computes the hash of the input using the SHA-256 algorithm. The input can either be STRING or BYTES. The string version treats the input as an array of bytes.

This function returns 32 bytes.

Return type

BYTES

Example

SELECT SHA256("Hello World") as sha256;

SHA512

SHA512(input)

Description

Computes the hash of the input using the SHA-512 algorithm. The input can either be STRING or BYTES. The string version treats the input as an array of bytes.

This function returns 32 bytes.

Return type

BYTES

Example

SELECT SHA512("Hello World") as sha512;

String functions

These string functions work on two different values: STRING and BYTES data types. STRING values must be well-formed UTF-8.

Functions that return position values, such as STRPOS, encode those positions as INT64. The value 1 refers to the first character (or byte), 2 refers to the second, and so on. The value 0 indicates an invalid index. When working on STRING types, the returned positions refer to character positions.

All string comparisons are done byte-by-byte, without regard to Unicode canonical equivalence.

BYTE_LENGTH

BYTE_LENGTH(value)

Description

Returns the length of the value in bytes, regardless of whether the type of the value is STRING or BYTES.

Return type

INT64

Examples

WITH example AS
  (SELECT "абвгд" AS characters, b"абвгд" AS bytes)

SELECT
  characters,
  BYTE_LENGTH(characters) AS string_example,
  bytes,
  BYTE_LENGTH(bytes) AS bytes_example
FROM example;

+------------+----------------+-------+---------------+
| characters | string_example | bytes | bytes_example |
+------------+----------------+-------+---------------+
| абвгд      | 10             | абвгд | 10            |
+------------+----------------+-------+---------------+

CHAR_LENGTH

CHAR_LENGTH(value)

Description

Returns the length of the STRING in characters.

Return type

INT64

Examples

WITH example AS
  (SELECT "абвгд" AS characters)

SELECT
  characters,
  CHAR_LENGTH(characters) AS char_length_example
FROM example;

+------------+---------------------+
| characters | char_length_example |
+------------+---------------------+
| абвгд      |                   5 |
+------------+---------------------+

CHARACTER_LENGTH

CHARACTER_LENGTH(value)

Description

Synonym for CHAR_LENGTH.

Return type

INT64

Examples

WITH example AS
  (SELECT "абвгд" AS characters)

SELECT
  characters,
  CHARACTER_LENGTH(characters) AS char_length_example
FROM example;

+------------+---------------------+
| characters | char_length_example |
+------------+---------------------+
| абвгд      |                   5 |
+------------+---------------------+

CODE_POINTS_TO_BYTES

CODE_POINTS_TO_BYTES(ascii_values)

Description

Takes an array of extended ASCII character values and returns BYTES.

To convert from BYTES to an array of code points, see TO_CODE_POINTS.

Return type

BYTES

Examples

The following is a basic example using CODE_POINTS_TO_BYTES.

SELECT CODE_POINTS_TO_BYTES([65, 98, 67, 100]) AS bytes;

+-------+
| bytes |
+-------+
| AbCd  |
+-------+

The following example uses a rotate-by-13 places (ROT13) algorithm to encode a string.

SELECT CODE_POINTS_TO_BYTES(ARRAY_AGG(
  (SELECT
      CASE
        WHEN chr BETWEEN b'a' and b'z'
          THEN TO_CODE_POINTS(b'a')[offset(0)] +
            MOD(code+13-TO_CODE_POINTS(b'a')[offset(0)],26)
        WHEN chr BETWEEN b'A' and b'Z'
          THEN TO_CODE_POINTS(b'A')[offset(0)] +
            MOD(code+13-TO_CODE_POINTS(b'A')[offset(0)],26)
        ELSE code
      END
   FROM
     (SELECT code, CODE_POINTS_TO_BYTES([code]) chr)
  ) ORDER BY OFFSET)) AS encoded_string
FROM UNNEST(TO_CODE_POINTS(b'Test String!')) code WITH OFFSET;

+----------------+
| encoded_string |
+----------------+
| Grfg Fgevat!   |
+----------------+

CODE_POINTS_TO_STRING

CODE_POINTS_TO_STRING(value)

Description

Takes an array of code points and returns a STRING.

To convert from a string to an array of code points, see TO_CODE_POINTS.

Return type

STRING

Example

The following is a basic example using CODE_POINTS_TO_STRING.

SELECT CODE_POINTS_TO_STRING([65, 255, 513, 1024]) AS string;

+--------+
| string |
+--------+
| AÿȁЀ   |
+--------+

The following example computes the frequency of letters in a set of words.

WITH Words AS (
  SELECT word
  FROM UNNEST(['foo', 'bar', 'baz', 'giraffe', 'llama']) AS word
)
SELECT
  CODE_POINTS_TO_STRING([code_point]) AS letter,
  COUNT(*) AS letter_count
FROM Words,
  UNNEST(TO_CODE_POINTS(word)) AS code_point
GROUP BY 1
ORDER BY 2 DESC;

+--------+--------------+
| letter | letter_count |
+--------+--------------+
| a      | 5            |
| f      | 3            |
| r      | 2            |
| b      | 2            |
| l      | 2            |
| o      | 2            |
| g      | 1            |
| z      | 1            |
| e      | 1            |
| m      | 1            |
| i      | 1            |
+--------+--------------+

CONCAT

CONCAT(value1[, ...])

Description

Concatenates one or more values into a single result.

Return type

STRING or BYTES

Examples

With Employees AS
  (SELECT
    "John" AS first_name,
    "Doe" AS last_name
  UNION ALL
  SELECT
    "Jane" AS first_name,
    "Smith" AS last_name
  UNION ALL
  SELECT
    "Joe" AS first_name,
    "Jackson" AS last_name)

SELECT
  CONCAT(first_name, " ", last_name)
  AS full_name
FROM Employees;

+---------------------+
| full_name           |
+---------------------+
| John Doe            |
| Jane Smith          |
| Joe Jackson         |
+---------------------+

ENDS_WITH

ENDS_WITH(value1, value2)

Description

Takes two values. Returns TRUE if the second value is a suffix of the first.

Return type

BOOL

Examples

WITH items AS
  (SELECT "apple" as item
  UNION ALL
  SELECT "banana" as item
  UNION ALL
  SELECT "orange" as item)

SELECT
  ENDS_WITH(item, "e") as example
FROM items;

+---------+
| example |
+---------+
|    True |
|   False |
|    True |
+---------+

FORMAT

BigQuery supports a FORMAT() function for formatting strings. This function is similar to the C printf function. It produces a STRING from a format string that contains zero or more format specifiers, along with a variable length list of additional arguments that matches the format specifiers. Here are some examples:

Description Statement Result
Simple integer 
format("%d", 10)
 
10
Integer with left blank padding 
format("|%10d|", 11)
 
|           11|
Integer with left zero padding 
format("+%010d+", 12)
 
+0000000012+
STRING 
format("-%s-", 'abcd efg')
 
-abcd efg-
FLOAT64 
format("%f %E", 1.1, 2.2)
 
1.100000 2.200000E+00
DATE 
format("%t", date "2015-09-01")
 
2015-09-01
TIMESTAMP 
format("%t", timestamp "2015-09-01 12:34:56
America/Los_Angeles")
 
2015-09-01 19:34:56+00

The FORMAT() function does not provide fully customizable formatting for all types and values, nor formatting that is sensitive to locale.

If custom formatting is necessary for a type, you must first format it using type-specific format functions, such as FORMAT_DATE() or FORMAT_TIMESTAMP(). For example:

FORMAT("date: %s!", FORMAT_DATE("%B %d, %Y", date '2015-01-02'));

Returns

date: January 02, 2015!

Syntax

The FORMAT() syntax takes a format string and variable length list of arguments and produces a STRING result:

FORMAT(<format_string>, ...)

The <format_string> expression can contain zero or more format specifiers. Each format specifier is introduced by the % symbol, and must map to one or more of the remaining arguments. For the most part, this is a one-to-one mapping, except when the * specifier is present. For example, %.*i maps to two arguments—a length argument and a signed integer argument. If the number of arguments related to the format specifiers is not the same as the number of arguments, an error occurs.

Supported format specifiers

The FORMAT() function format specifier follows this prototype:

%[flags][width][.precision]specifier

The supported format specifiers are identified in the following table. Extensions from printf() are identified in italics.

Specifier Description Examples Types
d or i Decimal integer 392 INT64
o Octal 610
INT64*
x Hexadecimal integer 7fa
INT64*
X Hexadecimal integer (uppercase) 7FA
INT64*
f Decimal floating point, lowercase 392.65
inf
NaN		 FLOAT64
F Decimal floating point, uppercase 392.65
inf
NAN		 FLOAT64
e Scientific notation (mantissa/exponent), lowercase 3.9265e+2
inf
NaN		 FLOAT64
E Scientific notation (mantissa/exponent), uppercase 3.9265E+2
inf
NAN		 FLOAT64
g Use the shortest representation, %e or %f 392.65 FLOAT64
G Use the shortest representation, %E or %F 392.65 FLOAT64
s String of characters sample STRING
t Returns a printable string representing the value. Often looks similar to casting the argument to STRING. See %t section below. sample
2014‑01‑01		 <any>
T Produces a string that is a valid BigQuery constant with a similar type to the value's type (maybe wider, or maybe string). See %T section below. 'sample'
b'bytes sample'
1234
2.3
date '2014‑01‑01'		 <any>
% '%%' produces a single '%' % n/a

*The specifiers o, x, and X raise an error if negative values are used.

The format specifier can optionally contain the sub-specifiers identified above in the specifier prototype.

These sub-specifiers must comply with the following specifications.

Flags
Flags Description
- Left-justify within the given field width; Right justification is the default (see width sub-specifier)
+ Forces to precede the result with a plus or minus sign (+ or -) even for positive numbers. By default, only negative numbers are preceded with a - sign
<space> If no sign is going to be written, a blank space is inserted before the value
# Used with o, x or X specifiers. Precedes the value with 0, 0x or 0X respectively for values different than zero
0 Left-pads the number with zeroes (0) instead of spaces when padding is specified (see width sub-specifier)
'

Formats integers using the appropriating grouping character. For example:

  • FORMAT("%'d", 12345678) returns 12,345,678
  • FORMAT("%'x", 12345678) returns bc:614e
  • FORMAT("%'o", 55555) returns 15,4403

    • This flag is only relevant for decimal, hex, and octal values.

Flags may be specified in any order. Duplicate flags are not an error. When flags are not relevant for some element type, they are ignored.

Width
Width Description
<number> Minimum number of characters to be printed. If the value to be printed is shorter than this number, the result is padded with blank spaces. The value is not truncated even if the result is larger
* The width is not specified in the format string, but as an additional integer value argument preceding the argument that has to be formatted
Precision
Precision Description
.<number> For integer specifiers (d, i, o, u, x, X): precision specifies the minimum number of digits to be written. If the value to be written is shorter than this number, the result is padded with trailing zeros. The value is not truncated even if the result is longer. A precision of 0 means that no character is written for the value 0. For a, A, e, E, f and F specifiers: this is the number of digits to be printed after the decimal point (by default, this is 6)
.* The precision is not specified in the format string, but as an additional integer value argument preceding the argument that has to be formatted

%t and %T behavior

The %t and %T format specifiers are defined for all types. The width, precision, and flags act as they do for %s: the width is the minimum width and the STRING will be padded to that size, and precision is the maximum width of content to show and the STRING will be truncated to that size, prior to padding to width.

%t is always meant to be a readable form of the value.

%T is always a valid SQL literal of a similar type, such as a wider numeric type. The literal will not include casts or a type name, except for the special case of non-finite floating point values.

The STRING is formatted as follows:

Type %t %T
NULL of any type NULL NULL
INT64
 123 123
FLOAT64 123.0 (always with .0)
123e+10
inf
-inf
NaN		 123.0 (always with .0)
123e+10
CAST("inf" AS <type>)
CAST("-inf" AS <type>)
CAST("nan" AS <type>)
STRING unquoted string value quoted string literal
BYTES unquoted escaped bytes
e.g. abc\x01\x02		 quoted bytes literal
e.g. b"abc\x01\x02"
DATE 2011-02-03 DATE "2011-02-03"
TIMESTAMP 2011-02-03 04:05:06+00 TIMESTAMP "2011-02-03 04:05:06+00"
ARRAY [value, value, ...]
where values are formatted with %t		 [value, value, ...]
where values are formatted with %T
STRUCT (value, value, ...)
where fields are formatted with %t		 (value, value, ...)
where fields are formatted with %T

Special cases:
Zero fields: STRUCT()
One field: STRUCT(value)

Error conditions

If a format specifier is invalid, or is not compatible with the related argument type, or the wrong number or arguments are provided, then an error is produced. For example, the following <format_string> expressions are invalid:

FORMAT('%s', 1)
FORMAT('%')

NULL argument handling

A NULL format string results in a NULL output STRING. Any other arguments are ignored in this case.

The function generally produces a NULL value if a NULL argument is present. For example, FORMAT('%i', <NULL expression>) produces a NULL STRING as output.

However, there are some exceptions: if the format specifier is %t or %T (both of which produce STRINGs that effectively match CAST and literal value semantics), a NULL value produces 'NULL' (without the quotes) in the result STRING. For example, the function:

FORMAT('00-%t-00', <NULL expression>);

Returns

00-NULL-00

Additional semantic rules

FLOAT64 values can be +/-inf or NaN. When an argument has one of those values, the result of the format specifiers %f, %F, %e, %E, %g, %G, and %t are inf, -inf, or nan (or the same in uppercase) as appropriate. This is consistent with how BigQuery casts these values to STRING. For %T, BigQuery returns quoted strings for FLOAT64 values that don't have non-string literal representations.

FROM_BASE64

FROM_BASE64(string_expr)

Description

Converts the base64-encoded input string_expr into BYTES format. To convert BYTES to a base64-encoded STRING, use TO_BASE64.

Return type

BYTES

Example

SELECT FROM_BASE64('Dave') as dave_base64;

+-------------+
| dave_base64 |
+-------------+
| \r\xab\xde  |
+-------------+

LENGTH

LENGTH(value)

Description

Returns the length of the value. The returned value is in characters for STRING arguments and in bytes for the BYTES argument.

Return type

INT64

Examples

WITH example AS
  (SELECT "абвгд" AS characters)

SELECT
  characters,
  LENGTH(characters) AS string_example,
  LENGTH(CAST(characters AS BYTES)) AS bytes_example
FROM example;

+------------+----------------+---------------+
| characters | string_example | bytes_example |
+------------+----------------+---------------+
| абвгд      |              5 |            10 |
+------------+----------------+---------------+

LPAD

LPAD(original_value, return_length[, pattern])

Description

Returns a value that consists of original_value prepended with pattern. The return_length is an INT64 that specifies the length of the returned value. If original_value is BYTES, return_length is the number of bytes. If original_value is STRING, return_length is the number of characters.

The default value of pattern is a blank space.

Both original_value and pattern must be the same data type.

If return_length is less than or equal to the original_value length, this function returns the original_value value, truncated to the value of return_length. For example, LPAD("hello world", 7); returns "hello w".

If original_value, return_length, or pattern is NULL, this function returns NULL.

This function returns an error if:

  • return_length is negative
  • pattern is empty

Return type

STRING or BYTES

Examples

SELECT t, len, FORMAT("%T", LPAD(t, len)) AS LPAD FROM UNNEST([
  STRUCT('abc' AS t, 5 AS len),
  ('abc', 2),
  ('例子', 4)
]);
t len LPAD
abc 5 "  abc"
abc 2 "ab"
例子 4 "  例子" 
SELECT t, len, pattern, FORMAT("%T", LPAD(t, len, pattern)) AS LPAD FROM UNNEST([
  STRUCT('abc' AS t, 8 AS len, 'def' AS pattern),
  ('abc', 5, '-'),
  ('例子', 5, '中文')
]);
t len pattern LPAD
abc 8 def "defdeabc"
abc 5 - "--abc"
例子 5 中文 "中文中例子" 
SELECT FORMAT("%T", t) AS t, len, FORMAT("%T", LPAD(t, len)) AS LPAD FROM UNNEST([
  STRUCT(b'abc' AS t, 5 AS len),
  (b'abc', 2),
  (b'\xab\xcd\xef', 4)
]);
t len LPAD
b"abc" 5 b"  abc"
b"abc" 2 b"ab"
b"\xab\xcd\xef" 4 b" \xab\xcd\xef" 
SELECT
  FORMAT("%T", t) AS t,
  len,
  FORMAT("%T", pattern) AS pattern,
  FORMAT("%T", LPAD(t, len, pattern)) AS LPAD
FROM UNNEST([
  STRUCT(b'abc' AS t, 8 AS len, b'def' AS pattern),
  (b'abc', 5, b'-'),
  (b'\xab\xcd\xef', 5, b'\x00')
]);
t len pattern LPAD
b"abc" 8 b"def" b"defdeabc"
b"abc" 5 b"-" b"--abc"
b"\xab\xcd\xef" 5 b"\x00" b"\x00\x00\xab\xcd\xef"

LOWER

LOWER(value)

Description

For STRING arguments, returns the original string with all alphabetic characters in lowercase. Mapping between lowercase and uppercase is done according to the Unicode Character Database without taking into account language-specific mappings.

For BYTES arguments, the argument is treated as ASCII text, with all bytes greater than 127 left intact.

Return type

STRING or BYTES

Examples

WITH items AS
  (SELECT
    "FOO" as item
  UNION ALL
  SELECT
    "BAR" as item
  UNION ALL
  SELECT
    "BAZ" as item)

SELECT
  LOWER(item) AS example
FROM items;

+---------+
| example |
+---------+
| foo     |
| bar     |
| baz     |
+---------+

LTRIM

LTRIM(value1[, value2])

Description

Identical to TRIM, but only removes leading characters.

Return type

STRING or BYTES

Examples

WITH items AS
  (SELECT "   apple   " as item
  UNION ALL
  SELECT "   banana   " as item
  UNION ALL
  SELECT "   orange   " as item)

SELECT
  CONCAT("#", LTRIM(item), "#") as example
FROM items;

+-------------+
| example     |
+-------------+
| #apple   #  |
| #banana   # |
| #orange   # |
+-------------+

WITH items AS
  (SELECT "***apple***" as item
  UNION ALL
  SELECT "***banana***" as item
  UNION ALL
  SELECT "***orange***" as item)

SELECT
  LTRIM(item, "*") as example
FROM items;

+-----------+
| example   |
+-----------+
| apple***  |
| banana*** |
| orange*** |
+-----------+

WITH items AS
  (SELECT "xxxapplexxx" as item
  UNION ALL
  SELECT "yyybananayyy" as item
  UNION ALL
  SELECT "zzzorangezzz" as item
  UNION ALL
  SELECT "xyzpearxyz" as item)

SELECT
  LTRIM(item, "xyz") as example
FROM items;

+---------+
| example |
+---------+
| apple   |
| banana  |
| orange  |
| pear    |
+---------+

REGEXP_CONTAINS

REGEXP_CONTAINS(value, regex)

Description

Returns TRUE if value is a partial match for the regular expression, regex. You can search for a full match by using ^ (beginning of text) and $ (end of text).

If the regex argument is invalid, the function returns an error.

Note: BigQuery provides regular expression support using the re2 library; see that documentation for its regular expression syntax.

Return type

BOOL

Examples

SELECT
  email,
  REGEXP_CONTAINS(email, r"@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+") AS is_valid
FROM
  (SELECT
    ["foo@example.com", "bar@example.org", "www.example.net"]
    AS addresses),
  UNNEST(addresses) AS email;

+-----------------+----------+
| email           | is_valid |
+-----------------+----------+
| foo@example.com | true     |
| bar@example.org | true     |
| www.example.net | false    |
+-----------------+----------+

# Performs a full match, using ^ and $.
SELECT
  email,
  REGEXP_CONTAINS(email, r"^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$")
    AS valid_email_address
FROM
  (SELECT
    ["foo@example.com", "bar@example.org", "www.example.net"]
    AS addresses),
  UNNEST(addresses) AS email;

+-----------------+---------------------+
| email           | valid_email_address |
+-----------------+---------------------+
| foo@example.com | true                |
| bar@example.org | true                |
| www.example.net | false               |
+-----------------+---------------------+

REGEXP_EXTRACT

REGEXP_EXTRACT(value, regex)

Description

Returns the first substring in value that matches the regular expression, regex. Returns NULL if there is no match.

If the regular expression contains a capturing group, the function returns the substring that is matched by that capturing group. If the expression does not contain a capturing group, the function returns the entire matching substring.

Returns an error if:

  • The regular expression is invalid
  • The regular expression has more than one capturing group

Note: BigQuery provides regular expression support using the re2 library; see that documentation for its regular expression syntax.

Return type

STRING or BYTES

Examples

WITH email_addresses AS
  (SELECT "foo@example.com" as email
  UNION ALL
  SELECT "bar@example.org" as email
  UNION ALL
  SELECT "baz@example.net" as email)

SELECT
  REGEXP_EXTRACT(email, r"^[a-zA-Z0-9_.+-]+")
  AS user_name
FROM email_addresses;

+-----------+
| user_name |
+-----------+
| foo       |
| bar       |
| baz       |
+-----------+

WITH email_addresses AS
  (SELECT "foo@example.com" as email
  UNION ALL
  SELECT "bar@example.org" as email
  UNION ALL
  SELECT "baz@example.net" as email)

SELECT
  REGEXP_EXTRACT(email, r"^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.([a-zA-Z0-9-.]+$)")
  AS top_level_domain
FROM email_addresses;

+------------------+
| top_level_domain |
+------------------+
| com              |
| org              |
| net              |
+------------------+

REGEXP_EXTRACT_ALL

REGEXP_EXTRACT_ALL(value, regex)

Description

Returns an array of all substrings of value that match the regular expression, regex.

The REGEXP_EXTRACT_ALL function only returns non-overlapping matches. For example, using this function to extract ana from banana returns only one substring, not two.

Note: BigQuery provides regular expression support using the re2 library; see that documentation for its regular expression syntax.

Return type

An ARRAY of either STRINGs or BYTES

Examples

WITH code_markdown AS
  (SELECT "Try `function(x)` or `function(y)`" as code)

SELECT
  REGEXP_EXTRACT_ALL(code, "`(.+?)`") AS example
FROM code_markdown;

+----------------------------+
| example                    |
+----------------------------+
| [function(x), function(y)] |
+----------------------------+

REGEXP_REPLACE

REGEXP_REPLACE(value, regex, replacement)

Description

Returns a STRING where all substrings of value that match regular expression regex are replaced with replacement.

You can use backslashed-escaped digits (\1 to \9) within the replacement argument to insert text matching the corresponding parenthesized group in the regex pattern. Use \0 to refer to the entire matching text.

Note: To add a backslash in your regular expression, you must first escape it. For example, SELECT REGEXP_REPLACE("abc", "b(.)", "X\\1"); returns aXc.

The REGEXP_REPLACE function only replaces non-overlapping matches. For example, replacing ana within banana results in only one replacement, not two.

If the regex argument is not a valid regular expression, this function returns an error.

Note: BigQuery provides regular expression support using the re2 library; see that documentation for its regular expression syntax.

Return type

STRING or BYTES

Examples

WITH markdown AS
  (SELECT "# Heading" as heading
  UNION ALL
  SELECT "# Another heading" as heading)

SELECT
  REGEXP_REPLACE(heading, r"^# ([a-zA-Z0-9\s]+$)", "<h1>\\1</h1>")
  AS html
FROM markdown;

+--------------------------+
| html                     |
+--------------------------+
| <h1>Heading</h1>         |
| <h1>Another heading</h1> |
+--------------------------+

REPLACE

REPLACE(original_value, from_value, to_value)

Description

Replaces all occurrences of from_value with to_value in original_value. If from_value is empty, no replacement is made.

Return type

STRING or BYTES

Examples

WITH desserts AS
  (SELECT "apple pie" as dessert
  UNION ALL
  SELECT "blackberry pie" as dessert
  UNION ALL
  SELECT "cherry pie" as dessert)

SELECT
  REPLACE (dessert, "pie", "cobbler") as example
FROM desserts;

+--------------------+
| example            |
+--------------------+
| apple cobbler      |
| blackberry cobbler |
| cherry cobbler     |
+--------------------+

REPEAT

REPEAT(original_value, repetitions)

Description

Returns a value that consists of original_value, repeated. The repetitions parameter specifies the number of times to repeat original_value. Returns NULL if either original_value or repetitions are NULL.

This function return an error if the repetitions value is negative.

Return type

STRING or BYTES

Examples

SELECT t, n, REPEAT(t, n) AS REPEAT FROM UNNEST([
  STRUCT('abc' AS t, 3 AS n),
  ('例子', 2),
  ('abc', null),
  (null, 3)
]);
t n REPEAT
abc 3 abcabcabc
例子 2 例子例子
abc NULL NULL
NULL 3 NULL

REVERSE

REVERSE(value)

Description

Returns the reverse of the input STRING or BYTES.

Return type

STRING or BYTES

Examples

WITH example AS (
  SELECT "foo" AS sample_string, b"bar" AS sample_bytes UNION ALL
  SELECT "абвгд" AS sample_string, b"123" AS sample_bytes
)
SELECT
  sample_string,
  REVERSE(sample_string) AS reverse_string,
  sample_bytes,
  REVERSE(sample_bytes) AS reverse_bytes
FROM example;

+---------------+----------------+--------------+---------------+
| sample_string | reverse_string | sample_bytes | reverse_bytes |
+---------------+----------------+--------------+---------------+
| foo           | oof            | bar          | rab           |
| абвгд         | дгвба          | 123          | 321           |
+---------------+----------------+--------------+---------------+

RPAD

RPAD(original_value, return_length[, pattern])

Description

Returns a value that consists of original_value appended with pattern. The return_length is an INT64 that specifies the length of the returned value. If original_value is BYTES, return_length is the number of bytes. If original_value is STRING, return_length is the number of characters.

The default value of pattern is a blank space.

Both original_value and pattern must be the same data type.

If return_length is less than or equal to the original_value length, this function returns the original_value value, truncated to the value of return_length. For example, RPAD("hello world", 7); returns "o world".

If original_value, return_length, or pattern is NULL, this function returns NULL.

This function returns an error if:

  • return_length is negative
  • pattern is empty

Return type

STRING or BYTES

Examples

SELECT t, len, FORMAT("%T", RPAD(t, len)) AS RPAD FROM UNNEST([
  STRUCT('abc' AS t, 5 AS len),
  ('abc', 2),
  ('例子', 4)
]);
t len RPAD
abc 5 "abc  "
abc 2 "ab"
例子 4 "例子  " 
SELECT t, len, pattern, FORMAT("%T", RPAD(t, len, pattern)) AS RPAD FROM UNNEST([
  STRUCT('abc' AS t, 8 AS len, 'def' AS pattern),
  ('abc', 5, '-'),
  ('例子', 5, '中文')
]);
t len pattern RPAD
abc 8 def "abcdefde"
abc 5 - "abc--"
例子 5 中文 "例子中文中" 
SELECT FORMAT("%T", t) AS t, len, FORMAT("%T", RPAD(t, len)) AS RPAD FROM UNNEST([
  STRUCT(b'abc' AS t, 5 AS len),
  (b'abc', 2),
  (b'\xab\xcd\xef', 4)
]);
t len RPAD
b"abc" 5 b"abc  "
b"abc" 2 b"ab"
b"\xab\xcd\xef" 4 b"\xab\xcd\xef " 
SELECT
  FORMAT("%T", t) AS t,
  len,
  FORMAT("%T", pattern) AS pattern,
  FORMAT("%T", RPAD(t, len, pattern)) AS RPAD
FROM UNNEST([
  STRUCT(b'abc' AS t, 8 AS len, b'def' AS pattern),
  (b'abc', 5, b'-'),
  (b'\xab\xcd\xef', 5, b'\x00')
]);
t len pattern RPAD
b"abc" 8 b"def" b"abcdefde"
b"abc" 5 b"-" b"abc--"
b"\xab\xcd\xef" 5 b"\x00" b"\xab\xcd\xef\x00\x00"

RTRIM

RTRIM(value1[, value2])

Description

Identical to TRIM, but only removes trailing characters.

Return type

STRING or BYTES

Examples

WITH items AS
  (SELECT "***apple***" as item
  UNION ALL
  SELECT "***banana***" as item
  UNION ALL
  SELECT "***orange***" as item)

SELECT
  RTRIM(item, "*") as example
FROM items;

+-----------+
| example   |
+-----------+
| ***apple  |
| ***banana |
| ***orange |
+-----------+

WITH items AS
  (SELECT "applexxx" as item
  UNION ALL
  SELECT "bananayyy" as item
  UNION ALL
  SELECT "orangezzz" as item
  UNION ALL
  SELECT "pearxyz" as item)

SELECT
  RTRIM(item, "xyz") as example
FROM items;

+---------+
| example |
+---------+
| apple   |
| banana  |
| orange  |
| pear    |
+---------+

SAFE_CONVERT_BYTES_TO_STRING

SAFE_CONVERT_BYTES_TO_STRING(value)

Description

Converts a sequence of bytes to a string. Any invalid UTF-8 characters are replaced with the Unicode replacement character, U+FFFD.

Return type

STRING

Examples

The following statement returns the Unicode replacement character, �.

SELECT SAFE_CONVERT_BYTES_TO_STRING(b'\xc2') as safe_convert;

SPLIT

SPLIT(value[, delimiter])

Description

Splits value using the delimiter argument.

For STRING, the default delimiter is ,.

For BYTES, you must specify a delimiter.

Splitting on an empty delimiter produces an array of UTF-8 characters for STRING values, and an array of BYTES for BYTES values.

Splitting an empty STRING returns an ARRAY with a single empty STRING.

Return type

ARRAY of type STRING or ARRAY of type BYTES

Examples

WITH letters AS
  (SELECT "a b c d" as letter_group
  UNION ALL
  SELECT "e f g h" as letter_group
  UNION ALL
  SELECT "i j k l" as letter_group)

SELECT SPLIT(letter_group, " ") as example
FROM letters;

+----------------------+
| example              |
+----------------------+
| [a, b, c, d]         |
| [e, f, g, h]         |
| [i, j, k, l]         |
+----------------------+

STARTS_WITH

STARTS_WITH(value1, value2)

Description

Takes two values. Returns TRUE if the second value is a prefix of the first.

Return type

BOOL

Examples

WITH items AS
  (SELECT "foo" as item
  UNION ALL
  SELECT "bar" as item
  UNION ALL
  SELECT "baz" as item)

SELECT
  STARTS_WITH(item, "b") as example
FROM items;

+---------+
| example |
+---------+
|   False |
|    True |
|    True |
+---------+

STRPOS

STRPOS(value1, value2)

Description

Returns the 1-based index of the first occurrence of value2 inside value1. Returns 0 if value2 is not found.

Return type

INT64

Examples

WITH email_addresses AS
  (SELECT
    "foo@example.com" AS email_address
  UNION ALL
  SELECT
    "foobar@example.com" AS email_address
  UNION ALL
  SELECT
    "foobarbaz@example.com" AS email_address
  UNION ALL
  SELECT
    "quxexample.com" AS email_address)

SELECT
  STRPOS(email_address, "@") AS example
FROM email_addresses;

+---------+
| example |
+---------+
|       4 |
|       7 |
|      10 |
|       0 |
+---------+

SUBSTR

SUBSTR(value, position[, length])

Description

Returns a substring of the supplied value. The position argument is an integer specifying the starting position of the substring, with position = 1 indicating the first character or byte. The length argument is the maximum number of characters for STRING arguments, or bytes for BYTES arguments.

If position is negative, the function counts from the end of value, with -1 indicating the last character.

If position is a position off the left end of the STRING (position = 0 or position < -LENGTH(value)), the function starts from position = 1. If length exceeds the length of value, returns fewer than length characters.

If length is less than 0, the function returns an error.

Return type

STRING or BYTES

Examples

WITH items AS
  (SELECT "apple" as item
  UNION ALL
  SELECT "banana" as item
  UNION ALL
  SELECT "orange" as item)

SELECT
  SUBSTR(item, 2) as example
FROM items;

+---------+
| example |
+---------+
| pple    |
| anana   |
| range   |
+---------+

WITH items AS
  (SELECT "apple" as item
  UNION ALL
  SELECT "banana" as item
  UNION ALL
  SELECT "orange" as item)

SELECT
  SUBSTR(item, 2, 2) as example
FROM items;

+---------+
| example |
+---------+
| pp      |
| an      |
| ra      |
+---------+

WITH items AS
  (SELECT "apple" as item
  UNION ALL
  SELECT "banana" as item
  UNION ALL
  SELECT "orange" as item)

SELECT
  SUBSTR(item, -2) as example
FROM items;

+---------+
| example |
+---------+
| le      |
| na      |
| ge      |
+---------+

TO_BASE64

TO_BASE64(bytes_expr)

Description

Converts a sequence of BYTES into a base64-encoded STRING. To convert a base64-encoded STRING into BYTES, use FROM_BASE64.

Return type

STRING

Example

SELECT TO_BASE64(b'\r\xab\xde') as base64_dave;

+-------------+
| base64_dave |
+-------------+
| Dave        |
+-------------+

TO_CODE_POINTS

TO_CODE_POINTS(value)

Description

Takes a value and returns an array of INT64.

  • If value is a STRING, each element in the returned array represents a code point. Each code point falls within the range of [0, 0xD7FF] and [0xE000, 0x10FFFF].
  • If value is BYTES, each element in the array is an extended ASCII character value in the range of [0, 255].

To convert from an array of code points to a STRING or BYTES, see CODE_POINTS_TO_STRING or CODE_POINTS_TO_BYTES.

Return type

ARRAY of INT64

Examples

The following example gets the code points for each element in an array of words.

SELECT word, TO_CODE_POINTS(word) AS code_points
FROM UNNEST(['foo', 'bar', 'baz', 'giraffe', 'llama']) AS word;

+---------+------------------------------------+
| word    | code_points                        |
+---------+------------------------------------+
| foo     | [102, 111, 111]                    |
| bar     | [98, 97, 114]                      |
| baz     | [98, 97, 122]                      |
| giraffe | [103, 105, 114, 97, 102, 102, 101] |
| llama   | [108, 108, 97, 109, 97]            |
+---------+------------------------------------+

The following example converts integer representations of BYTES to their corresponding ASCII character values.

SELECT word, TO_CODE_POINTS(word) AS bytes_value_as_integer
FROM UNNEST([b'\x00\x01\x10\xff', b'\x66\x6f\x6f']) AS word;

+------------------+------------------------+
| word             | bytes_value_as_integer |
+------------------+------------------------+
| \x00\x01\x10\xff | [0, 1, 16, 255]        |
| foo              | [102, 111, 111]        |
+------------------+------------------------+

The following example demonstrates the difference between a BYTES result and a STRING result.

SELECT TO_CODE_POINTS(b'Ā') AS b_result, TO_CODE_POINTS('Ā') AS s_result;

+------------+----------+
| b_result   | s_result |
+------------+----------+
| [196, 128] | [256]    |
+------------+----------+

Notice that the character, Ā, is represented as a two-byte Unicode sequence. As a result, the BYTES version of TO_CODE_POINTS returns an array with two elements, while the STRING version returns an array with a single element.

TRIM

TRIM(value1[, value2])

Description

Removes all leading and trailing characters that match value2. If value2 is not specified, all leading and trailing whitespace characters (as defined by the Unicode standard) are removed. If the first argument is of type BYTES, the second argument is required.

If value2 contains more than one character or byte, the function removes all leading or trailing characters or bytes contained in value2.

Return type

STRING or BYTES

Examples

WITH items AS
  (SELECT "   apple   " as item
  UNION ALL
  SELECT "   banana   " as item
  UNION ALL
  SELECT "   orange   " as item)

SELECT
  CONCAT("#", TRIM(item), "#") as example
FROM items;

+----------+
| example  |
+----------+
| #apple#  |
| #banana# |
| #orange# |
+----------+

WITH items AS
  (SELECT "***apple***" as item
  UNION ALL
  SELECT "***banana***" as item
  UNION ALL
  SELECT "***orange***" as item)

SELECT
  TRIM(item, "*") as example
FROM items;

+---------+
| example |
+---------+
| apple   |
| banana  |
| orange  |
+---------+

WITH items AS
  (SELECT "xxxapplexxx" as item
  UNION ALL
  SELECT "yyybananayyy" as item
  UNION ALL
  SELECT "zzzorangezzz" as item
  UNION ALL
  SELECT "xyzpearxyz" as item)

SELECT
  TRIM(item, "xyz") as example
FROM items;

+---------+
| example |
+---------+
| apple   |
| banana  |
| orange  |
| pear    |
+---------+

UPPER

UPPER(value)

Description

For STRING arguments, returns the original string with all alphabetic characters in uppercase. Mapping between uppercase and lowercase is done according to the Unicode Character Database without taking into account language-specific mappings.

For BYTES arguments, the argument is treated as ASCII text, with all bytes greater than 127 left intact.

Return type

STRING or BYTES

Examples

WITH items AS
  (SELECT
    "foo" as item
  UNION ALL
  SELECT
    "bar" as item
  UNION ALL
  SELECT
    "baz" as item)

SELECT
  UPPER(item) AS example
FROM items;

+---------+
| example |
+---------+
| FOO     |
| BAR     |
| BAZ     |
+---------+

JSON functions

BigQuery supports functions that help you retrieve data stored in JSON-formatted strings and functions that help you transform data into JSON-formatted strings.

JSON_EXTRACT or JSON_EXTRACT_SCALAR

JSON_EXTRACT(json_string_expr, json_path_string_literal), which returns JSON values as STRINGs.

JSON_EXTRACT_SCALAR(json_string_expr, json_path_string_literal), which returns scalar JSON values as STRINGs.

Description

The json_string_expr parameter must be a JSON-formatted string. For example:

{"class" : {"students" : [{"name" : "Jane"}]}}

The json_path_string_literal parameter identifies the value or values you want to obtain from the JSON-formatted string. You construct this parameter using the JSONPath format. As part of this format, this parameter must start with a $ symbol, which refers to the outermost level of the JSON-formatted string. You can identify child values using dot or bracket notation. If the JSON object is an array, you can use brackets to specify the array index.

JSONPath Description
$ Root object or element
. or [] Child operator
[] Subscript operator

Both functions return NULL if the json_path_string_literal parameter does not match a value in json_string_expr. If the selected value for JSON_EXTRACT_SCALAR is not scalar, such as an object or an array, the function returns NULL.

If the JSONPath is invalid, these functions raise an error.

In cases where a JSON key uses invalid JSONPath characters, you can escape those characters using single quotes and brackets, [' ']. For example:

SELECT JSON_EXTRACT_SCALAR('{"a.b": {"c": "world"}}', "$['a.b'].c") as hello;

+-------+
| hello |
+-------+
| world |
+-------+

Examples

SELECT JSON_EXTRACT(json_text, '$') AS json_text_string
FROM UNNEST([
  '{"class" : {"students" : [{"name" : "Jane"}]}}',
  '{"class" : {"students" : []}}',
  '{"class" : {"students" : [{"name" : "John"}, {"name": "Jamie"}]}}'
  ]) AS json_text;

+-----------------------------------------------------------+
| json_text_string                                          |
+-----------------------------------------------------------+
| {"class":{"students":[{"name":"Jane"}]}}                  |
| {"class":{"students":[]}}                                 |
| {"class":{"students":[{"name":"John"},{"name":"Jamie"}]}} |
+-----------------------------------------------------------+
SELECT JSON_EXTRACT(json_text, '$.class.students[0]') AS first_student
FROM UNNEST([
  '{"class" : {"students" : [{"name" : "Jane"}]}}',
  '{"class" : {"students" : []}}',
  '{"class" : {"students" : [{"name" : "John"}, {"name": "Jamie"}]}}'
  ]) AS json_text;

+-----------------+
| first_student   |
+-----------------+
| {"name":"Jane"} |
| NULL            |
| {"name":"John"} |
+-----------------+
SELECT JSON_EXTRACT(json_text, '$.class.students[1].name') AS second_student_name
FROM UNNEST([
  '{"class" : {"students" : [{"name" : "Jane"}]}}',
  '{"class" : {"students" : []}}',
  '{"class" : {"students" : [{"name" : "John"}, {"name": "Jamie"}]}}'
  ]) AS json_text;

+-------------------+
| second_student    |
+-------------------+
| NULL              |
| NULL              |
| {"first":"Jamie"} |
+-------------------+
SELECT JSON_EXTRACT(json_text, "$.class['students']") AS student_names
FROM UNNEST([
  '{"class" : {"students" : [{"name" : "Jane"}]}}',
  '{"class" : {"students" : []}}',
  '{"class" : {"students" : [{"name" : "John"}, {"name": "Jamie"}]}}'
  ]) AS json_text;

+------------------------------------+
| student_names                      |
+------------------------------------+
| [{"name":"Jane"}]                  |
| []                                 |
| [{"name":"John"},{"name":"Jamie"}] |
+------------------------------------+
SELECT JSON_EXTRACT('{ "name" : "Jakob", "age" : "6" }', '$.name') as json_name,
  JSON_EXTRACT_SCALAR('{ "name" : "Jakob", "age" : "6" }', '$.name') as scalar_name,
  JSON_EXTRACT('{ "name" : "Jakob", "age" : "6" }', '$.age') as json_age,
  JSON_EXTRACT_SCALAR('{ "name" : "Jakob", "age" : "6" }', '$.age') as scalar;

+-----------+-------------+----------+--------+
| json_name | scalar_name | json_age | scalar |
+-----------+-------------+----------+--------+
| "Jakob"   | Jakob       | "6"      | 6      |
+-----------+-------------+----------+--------+

TO_JSON_STRING

TO_JSON_STRING(value[, pretty_print])

Description

Returns a JSON-formatted string representation of value. This function supports an optional pretty_print parameter. If pretty_print is present, the returned value is formatted for easy readability.

Input data type Returned value
NULL of any type null
BOOL true or false.
INT64

Same as CAST(value AS STRING) when value is in the range of [-253, 253], which is the range of integers that can be represented losslessly as IEEE 754 double-precision floating point numbers. Values outside of this range are represented as quoted strings. For example:

-1
0
12345678901
9007199254740992
-9007199254740992
"9007199254740993"

9007199254740993 is greater than 253, so it is represented as a quoted string.
FLOAT64 +/-inf and NaN are represented as Infinity, -Infinity, and NaN, respectively.

Otherwise, the same as CAST(value AS STRING).
STRING Quoted string value, escaped according to the JSON standard. Specifically, ", \, and the control characters from U+0000 to U+001F are escaped.
BYTES

Quoted RFC 4648 base64-escaped value. For example:

"R29vZ2xl" is the base64 representation of bytes b"Google"
DATE

Quoted date. For example:

"2017-03-06"
TIMESTAMP

Quoted ISO 8601 date-time, where T separates the date and time and Zulu/UTC represents the time zone. For example:

"2017-03-06T12:34:56.789012Z"
DATETIME

Quoted ISO 8601 date-time, where T separates the date and time. For example:

"2017-03-06T12:34:56.789012"
TIME

Quoted ISO 8601 time. For example:

"12:34:56.789012"
ARRAY

[elem1,elem2,...], where each elem is formatted according to the element type.

Example with formatting: 
[
  elem1,
  elem2,
  ...
]

Where each elem is formatted according to the element type. The empty array is represented as [].
STRUCT {"field_name1":field_value1,"field_name2":field_value2,...}

Where each field_value is formatted according to its type.

Example with formatting: 
{
  "field_name1": field_value1,
  "field_name2": field_value2,
  ...
}

Where each field_value is formatted according to its type. If a field_value is a non-empty ARRAY or STRUCT, elements are indented to the appropriate level. The empty struct is represented as {}.

Fields with duplicate names might result in unparseable JSON. Anonymous fields are represented with "".

Invalid UTF-8 field names might result in unparseable JSON. String values are escaped according to the JSON standard. Specifically, ", \, and the control characters from U+0000 to U+001F are escaped.

Return type

JSON string representation of the value.

Examples

Convert rows in a table to JSON.

WITH Input AS (
  SELECT [1, 2] AS x, 'foo' AS y, STRUCT(true AS a, DATE '2017-04-05' AS b) AS s UNION ALL
  SELECT NULL AS x, '' AS y, STRUCT(false AS a, DATE '0001-01-01' AS b) AS s UNION ALL
  SELECT [3] AS x, 'bar' AS y, STRUCT(NULL AS a, DATE '2016-12-05' AS b) AS s
)
SELECT
  t,
  TO_JSON_STRING(t) AS json_row
FROM Input AS t;
+-----------------------------------+-------------------------------------------------------+
| t                                 | json_row                                              |
+-----------------------------------+-------------------------------------------------------+
| {[1, 2], foo, {true, 2017-04-05}} | {"x":[1,2],"y":"foo","s":{"a":true,"b":"2017-04-05"}} |
| {NULL, , {false, 0001-01-01}}     | {"x":null,"y":"","s":{"a":false,"b":"0001-01-01"}}    |
| {[3], bar, {NULL, 2016-12-05}}    | {"x":[3],"y":"bar","s":{"a":null,"b":"2016-12-05"}}   |
+-----------------------------------+-------------------------------------------------------+

Convert rows in a table to JSON with formatting.

WITH Input AS (
  SELECT [1, 2] AS x, 'foo' AS y, STRUCT(true AS a, DATE '2017-04-05' AS b) AS s UNION ALL
  SELECT NULL AS x, '' AS y, STRUCT(false AS a, DATE '0001-01-01' AS b) AS s UNION ALL
  SELECT [3] AS x, 'bar' AS y, STRUCT(NULL AS a, DATE '2016-12-05' AS b) AS s
)
SELECT
  TO_JSON_STRING(t, true) AS json_row
FROM Input AS t;
+-----------------------+
| json_row              |
+-----------------------+
| {                     |
|  "x": [               |
|    1,                 |
|    2                  |
|  ],                   |
|  "y": "foo",          |
|  "s": {               |
|    "a": true,         |
|    "b": "2017-04-05"  |
|  }                    |
|}                      |
| {                     |
|  "x": null,           |
|  "y": "",             |
|  "s": {               |
|    "a": false,        |
|    "b": "0001-01-01"  |
|  }                    |
|}                      |
| {                     |
|  "x": [               |
|    3                  |
|  ],                   |
|  "y": "bar",          |
|  "s": {               |
|    "a": null,         |
|    "b": "2016-12-05"  |
|  }                    |
|}                      |
+-----------------------+

Array functions

ARRAY_CONCAT

ARRAY_CONCAT(array_expression_1 [, array_expression_n])

Description

Concatenates one or more arrays into a single array.

Return type

ARRAY

Examples

SELECT ARRAY_CONCAT([1, 2], [3, 4], [5, 6]) as count_to_six;

+--------------------------------------------------+
| count_to_six                                     |
+--------------------------------------------------+
| [1, 2, 3, 4, 5, 6]                               |
+--------------------------------------------------+

ARRAY_LENGTH

ARRAY_LENGTH(array_expression)

Description

Returns the size of the array. Returns 0 for an empty array. Returns NULL if the array_expression is NULL.

Return type

INT64

Examples

WITH items AS
  (SELECT ["apples", "bananas", NULL, "grapes"] as list
  UNION ALL
  SELECT ["coffee", "tea", "milk" ] as list
  UNION ALL
  SELECT ["cake", "pie"] as list)

SELECT list, ARRAY_LENGTH(list) AS size
FROM items
ORDER BY size DESC;

+---------------------------------+------+
| list                            | size |
+---------------------------------+------+
| [apples, bananas, NULL, grapes] | 4    |
| [coffee, tea, milk]             | 3    |
| [cake, pie]                     | 2    |
+---------------------------------+------+

ARRAY_TO_STRING

ARRAY_TO_STRING(array_expression, delimiter[, null_text])

Description

Returns a concatenation of the elements in array_expression as a STRING. The value for array_expression can either be an array of STRING or BYTES data types.

If the null_text parameter is used, the function replaces any NULL values in the array with the value of null_text.

If the null_text parameter is not used, the function omits the NULL value and its preceding delimiter.

Examples

WITH items AS
  (SELECT ["apples", "bananas", "pears", "grapes"] as list
  UNION ALL
  SELECT ["coffee", "tea", "milk" ] as list
  UNION ALL
  SELECT ["cake", "pie", NULL] as list)

SELECT ARRAY_TO_STRING(list, '--') AS text
FROM items;

+--------------------------------+
| text                           |
+--------------------------------+
| apples--bananas--pears--grapes |
| coffee--tea--milk              |
| cake--pie                      |
+--------------------------------+

WITH items AS
  (SELECT ["apples", "bananas", "pears", "grapes"] as list
  UNION ALL
  SELECT ["coffee", "tea", "milk" ] as list
  UNION ALL
  SELECT ["cake", "pie", NULL] as list)

SELECT ARRAY_TO_STRING(list, '--', 'MISSING') AS text
FROM items;

+--------------------------------+
| text                           |
+--------------------------------+
| apples--bananas--pears--grapes |
| coffee--tea--milk              |
| cake--pie--MISSING             |
+--------------------------------+

GENERATE_ARRAY

GENERATE_ARRAY(start_expression, end_expression[, step_expression])

Description

Returns an array of values. The start_expression and end_expression parameters determine the inclusive start and end of the array.

The GENERATE_ARRAY function accepts the following data types as inputs:

  • FLOAT64
  • INT64

The step_expression parameter determines the increment used to generate array values. The default value for this parameter is 1.

This function returns an error if step_expression is set to 0, or if any input is NaN.

Return Data Type

ARRAY

Examples

The following returns an array of integers, with a default step of 1.

SELECT GENERATE_ARRAY(1, 5) AS example_array;

+-----------------+
| example_array   |
+-----------------+
| [1, 2, 3, 4, 5] |
+-----------------+

The following returns an array using a user-specified step size.

SELECT GENERATE_ARRAY(0, 10, 3) AS example_array;

+---------------+
| example_array |
+---------------+
| [0, 3, 6, 9]  |
+---------------+

The following returns an array using a negative value, -3 for its step size.

SELECT GENERATE_ARRAY(10, 0, -3) AS example_array;

+---------------+
| example_array |
+---------------+
| [10, 7, 4, 1] |
+---------------+

The following returns an array using the same value for the start_expression and end_expression.

SELECT GENERATE_ARRAY(4, 4, 10) AS example_array;

+---------------+
| example_array |
+---------------+
| [4]           |
+---------------+

The following returns an empty array, because the start_expression is greater than the end_expression, and the step_expression value is positive.

SELECT GENERATE_ARRAY(10, 0, 3) AS example_array;

+---------------+
| example_array |
+---------------+
| []            |
+---------------+

The following returns a NULL array, because one of its inputs is NULL.

SELECT GENERATE_ARRAY(5, NULL, 1) AS example_array;

+---------------+
| example_array |
+---------------+
| NULL          |
+---------------+

The following returns multiple arrays.

SELECT GENERATE_ARRAY(start, 5) AS example_array
FROM UNNEST([3, 4, 5]) AS start;

+---------------+
| example_array |
+---------------+
| [3, 4, 5]     |
| [4, 5]        |
| [5]           |
+---------------+

GENERATE_DATE_ARRAY

GENERATE_DATE_ARRAY(start_date, end_date[, INTERVAL INT64_expr date_part])

Description

Returns an array of dates. The start_date and end_date parameters determine the inclusive start and end of the array.

The GENERATE_DATE_ARRAY function accepts the following data types as inputs:

  • start_date must be a DATE
  • end_date must be a DATE
  • INT64_expr must be an INT64
  • date_part must be either DAY, WEEK, MONTH, QUARTER, or YEAR.

The INT64_expr parameter determines the increment used to generate dates. The default value for this parameter is 1 day.

This function returns an error if INT64_expr is set to 0.

Return Data Type

An ARRAY containing 0 or more DATE values.

Examples

The following returns an array of dates, with a default step of 1.

SELECT GENERATE_DATE_ARRAY('2016-10-05', '2016-10-08') AS example;

+--------------------------------------------------+
| example                                          |
+--------------------------------------------------+
| [2016-10-05, 2016-10-06, 2016-10-07, 2016-10-08] |
+--------------------------------------------------+

The following returns an array using a user-specified step size.

SELECT GENERATE_DATE_ARRAY(
 '2016-10-05', '2016-10-09', INTERVAL 2 DAY) AS example;

+--------------------------------------+
| example                              |
+--------------------------------------+
| [2016-10-05, 2016-10-07, 2016-10-09] |
+--------------------------------------+

The following returns an array using a negative value, -3 for its step size.

SELECT GENERATE_DATE_ARRAY('2016-10-05',
  '2016-10-01', INTERVAL -3 DAY) AS example;

+--------------------------+
| example                  |
+--------------------------+
| [2016-10-05, 2016-10-02] |
+--------------------------+

The following returns an array using the same value for the start_dateand end_date.

SELECT GENERATE_DATE_ARRAY('2016-10-05',
  '2016-10-05', INTERVAL 8 DAY) AS example;

+--------------+
| example      |
+--------------+
| [2016-10-05] |
+--------------+

The following returns an empty array, because the start_date is greater than the end_date, and the step value is positive.

SELECT GENERATE_DATE_ARRAY('2016-10-05',
  '2016-10-01', INTERVAL 1 DAY) AS example;

+---------+
| example |
+---------+
| []      |
+---------+

The following returns a NULL array, because one of its inputs is NULL.

SELECT GENERATE_DATE_ARRAY('2016-10-05', NULL) AS example;

+---------+
| example |
+---------+
| NULL    |
+---------+

The following returns an array of dates, using MONTH as the date_part interval:

SELECT GENERATE_DATE_ARRAY('2016-01-01',
  '2016-12-31', INTERVAL 2 MONTH) AS example;

+--------------------------------------------------------------------------+
| example                                                                  |
+--------------------------------------------------------------------------+
| [2016-01-01, 2016-03-01, 2016-05-01, 2016-07-01, 2016-09-01, 2016-11-01] |
+--------------------------------------------------------------------------+

The following uses non-constant dates to generate an array.

WITH StartsAndEnds AS (
  SELECT DATE '2016-01-01' AS date_start, DATE '2016-01-31' AS date_end
  UNION ALL SELECT DATE "2016-04-01", DATE "2016-04-30"
  UNION ALL SELECT DATE "2016-07-01", DATE "2016-07-31"
  UNION ALL SELECT DATE "2016-10-01", DATE "2016-10-31"
)
SELECT GENERATE_DATE_ARRAY(date_start, date_end, INTERVAL 1 WEEK) AS date_range
FROM StartsAndEnds;

+--------------------------------------------------------------+
| date_range                                                   |
+--------------------------------------------------------------+
| [2016-01-01, 2016-01-08, 2016-01-15, 2016-01-22, 2016-01-29] |
| [2016-04-01, 2016-04-08, 2016-04-15, 2016-04-22, 2016-04-29] |
| [2016-07-01, 2016-07-08, 2016-07-15, 2016-07-22, 2016-07-29] |
| [2016-10-01, 2016-10-08, 2016-10-15, 2016-10-22, 2016-10-29] |
+--------------------------------------------------------------+

OFFSET and ORDINAL

array_expression[OFFSET(zero_based_offset)]
array_expression[ORDINAL(one_based_offset)]

Description

Accesses an ARRAY element by position and returns the element. OFFSET means that the numbering starts at zero, ORDINAL means that the numbering starts at one.

A given array can be interpreted as either 0-based or 1-based. When accessing an array element, you must preface the array position with OFFSET or ORDINAL, respectively; there is no default behavior.

Both OFFSET and ORDINAL generate an error if the index is out of range.

Return type

Varies depending on the elements in the ARRAY.

Examples

WITH items AS
  (SELECT ["apples", "bananas", "pears", "grapes"] as list
  UNION ALL
  SELECT ["coffee", "tea", "milk" ] as list
  UNION ALL
  SELECT ["cake", "pie"] as list)

SELECT list, list[OFFSET(1)] as offset_1, list[ORDINAL(1)] as ordinal_1
FROM items;

+----------------------------------+-----------+-----------+
| list                             | offset_1  | ordinal_1 |
+----------------------------------+-----------+-----------+
| [apples, bananas, pears, grapes] | bananas   | apples    |
| [coffee, tea, milk]              | tea       | coffee    |
| [cake, pie]                      | pie       | cake      |
+----------------------------------+-----------+-----------+

ARRAY_REVERSE

ARRAY_REVERSE(value)

Description

Returns the input ARRAY with elements in reverse order.

Return type

ARRAY

Examples

WITH example AS (
  SELECT [1, 2, 3] AS arr UNION ALL
  SELECT [4, 5] AS arr UNION ALL
  SELECT [] AS arr
)
SELECT
  arr,
  ARRAY_REVERSE(arr) AS reverse_arr
FROM example;

+-----------+-------------+
| arr       | reverse_arr |
+-----------+-------------+
| [1, 2, 3] | [3, 2, 1]   |
| [4, 5]    | [5, 4]      |
| []        | []          |
+-----------+-------------+

SAFE_OFFSET and SAFE_ORDINAL

array_expression[SAFE_OFFSET(zero_based_offset)]
array_expression[SAFE_ORDINAL(one_based_offset)]

Description

Identical to OFFSET and ORDINAL, except returns NULL if the index is out of range.

Return type

Varies depending on the elements in the ARRAY.

Example

WITH items AS
  (SELECT ["apples", "bananas", "pears", "grapes"] as list
  UNION ALL
  SELECT ["coffee", "tea", "milk" ] as list
  UNION ALL
  SELECT ["cake", "pie"] as list)

SELECT list,
  list[SAFE_OFFSET(3)] as safe_offset_3,
  list[SAFE_ORDINAL(3)] as safe_ordinal_3
FROM items;

+----------------------------------+---------------+----------------+
| list                             | safe_offset_3 | safe_ordinal_3 |
+----------------------------------+---------------+----------------+
| [apples, bananas, pears, grapes] | grapes        | pears          |
| [coffee, tea, milk]              | NULL          | milk           |
| [cake, pie]                      | NULL          | NULL           |
+----------------------------------+---------------+----------------+

DATE functions

BigQuery supports the following DATE functions.

CURRENT_DATE

CURRENT_DATE([time_zone])

Description

Returns the current date as of the specified or default timezone.

This function supports an optional time_zone parameter. This parameter is a string representing the timezone to use. If no timezone is specified, the default timezone, UTC, is used. See Timezone definitions for information on how to specify a time zone.

If the time_zone parameter evaluates to NULL, this function returns NULL.

Return Data Type

DATE

Example

SELECT CURRENT_DATE() as the_date;

+--------------+
| the_date     |
+--------------+
| 2016-12-25   |
+--------------+

EXTRACT

EXTRACT(part FROM date_expression)

Description

Returns the value corresponding to the specified date part. The part must be one of:

  • DAYOFWEEK (Returns 1-7, where 1=Sunday ... 7=Saturday)
  • DAY
  • DAYOFYEAR
  • MONTH
  • QUARTER (Returns 1-4)
  • YEAR

Return Data Type

INT64

Example

SELECT EXTRACT(DAY FROM DATE '2013-12-25') as the_day;

+---------+
| the_day |
+---------+
| 25      |
+---------+

DATE

1. DATE(year, month, day)
2. DATE(timestamp_expression[, timezone])

Description

  1. Constructs a DATE from INT64 values representing the year, month, and day.
  2. Converts a timestamp_expression to a DATE data type. It supports an optional parameter to specify a timezone. If no timezone is specified, the default timezone, UTC, is used.

Return Data Type

DATE

Example

SELECT
  DATE(2016, 12, 25) as date_ymd,
  DATE(TIMESTAMP "2016-12-25 05:30:00+07", "America/Los_Angeles") as date_tstz;

+------------+------------+
| date_ymd   | date_tstz  |
+------------+------------+
| 2016-12-25 | 2016-12-24 |
+------------+------------+

DATE_ADD

DATE_ADD(date_expression, INTERVAL INT64_expr date_part)

Description

Adds a specified time interval to a DATE.

DATE_ADD supports the following date_part values:

  • DAY
  • WEEK. Equivalent to 7 DAYs.
  • MONTH
  • QUARTER
  • YEAR

Special handling is required for MONTH, QUARTER, and YEAR parts when the date is at (or near) the last day of the month. If the resulting month has fewer days than the original date's day, then the result day is the last day of the new month.

Return Data Type

DATE

Example

SELECT DATE_ADD(DATE "2008-12-25", INTERVAL 5 DAY) as five_days_later;

+--------------------+
| five_days_later    |
+--------------------+
| 2008-12-30         |
+--------------------+

DATE_SUB

DATE_SUB(date_expression, INTERVAL INT64_expr date_part)

Description

Subtracts a specified time interval from a DATE.

DATE_SUB supports the following date_part values:

  • DAY
  • WEEK. Equivalent to 7 DAYs.
  • MONTH
  • QUARTER
  • YEAR

Special handling is required for MONTH, QUARTER, and YEAR parts when the date is at (or near) the last day of the month. If the resulting month has fewer days than the original date's day, then the result day is the last day of the new month.

Return Data Type

DATE

Example

SELECT DATE_SUB(DATE "2008-12-25", INTERVAL 5 DAY) as five_days_ago;

+---------------+
| five_days_ago |
+---------------+
| 2008-12-20    |
+---------------+

DATE_DIFF

DATE_DIFF(date_expression, date_expression, date_part)

Description

Computes the number of specified date_part differences between two date expressions. This can be thought of as the number of date_part boundaries crossed between the two dates. If the first date occurs before the second date, then the result is negative.

DATE_DIFF supports the following date_part values:

  • DAY
  • MONTH
  • QUARTER
  • YEAR

Return Data Type

INT64

Example

SELECT DATE_DIFF(DATE "2010-07-07", DATE "2008-12-25", DAY) as days_diff;

+-----------+
| days_diff |
+-----------+
| 559       |
+-----------+

DATE_TRUNC

DATE_TRUNC(date_expression, date_part)

Description

Truncates the date to the specified granularity.

DATE_TRUNC supports the following values for date_part:

  • DAY
  • WEEK
  • MONTH
  • QUARTER
  • YEAR

Return Data Type

DATE

Example

SELECT DATE_TRUNC(DATE '2008-12-25', MONTH) as month;

+------------+
| month      |
+------------+
| 2008-12-01 |
+------------+

DATE_FROM_UNIX_DATE

DATE_FROM_UNIX_DATE(INT64_expression)

Description

Interprets INT64_expression as the number of days since 1970-01-01.

Return Data Type

DATE

Example

SELECT DATE_FROM_UNIX_DATE(14238) as date_from_epoch;

+-----------------+
| date_from_epoch |
+-----------------+
| 2008-12-25      |
+-----------------+

FORMAT_DATE

FORMAT_DATE(format_string, date_expr)

Description

Formats the date_expr according to the specified format_string.

See Supported Format Elements For DATE for a list of format elements that this function supports.

Return Data Type

STRING

Example

SELECT FORMAT_DATE("%x", DATE "2008-12-25") as US_format;

+------------+
| US_format  |
+------------+
| 12/25/08   |
+------------+

PARSE_DATE

PARSE_DATE(format_string, date_string)

Description

Uses a format_string and a string representation of a date to return a DATE object.

When using PARSE_DATE, keep the following in mind:

  • Unspecified fields. Any unspecified field is initialized from 1970-01-01.
  • Case insensitive names. Names, such as Monday, February, and so on, are case insensitive.
  • Whitespace. One or more consecutive white spaces in the format string matches zero or more consecutive white spaces in the date string. In addition, leading and trailing white spaces in the date string are always allowed -- even if they are not in the format string.
  • Format precedence. When two (or more) format elements have overlapping information (for example both %F and %Y affect the year), the last one generally overrides any earlier ones.

See Supported Format Elements For DATE for a list of format elements that this function supports.

Return Data Type

DATE

Example

SELECT PARSE_DATE("%x", "12/25/08") as parsed;

+------------+
| parsed     |
+------------+
| 2008-12-25 |
+------------+

UNIX_DATE

UNIX_DATE(date_expression)

Description

Returns the number of days since 1970-01-01.

Return Data Type

INT64

Example

SELECT UNIX_DATE(DATE "2008-12-25") as days_from_epoch;

+-----------------+
| days_from_epoch |
+-----------------+
| 14238           |
+-----------------+

Supported Format Elements for DATE

Unless otherwise noted, DATE functions that use format strings support the following elements:

Format element Description
%A The full weekday name.
%a The abbreviated weekday name.
%B The full month name.
%b or %h The abbreviated month name.
%C The century (a year divided by 100 and truncated to an integer) as a decimal number (00-99).
%D The date in the format %m/%d/%y.
%d The day of the month as a decimal number (01-31).
%e The day of month as a decimal number (1-31); single digits are preceded by a space.
%F The date in the format %Y-%m-%d.
%G The ISO 8601 year with century as a decimal number.
%g The ISO 8601 year without century as a decimal number (00-99).
%j The day of the year as a decimal number (001-366).
%m The month as a decimal number (01-12).
%n A newline character.
%t A tab character.
%U The week number of the year (Sunday as the first day of the week) as a decimal number (00-53).
%u The weekday (Monday as the first day of the week) as a decimal number (1-7).
%V The week number of the year (Monday as the first day of the week) as a decimal number (01-53). If the week containing January 1 has four or more days in the new year, then it is week 1; otherwise it is week 53 of the previous year, and the next week is week 1.
%W The week number of the year (Monday as the first day of the week) as a decimal number (00-53).
%w The weekday (Sunday as the first day of the week) as a decimal number (0-6).
%x The date representation in MM/DD/YY format.
%Y The year with century as a decimal number.
%y The year without century as a decimal number (00-99), with an optional leading zero. Can be mixed with %C. If %C is not specified, years 00-68 are 2000s, while years 69-99 are 1900s.
%E4Y Four-character years (0001 ... 9999). Note that %Y produces as many characters as it takes to fully render the year.

DATETIME functions

BigQuery supports the following DATETIME functions.

CURRENT_DATETIME

CURRENT_DATETIME([timezone])

Description

Returns the current time as a DATETIME object.

This function supports an optional timezone parameter. See Timezone definitions for information on how to specify a time zone.

Return Data Type

DATETIME

Example

SELECT CURRENT_DATETIME() as now;

+----------------------------+
| now                        |
+----------------------------+
| 2016-05-19 10:38:47.046465 |
+----------------------------+

DATETIME

1. DATETIME(year, month, day, hour, minute, second)
2. DATETIME(date_expression, time_expression)
3. DATETIME(timestamp_expression [, timezone])

Description

  1. Constructs a DATETIME object using INT64 values representing the year, month, day, hour, minute, and second.
  2. Constructs a DATETIME object using a DATE object and a TIME object.
  3. Constructs a DATETIME object using a TIMESTAMP object. It supports an optional parameter to specify a timezone. If no timezone is specified, the default timezone, UTC, is used.

Return Data Type

DATETIME

Example

SELECT
  DATETIME(2008, 12, 25, 05, 30, 00) as datetime_ymdhms,
  DATETIME(TIMESTAMP "2008-12-25 05:30:00+00", "America/Los_Angeles") as datetime_tstz;

+---------------------+---------------------+
| datetime_ymdhms     | datetime_tstz       |
+---------------------+---------------------+
| 2008-12-25 05:30:00 | 2008-12-24 21:30:00 |
+---------------------+---------------------+

DATETIME_ADD

DATETIME_ADD(datetime_expression, INTERVAL INT64_expr part)

Description

Adds INT64_expr units of part to the DATETIME object.

DATETIME_ADD supports the following values for part:

  • MICROSECOND
  • MILLISECOND
  • SECOND
  • MINUTE
  • HOUR
  • DAY
  • WEEK. Equivalent to 7 DAYs.
  • MONTH
  • QUARTER
  • YEAR

Special handling is required for MONTH, QUARTER, and YEAR parts when the date is at (or near) the last day of the month. If the resulting month has fewer days than the original DATETIME's day, then the result day is the last day of the new month.

Return Data Type

DATETIME

Example

SELECT
  DATETIME "2008-12-25 15:30:00" as original_date,
  DATETIME_ADD(DATETIME "2008-12-25 15:30:00", INTERVAL 10 MINUTE) as later;

+-----------------------------+------------------------+
| original_date               | later                  |
+-----------------------------+------------------------+
| 2008-12-25 15:30:00         | 2008-12-25 15:40:00    |
+-----------------------------+------------------------+

DATETIME_SUB

DATETIME_SUB(datetime_expression, INTERVAL INT64_expr part)

Description

Subtracts INT64_expr units of part from the DATETIME.

DATETIME_SUB supports the following values for part:

  • MICROSECOND
  • MILLISECOND
  • SECOND
  • MINUTE
  • HOUR
  • DAY
  • WEEK. Equivalent to 7 DAYs.
  • MONTH
  • QUARTER
  • YEAR

Special handling is required for MONTH, QUARTER, and YEAR parts when the date is at (or near) the last day of the month. If the resulting month has fewer days than the original DATETIME's day, then the result day is the last day of the new month.

Return Data Type

DATETIME

Example

SELECT
  DATETIME "2008-12-25 15:30:00" as original_date,
  DATETIME_SUB(DATETIME "2008-12-25 15:30:00", INTERVAL 10 MINUTE) as earlier;

+-----------------------------+------------------------+
| original_date               | earlier                |
+-----------------------------+------------------------+
| 2008-12-25 15:30:00         | 2008-12-25 15:20:00    |
+-----------------------------+------------------------+

DATETIME_DIFF

DATETIME_DIFF(datetime_expression, datetime_expression, part)

Description

Returns the number of whole specified part intervals between two DATETIME objects. Throws an error if the computation overflows the result type, such as if the difference in microseconds between the two DATETIME objects would overflow an INT64 value.

DATETIME_DIFF supports the following values for part:

  • MICROSECOND
  • MILLISECOND
  • SECOND
  • MINUTE
  • HOUR
  • DAY
  • MONTH
  • QUARTER
  • YEAR

Return Data Type

INT64

Example

SELECT
  DATETIME "2010-07-07 10:20:00" as first_datetime,
  DATETIME "2008-12-25 15:30:00" as second_datetime,
  DATETIME_DIFF(DATETIME "2010-07-07 10:20:00",
    DATETIME "2008-12-25 15:30:00", DAY) as difference;

+----------------------------+------------------------+------------------------+
| first_datetime             | second_datetime        | difference             |
+----------------------------+------------------------+------------------------+
| 2010-07-07 10:20:00        | 2008-12-25 15:30:00    | 559                    |
+----------------------------+------------------------+------------------------+

DATETIME_TRUNC

DATETIME_TRUNC(datetime_expression, part)

Description

Truncates a DATETIME object to the granularity of part.

DATETIME_TRUNC supports the following values for part:

  • MICROSECOND
  • MILLISECOND
  • SECOND
  • MINUTE
  • HOUR
  • DAY
  • WEEK
  • MONTH
  • QUARTER
  • YEAR

Return Data Type

DATETIME

Example

SELECT
  DATETIME "2008-12-25 15:30:00" as original,
  DATETIME_TRUNC(DATETIME "2008-12-25 15:30:00", DAY) as truncated;

+----------------------------+------------------------+
| original                   | truncated              |
+----------------------------+------------------------+
| 2008-12-25 15:30:00        | 2008-12-25 00:00:00    |
+----------------------------+------------------------+

FORMAT_DATETIME

FORMAT_DATETIME(format_string, datetime_expression)

Description

Formats a DATETIME object according to the specified format_string. See Supported Format Elements For DATETIME for a list of format elements that this function supports.

Return Data Type

STRING

Example

SELECT
  FORMAT_DATETIME("%c", DATETIME "2008-12-25 15:30:00")
  AS formatted;

PARSE_DATETIME

PARSE_DATETIME(format_string, string)

Description

Uses a format_string and a string representation of a timestamp to return a TIMESTAMP object. See Supported Format Elements For DATETIME for a list of format elements that this function supports.

When using PARSE_DATETIME, keep the following in mind:

  • Unspecified fields. Any unspecified field is initialized from 1970-01-01 00:00:00.0. For instance, if the year is unspecified then it defaults to 1970, and so on.
  • Case insensitive names. Names, such as Monday, February, and so on, are case insensitive.
  • Whitespace. One or more consecutive white spaces in the format string matches zero or more consecutive white spaces in the DATETIME string. In addition, leading and trailing white spaces in the DATETIME string are always allowed—even if they are not in the format string.
  • Format precedence. When two (or more) format elements have overlapping information (for example both %F and %Y affect the year), the last one generally overrides any earlier ones, with some exceptions (see the descriptions of %s, %C, and %y).

Return Data Type

DATETIME

Supported format elements for DATETIME

Unless otherwise noted, DATETIME functions that use format strings support the following elements:

Format element Description
%A The full weekday name.
%a The abbreviated weekday name.
%B The full month name.
%b or %h The abbreviated month name.
%C The century (a year divided by 100 and truncated to an integer) as a decimal number (00-99).
%c The date and time representation.
%D The date in the format %m/%d/%y.
%d The day of the month as a decimal number (01-31).
%e The day of month as a decimal number (1-31); single digits are preceded by a space.
%F The date in the format %Y-%m-%d.
%G The ISO 8601 year with century as a decimal number.
%g The ISO 8601 year without century as a decimal number (00-99).
%H The hour (24-hour clock) as a decimal number (00-23).
%I The hour (12-hour clock) as a decimal number (01-12).
%j The day of the year as a decimal number (001-366).
%k The hour (24-hour clock) as a decimal number (0-23); single digits are preceded by a space.
%l The hour (12-hour clock) as a decimal number (1-12); single digits are preceded by a space.
%M The minute as a decimal number (00-59).
%m The month as a decimal number (01-12).
%n A newline character.
%P Either am or pm.
%p Either AM or PM.
%R The time in the format %H:%M.
%r The 12-hour clock time using AM/PM notation.
%S The second as a decimal number (00-60).
%s The number of seconds since 1970-01-01 00:00:00. Always overrides all other format elements, independent of where %s appears in the string. If multiple %s elements appear, then the last one takes precedence.
%T The time in the format %H:%M:%S.
%t A tab character.
%U The week number of the year (Sunday as the first day of the week) as a decimal number (00-53).
%u The weekday (Monday as the first day of the week) as a decimal number (1-7).
%V The week number of the year (Monday as the first day of the week) as a decimal number (01-53). If the week containing January 1 has four or more days in the new year, then it is week 1; otherwise it is week 53 of the previous year, and the next week is week 1.
%W The week number of the year (Monday as the first day of the week) as a decimal number (00-53).
%w The weekday (Sunday as the first day of the week) as a decimal number (0-6).
%X The time representation in HH:MM:SS format.
%x The date representation in MM/DD/YY format.
%Y The year with century as a decimal number.
%y The year without century as a decimal number (00-99), with an optional leading zero. Can be mixed with %C. If %C is not specified, years 00-68 are 2000s, while years 69-99 are 1900s.
%% A single % character.
%E#S Seconds with # digits of fractional precision.
%E*S Seconds with full fractional precision (a literal '*').
%E4Y Four-character years (0001 ... 9999). Note that %Y produces as many characters as it takes to fully render the year.

TIME functions

BigQuery supports the following TIME functions.

CURRENT_TIME

CURRENT_TIME()

Description

Returns the current time as a TIME object.

Return Data Type

TIME

Example

SELECT CURRENT_TIME() as now;

+----------------------------+
| now                        |
+----------------------------+
| 15:31:38.776361            |
+----------------------------+

TIME

1. TIME(hour, minute, second)
2. TIME(timestamp, [timezone])

Description

  1. Constructs a TIME object using INT64 values representing the hour, minute, and second.
  2. Constructs a TIME object using a TIMESTAMP object. It supports an optional parameter to specify a timezone. If no timezone is specified, the default timezone, UTC, is used.

Return Data Type

TIME

Example

SELECT
  TIME(15, 30, 00) as time_hms,
  TIME(TIMESTAMP "2008-12-25 15:30:00+08", "America/Los_Angeles") as time_tstz;

+----------+-----------+
| time_hms | time_tstz |
+----------+-----------+
| 15:30:00 | 23:30:00  |
+----------+-----------+

TIME_ADD

TIME_ADD(time_expression, INTERVAL INT64_expr part)

Description

Adds INT64_expr units of part to the TIME object.

TIME_ADD supports the following values for part:

  • MICROSECOND
  • MILLISECOND
  • SECOND
  • MINUTE
  • HOUR

This function automatically adjusts when values fall outside of the 00:00:00 to 24:00:00 boundary. For example, if you add an hour to 23:30:00, the returned value is 00:30:00.

Return Data Types

TIME

Example

SELECT
  TIME "15:30:00" as original_time,
  TIME_ADD(TIME "15:30:00", INTERVAL 10 MINUTE) as later;

+-----------------------------+------------------------+
| original_time               | later                  |
+-----------------------------+------------------------+
| 15:30:00                    | 15:40:00               |
+-----------------------------+------------------------+

TIME_SUB

TIME_SUB(time_expression, INTERVAL INT_expr part)

Description

Subtracts INT64_expr units of part from the TIME object.

TIME_SUB supports the following values for part:

  • MICROSECOND
  • MILLISECOND
  • SECOND
  • MINUTE
  • HOUR

This function automatically adjusts when values fall outside of the 00:00:00 to 24:00:00 boundary. For example, if you subtract an hour from 00:30:00, the returned value is 23:30:00.

Return Data Type

TIME

Example

SELECT
  TIME "15:30:00" as original_date,
  TIME_SUB(TIME "15:30:00", INTERVAL 10 MINUTE) as earlier;

+-----------------------------+------------------------+
| original_date                | earlier                |
+-----------------------------+------------------------+
| 15:30:00                    | 15:20:00               |
+-----------------------------+------------------------+

TIME_DIFF

TIME_DIFF(time_expression, time_expression, part)

Description

Returns the number of whole specified part intervals between two TIME objects. Throws an error if the computation overflows the result type, such as if the difference in microseconds between the two time objects would overflow an INT64 value.

TIME_DIFF supports the following values for part:

  • MICROSECOND
  • MILLISECOND
  • SECOND
  • MINUTE
  • HOUR

Return Data Type

INT64

Example

SELECT
  TIME "15:30:00" as first_time,
  TIME "14:35:00" as second_time,
  TIME_DIFF(TIME "15:30:00", TIME "14:35:00", MINUTE) as difference;

+----------------------------+------------------------+------------------------+
| first_time                 | second_time            | difference             |
+----------------------------+------------------------+------------------------+
| 15:30:00                   | 14:35:00               | 55                     |
+----------------------------+------------------------+------------------------+

TIME_TRUNC

TIME_TRUNC(time_expression, part)

Description

Truncates a TIME object to the granularity of part.

TIME_TRUNC supports the following values for part:

  • MICROSECOND
  • MILLISECOND
  • SECOND
  • MINUTE
  • HOUR

Return Data Type

TIME

Example

SELECT
  TIME "15:30:00" as original,
  TIME_TRUNC(TIME "15:30:00", HOUR) as truncated;

+----------------------------+------------------------+
| original                   | truncated              |
+----------------------------+------------------------+
| 15:30:00                   | 15:00:00               |
+----------------------------+------------------------+

FORMAT_TIME

FORMAT_TIME(format_string, time_object)

Description Formats a TIME object according to the specified format_string. See Supported Format Elements For TIME for a list of format elements that this function supports.

Return Data Type

STRING

Example

SELECT FORMAT_TIME("%R", TIME "15:30:00") as formatted_time;

+----------------+
| formatted_time |
+----------------+
| 15:30          |
+----------------+

PARSE_TIME

PARSE_TIME(format_string, string)

Description

Uses a format_string and a string to return a TIME object. See Supported Format Elements For TIME for a list of format elements that this function supports.

When using PARSE_TIME, keep the following in mind:

  • Unspecified fields. Any unspecified field is initialized from 00:00:00.0. For instance, if seconds is unspecified then it defaults to 00, and so on.
  • Whitespace. One or more consecutive white spaces in the format string matches zero or more consecutive white spaces in the TIME string. In addition, leading and trailing white spaces in the TIME string are always allowed—even if they are not in the format string.
  • Format precedence. When two (or more) format elements have overlapping information, the last one generally overrides any earlier ones.

Return Data Type

TIME

Example

SELECT PARSE_TIME("%H", "15") as parsed_time;

+-------------+
| parsed_time |
+-------------+
| 15:00:00    |
+-------------+

Supported format elements for TIME

Unless otherwise noted, TIME functions that use format strings support the following elements:

Format element Description
%H The hour (24-hour clock) as a decimal number (00-23).
%I The hour (12-hour clock) as a decimal number (01-12).
%j The day of the year as a decimal number (001-366).
%k The hour (24-hour clock) as a decimal number (0-23); single digits are preceded by a space.
%l The hour (12-hour clock) as a decimal number (1-12); single digits are preceded by a space.
%M The minute as a decimal number (00-59).
%n A newline character.
%P Either am or pm.
%p Either AM or PM.
%R The time in the format %H:%M.
%r The 12-hour clock time using AM/PM notation.
%S The second as a decimal number (00-60).
%T The time in the format %H:%M:%S.
%t A tab character.
%X The time representation in HH:MM:SS format.
%% A single % character.
%E#S Seconds with # digits of fractional precision.
%E*S Seconds with full fractional precision (a literal '*').

TIMESTAMP functions

BigQuery supports the following TIMESTAMP functions.

NOTE: These functions return a runtime error if overflow occurs; result values are bounded by the defined date and timestamp min/max values.

CURRENT_TIMESTAMP

CURRENT_TIMESTAMP()

Description

Parentheses are optional. This function handles leap seconds by smearing them across a window of 20 hours around the inserted leap second. CURRENT_TIMESTAMP() produces a TIMESTAMP value that is continuous, non-ambiguous, has exactly 60 seconds per minute and does not repeat values over the leap second.

Supported Input Types

Not applicable

Result Data Type

TIMESTAMP

Example

SELECT CURRENT_TIMESTAMP() as now;

+-------------------------------+
| now                           |
+-------------------------------+
| 2016-05-16 18:12:47.145482+00 |
+-------------------------------+

EXTRACT

EXTRACT(part FROM timestamp_expression [AT TIME ZONE tz_spec])

Description

Returns an INT64 value that corresponds to the specified part from a supplied timestamp_expression.

Allowed part values are:

  • MICROSECOND
  • MILLISECOND
  • SECOND
  • MINUTE
  • HOUR
  • DAYOFWEEK
  • DAY
  • DAYOFYEAR
  • MONTH
  • QUARTER
  • YEAR
  • DATE
  • DATETIME
  • TIME

Returned values truncate lower order time periods—for example, when extracting seconds, the millisecond and microsecond values are truncated.

See Timezone definitions for information on how to specify a time zone.

Return Data Type

Generally INT64 . Returns DATE if part is DATE.

Example

SELECT EXTRACT(DAY
  FROM TIMESTAMP "2008-12-25 15:30:00" AT TIME ZONE "America/Los_Angeles")
  AS the_day;

+------------+
| the_day    |
+------------+
| 25         |
+------------+

STRING

STRING(timestamp_expression[, timezone])

Description

Converts a timestamp_expression to a STRING data type. Supports an optional parameter to specify a timezone. See Timezone definitions for information on how to specify a time zone.

Return Data Type

STRING

Example

SELECT STRING(TIMESTAMP "2008-12-25 15:30:00", "America/Los_Angeles") as string;

+-------------------------------+
| string                        |
+-------------------------------+
| 2008-12-25 15:30:00-08        |
+-------------------------------+

TIMESTAMP

1. TIMESTAMP(string_expression[, timezone])
2. TIMESTAMP(date_expression[, timezone])
3. TIMESTAMP(datetime_expression[, timezone])

Description

  1. Converts a STRING expression to a TIMESTAMP data type.

  2. Converts a DATE object to a TIMESTAMP data type.

  3. Converts a DATETIME object to a TIMESTAMP data type.

This function supports an optional parameter to specify a timezone. If no timezone is specified, the default timezone, UTC, is used.

Return Data Type

TIMESTAMP

Example

SELECT
  CAST(TIMESTAMP("2008-12-25 15:30:00", "America/Los_Angeles") AS STRING) AS timestamp_str,
  CAST(TIMESTAMP(DATE "2008-12-25", "America/Los_Angeles") AS STRING) AS timestamp_date,
  CAST(TIMESTAMP(DATETIME "2008-12-25 15:30:00", "America/Los_Angeles") AS STRING) AS timestamp_datetime;

+------------------------+------------------------+------------------------+
| timestamp_str          | timestamp_date         | timestamp_datetime     |
+------------------------+------------------------+------------------------+
| 2008-12-25 23:30:00+00 | 2008-12-25 08:00:00+00 | 2008-12-25 23:30:00+00 |
+------------------------+------------------------+------------------------+

TIMESTAMP_ADD

TIMESTAMP_ADD(timestamp_expression, INTERVAL INT64_expr date_part)

Description

Adds INT64_expr units of date_part to the timestamp, independent of any time zone.

TIMESTAMP_ADD supports the following values for date_part:

  • MICROSECOND
  • MILLISECOND
  • SECOND
  • MINUTE
  • HOUR. Equivalent to 60 MINUTEs.

Return Data Types

TIMESTAMP

Example

SELECT
  TIMESTAMP "2008-12-25 15:30:00 UTC" as original,
  TIMESTAMP_ADD(TIMESTAMP "2008-12-25 15:30:00 UTC", INTERVAL 10 MINUTE) AS later;

+------------------------+------------------------+
| original               | later                  |
+------------------------+------------------------+
| 2008-12-25 15:30:00+00 | 2008-12-25 15:40:00+00 |
+------------------------+------------------------+

TIMESTAMP_SUB

TIMESTAMP_SUB(timestamp_expression, INTERVAL INT_expr date_part)

Description

Subtracts INT64_expr units of date_part from the timestamp, independent of any time zone.

TIMESTAMP_SUB supports the following values for date_part:

  • MICROSECOND
  • MILLISECOND
  • SECOND
  • MINUTE
  • HOUR. Equivalent to 60 MINUTEs.

Return Data Type

TIMESTAMP

Example

SELECT
  TIMESTAMP "2008-12-25 15:30:00 UTC" as original,
  TIMESTAMP_SUB(TIMESTAMP "2008-12-25 15:30:00 UTC", INTERVAL 10 MINUTE) AS earlier;

+------------------------+------------------------+
| original               | earlier                |
+------------------------+------------------------+
| 2008-12-25 15:30:00+00 | 2008-12-25 15:20:00+00 |
+------------------------+------------------------+

TIMESTAMP_DIFF

TIMESTAMP_DIFF(timestamp_expression, timestamp_expression, date_part)

Description

Returns the number of whole specified date_part intervals between two timestamps. Throws an error if the computation overflows the result type, such as if the difference in microseconds between the two timestamps would overflow an INT64 value.

TIMESTAMP_DIFF supports the following values for date_part:

  • MICROSECOND
  • MILLISECOND
  • SECOND
  • MINUTE
  • HOUR. Equivalent to 60 MINUTEs.

Return Data Type

INT64

Example

SELECT
  TIMESTAMP "2010-07-07 10:20:00 UTC" as first_timestamp,
  TIMESTAMP "2008-12-25 15:30:00 UTC" as second_timestamp,
  TIMESTAMP_DIFF(TIMESTAMP "2010-07-07 10:20:00 UTC",
    TIMESTAMP "2008-12-25 15:30:00 UTC", HOUR) AS hours;

+------------------------+------------------------+-------+
| first_timestamp        | second_timestamp       | hours |
+------------------------+------------------------+-------+
| 2010-07-07 10:20:00+00 | 2008-12-25 15:30:00+00 | 13410 |
+------------------------+------------------------+-------+

TIMESTAMP_TRUNC

TIMESTAMP_TRUNC(timestamp_expression, date_part, [, time_zone])

Description

Truncates a timestamp to the granularity of date_part.

TIMESTAMP_TRUNC supports the following values for date_part:

  • MICROSECOND
  • MILLISECOND
  • SECOND
  • MINUTE
  • HOUR
  • DAY
  • WEEK
  • MONTH
  • QUARTER
  • YEAR

In addition, TIMESTAMP_TRUNC function supports an optional time_zone parameter. This parameter applies to the following date_parts:

  • MINUTE
  • HOUR
  • DAY
  • WEEK
  • MONTH
  • QUARTER
  • YEAR

Use this parameter if you want to use a time zone other than the default timezone, UTC, as part of the truncate operation.

Return Data Type

TIMESTAMP

Example

SELECT
  TIMESTAMP_TRUNC(TIMESTAMP '2008-12-25 15:30:00', DAY, 'UTC') as utc,
  TIMESTAMP_TRUNC(TIMESTAMP '2008-12-25 15:30:00', DAY, 'America/Los_Angeles') as la;

+------------------------+------------------------+
| utc                    | la                     |
+------------------------+------------------------+
| 2008-12-25 00:00:00+00 | 2008-12-25 08:00:00+00 |
+------------------------+------------------------+

FORMAT_TIMESTAMP

FORMAT_TIMESTAMP(format_string, timestamp[, time_zone])

Description

Formats a timestamp according to the specified format_string.

See Supported Format Elements For TIMESTAMP for a list of format elements that this function supports.

Return Data Type

STRING

Example

SELECT FORMAT_TIMESTAMP("%c", TIMESTAMP "2008-12-25 15:30:00", "America/Los_Angeles")
  AS formatted;

+--------------------------+
| formatted                |
+--------------------------+
| Thu Dec 25 07:30:00 2008 |
+--------------------------+

PARSE_TIMESTAMP

PARSE_TIMESTAMP(format_string, string[, time_zone])

Description

Uses a format_string and a string representation of a timestamp to return a TIMESTAMP object.

When using PARSE_TIMESTAMP, keep the following in mind:

  • Unspecified fields. Any unspecified field is initialized from 1970-01-01 00:00:00.0. This initialization value uses the time zone specified by the function's time zone argument, if present. If not, the initialization value uses the default time zone, UTC. For instance, if the year is unspecified then it defaults to 1970, and so on.
  • Case insensitive names. Names, such as Monday, February, and so on, are case insensitive.
  • Whitespace. One or more consecutive white spaces in the format string matches zero or more consecutive white spaces in the timestamp string. In addition, leading and trailing white spaces in the timestamp string are always allowed -- even if they are not in the format string.
  • Format precedence. When two (or more) format elements have overlapping information (for example both %F and %Y affect the year), the last one generally overrides any earlier ones, with some exceptions (see the descriptions of %s, %C, and %y).

See Supported Format Elements For TIMESTAMP for a list of format elements that this function supports.

Return Data Type

TIMESTAMP

Example

SELECT PARSE_TIMESTAMP("%c", "Thu Dec 25 07:30:00 2008", "America/Los_Angeles") as parsed;

+-------------------------+
| parsed                  |
+-------------------------+
| 2008-12-25 15:30:00 UTC |
+-------------------------+

TIMESTAMP_SECONDS

TIMESTAMP_SECONDS(INT64_expression)

Description

Interprets INT64_expression as the number of seconds since 1970-01-01 00:00:00 UTC.

Return Data Type

TIMESTAMP

Example

SELECT TIMESTAMP_SECONDS(1230219000) as timestamp;

+-------------------------+
| timestamp               |
+-------------------------+
| 2008-12-25 15:30:00 UTC |
+-------------------------+

TIMESTAMP_MILLIS

TIMESTAMP_MILLIS(INT64_expression)

Description

Interprets INT64_expression as the number of milliseconds since 1970-01-01 00:00:00 UTC.

Return Data Type

TIMESTAMP

Example

SELECT TIMESTAMP_MILLIS(1230219000000) as timestamp;

+-------------------------+
| timestamp               |
+-------------------------+
| 2008-12-25 15:30:00 UTC |
+-------------------------+

TIMESTAMP_MICROS

TIMESTAMP_MICROS(INT64_expression)

Description

Interprets INT64_expression as the number of microseconds since 1970-01-01 00:00:00 UTC.

Return Data Type

TIMESTAMP

Example

SELECT TIMESTAMP_MICROS(1230219000000000) as timestamp;

+-------------------------+
| timestamp               |
+-------------------------+
| 2008-12-25 15:30:00 UTC |
+-------------------------+

UNIX_SECONDS

UNIX_SECONDS(timestamp_expression)

Description

Returns the number of seconds since 1970-01-01 00:00:00 UTC. Truncates higher levels of precision.

Return Data Type

INT64

Example

SELECT UNIX_SECONDS(TIMESTAMP "2008-12-25 15:30:00") as seconds;

+------------+
| seconds    |
+------------+
| 1230219000 |
+------------+

UNIX_MILLIS

UNIX_MILLIS(timestamp_expression)

Description

Returns the number of milliseconds since 1970-01-01 00:00:00 UTC. Truncates higher levels of precision.

Return Data Type

INT64

Example

SELECT UNIX_MILLIS(TIMESTAMP "2008-12-25 15:30:00 UTC") as millis;

+---------------+
| millis        |
+---------------+
| 1230219000000 |
+---------------+

UNIX_MICROS

UNIX_MICROS(timestamp_expression)

Description

Returns the number of microseconds since 1970-01-01 00:00:00 UTC. Truncates higher levels of precision.

Return Data Type

INT64

Example

SELECT UNIX_MICROS(TIMESTAMP "2008-12-25 15:30:00") as micros;

+------------------+
| micros           |
+------------------+
| 1230219000000000 |
+------------------+

Supported format elements for TIMESTAMP

Unless otherwise noted, TIMESTAMP functions that use format strings support the following elements:

Format element Description
%A The full weekday name.
%a The abbreviated weekday name.
%B The full month name.
%b or %h The abbreviated month name.
%C The century (a year divided by 100 and truncated to an integer) as a decimal number (00-99).
%c The date and time representation.
%D The date in the format %m/%d/%y.
%d The day of the month as a decimal number (01-31).
%e The day of month as a decimal number (1-31); single digits are preceded by a space.
%F The date in the format %Y-%m-%d.
%G The ISO 8601 year with century as a decimal number.
%g The ISO 8601 year without century as a decimal number (00-99).
%H The hour (24-hour clock) as a decimal number (00-23).
%I The hour (12-hour clock) as a decimal number (01-12).
%j The day of the year as a decimal number (001-366).
%k The hour (24-hour clock) as a decimal number (0-23); single digits are preceded by a space.
%l The hour (12-hour clock) as a decimal number (1-12); single digits are preceded by a space.
%M The minute as a decimal number (00-59).
%m The month as a decimal number (01-12).
%n A newline character.
%P Either am or pm.
%p Either AM or PM.
%R The time in the format %H:%M.
%r The 12-hour clock time using AM/PM notation.
%S The second as a decimal number (00-60).
%s The number of seconds since 1970-01-01 00:00:00 UTC. Always overrides all other format elements, independent of where %s appears in the string. If multiple %s elements appear, then the last one takes precedence.
%T The time in the format %H:%M:%S.
%t A tab character.
%U The week number of the year (Sunday as the first day of the week) as a decimal number (00-53).
%u The weekday (Monday as the first day of the week) as a decimal number (1-7).
%V The week number of the year (Monday as the first day of the week) as a decimal number (01-53). If the week containing January 1 has four or more days in the new year, then it is week 1; otherwise it is week 53 of the previous year, and the next week is week 1.
%W The week number of the year (Monday as the first day of the week) as a decimal number (00-53).
%w The weekday (Sunday as the first day of the week) as a decimal number (0-6).
%X The time representation in HH:MM:SS format.
%x The date representation in MM/DD/YY format.
%Y The year with century as a decimal number.
%y The year without century as a decimal number (00-99), with an optional leading zero. Can be mixed with %C. If %C is not specified, years 00-68 are 2000s, while years 69-99 are 1900s.
%Z The time zone name.
%z The offset from the Prime Meridian in the format +HHMM or -HHMM as appropriate, with positive values representing locations east of Greenwich.
%% A single % character.
%Ez RFC3339-compatible numeric time zone (+HH:MM or -HH:MM).
%E#S Seconds with # digits of fractional precision.
%E*S Seconds with full fractional precision (a literal '*').
%E4Y Four-character years (0001 ... 9999). Note that %Y produces as many characters as it takes to fully render the year.

Timezone definitions

Certain date and timestamp functions allow you to override the default time zone and specify a different one. You can specify a timezone by supplying its UTC offset using the following format:

(+|-)H[H][:M[M]]

For example:

-08:00

Security functions

BigQuery supports the following security functions.

SESSION_USER

SESSION_USER()

Description

Returns the email address of whoever is executing the query.

Return Data Type

STRING

Example

SELECT SESSION_USER() as user;

+----------------------+
| user                 |
+----------------------+
| jdoe@example.com     |
+----------------------+

NET functions

NET.IP_FROM_STRING

NET.IP_FROM_STRING(addr_str)

Description

Converts an IPv4 or IPv6 address from text (STRING) format to binary (BYTES) format in network byte order.

This function supports the following formats for addr_str:

  • IPv4: Dotted-quad format. For example, 10.1.2.3.
  • IPv6: Colon-separated format. For example, 1234:5678:90ab:cdef:1234:5678:90ab:cdef. For more examples, see the [IP Version 6 Addressing Architecture][net-link-to-ipv6-architecture].

This function does not support CIDR notation, such as 10.1.2.3/32.

If this function receives a NULL input, it returns NULL. If the input is considered invalid, an OUT_OF_RANGE error occurs.

Return Data Type

BYTES

Example

SELECT
  addr_str, FORMAT("%T", NET.IP_FROM_STRING(addr_str)) AS ip_from_string
FROM UNNEST([
  '48.49.50.51',
  '::1',
  '3031:3233:3435:3637:3839:4041:4243:4445',
  '::ffff:192.0.2.128'
]) AS addr_str;
addr_str ip_from_string
48.49.50.51 b"0123"
::1 b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01"
3031:3233:3435:3637:3839:4041:4243:4445 b"0123456789@ABCDE"
::ffff:192.0.2.128 b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xc0\x00\x02\x80"

NET.SAFE_IP_FROM_STRING

NET.SAFE_IP_FROM_STRING(addr_str)

Description

Similar to [NET.IP_FROM_STRING][net-link-to-net-ip-from-string], but returns NULL instead of throwing an error if the input is invalid.

Return Data Type

BYTES

Example

SELECT
  addr_str,
  FORMAT("%T", NET.SAFE_IP_FROM_STRING(addr_str)) AS safe_ip_from_string
FROM UNNEST([
  '48.49.50.51',
  '::1',
  '3031:3233:3435:3637:3839:4041:4243:4445',
  '::ffff:192.0.2.128',
  '48.49.50.51/32',
  '48.49.50',
  '::wxyz'
]) AS addr_str;
addr_str safe_ip_from_string
48.49.50.51 b"0123"
::1 b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01"
3031:3233:3435:3637:3839:4041:4243:4445 b"0123456789@ABCDE"
::ffff:192.0.2.128 b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xc0\x00\x02\x80"
48.49.50.51/32 NULL
48.49.50 NULL
::wxyz NULL

NET.IP_TO_STRING

NET.IP_TO_STRING(addr_bin)

Description Converts an IPv4 or IPv6 address from binary (BYTES) format in network byte order to text (STRING) format.

If the input is 4 bytes, this function returns an IPv4 address as a STRING. If the input is 16 bytes, it returns an IPv6 address as a STRING.

If this function receives a NULL input, it returns NULL. If the input has a length different from 4 or 16, an OUT_OF_RANGE error occurs.

Return Data Type

STRING

Example

SELECT FORMAT("%T", x) AS addr_bin, NET.IP_TO_STRING(x) AS ip_to_string
FROM UNNEST([
  b"0123",
  b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01",
  b"0123456789@ABCDE",
  b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xc0\x00\x02\x80"
]) AS x;
addr_bin ip_to_string
b"0123" 48.49.50.51
b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01" ::1
b"0123456789@ABCDE" 3031:3233:3435:3637:3839:4041:4243:4445
b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xc0\x00\x02\x80" ::ffff:192.0.2.128

NET.IP_NET_MASK

NET.IP_NET_MASK(num_output_bytes, prefix_length)

Description

Returns a network mask: a byte sequence with length equal to num_output_bytes, where the first prefix_length bits are set to 1 and the other bits are set to 0. num_output_bytes and prefix_length are INT64. This function throws an error if num_output_bytes is not 4 (for IPv4) or 16 (for IPv6). It also throws an error if prefix_length is negative or greater than 8 * num_output_bytes.

Return Data Type

BYTES

Example

SELECT x, y, FORMAT("%T", NET.IP_NET_MASK(x, y)) AS ip_net_mask
FROM UNNEST([
  STRUCT(4 as x, 0 as y),
  (4, 20),
  (4, 32),
  (16, 0),
  (16, 1),
  (16, 128)
]);
x y ip_net_mask
4 0 b"\x00\x00\x00\x00"
4 20 b"\xff\xff\xf0\x00"
4 32 b"\xff\xff\xff\xff"
16 0 b"\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
16 1 b"\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
16 128 b"\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"

NET.IP_TRUNC

NET.IP_TRUNC(addr_bin, prefix_length)

Description Takes addr_bin, an IPv4 or IPv6 address in binary (BYTES) format in network byte order, and returns a subnet address in the same format. The result has the same length as addr_bin, where the first prefix_length bits are equal to those in addr_bin and the remaining bits are 0.

This function throws an error if LENGTH(addr_bin) is not 4 or 16, or if prefix_len is negative or greater than LENGTH(addr_bin) * 8.

Return Data Type

BYTES

Example

SELECT
  FORMAT("%T", x) as addr_bin, prefix_length,
  FORMAT("%T", NET.IP_TRUNC(x, prefix_length)) AS ip_trunc
FROM UNNEST([
  STRUCT(b"\xAA\xBB\xCC\xDD" as x, 0 as prefix_length),
  (b"\xAA\xBB\xCC\xDD", 11), (b"\xAA\xBB\xCC\xDD", 12),
  (b"\xAA\xBB\xCC\xDD", 24), (b"\xAA\xBB\xCC\xDD", 32),
  (b'0123456789@ABCDE', 80)
]);
addr_bin prefix_length ip_trunc
b"\xaa\xbb\xcc\xdd" 0 b"\x00\x00\x00\x00"
b"\xaa\xbb\xcc\xdd" 11 b"\xaa\xa0\x00\x00"
b"\xaa\xbb\xcc\xdd" 12 b"\xaa\xb0\x00\x00"
b"\xaa\xbb\xcc\xdd" 24 b"\xaa\xbb\xcc\x00"
b"\xaa\xbb\xcc\xdd" 32 b"\xaa\xbb\xcc\xdd"
b"0123456789@ABCDE" 80 b"0123456789\x00\x00\x00\x00\x00\x00"

NET.IPV4_FROM_INT64

NET.IPV4_FROM_INT64(integer_value)

Description

Converts an IPv4 address from integer format to binary (BYTES) format in network byte order. In the integer input, the least significant bit of the IP address is stored in the least significant bit of the integer, regardless of host or client architecture. For example, 1 means 0.0.0.1, and 0x1FF means 0.0.1.255.

This function checks that either all the most significant 32 bits are 0, or all the most significant 33 bits are 1 (sign-extended from a 32 bit integer). In other words, the input should be in the range [-0x80000000, 0xFFFFFFFF]; otherwise, this function throws an error.

This function does not support IPv6.

Return Data Type

BYTES

Example

SELECT x, x_hex, FORMAT("%T", NET.IPV4_FROM_INT64(x)) AS ipv4_from_int64
FROM (
  SELECT CAST(x_hex AS INT64) x, x_hex
  FROM UNNEST(["0x0", "0xABCDEF", "0xFFFFFFFF", "-0x1", "-0x2"]) AS x_hex
);
x x_hex ipv4_from_int64
0 0x0 b"\x00\x00\x00\x00"
11259375 0xABCDEF b"\x00\xab\xcd\xef"
4294967295 0xFFFFFFFF b"\xff\xff\xff\xff"
-1 -0x1 b"\xff\xff\xff\xff"
-2 -0x2 b"\xff\xff\xff\xfe"

NET.IPV4_TO_INT64

NET.IPV4_TO_INT64(addr_bin)

Description

Converts an IPv4 address from binary (BYTES) format in network byte order to integer format. In the integer output, the least significant bit of the IP address is stored in the least significant bit of the integer, regardless of host or client architecture. For example, 1 means 0.0.0.1, and 0x1FF means 0.0.1.255. The output is in the range [0, 0xFFFFFFFF].

If the input length is not 4, this function throws an error.

This function does not support IPv6.

Return Data Type

INT64

Example

SELECT
  FORMAT("%T", x) AS addr_bin,
  FORMAT("0x%X", NET.IPV4_TO_INT64(x)) AS ipv4_to_int64
FROM
UNNEST([b"\x00\x00\x00\x00", b"\x00\xab\xcd\xef", b"\xff\xff\xff\xff"]) AS x;
addr_bin ipv4_to_int64
b"\x00\x00\x00\x00" 0x0
b"\x00\xab\xcd\xef" 0xABCDEF
b"\xff\xff\xff\xff" 0xFFFFFFFF

NET.HOST

NET.HOST(url)

Description

Takes a URL as a STRING and returns the host as a STRING. For best results, URL values should comply with the format as defined by RFC 3986. If the URL value does not comply with RFC 3986 formatting, this function makes a best effort to parse the input and return a relevant result. If the function cannot parse the input, it returns NULL.

Note: The function does not perform any normalization.

Return Data Type

STRING

Example

SELECT
  FORMAT("%T", input) AS input,
  description,
  FORMAT("%T", NET.HOST(input)) AS host,
  FORMAT("%T", NET.PUBLIC_SUFFIX(input)) AS suffix,
  FORMAT("%T", NET.REG_DOMAIN(input)) AS domain
FROM (
  SELECT "" AS input, "invalid input" AS description
  UNION ALL SELECT "http://abc.xyz", "standard URL"
  UNION ALL SELECT "//user:password@a.b:80/path?query",
                   "standard URL with relative scheme, port, path and query, but no public suffix"
  UNION ALL SELECT "https://[::1]:80", "standard URL with IPv6 host"
  UNION ALL SELECT "http://例子.卷筒纸.中国", "standard URL with internationalized domain name"
  UNION ALL SELECT "    www.Google.Co.UK    ",
                   "non-standard URL with spaces, upper case letters, and without scheme"
  UNION ALL SELECT "mailto:?to=&subject=&body=", "URI rather than URL--unsupported"
);
input description host suffix domain
"" invalid input NULL NULL NULL
"http://abc.xyz" standard URL "abc.xyz" "xyz" "abc.xyz"
"//user:password@a.b:80/path?query" standard URL with relative scheme, port, path and query, but no public suffix "a.b" NULL NULL
"https://[::1]:80" standard URL with IPv6 host "[::1]" NULL NULL
"http://例子.卷筒纸.中国" standard URL with internationalized domain name "例子.卷筒纸.中国" "中国" "卷筒纸.中国"
"    www.Google.Co.UK    " non-standard URL with spaces, upper case letters, and without scheme "www.Google.Co.UK" "Co.UK" "Google.Co.UK"
"mailto:?to=&subject=&body=" URI rather than URL--unsupported "mailto" NULL NULL

NET.PUBLIC_SUFFIX

NET.PUBLIC_SUFFIX(url)

Description

Takes a URL as a STRING and returns the public suffix (such as com, org, or net) as a STRING. For best results, URL values should comply with the format as defined by RFC 3986. If the URL value does not comply with RFC 3986 formatting, this function makes a best effort to parse the input and return a relevant result.

This function returns NULL if any of the following is true:

  • It cannot parse the host from the input;
  • The parsed host contains adjacent dots in the middle (not leading or trailing);
  • The parsed host does not contain any public suffix.

Before looking up the public suffix, this function temporarily normalizes the host by converting upper case English letters to lower case and encoding all non-ASCII characters with Punycode. The function then returns the public suffix as part of the original host instead of the normalized host.

Note: The function does not perform Unicode normalization.

Note: The public suffix data may change over time. Consequently, input that produces a NULL result now may produce a non-NULL value in the future.

Return Data Type

STRING

Example

SELECT
  FORMAT("%T", input) AS input,
  description,
  FORMAT("%T", NET.HOST(input)) AS host,
  FORMAT("%T", NET.PUBLIC_SUFFIX(input)) AS suffix,
  FORMAT("%T", NET.REG_DOMAIN(input)) AS domain
FROM (
  SELECT "" AS input, "invalid input" AS description
  UNION ALL SELECT "http://abc.xyz", "standard URL"
  UNION ALL SELECT "//user:password@a.b:80/path?query",
                   "standard URL with relative scheme, port, path and query, but no public suffix"
  UNION ALL SELECT "https://[::1]:80", "standard URL with IPv6 host"
  UNION ALL SELECT "http://例子.卷筒纸.中国", "standard URL with internationalized domain name"
  UNION ALL SELECT "    www.Google.Co.UK    ",
                   "non-standard URL with spaces, upper case letters, and without scheme"
  UNION ALL SELECT "mailto:?to=&subject=&body=", "URI rather than URL--unsupported"
);
input description host suffix domain
"" invalid input NULL NULL NULL
"http://abc.xyz" standard URL "abc.xyz" "xyz" "abc.xyz"
"//user:password@a.b:80/path?query" standard URL with relative scheme, port, path and query, but no public suffix "a.b" NULL NULL
"https://[::1]:80" standard URL with IPv6 host "[::1]" NULL NULL
"http://例子.卷筒纸.中国" standard URL with internationalized domain name "例子.卷筒纸.中国" "中国" "卷筒纸.中国"
"    www.Google.Co.UK    " non-standard URL with spaces, upper case letters, and without scheme "www.Google.Co.UK" "Co.UK" "Google.Co.UK"
"mailto:?to=&subject=&body=" URI rather than URL--unsupported "mailto" NULL NULL

NET.REG_DOMAIN

NET.REG_DOMAIN(url)

Description

Takes a URL as a STRING and returns the registered or registerable domain (the public suffix plus one preceding label), as a STRING. For best results, URL values should comply with the format as defined by RFC 3986. If the URL value does not comply with RFC 3986 formatting, this function makes a best effort to parse the input and return a relevant result.

This function returns NULL if any of the following is true:

  • It cannot parse the host from the input;
  • The parsed host contains adjacent dots in the middle (not leading or trailing);
  • The parsed host does not contain any public suffix;
  • The parsed host contains only a public suffix without any preceding label.

Before looking up the public suffix, this function temporarily normalizes the host by converting upper case English letters to lowercase and encoding all non-ASCII characters with [Punycode][net-link-to-ipv6-architecture1]. The function then returns the registered or registerable domain as part of the original host instead of the normalized host.

Note: The function does not perform Unicode normalization.

Note: The public suffix data may change over time. Consequently, input that produces a NULL result now may produce a non-NULL value in the future.

Return Data Type

STRING

Example

SELECT
  FORMAT("%T", input) AS input,
  description,
  FORMAT("%T", NET.HOST(input)) AS host,
  FORMAT("%T", NET.PUBLIC_SUFFIX(input)) AS suffix,
  FORMAT("%T", NET.REG_DOMAIN(input)) AS domain
FROM (
  SELECT "" AS input, "invalid input" AS description
  UNION ALL SELECT "http://abc.xyz", "standard URL"
  UNION ALL SELECT "//user:password@a.b:80/path?query",
                   "standard URL with relative scheme, port, path and query, but no public suffix"
  UNION ALL SELECT "https://[::1]:80", "standard URL with IPv6 host"
  UNION ALL SELECT "http://例子.卷筒纸.中国", "standard URL with internationalized domain name"
  UNION ALL SELECT "    www.Google.Co.UK    ",
                   "non-standard URL with spaces, upper case letters, and without scheme"
  UNION ALL SELECT "mailto:?to=&subject=&body=", "URI rather than URL--unsupported"
);
input description host suffix domain
"" invalid input NULL NULL NULL
"http://abc.xyz" standard URL "abc.xyz" "xyz" "abc.xyz"
"//user:password@a.b:80/path?query" standard URL with relative scheme, port, path and query, but no public suffix "a.b" NULL NULL
"https://[::1]:80" standard URL with IPv6 host "[::1]" NULL NULL
"http://例子.卷筒纸.中国" standard URL with internationalized domain name "例子.卷筒纸.中国" "中国" "卷筒纸.中国"
"    www.Google.Co.UK    " non-standard URL with spaces, upper case letters, and without scheme "www.Google.Co.UK" "Co.UK" "Google.Co.UK"
"mailto:?to=&subject=&body=" URI rather than URL--unsupported "mailto" NULL NULL

Operators

Operators are represented by special characters or keywords; they do not use function call syntax. An operator manipulates any number of data inputs, also called operands, and returns a result.

Common conventions:

  • Unless otherwise specified, all operators return NULL when one of the operands is NULL.
  • All operators will throw an error if the computation result overflows.
  • For all floating point operations, +/-inf and NaN may only be returned if one of the operands is +/-inf or NaN. In other cases, an error is returned.

The following table lists all BigQuery operators from highest to lowest precedence, i.e the order in which they will be evaluated within a statement.

Order of Precedence Operator Input Data Types Name Operator Arity
1 . STRUCT
 Member field access operator Binary
  [ ] ARRAY Array position. Must be used with OFFSET or ORDINAL—see ARRAY Functions. Binary
2 - Numeric Unary minus Unary
  ~ Integer or BYTES Bitwise not Unary
3 * Numeric Multiplication Binary
  / Numeric Division Binary
4 + Numeric Addition Binary
  - Numeric Subtraction Binary
5 << Integer or BYTES Bitwise left-shift Binary
  >> Integer or BYTES Bitwise right-shift Binary
6 & Integer or BYTES Bitwise and Binary
7 ^ Integer or BYTES Bitwise xor Binary
8 | Integer or BYTES Bitwise or Binary
9 (Comparison Operators) = Any comparable type. See Data Types for a complete list. Equal Binary
  < Any comparable type. See Data Types for a complete list. Less than Binary
  > Any comparable type. See Data Types for a complete list. Greater than Binary
  <= Any comparable type. See Data Types for a complete list. Less than or equal to Binary
  >= Any comparable type. See Data Types for a complete list. Greater than or equal to Binary
  !=, <> Any comparable type. See Data Types for a complete list. Not equal Binary
  [NOT] LIKE STRING and byte Value does [not] match the pattern specified Binary
  [NOT] BETWEEN Any comparable types. See Data Types for list. Value is [not] within the range specified Binary
  [NOT] IN Any comparable types. See Data Types for list. Value is [not] in the set of values specified Binary
  IS [NOT] NULL All Value is [not] NULL Unary
  IS [NOT] TRUE BOOL Value is [not] TRUE. Unary
  IS [NOT] FALSE BOOL Value is [not] FALSE. Unary
10 NOT BOOL Logical NOT Unary
11 AND BOOL Logical AND Binary
12 OR BOOL Logical OR Binary

Operators with the same precedence are left associative. This means that those operators are grouped together starting from the left and moving right. For example, the expression:

x AND y AND z

is interpreted as

( ( x AND y ) AND z )

The expression:

x * y / z

is interpreted as:

( ( x * y ) / z )

All comparison operators have the same priority and are grouped using left associativity. However, comparison operators are not associative. As a result, it is recommended that you use parentheses to improve readability and ensure expressions are resolved as desired. For example:

(x < y) IS FALSE

is recommended over:

x < y IS FALSE

Element access operators

Operator Syntax Input Data Types Result Data Type Description
. expression.fieldname1... STRUCT
 Type T stored in fieldname1 Dot operator. Can be used to access nested fields, e.g.expression.fieldname1.fieldname2...
[ ] array_expression [position_keyword (int_expression ) ] See ARRAY Functions. Type T stored in ARRAY position_keyword is either OFFSET or ORDINAL. See ARRAY Functions for the two functions that use this operator.

Arithmetic operators

All arithmetic operators accept input of numeric type T, and the result type has type T unless otherwise indicated in the description below:

Name Syntax
Addition X + Y
Subtraction X - Y
Multiplication X * Y
Division X / Y
Unary Minus - X

NOTE: Divide by zero operations return an error. To return a different result, consider the IEEE_DIVIDE or SAFE_DIVIDE functions.

Result types for Addition and Multiplication:

 FLOAT64INT64
FLOAT64FLOAT64FLOAT64
INT64FLOAT64INT64

Result types for Subtraction:

 FLOAT64INT64
FLOAT64FLOAT64FLOAT64
INT64FLOAT64INT64

Result types for Division:

 FLOAT64INT64
FLOAT64FLOAT64FLOAT64
INT64FLOAT64FLOAT64

Result types for Unary Minus:

Input Data Type Result Data Type
INT64 INT64
FLOAT64 FLOAT64

Bitwise operators

All bitwise operators return the same type and the same lengthas the first operand.

Name Syntax Input Data Type Description
Bitwise not ~ X Integer or BYTES Performs logical negation on each bit, forming the ones' complement of the given binary value.
Bitwise or X | Y X: Integer or BYTES
Y: Same type as X		 Takes two bit patterns of equal length and performs the logical inclusive OR operation on each pair of the corresponding bits. This operator throws an error if X and Y are BYTES of different lengths.
Bitwise xor X ^ Y X: Integer or BYTES
Y: Same type as X		 Takes two bit patterns of equal length and performs the logical exclusive OR operation on each pair of the corresponding bits. This operator throws an error if X and Y are BYTES of different lengths.
Bitwise and X & Y X: Integer or BYTES
Y: Same type as X		 Takes two bit patterns of equal length and performs the logical AND operation on each pair of the corresponding bits. This operator throws an error if X and Y are BYTES of different lengths.
Left shift X << Y X: Integer or BYTES
Y: INT64		 Shifts the first operand X to the left. This operator returns 0 or a byte sequence of b'\x00' if the second operand Y is greater than or equal to the bit length of the first operand X (for example, 64 if X has the type INT64). This operator throws an error if Y is negative.
Right shift X >> Y X: Integer or BYTES
Y: INT64		 Shifts the first operand X to the right. This operator does not do sign bit extension with a signed type (i.e. it fills vacant bits on the left with 0). This operator returns 0 or a byte sequence of b'\x00' if the second operand Y is greater than or equal to the bit length of the first operand X (for example, 64 if X has the type INT64). This operator throws an error if Y is negative.

Logical operators

All logical operators allow only BOOL input.

Name Syntax Description
Logical NOT NOT X Returns FALSE if input is TRUE. Returns TRUE if input is FALSE. Returns NULL otherwise.
Logical AND X AND Y Returns FALSE if at least one input is FALSE. Returns TRUE if both X and Y are TRUE. Returns NULL otherwise.
Logical OR X OR Y Returns FALSE if both X or Y are FALSE. Returns TRUE if at least one input is TRUE. Returns NULL otherwise.

Comparison operators

Comparisons always return BOOL. Comparisons require both operands to be the same type; if they are different, they are coerced to a common type for the comparison. Comparable data types are defined in Data Types.

STRUCTs support only 4 comparison operators: equal (=), not equal (!= and <>), and IN.

The following rules apply when comparing these data types:

  • FLOAT64 : All comparisons with NaN return FALSE, except for != and <>, which return TRUE.
  • BOOL: FALSE is less than TRUE.
  • STRING: Strings are compared codepoint-by-codepoint, which means that canonically equivalent strings are only guaranteed to compare as equal if they have been normalized first.
  • NULL: The convention holds here: any operation with a NULL input returns NULL.
Name Syntax Description
Less Than X < Y Returns TRUE if X is less than Y.
Less Than or Equal To X <= Y Returns TRUE if X is less than or equal to Y.
Greater Than X > Y Returns TRUE if X is greater than Y.
Greater Than or Equal To X >= Y Returns TRUE if X is greater than or equal to Y.
Equal X = Y Returns TRUE if X is equal to Y.
Not Equal X != Y
X <> Y		 Returns TRUE if X is not equal to Y.
BETWEEN X [NOT] BETWEEN Y AND Z Returns TRUE if X is [not] within the range specified. The result of "X BETWEEN Y AND Z" is equivalent to "Y <= X AND X <= Z" but X is evaluated only once in the former.
LIKE X [NOT] LIKE Y Checks if the STRING in the first operand X matches a pattern specified by the second operand Y. Expressions can contain these characters:
  • A percent sign "%" matches any number of characters or bytes
  • An underscore "_" matches a single character or byte
  • You can escape "\", "_", or "%" using two backslashes. For example, "\\%". If you are using raw strings, only a single backslash is required. For example, r"\%".
IN Multiple - see below Returns FALSE if the right operand is empty. Returns NULL if the left operand is NULL. Returns TRUE or NULL, never FALSE, if the right operand contains NULL. Arguments on either side of IN are general expressions. Neither operand is required to be a literal, although using a literal on the right is most common. X is evaluated only once.

When testing values that have a STRUCT data type for equality, it's possible that one or more fields are NULL. In such cases:

  • If all non-NULL field values are equal, the comparison returns NULL.
  • If any non-NULL field values are not equal, the comparison returns false.

The following table demonstrates how STRUCT data types are compared when they have fields that are NULL valued.

Struct1 Struct2 Struct1 = Struct2
STRUCT(1, NULL) STRUCT(1, NULL) NULL
STRUCT(1, NULL) STRUCT(2, NULL) FALSE
STRUCT(1,2) STRUCT(1, NULL) NULL

IN operators

The IN operator supports the following syntaxes:

x [NOT] IN (y, z, ... ) # Requires at least one element
x [NOT] IN (<subquery>)
x [NOT] IN UNNEST(<array expression>) # analysis error if the expression
                                      # does not return an ARRAY type.

Arguments on either side of the IN operator are general expressions. It is common to use literals on the right side expression; however, this is not required.

The semantics of:

x IN (y, z, ...)

are defined as equivalent to:

(x = y) OR (x = z) OR ...

and the subquery and array forms are defined similarly.

x NOT IN ...

is equivalent to:

NOT(x IN ...)

The UNNEST form treats an array scan like UNNEST in the FROM clause:

x [NOT] IN UNNEST(<array expression>)

This form is often used with ARRAY parameters. For example:

x IN UNNEST(@array_parameter)

Note: A NULL ARRAY will be treated equivalently to an empty ARRAY.

When using the IN operator, the following semantics apply:

  • IN with an empty right side expression is always FALSE
  • IN with a NULL left side expression and a non-empty right side expression is always NULL
  • IN with a NULL in the IN-list can only return TRUE or NULL, never FALSE
  • NULL IN (NULL) returns NULL
  • IN UNNEST(<NULL array>) returns FALSE (not NULL)

IN can be used with multi-part keys by using the struct constructor syntax. For example:

(Key1, Key2) IN ( (12,34), (56,78) )
(Key1, Key2) IN ( SELECT (table.a, table.b) FROM table )

See the Struct Type section of the Data Types topic for more information on this syntax.

IS operators

IS operators return TRUE or FALSE for the condition they are testing. They never return NULL, even for NULL inputs, unlike the IS_INF and IS_NAN functions defined in Mathematical Functions. If NOT is present, the output BOOL value is inverted.

Function Syntax Input Data Type Result Data Type Description
X IS [NOT] NULL Any value type BOOL Returns TRUE if the operand X evaluates to NULL, and returns FALSE otherwise.
X IS [NOT] TRUE BOOL BOOL Returns TRUE if the BOOL operand evaluates to TRUE. Returns FALSE otherwise.
X IS [NOT] FALSE BOOL BOOL Returns TRUE if the BOOL operand evaluates to FALSE. Returns FALSE otherwise.

Conditional expressions

Conditional expressions impose constraints on the evaluation order of their inputs. In essence, they are evaluated left to right, with short-circuiting, and only evaluate the output value that was chosen. In contrast, all inputs to regular functions are evaluated before calling the function. Short-circuiting in conditional expressions can be exploited for error handling or performance tuning.

Syntax Input Data Types Result Data Type Description
CASE expr WHEN value THEN result [WHEN ...] [ELSE else_result] END expr and value: Any type result and else_result: Supertype of input types. Compares expr to value of each successive WHEN clause and returns the first result where this comparison returns true. The remaining WHEN clauses and else_result are not evaluated. If the expr = value comparison returns false or NULL for all WHEN clauses, returns else_result. expr and value expressions must be implicitly coercible to a common supertype; equality comparisons are done on coerced values. result and else_result expressions must be coercible to a common supertype.
CASE WHEN cond1 THEN result [WHEN cond2...] [ELSE else_result] END cond: BOOL result and else_result: Supertype of input types. Evaluates condition cond of each successive WHEN clause and returns the first result where the condition is true; any remaining WHEN clauses and else_result are not evaluated. If all conditions are false or NULL, returns else_result. result and else_result expressions must be implicitly coercible to a common supertype.
COALESCE(expr1, ..., exprN) Any type Supertype of input types Returns the value of the first non-null expression. The remaining expressions are not evaluated. All input expressions must be implicitly coercible to a common supertype.
IF(cond, true_result, else_result) cond: BOOL true_result and else_result: Any type. If cond is true, returns true_result, else returns else_result. else_result is not evaluated if cond is true. true_result is not evaluated if cond is false or NULL. true_result and else_result must be coercible to a common supertype.
IFNULL(expr, null_result) Any type Any type or supertype of input types. If expr is NULL, return null_result. Otherwise, return expr. If expr is not NULL, null_result is not evaluated. expr and null_result must be implicitly coercible to a common supertype. Synonym for COALESCE(expr, null_result).
NULLIF(expression, expression_to_match) Any type T or subtype of T Any type T or subtype of T Returns NULL if expression = expression_to_match is true, otherwise returns expression. expression and expression_to_match must be implicitly coercible to a common supertype; equality comparison is done on coerced values.

Expression subqueries

There are four types of expression subqueries, i.e. subqueries that are used as expressions. Expression subqueries return NULL or a single value, as opposed to a column or table, and must be surrounded by parentheses. For a fuller discussion of subqueries, see Subqueries.

Type of Subquery Result Data Type Description
Scalar Any type T A subquery in parentheses inside an expression (e.g. in the SELECT list or WHERE clause) is interpreted as a scalar subquery. The SELECT list in a scalar subquery must have exactly one field. If the subquery returns exactly one row, that single value is the scalar subquery result. If the subquery returns zero rows, the scalar subquery value is NULL. If the subquery returns more than one row, the query fails with a runtime error. When the subquery is written with SELECT AS STRUCT , it can include multiple columns, and the returned value is the constructed STRUCT. Selecting multiple columns without using SELECT AS is an error.
ARRAY ARRAY Can use SELECT AS STRUCT to build arrays of structs, and conversely, selecting multiple columns without using SELECT AS is an error. Returns an empty ARRAY if the subquery returns zero rows. Never returns a NULL ARRAY.
IN BOOL Occurs in an expression following the IN operator. The subquery must produce a single column whose type is equality-compatible with the expression on the left side of the IN operator. Returns FALSE if the subquery returns zero rows. x IN () is equivalent to x IN (value, value, ...) See the IN operator in Comparison Operators for full semantics.
EXISTS BOOL Returns TRUE if the subquery produced one or more rows. Returns FALSE if the subquery produces zero rows. Never returns NULL. Unlike all other expression subqueries, there are no rules about the column list. Any number of columns may be selected and it will not affect the query result.

Examples

The following examples of expression subqueries assume that t.int_array has type ARRAY<INT64>.

Type Subquery Result Data Type Notes
Scalar (SELECT COUNT(*) FROM t.int_array) INT64  
(SELECT DISTINCT i FROM t.int_array i) INT64, possibly runtime error  
(SELECT i FROM t.int_array i WHERE i=5) INT64, possibly runtime error  
(SELECT ARRAY_AGG(i) FROM t.int_array i) ARRAY Uses the ARRAY_AGG aggregation function to return an ARRAY.
(SELECT 'xxx' a) STRING  
(SELECT 'xxx' a, 123 b) Error Returns an error because there is more than one column
(SELECT AS STRUCT 'xxx' a, 123 b) STRUCT  
(SELECT AS STRUCT 'xxx' a) STRUCT  
ARRAY ARRAY(SELECT COUNT(*) FROM t.int_array) ARRAY of size 1  
ARRAY(SELECT x FROM t) ARRAY  
ARRAY(SELECT 5 a, COUNT(*) b FROM t.int_array) Error Returns an error because there is more than one column
ARRAY(SELECT AS STRUCT 5 a, COUNT(*) b FROM t.int_array) ARRAY  
ARRAY(SELECT AS STRUCT i FROM t.int_array i) ARRAY Makes an ARRAY of one-field STRUCTs
ARRAY(SELECT AS STRUCT 1 x, 2, 3 x) ARRAY Returns an ARRAY of STRUCTs with anonymous or duplicate fields.
ARRAY(SELECT AS TypeName SUM(x) a, SUM(y) b, SUM(z) c from t) array<TypeName> Selecting into a named type. Assume TypeName is a STRUCT type with fields a,b,c.
STRUCT (SELECT AS STRUCT 1 x, 2, 3 x) STRUCT Constructs a STRUCT with anonymous or duplicate fields.
EXISTS EXISTS(SELECT x,y,z FROM table WHERE y=z) BOOL  
NOT EXISTS(SELECT x,y,z FROM table WHERE y=z) BOOL  
IN x IN (SELECT y FROM table WHERE z) BOOL  
x NOT IN (SELECT y FROM table WHERE z) BOOL  

Send feedback about...

This page
Documentation feedback
BigQuery Documentation
Product feedback