This page explains BigQuery expressions, including functions and operators.
Function call rules
The following rules apply to all functions unless explicitly indicated otherwise in the function description:
- For functions that accept numeric types, if one operand is a floating point operand and the other operand is another numeric type, both operands are converted to FLOAT64 before the function is evaluated.
- If an operand is
NULL
, the result isNULL
, with the exception of the IS operator. - For functions that are time zone sensitive (as indicated in the function description), the default time zone, UTC, is used if a time zone is not specified.
SAFE. prefix
Syntax:
SAFE.function_name()
Description
If you begin a function with the SAFE.
prefix, it will return NULL
instead
of an error. The SAFE.
prefix only prevents errors from the prefixed function
itself: it does not prevent errors that occur while evaluating argument
expressions. The SAFE.
prefix only prevents errors that occur because of the
value of the function inputs, such as "value out of range" errors; other
errors, such as internal or system errors, may still occur. If the function
does not return an error, SAFE.
has no effect on the output. If the function
never returns an error, like RAND
, then SAFE.
has no effect.
Operators, such as +
and =
, do not support the SAFE.
prefix. To prevent errors from a division
operation, use SAFE_DIVIDE. Some operators,
such as IN
, ARRAY
, and UNNEST
, resemble functions, but do not support the
SAFE.
prefix. The CAST
and EXTRACT
functions also do not support the
SAFE.
prefix. To prevent errors from casting, use
SAFE_CAST.
Example
In the following example, the first use of the SUBSTR
function would normally
return an error, because the function does not support length arguments with
negative values. However, the SAFE.
prefix causes the function to return
NULL
instead. The second use of the SUBSTR
function provides the expected
output: the SAFE.
prefix has no effect.
SELECT SAFE.SUBSTR('foo', 0, -2) AS safe_output UNION ALL
SELECT SAFE.SUBSTR('bar', 0, 2) AS safe_output;
+-------------+
| safe_output |
+-------------+
| NULL |
| ba |
+-------------+
Supported functions
BigQuery supports the use of the SAFE.
prefix with most scalar
functions that can raise errors, including STRING functions,
math functions, DATE functions,
DATETIME functions, and
TIMESTAMP functions. BigQuery does not support
the use of the SAFE.
prefix with aggregate,
analytic, or user-defined functions.
Calling persistent user-defined functions (UDFs)
After creating a persistent UDF, you can call it as you would any other function, prepended with the name of the dataset in which it is defined as a prefix.
Syntax
[`project_name`].dataset_name.function_name([parameter_value[, ...]])
To call a UDF in a project other than the project that you are using to run
the query, project_name
is required.
Examples
The following example creates a UDF named multiply_by_three
and calls it
from the same project.
CREATE FUNCTION my_dataset.multiply_by_three(x INT64) AS (x * 3);
SELECT my_dataset.multiply_by_three(5) AS result; -- returns 15
The following example calls a persistent UDF from a different project.
CREATE `other_project`.other_dataset.other_function(x INT64, y INT64)
AS (x * y * 2);
SELECT `other_project`.other_dataset.other_function(3, 4); --returns 24
Conversion rules
"Conversion" includes, but is not limited to, casting and coercion.
- Casting is explicit conversion and uses the
CAST()
function. - Coercion is implicit conversion, which BigQuery performs automatically under the conditions described below.
- There is a third group of conversion
functions that have their own function
names, such as
UNIX_DATE()
.
The table below summarizes all possible CAST
and coercion possibilities for
BigQuery data types. "Coercion To" applies to all expressions of a
given data type, (for example, a
column), but literals
and parameters can also be coerced. See Literal Coercion and
Parameter Coercion for details.
From Type | CAST to | Coercion To |
---|---|---|
INT64 | BOOL INT64 NUMERIC BIGNUMERIC FLOAT64 STRING |
NUMERIC BIGNUMERIC FLOAT64 |
NUMERIC | INT64 NUMERIC BIGNUMERIC FLOAT64 STRING |
BIGNUMERIC FLOAT64 |
BIGNUMERIC | INT64 NUMERIC BIGNUMERIC FLOAT64 STRING |
FLOAT64 |
FLOAT64 | INT64 NUMERIC BIGNUMERIC FLOAT64 STRING |
|
BOOL | BOOL INT64 STRING |
|
STRING | BOOL INT64 NUMERIC BIGNUMERIC FLOAT64 STRING BYTES DATE DATETIME TIME TIMESTAMP |
|
BYTES | STRING BYTES |
|
DATE | STRING DATE DATETIME TIMESTAMP |
DATETIME |
DATETIME | STRING DATE DATETIME TIME TIMESTAMP |
|
TIME | STRING TIME |
|
TIMESTAMP | STRING DATE DATETIME TIME TIMESTAMP |
|
ARRAY | ARRAY | |
STRUCT | STRUCT |
Casting
Syntax:
CAST(expr AS typename)
Cast syntax is used in a query to indicate that the result type of an expression should be converted to some other type.
Example:
CAST(x=1 AS STRING)
This results in "true"
if x
is 1
, "false"
for any other non-NULL
value, and NULL
if x
is NULL
.
Casts between supported types that do not successfully map from the original value to the target domain produce runtime errors. For example, casting BYTES to STRING where the byte sequence is not valid UTF-8 results in a runtime error.
When casting an expression x
of the following types, these rules apply:
From | To | Rule(s) when casting x |
---|---|---|
INT64 | FLOAT64 | Returns a close but potentially not exact FLOAT64 value. |
INT64 | BOOL | Returns FALSE if x is 0 , TRUE otherwise. |
NUMERIC | Floating Point | NUMERIC will convert to the closest floating point number with a possible loss of precision. |
BIGNUMERIC | Floating Point | BIGNUMERIC will convert to the closest floating point number with a possible loss of precision. |
FLOAT64 | INT64 | Returns the closest INT64 value. Halfway cases such as 1.5 or -0.5 round away from zero. |
FLOAT64 | STRING | Returns an approximate string representation. |
FLOAT64 | NUMERIC | The floating point number will round
half away from zero. Casting a NaN , +inf or
-inf will return an error. Casting a value outside the range of
NUMERIC
will return an overflow error.
|
FLOAT64 | BIGNUMERIC | The floating point number will round
half away from zero. Casting a NaN , +inf or
-inf will return an error. Casting a value outside the range of
BIGNUMERIC
will return an overflow error.
|
BOOL | INT64 | Returns 1 if x is TRUE , 0 otherwise. |
BOOL | STRING | Returns "true" if x is TRUE , "false" otherwise. |
STRING | FLOAT64 |
Returns x as a
FLOAT64
value, interpreting it as having the same form as a valid
FLOAT64
literal.
Also supports casts from "[+,-]inf" to [,-]Infinity ,
"[+,-]infinity" to [,-]Infinity , and
"[+,-]nan" to NaN .
Conversions are case-insensitive.
|
STRING | NUMERIC | The numeric literal contained in the STRING must not exceed the
maximum precision or range of the
NUMERIC
type, or an error will occur. If the number of digits
after the decimal point exceeds nine, then the resulting NUMERIC
value will round
half
away from zero to have nine digits after the decimal point.
|
STRING | BIGNUMERIC | The numeric literal contained in the STRING must not exceed the
maximum precision or range of the
BIGNUMERIC
type, or an error will occur. If the number of digits
after the decimal point exceeds 38, then the resulting BIGNUMERIC
value will round
half
away from zero to have 38 digits after the decimal point.
|
STRING | BOOL | Returns TRUE if x is "true" and
FALSE if x is "false" All other values of x are invalid and throw an error instead of
casting to BOOL.STRINGs are case-insensitive when converting to BOOL. |
STRING | BYTES | STRINGs are cast to BYTES using UTF-8 encoding. For example, the STRING "©", when cast to BYTES, would become a 2-byte sequence with the hex values C2 and A9. |
BYTES | STRING | Returns x interpreted as a UTF-8 STRING.For example, the BYTES literal b'\xc2\xa9' , when cast to STRING, is interpreted as UTF-8 and
becomes the unicode character "©".An error occurs if x is not valid UTF-8. |
ARRAY | ARRAY | Must be the exact same ARRAY type. |
STRUCT | STRUCT | Allowed if the following conditions are met:
|
Safe casting
When using CAST
, a query can fail if BigQuery is unable to perform
the cast. For example, the following query generates an error:
SELECT CAST("apple" AS INT64) AS not_a_number;
If you want to protect your queries from these types of errors, you can use
SAFE_CAST
. SAFE_CAST
is identical to CAST
, except it returns NULL instead
of raising an error.
SELECT SAFE_CAST("apple" AS INT64) AS not_a_number;
+--------------+
| not_a_number |
+--------------+
| NULL |
+--------------+
If you are casting from bytes to strings, you can also use the
function, SAFE_CONVERT_BYTES_TO_STRING
. Any invalid UTF-8 characters are
replaced with the unicode replacement character, U+FFFD
. See
SAFE_CONVERT_BYTES_TO_STRING for more
information.
Casting hex strings to integers
If you are working with hex strings (0x123
), you can cast those strings as
integers:
SELECT '0x123' as hex_value, CAST('0x123' as INT64) as hex_to_int;
+-----------+------------+
| hex_value | hex_to_int |
+-----------+------------+
| 0x123 | 291 |
+-----------+------------+
SELECT '-0x123' as hex_value, CAST('-0x123' as INT64) as hex_to_int;
+-----------+------------+
| hex_value | hex_to_int |
+-----------+------------+
| -0x123 | -291 |
+-----------+------------+
Casting time types
BigQuery supports casting time types to/from strings as follows:
CAST(time_expression AS STRING)
CAST(string_expression AS TIME)
Casting from a time type to a string is independent of time zone and is of the
form HH:MM:SS
. When casting from string to time, the string must conform to
the supported time literal format, and is independent of time zone. If the
string expression is invalid or represents a time that is outside of the
supported min/max range, then an error is produced.
Casting date types
BigQuery supports casting date types to/from strings as follows:
CAST(date_expression AS STRING)
CAST(string_expression AS DATE)
Casting from a date type to a string is independent of time zone and is of the
form YYYY-MM-DD
. When casting from string to date, the string must conform to
the supported date literal format, and is independent of time zone. If the
string expression is invalid or represents a date that is outside of the
supported min/max range, then an error is produced.
Casting datetime types
BigQuery supports casting datetime types to/from strings as follows:
CAST(datetime_expression AS STRING)
CAST(string_expression AS DATETIME)
Casting from a datetime type to a string is independent of time zone and is of
the form YYYY-MM-DD HH:MM:SS
. When casting from string to datetime, the
string must conform to the supported datetime literal format, and is independent
of time zone. If the string expression is invalid or represents a datetime that
is outside of the supported min/max range, then an error is produced.
Casting timestamp types
BigQuery supports casting timestamp types to/from strings as follows:
CAST(timestamp_expression AS STRING)
CAST(string_expression AS TIMESTAMP)
When casting from timestamp types to string, the timestamp is interpreted using the default time zone, UTC. The number of subsecond digits produced depends on the number of trailing zeroes in the subsecond part: the CAST function will truncate zero, three, or six digits.
When casting from string to a timestamp, string_expression
must conform to
the supported timestamp literal formats, or else a runtime error
occurs. The string_expression
may itself contain a time_zone
—see
time zones.
If there is a time zone in the string_expression
, that time zone is used for
conversion, otherwise the default time zone, UTC, is used.
If the string has fewer than six digits, then it is implicitly widened.
An error is produced if the string_expression
is invalid, has more than six
subsecond digits (i.e. precision greater than microseconds), or represents a
time outside of the supported timestamp range.
Casting between date, datetime and timestamp types
BigQuery supports casting between date, datetime and timestamp types as shown in the conversion rules table.
CAST(date_expression AS TIMESTAMP)
CAST(timestamp_expression AS DATE)
Casting from a date to a timestamp interprets date_expression
as of midnight
(start of the day) in the default time zone, UTC. Casting
from a timestamp to date effectively truncates the timestamp as of the default
time zone.
CAST(datetime_expression AS TIMESTAMP)
CAST(timestamp_expression AS DATETIME)
Casting from a datetime to a timestamp interprets datetime_expression
as of
midnight (start of the day) in the default time zone, UTC.
Coercion
BigQuery coerces the result type of an argument expression to another type if needed to match function signatures. For example, if function func() is defined to take a single argument of type FLOAT64 and an expression is used as an argument that has a result type of INT64, then the result of the expression will be coerced to FLOAT64 type before func() is computed.
Literal coercion
BigQuery supports the following literal coercions:
Input Data Type | Result Data Type | Notes |
---|---|---|
STRING literal | DATE DATETIME TIME TIMESTAMP |
Literal coercion is needed when the actual literal type is different from the
type expected by the function in question. For
example, if function func()
takes a DATE argument, then the expression
func("2014-09-27")
is valid because the STRING literal "2014-09-27"
is coerced to DATE.
Literal conversion is evaluated at analysis time, and gives an error if the input literal cannot be converted successfully to the target type.
Note: String literals do not coerce to numeric types.
Parameter coercion
BigQuery supports the following parameter coercions:
Input Data Type | Result Data Type |
---|---|
STRING parameter |
If the parameter value cannot be coerced successfully to the target type, an error is provided.
Additional conversion functions
BigQuery provides the following additional conversion functions:
Aggregate functions
An aggregate function is a function that summarizes the rows of a group into a
single value. COUNT
, MIN
and MAX
are examples of aggregate functions.
SELECT COUNT(*) as total_count, COUNT(fruit) as non_null_count,
MIN(fruit) as min, MAX(fruit) as max
FROM (SELECT NULL as fruit UNION ALL
SELECT "apple" as fruit UNION ALL
SELECT "pear" as fruit UNION ALL
SELECT "orange" as fruit)
+-------------+----------------+-------+------+
| total_count | non_null_count | min | max |
+-------------+----------------+-------+------+
| 4 | 3 | apple | pear |
+-------------+----------------+-------+------+
When used in conjunction with a GROUP BY
clause, the groups summarized
typically have at least one row. When the associated SELECT
has no GROUP BY
clause or when certain aggregate function modifiers filter rows from the group
to be summarized it is possible that the aggregate function needs to summarize
an empty group. In this case, the COUNT
and COUNTIF
functions return 0
,
while all other aggregate functions return NULL
.
The following sections describe the aggregate functions that BigQuery supports.
ANY_VALUE
ANY_VALUE(expression) [OVER (...)]
Description
Returns expression
for some row chosen from the group. Which row is chosen is
nondeterministic, not random. Returns NULL
when the input produces no
rows. Returns NULL
when expression
is NULL
for all rows in the group.
ANY_VALUE
behaves as if RESPECT NULLS
is specified;
Rows for which expression
is NULL
are considered and may be selected.
Supported Argument Types
Any
Optional Clause
OVER
: Specifies a window. See
Analytic Functions.
Returned Data Types
Matches the input data type.
Examples
SELECT ANY_VALUE(fruit) as any_value
FROM UNNEST(["apple", "banana", "pear"]) as fruit;
+-----------+
| any_value |
+-----------+
| apple |
+-----------+
SELECT
fruit,
ANY_VALUE(fruit) OVER (ORDER BY LENGTH(fruit) ROWS BETWEEN 1 PRECEDING AND CURRENT ROW) AS any_value
FROM UNNEST(["apple", "banana", "pear"]) as fruit;
+--------+-----------+
| fruit | any_value |
+--------+-----------+
| pear | pear |
| apple | pear |
| banana | apple |
+--------+-----------+
ARRAY_AGG
ARRAY_AGG([DISTINCT] expression [{IGNORE|RESPECT} NULLS]
[ORDER BY key [{ASC|DESC}] [, ... ]] [LIMIT n])
[OVER (...)]
Description
Returns an ARRAY of expression
values.
Supported Argument Types
All data types except ARRAY.
Optional Clauses
The clauses are applied in the following order:
OVER
: Specifies a window. See Analytic Functions. This clause is currently incompatible with all other clauses withinARRAY_AGG()
.DISTINCT
: Each distinct value ofexpression
is aggregated only once into the result.IGNORE NULLS
orRESPECT NULLS
: IfIGNORE NULLS
is specified, theNULL
values are excluded from the result. IfRESPECT NULLS
is specified, theNULL
values are included in the result. If neither is specified, theNULL
values are included in the result. An error is raised if an array in the final query result contains aNULL
element.ORDER BY
: Specifies the order of the values.- For each sort key, the default sort direction is
ASC
. - NULLs: In the context of the
ORDER BY
clause, NULLs are the minimum possible value; that is, NULLs appear first inASC
sorts and last inDESC
sorts. - Floating point data types: see Floating Point Semantics on ordering and grouping.
- If
DISTINCT
is also specified, then the sort key must be the same asexpression
. - If
ORDER BY
is not specified, the order of the elements in the output array is non-deterministic, which means you might receive a different result each time you use this function.
- For each sort key, the default sort direction is
LIMIT
: Specifies the maximum number ofexpression
inputs in the result. The limitn
must be a constant INT64.
Returned Data Types
ARRAY
If there are zero input rows, this function returns NULL
.
Examples
SELECT FORMAT("%T", ARRAY_AGG(x)) AS array_agg
FROM UNNEST([2, 1, -2, 3, -2, 1, 2]) AS x;
+-------------------------+
| array_agg |
+-------------------------+
| [2, 1, -2, 3, -2, 1, 2] |
+-------------------------+
SELECT FORMAT("%T", ARRAY_AGG(DISTINCT x)) AS array_agg
FROM UNNEST([2, 1, -2, 3, -2, 1, 2]) AS x;
+---------------+
| array_agg |
+---------------+
| [2, 1, -2, 3] |
+---------------+
SELECT FORMAT("%T", ARRAY_AGG(x IGNORE NULLS)) AS array_agg
FROM UNNEST([NULL, 1, -2, 3, -2, 1, NULL]) AS x;
+-------------------+
| array_agg |
+-------------------+
| [1, -2, 3, -2, 1] |
+-------------------+
SELECT FORMAT("%T", ARRAY_AGG(x ORDER BY ABS(x))) AS array_agg
FROM UNNEST([2, 1, -2, 3, -2, 1, 2]) AS x;
+-------------------------+
| array_agg |
+-------------------------+
| [1, 1, 2, -2, -2, 2, 3] |
+-------------------------+
SELECT FORMAT("%T", ARRAY_AGG(x LIMIT 5)) AS array_agg
FROM UNNEST([2, 1, -2, 3, -2, 1, 2]) AS x;
+-------------------+
| array_agg |
+-------------------+
| [2, 1, -2, 3, -2] |
+-------------------+
SELECT FORMAT("%T", ARRAY_AGG(DISTINCT x IGNORE NULLS ORDER BY x LIMIT 2)) AS array_agg
FROM UNNEST([NULL, 1, -2, 3, -2, 1, NULL]) AS x;
+-----------+
| array_agg |
+-----------+
| [-2, 1] |
+-----------+
SELECT
x,
FORMAT("%T", ARRAY_AGG(x) OVER (ORDER BY ABS(x))) AS array_agg
FROM UNNEST([2, 1, -2, 3, -2, 1, 2]) AS x;
+----+-------------------------+
| x | array_agg |
+----+-------------------------+
| 1 | [1, 1] |
| 1 | [1, 1] |
| 2 | [1, 1, 2, -2, -2, 2] |
| -2 | [1, 1, 2, -2, -2, 2] |
| -2 | [1, 1, 2, -2, -2, 2] |
| 2 | [1, 1, 2, -2, -2, 2] |
| 3 | [1, 1, 2, -2, -2, 2, 3] |
+----+-------------------------+
ARRAY_CONCAT_AGG
ARRAY_CONCAT_AGG(expression [ORDER BY key [{ASC|DESC}] [, ... ]] [LIMIT n])
Description
Concatenates elements from expression
of type
ARRAY, returning a single
ARRAY as a result. This function ignores NULL input
arrays, but respects the NULL elements in non-NULL input arrays
(an error is raised, however, if an array in the final query result contains a NULL element).
Supported Argument Types
ARRAY
Optional Clauses
The clauses are applied in the following order:
ORDER BY
: Specifies the order of the values.- For each sort key, the default sort direction is
ASC
. - Array ordering is not supported, and thus the sort key cannot be the
same as
expression
. - NULLs: In the context of the
ORDER BY
clause, NULLs are the minimum possible value; that is, NULLs appear first inASC
sorts and last inDESC
sorts. - Floating point data types: see Floating Point Semantics on ordering and grouping.
- If
ORDER BY
is not specified, the order of the elements in the output array is non-deterministic, which means you might receive a different result each time you use this function.
- For each sort key, the default sort direction is
LIMIT
: Specifies the maximum number ofexpression
inputs in the result. The limit applies to the number of input arrays, not the number of elements in the arrays. An empty array counts as 1. A NULL array is not counted. The limitn
must be a constant INT64.
Returned Data Types
ARRAY
Returns NULL
if there are zero input
rows or expression
evaluates to NULL for all rows.
Examples
SELECT FORMAT("%T", ARRAY_CONCAT_AGG(x)) AS array_concat_agg FROM (
SELECT [NULL, 1, 2, 3, 4] AS x
UNION ALL SELECT NULL
UNION ALL SELECT [5, 6]
UNION ALL SELECT [7, 8, 9]
);
+-----------------------------------+
| array_concat_agg |
+-----------------------------------+
| [NULL, 1, 2, 3, 4, 5, 6, 7, 8, 9] |
+-----------------------------------+
SELECT FORMAT("%T", ARRAY_CONCAT_AGG(x ORDER BY ARRAY_LENGTH(x))) AS array_concat_agg FROM (
SELECT [1, 2, 3, 4] AS x
UNION ALL SELECT [5, 6]
UNION ALL SELECT [7, 8, 9]
);
+-----------------------------------+
| array_concat_agg |
+-----------------------------------+
| [5, 6, 7, 8, 9, 1, 2, 3, 4] |
+-----------------------------------+
SELECT FORMAT("%T", ARRAY_CONCAT_AGG(x LIMIT 2)) AS array_concat_agg FROM (
SELECT [1, 2, 3, 4] AS x
UNION ALL SELECT [5, 6]
UNION ALL SELECT [7, 8, 9]
);
+--------------------------+
| array_concat_agg |
+--------------------------+
| [1, 2, 3, 4, 5, 6] |
+--------------------------+
SELECT FORMAT("%T", ARRAY_CONCAT_AGG(x ORDER BY ARRAY_LENGTH(x) LIMIT 2)) AS array_concat_agg FROM (
SELECT [1, 2, 3, 4] AS x
UNION ALL SELECT [5, 6]
UNION ALL SELECT [7, 8, 9]
);
+------------------+
| array_concat_agg |
+------------------+
| [5, 6, 7, 8, 9] |
+------------------+
AVG
AVG([DISTINCT] expression) [OVER (...)]
Description
Returns the average of non-NULL
input values, or NaN
if the input contains a
NaN
.
Supported Argument Types
Any numeric input type, such as INT64. Note that, for floating point input types, the return result is non-deterministic, which means you might receive a different result each time you use this function.
Optional Clauses
The clauses are applied in the following order:
OVER
: Specifies a window. See Analytic Functions. This clause is currently incompatible with all other clauses withinAVG()
.DISTINCT
: Each distinct value ofexpression
is aggregated only once into the result.
Returned Data Types
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
OUTPUT | FLOAT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
Examples
SELECT AVG(x) as avg
FROM UNNEST([0, 2, 4, 4, 5]) as x;
+-----+
| avg |
+-----+
| 3 |
+-----+
SELECT AVG(DISTINCT x) AS avg
FROM UNNEST([0, 2, 4, 4, 5]) AS x;
+------+
| avg |
+------+
| 2.75 |
+------+
SELECT
x,
AVG(x) OVER (ORDER BY x ROWS BETWEEN 1 PRECEDING AND CURRENT ROW) AS avg
FROM UNNEST([0, 2, NULL, 4, 4, 5]) AS x;
+------+------+
| x | avg |
+------+------+
| NULL | NULL |
| 0 | 0 |
| 2 | 1 |
| 4 | 3 |
| 4 | 4 |
| 5 | 4.5 |
+------+------+
BIT_AND
BIT_AND(expression)
Description
Performs a bitwise AND operation on expression
and returns the result.
Supported Argument Types
- INT64
Returned Data Types
INT64
Examples
SELECT BIT_AND(x) as bit_and FROM UNNEST([0xF001, 0x00A1]) as x;
+---------+
| bit_and |
+---------+
| 1 |
+---------+
BIT_OR
BIT_OR(expression)
Description
Performs a bitwise OR operation on expression
and returns the result.
Supported Argument Types
- INT64
Returned Data Types
INT64
Examples
SELECT BIT_OR(x) as bit_or FROM UNNEST([0xF001, 0x00A1]) as x;
+--------+
| bit_or |
+--------+
| 61601 |
+--------+
BIT_XOR
BIT_XOR([DISTINCT] expression)
Description
Performs a bitwise XOR operation on expression
and returns the result.
Supported Argument Types
- INT64
Optional Clause
DISTINCT
: Each distinct value of
expression
is aggregated only once into the result.
Returned Data Types
INT64
Examples
SELECT BIT_XOR(x) AS bit_xor FROM UNNEST([5678, 1234]) AS x;
+---------+
| bit_xor |
+---------+
| 4860 |
+---------+
SELECT BIT_XOR(x) AS bit_xor FROM UNNEST([1234, 5678, 1234]) AS x;
+---------+
| bit_xor |
+---------+
| 5678 |
+---------+
SELECT BIT_XOR(DISTINCT x) AS bit_xor FROM UNNEST([1234, 5678, 1234]) AS x;
+---------+
| bit_xor |
+---------+
| 4860 |
+---------+
COUNT
1.
COUNT(*) [OVER (...)]
2.
COUNT([DISTINCT] expression) [OVER (...)]
Description
- Returns the number of rows in the input.
- Returns the number of rows with
expression
evaluated to any value other thanNULL
.
Supported Argument Types
expression
can be any data type. If
DISTINCT
is present, expression
can only be a data type that is
groupable.
Optional Clauses
The clauses are applied in the following order:
OVER
: Specifies a window. See Analytic Functions.DISTINCT
: Each distinct value ofexpression
is aggregated only once into the result.
Return Data Types
INT64
Examples
SELECT
COUNT(*) AS count_star,
COUNT(DISTINCT x) AS count_dist_x
FROM UNNEST([1, 4, 4, 5]) AS x;
+------------+--------------+
| count_star | count_dist_x |
+------------+--------------+
| 4 | 3 |
+------------+--------------+
SELECT
x,
COUNT(*) OVER (PARTITION BY MOD(x, 3)) AS count_star,
COUNT(DISTINCT x) OVER (PARTITION BY MOD(x, 3)) AS count_dist_x
FROM UNNEST([1, 4, 4, 5]) AS x;
+------+------------+--------------+
| x | count_star | count_dist_x |
+------+------------+--------------+
| 1 | 3 | 2 |
| 4 | 3 | 2 |
| 4 | 3 | 2 |
| 5 | 1 | 1 |
+------+------------+--------------+
SELECT
x,
COUNT(*) OVER (PARTITION BY MOD(x, 3)) AS count_star,
COUNT(x) OVER (PARTITION BY MOD(x, 3)) AS count_x
FROM UNNEST([1, 4, NULL, 4, 5]) AS x;
+------+------------+---------+
| x | count_star | count_x |
+------+------------+---------+
| NULL | 1 | 0 |
| 1 | 3 | 3 |
| 4 | 3 | 3 |
| 4 | 3 | 3 |
| 5 | 1 | 1 |
+------+------------+---------+
COUNTIF
COUNTIF(expression) [OVER (...)]
Description
Returns the count of TRUE
values for expression
. Returns 0
if there are
zero input rows, or if expression
evaluates to FALSE
or NULL
for all rows.
Supported Argument Types
BOOL
Optional Clause
OVER
: Specifies a window. See
Analytic Functions.
Return Data Types
INT64
Examples
SELECT COUNTIF(x<0) AS num_negative, COUNTIF(x>0) AS num_positive
FROM UNNEST([5, -2, 3, 6, -10, -7, 4, 0]) AS x;
+--------------+--------------+
| num_negative | num_positive |
+--------------+--------------+
| 3 | 4 |
+--------------+--------------+
SELECT
x,
COUNTIF(x<0) OVER (ORDER BY ABS(x) ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING) AS num_negative
FROM UNNEST([5, -2, 3, 6, -10, NULL, -7, 4, 0]) AS x;
+------+--------------+
| x | num_negative |
+------+--------------+
| NULL | 0 |
| 0 | 1 |
| -2 | 1 |
| 3 | 1 |
| 4 | 0 |
| 5 | 0 |
| 6 | 1 |
| -7 | 2 |
| -10 | 2 |
+------+--------------+
LOGICAL_AND
LOGICAL_AND(expression)
Description
Returns the logical AND of all non-NULL
expressions. Returns NULL
if there
are zero input rows or expression
evaluates to NULL
for all rows.
Supported Argument Types
BOOL
Return Data Types
BOOL
Examples
SELECT LOGICAL_AND(x) AS logical_and FROM UNNEST([true, false, true]) AS x;
+-------------+
| logical_and |
+-------------+
| false |
+-------------+
LOGICAL_OR
LOGICAL_OR(expression)
Description
Returns the logical OR of all non-NULL
expressions. Returns NULL
if there
are zero input rows or expression
evaluates to NULL
for all rows.
Supported Argument Types
BOOL
Return Data Types
BOOL
Examples
SELECT LOGICAL_OR(x) AS logical_or FROM UNNEST([true, false, true]) AS x;
+------------+
| logical_or |
+------------+
| true |
+------------+
MAX
MAX(expression) [OVER (...)]
Description
Returns the maximum value of non-NULL
expressions. Returns NULL
if there
are zero input rows or expression
evaluates to NULL
for all rows.
Returns NaN
if the input contains a NaN
.
Supported Argument Types
Any data type except: ARRAY
STRUCT
GEOGRAPHY
Optional Clause
OVER
: Specifies a window. See
Analytic Functions.
Return Data Types
Same as the data type used as the input values.
Examples
SELECT MAX(x) AS max
FROM UNNEST([8, 37, 4, 55]) AS x;
+-----+
| max |
+-----+
| 55 |
+-----+
SELECT x, MAX(x) OVER (PARTITION BY MOD(x, 2)) AS max
FROM UNNEST([8, NULL, 37, 4, NULL, 55]) AS x;
+------+------+
| x | max |
+------+------+
| NULL | NULL |
| NULL | NULL |
| 8 | 8 |
| 4 | 8 |
| 37 | 55 |
| 55 | 55 |
+------+------+
MIN
MIN(expression) [OVER (...)]
Description
Returns the minimum value of non-NULL
expressions. Returns NULL
if there
are zero input rows or expression
evaluates to NULL
for all rows.
Returns NaN
if the input contains a NaN
.
Supported Argument Types
Any data type except: ARRAY
STRUCT
GEOGRAPHY
Optional Clause
OVER
: Specifies a window. See
Analytic Functions.
Return Data Types
Same as the data type used as the input values.
Examples
SELECT MIN(x) AS min
FROM UNNEST([8, 37, 4, 55]) AS x;
+-----+
| min |
+-----+
| 4 |
+-----+
SELECT x, MIN(x) OVER (PARTITION BY MOD(x, 2)) AS min
FROM UNNEST([8, NULL, 37, 4, NULL, 55]) AS x;
+------+------+
| x | min |
+------+------+
| NULL | NULL |
| NULL | NULL |
| 8 | 4 |
| 4 | 4 |
| 37 | 37 |
| 55 | 37 |
+------+------+
STRING_AGG
STRING_AGG([DISTINCT] expression [, delimiter] [ORDER BY key [{ASC|DESC}] [, ... ]] [LIMIT n])
[OVER (...)]
Description
Returns a value (either STRING or BYTES) obtained by concatenating non-null values.
If a delimiter
is specified, concatenated values are separated by that
delimiter; otherwise, a comma is used as a delimiter.
Supported Argument Types
STRING BYTES
Optional Clauses
The clauses are applied in the following order:
OVER
: Specifies a window. See Analytic Functions. This clause is currently incompatible with all other clauses withinSTRING_AGG()
.DISTINCT
: Each distinct value ofexpression
is aggregated only once into the result.ORDER BY
: Specifies the order of the values.- For each sort key, the default sort direction is
ASC
. - NULLs: In the context of the
ORDER BY
clause, NULLs are the minimum possible value; that is, NULLs appear first inASC
sorts and last inDESC
sorts. - Floating point data types: see Floating Point Semantics on ordering and grouping.
- If
DISTINCT
is also specified, then the sort key must be the same asexpression
. - If
ORDER BY
is not specified, the order of the elements in the output array is non-deterministic, which means you might receive a different result each time you use this function.
- For each sort key, the default sort direction is
LIMIT
: Specifies the maximum number ofexpression
inputs in the result. The limit applies to the number of input strings, not the number of characters or bytes in the inputs. An empty string counts as 1. A NULL string is not counted. The limitn
must be a constant INT64.
Return Data Types
STRING BYTES
Examples
SELECT STRING_AGG(fruit) AS string_agg
FROM UNNEST(["apple", NULL, "pear", "banana", "pear"]) AS fruit;
+------------------------+
| string_agg |
+------------------------+
| apple,pear,banana,pear |
+------------------------+
SELECT STRING_AGG(fruit, " & ") AS string_agg
FROM UNNEST(["apple", "pear", "banana", "pear"]) AS fruit;
+------------------------------+
| string_agg |
+------------------------------+
| apple & pear & banana & pear |
+------------------------------+
SELECT STRING_AGG(DISTINCT fruit, " & ") AS string_agg
FROM UNNEST(["apple", "pear", "banana", "pear"]) AS fruit;
+-----------------------+
| string_agg |
+-----------------------+
| apple & pear & banana |
+-----------------------+
SELECT STRING_AGG(fruit, " & " ORDER BY LENGTH(fruit)) AS string_agg
FROM UNNEST(["apple", "pear", "banana", "pear"]) AS fruit;
+------------------------------+
| string_agg |
+------------------------------+
| pear & pear & apple & banana |
+------------------------------+
SELECT STRING_AGG(fruit, " & " LIMIT 2) AS string_agg
FROM UNNEST(["apple", "pear", "banana", "pear"]) AS fruit;
+--------------+
| string_agg |
+--------------+
| apple & pear |
+--------------+
SELECT STRING_AGG(DISTINCT fruit, " & " ORDER BY fruit DESC LIMIT 2) AS string_agg
FROM UNNEST(["apple", "pear", "banana", "pear"]) AS fruit;
+---------------+
| string_agg |
+---------------+
| pear & banana |
+---------------+
SELECT
fruit,
STRING_AGG(fruit, " & ") OVER (ORDER BY LENGTH(fruit)) AS string_agg
FROM UNNEST(["apple", NULL, "pear", "banana", "pear"]) AS fruit;
+--------+------------------------------+
| fruit | string_agg |
+--------+------------------------------+
| NULL | NULL |
| pear | pear & pear |
| pear | pear & pear |
| apple | pear & pear & apple |
| banana | pear & pear & apple & banana |
+--------+------------------------------+
SUM
SUM([DISTINCT] expression) [OVER (...)]
Description
Returns the sum of non-null values.
If the expression is a floating point value, the sum is non-deterministic, which means you might receive a different result each time you use this function.
Supported Argument Types
Any supported numeric data types.
Optional Clauses
The clauses are applied in the following order:
OVER
: Specifies a window. See Analytic Functions.DISTINCT
: Each distinct value ofexpression
is aggregated only once into the result.
Return Data Types
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
OUTPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
Special cases:
Returns NULL
if the input contains only NULL
s.
Returns NULL
if the input contains no rows.
Returns Inf
if the input contains Inf
.
Returns -Inf
if the input contains -Inf
.
Returns NaN
if the input contains a NaN
.
Returns NaN
if the input contains a combination of Inf
and -Inf
.
Examples
SELECT SUM(x) AS sum
FROM UNNEST([1, 2, 3, 4, 5, 4, 3, 2, 1]) AS x;
+-----+
| sum |
+-----+
| 25 |
+-----+
SELECT SUM(DISTINCT x) AS sum
FROM UNNEST([1, 2, 3, 4, 5, 4, 3, 2, 1]) AS x;
+-----+
| sum |
+-----+
| 15 |
+-----+
SELECT
x,
SUM(x) OVER (PARTITION BY MOD(x, 3)) AS sum
FROM UNNEST([1, 2, 3, 4, 5, 4, 3, 2, 1]) AS x;
+---+-----+
| x | sum |
+---+-----+
| 3 | 6 |
| 3 | 6 |
| 1 | 10 |
| 4 | 10 |
| 4 | 10 |
| 1 | 10 |
| 2 | 9 |
| 5 | 9 |
| 2 | 9 |
+---+-----+
SELECT
x,
SUM(DISTINCT x) OVER (PARTITION BY MOD(x, 3)) AS sum
FROM UNNEST([1, 2, 3, 4, 5, 4, 3, 2, 1]) AS x;
+---+-----+
| x | sum |
+---+-----+
| 3 | 3 |
| 3 | 3 |
| 1 | 5 |
| 4 | 5 |
| 4 | 5 |
| 1 | 5 |
| 2 | 7 |
| 5 | 7 |
| 2 | 7 |
+---+-----+
SELECT SUM(x) AS sum
FROM UNNEST([]) AS x;
+------+
| sum |
+------+
| NULL |
+------+
Statistical aggregate functions
BigQuery supports the following statistical aggregate functions.
CORR
CORR(X1, X2) [OVER (...)]
Description
Returns the Pearson coefficient
of correlation of a set of number pairs. For each number pair, the first number
is the dependent variable and the second number is the independent variable.
The return result is between -1
and 1
. A result of 0
indicates no
correlation.
All numeric types are supported. If the
input is NUMERIC
or BIGNUMERIC
then the internal aggregation is
stable with the final output converted to a FLOAT64
.
Otherwise the input is converted to a FLOAT64
before aggregation, resulting in a potentially unstable result.
This function ignores any input pairs that contain one or more NULL values. If there are fewer than two input pairs without NULL values, this function returns NULL.
Optional Clause
OVER
: Specifies a window. See
Analytic Functions.
Return Data Type
FLOAT64
COVAR_POP
COVAR_POP(X1, X2) [OVER (...)]
Description
Returns the population covariance of
a set of number pairs. The first number is the dependent variable; the second
number is the independent variable. The return result is between -Inf
and
+Inf
.
All numeric types are supported. If the
input is NUMERIC
or BIGNUMERIC
then the internal aggregation is
stable with the final output converted to a FLOAT64
.
Otherwise the input is converted to a FLOAT64
before aggregation, resulting in a potentially unstable result.
This function ignores any input pairs that contain one or more NULL values. If there is no input pair without NULL values, this function returns NULL. If there is exactly one input pair without NULL values, this function returns 0.
Optional Clause
OVER
: Specifies a window. See
Analytic Functions.
Return Data Type
FLOAT64
COVAR_SAMP
COVAR_SAMP(X1, X2) [OVER (...)]
Description
Returns the sample covariance of a
set of number pairs. The first number is the dependent variable; the second
number is the independent variable. The return result is between -Inf
and
+Inf
.
All numeric types are supported. If the
input is NUMERIC
or BIGNUMERIC
then the internal aggregation is
stable with the final output converted to a FLOAT64
.
Otherwise the input is converted to a FLOAT64
before aggregation, resulting in a potentially unstable result.
This function ignores any input pairs that contain one or more NULL values. If there are fewer than two input pairs without NULL values, this function returns NULL.
Optional Clause
OVER
: Specifies a window. See
Analytic Functions.
Return Data Type
FLOAT64
STDDEV_POP
STDDEV_POP([DISTINCT] expression) [OVER (...)]
Description
Returns the population (biased) standard deviation of the values. The return
result is between 0
and +Inf
.
All numeric types are supported. If the
input is NUMERIC
or BIGNUMERIC
then the internal aggregation is
stable with the final output converted to a FLOAT64
.
Otherwise the input is converted to a FLOAT64
before aggregation, resulting in a potentially unstable result.
This function ignores any NULL inputs. If all inputs are ignored, this function returns NULL.
If this function receives a single non-NULL input, it returns 0
.
Optional Clauses
The clauses are applied in the following order:
OVER
: Specifies a window. See Analytic Functions. This clause is currently incompatible with all other clauses withinSTDDEV_POP()
.DISTINCT
: Each distinct value ofexpression
is aggregated only once into the result.
Return Data Type
FLOAT64
STDDEV_SAMP
STDDEV_SAMP([DISTINCT] expression) [OVER (...)]
Description
Returns the sample (unbiased) standard deviation of the values. The return
result is between 0
and +Inf
.
All numeric types are supported. If the
input is NUMERIC
or BIGNUMERIC
then the internal aggregation is
stable with the final output converted to a FLOAT64
.
Otherwise the input is converted to a FLOAT64
before aggregation, resulting in a potentially unstable result.
This function ignores any NULL inputs. If there are fewer than two non-NULL inputs, this function returns NULL.
Optional Clauses
The clauses are applied in the following order:
OVER
: Specifies a window. See Analytic Functions. This clause is currently incompatible with all other clauses withinSTDDEV_SAMP()
.DISTINCT
: Each distinct value ofexpression
is aggregated only once into the result.
Return Data Type
FLOAT64
STDDEV
STDDEV([DISTINCT] expression) [OVER (...)]
Description
An alias of STDDEV_SAMP.
VAR_POP
VAR_POP([DISTINCT] expression) [OVER (...)]
Description
Returns the population (biased) variance of the values. The return result is
between 0
and +Inf
.
All numeric types are supported. If the
input is NUMERIC
or BIGNUMERIC
then the internal aggregation is
stable with the final output converted to a FLOAT64
.
Otherwise the input is converted to a FLOAT64
before aggregation, resulting in a potentially unstable result.
This function ignores any NULL inputs. If all inputs are ignored, this function returns NULL.
If this function receives a single non-NULL input, it returns 0
.
Optional Clauses
The clauses are applied in the following order:
OVER
: Specifies a window. See Analytic Functions. This clause is currently incompatible with all other clauses withinVAR_POP()
.DISTINCT
: Each distinct value ofexpression
is aggregated only once into the result.
Return Data Type
FLOAT64
VAR_SAMP
VAR_SAMP([DISTINCT] expression) [OVER (...)]
Description
Returns the sample (unbiased) variance of the values. The return result is
between 0
and +Inf
.
All numeric types are supported. If the
input is NUMERIC
or BIGNUMERIC
then the internal aggregation is
stable with the final output converted to a FLOAT64
.
Otherwise the input is converted to a FLOAT64
before aggregation, resulting in a potentially unstable result.
This function ignores any NULL inputs. If there are fewer than two non-NULL inputs, this function returns NULL.
Optional Clauses
The clauses are applied in the following order:
OVER
: Specifies a window. See Analytic Functions. This clause is currently incompatible with all other clauses withinVAR_SAMP()
.DISTINCT
: Each distinct value ofexpression
is aggregated only once into the result.
Return Data Type
FLOAT64
VARIANCE
VARIANCE([DISTINCT] expression) [OVER (...)]
Description
An alias of VAR_SAMP.
Approximate aggregate functions
Approximate aggregate functions are scalable in terms of memory usage and time,
but produce approximate results instead of exact results. These functions
typically require less memory than exact aggregation functions
like COUNT(DISTINCT ...)
, but also introduce statistical uncertainty.
This makes approximate aggregation appropriate for large data streams for
which linear memory usage is impractical, as well as for data that is
already approximate.
The approximate aggregate functions in this section work directly on the input data, rather than an intermediate estimation of the data. These functions do not allow users to specify the precision for the estimation with sketches. If you would like specify precision with sketches, see:
- HyperLogLog++ functions to estimate cardinality.
APPROX_COUNT_DISTINCT
APPROX_COUNT_DISTINCT(expression)
Description
Returns the approximate result for COUNT(DISTINCT expression)
. The value
returned is a statistical estimate—not necessarily the actual value.
This function is less accurate than COUNT(DISTINCT expression)
, but performs
better on huge input.
Supported Argument Types
Any data type except:
ARRAY
STRUCT
Returned Data Types
INT64
Examples
SELECT APPROX_COUNT_DISTINCT(x) as approx_distinct
FROM UNNEST([0, 1, 1, 2, 3, 5]) as x;
+-----------------+
| approx_distinct |
+-----------------+
| 5 |
+-----------------+
APPROX_QUANTILES
APPROX_QUANTILES([DISTINCT] expression, number [{IGNORE|RESPECT} NULLS])
Description
Returns the approximate boundaries for a group of expression
values, where
number
represents the number of quantiles to create. This function returns
an array of number
+ 1 elements, where the first element is the approximate
minimum and the last element is the approximate maximum.
Supported Argument Types
expression
can be any supported data type except:
ARRAY
STRUCT
number
must be INT64.
Optional Clauses
The clauses are applied in the following order:
DISTINCT
: Each distinct value ofexpression
is aggregated only once into the result.IGNORE NULLS
orRESPECT NULLS
: IfIGNORE NULLS
is specified, theNULL
values are excluded from the result. IfRESPECT NULLS
is specified, theNULL
values are included in the result. If neither is specified, theNULL
values are excluded from the result. An error is raised if an array in the final query result contains aNULL
element.
Returned Data Types
An ARRAY of the type specified by the expression
parameter.
Returns NULL
if there are zero input
rows or expression
evaluates to NULL for all rows.
Examples
SELECT APPROX_QUANTILES(x, 2) AS approx_quantiles
FROM UNNEST([1, 1, 1, 4, 5, 6, 7, 8, 9, 10]) AS x;
+------------------+
| approx_quantiles |
+------------------+
| [1, 5, 10] |
+------------------+
SELECT APPROX_QUANTILES(x, 100)[OFFSET(90)] AS percentile_90
FROM UNNEST([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]) AS x;
+---------------+
| percentile_90 |
+---------------+
| 9 |
+---------------+
SELECT APPROX_QUANTILES(DISTINCT x, 2) AS approx_quantiles
FROM UNNEST([1, 1, 1, 4, 5, 6, 7, 8, 9, 10]) AS x;
+------------------+
| approx_quantiles |
+------------------+
| [1, 6, 10] |
+------------------+
SELECT FORMAT("%T", APPROX_QUANTILES(x, 2 RESPECT NULLS)) AS approx_quantiles
FROM UNNEST([NULL, NULL, 1, 1, 1, 4, 5, 6, 7, 8, 9, 10]) AS x;
+------------------+
| approx_quantiles |
+------------------+
| [NULL, 4, 10] |
+------------------+
SELECT FORMAT("%T", APPROX_QUANTILES(DISTINCT x, 2 RESPECT NULLS)) AS approx_quantiles
FROM UNNEST([NULL, NULL, 1, 1, 1, 4, 5, 6, 7, 8, 9, 10]) AS x;
+------------------+
| approx_quantiles |
+------------------+
| [NULL, 6, 10] |
+------------------+
APPROX_TOP_COUNT
APPROX_TOP_COUNT(expression, number)
Description
Returns the approximate top elements of expression
. The number
parameter
specifies the number of elements returned.
Supported Argument Types
expression
can be of any data type that the GROUP BY
clause supports.
number
must be INT64.
Returned Data Types
An ARRAY of type STRUCT.
The STRUCT contains two fields. The first field
(named value
) contains an input value. The second field (named count
)
contains an INT64 specifying the number of times the
value was returned.
Returns NULL
if there are zero input rows.
Examples
SELECT APPROX_TOP_COUNT(x, 2) as approx_top_count
FROM UNNEST(["apple", "apple", "pear", "pear", "pear", "banana"]) as x;
+-------------------------+
| approx_top_count |
+-------------------------+
| [{pear, 3}, {apple, 2}] |
+-------------------------+
NULL handling
APPROX_TOP_COUNT does not ignore NULLs in the input. For example:
SELECT APPROX_TOP_COUNT(x, 2) as approx_top_count
FROM UNNEST([NULL, "pear", "pear", "pear", "apple", NULL]) as x;
+------------------------+
| approx_top_count |
+------------------------+
| [{pear, 3}, {NULL, 2}] |
+------------------------+
APPROX_TOP_SUM
APPROX_TOP_SUM(expression, weight, number)
Description
Returns the approximate top elements of expression
, based on the sum of an
assigned weight
. The number
parameter specifies the number of elements
returned.
If the weight
input is negative or NaN
, this function returns an error.
Supported Argument Types
expression
can be of any data type that the GROUP BY
clause supports.
weight
must be one of the following:
- INT64
- NUMERIC
- BIGNUMERIC
- FLOAT64
number
must be INT64.
Returned Data Types
An ARRAY of type STRUCT.
The STRUCT contains two fields: value
and sum
.
The value
field contains the value of the input expression. The sum
field is
the same type as weight
, and is the approximate sum of the input weight
associated with the value
field.
Returns NULL
if there are zero input rows.
Examples
SELECT APPROX_TOP_SUM(x, weight, 2) AS approx_top_sum FROM
UNNEST([
STRUCT("apple" AS x, 3 AS weight),
("pear", 2),
("apple", 0),
("banana", 5),
("pear", 4)
]);
+--------------------------+
| approx_top_sum |
+--------------------------+
| [{pear, 6}, {banana, 5}] |
+--------------------------+
NULL handling
APPROX_TOP_SUM does not ignore NULL values for the expression
and weight
parameters.
SELECT APPROX_TOP_SUM(x, weight, 2) AS approx_top_sum FROM
UNNEST([STRUCT("apple" AS x, NULL AS weight), ("pear", 0), ("pear", NULL)]);
+----------------------------+
| approx_top_sum |
+----------------------------+
| [{pear, 0}, {apple, NULL}] |
+----------------------------+
SELECT APPROX_TOP_SUM(x, weight, 2) AS approx_top_sum FROM
UNNEST([STRUCT("apple" AS x, 0 AS weight), (NULL, 2)]);
+-------------------------+
| approx_top_sum |
+-------------------------+
| [{NULL, 2}, {apple, 0}] |
+-------------------------+
SELECT APPROX_TOP_SUM(x, weight, 2) AS approx_top_sum FROM
UNNEST([STRUCT("apple" AS x, 0 AS weight), (NULL, NULL)]);
+----------------------------+
| approx_top_sum |
+----------------------------+
| [{apple, 0}, {NULL, NULL}] |
+----------------------------+
HyperLogLog++ functions
The HyperLogLog++ algorithm (HLL++) estimates cardinality from sketches. If you do not want to work with sketches and do not need customized precision, consider using approximate aggregate functions with system-defined precision.
HLL++ functions are approximate aggregate functions.
Approximate aggregation typically requires less
memory than exact aggregation functions,
like COUNT(DISTINCT)
, but also introduces statistical uncertainty.
This makes HLL++ functions appropriate for large data streams for
which linear memory usage is impractical, as well as for data that is
already approximate.
BigQuery supports the following HLL++ functions:
HLL_COUNT.INIT
HLL_COUNT.INIT(input [, precision])
Description
An aggregate function that takes one or more input
values and aggregates them
into a HLL++ sketch. Each sketch
is represented using the BYTES
data type. You can then merge sketches using
HLL_COUNT.MERGE
or HLL_COUNT.MERGE_PARTIAL
. If no merging is needed,
you can extract the final count of distinct values from the sketch using
HLL_COUNT.EXTRACT
.
This function supports an optional parameter, precision
. This parameter
defines the accuracy of the estimate at the cost of additional memory required
to process the sketches or store them on disk. The following table shows the
allowed precision values, the maximum sketch size per group, and confidence
interval (CI) of typical precisions:
Precision | Max. Sketch Size (KiB) | 65% CI | 95% CI | 99% CI |
---|---|---|---|---|
10 | 1 | ±3.25% | ±6.50% | ±9.75% |
11 | 2 | ±2.30% | ±4.60% | ±6.89% |
12 | 4 | ±1.63% | ±3.25% | ±4.88% |
13 | 8 | ±1.15% | ±2.30% | ±3.45% |
14 | 16 | ±0.81% | ±1.63% | ±2.44% |
15 (default) | 32 | ±0.57% | ±1.15% | ±1.72% |
16 | 64 | ±0.41% | ±0.81% | ±1.22% |
17 | 128 | ±0.29% | ±0.57% | ±0.86% |
18 | 256 | ±0.20% | ±0.41% | ±0.61% |
19 | 512 | ±0.14% | ±0.29% | ±0.43% |
20 | 1024 | ±0.10% | ±0.20% | ±0.30% |
21 | 2048 | ±0.07% | ±0.14% | ±0.22% |
22 | 4096 | ±0.05% | ±0.10% | ±0.15% |
23 | 8192 | ±0.04% | ±0.07% | ±0.11% |
24 | 16384 | ±0.03% | ±0.05% | ±0.08% |
If the input is NULL, this function returns NULL.
For more information, see HyperLogLog in Practice: Algorithmic Engineering of a State of The Art Cardinality Estimation Algorithm.
Supported input types
INT64, NUMERIC, BIGNUMERIC, STRING, BYTES
Return type
BYTES
Example
SELECT
HLL_COUNT.INIT(respondent) AS respondents_hll,
flavor,
country
FROM UNNEST([
STRUCT(1 AS respondent, "Vanilla" AS flavor, "CH" AS country),
(1, "Chocolate", "CH"),
(2, "Chocolate", "US"),
(2, "Strawberry", "US")])
GROUP BY flavor, country;
HLL_COUNT.MERGE
HLL_COUNT.MERGE(sketch)
Description
An aggregate function that returns the cardinality of several HLL++ set sketches by computing their union.
Each sketch
must have the same precision and be initialized on the same type.
Attempts to merge sketches with different precisions or for different types
results in an error. For example, you cannot merge a sketch initialized
from INT64 data with one initialized from STRING data.
This function ignores NULL values when merging sketches. If the merge happens
over zero rows or only over NULL values, the function returns 0
.
Supported input types
BYTES
Return type
INT64
Example
SELECT HLL_COUNT.MERGE(respondents_hll) AS num_respondents, flavor
FROM (
SELECT
HLL_COUNT.INIT(respondent) AS respondents_hll,
flavor,
country
FROM UNNEST([
STRUCT(1 AS respondent, "Vanilla" AS flavor, "CH" AS country),
(1, "Chocolate", "CH"),
(2, "Chocolate", "US"),
(2, "Strawberry", "US")])
GROUP BY flavor, country)
GROUP BY flavor;
HLL_COUNT.MERGE_PARTIAL
HLL_COUNT.MERGE_PARTIAL(sketch)
Description
An aggregate function that takes one or more
HLL++ sketch
inputs and merges them into a new sketch.
This function returns NULL if there is no input or all inputs are NULL.
Supported input types
BYTES
Return type
BYTES
Example
SELECT HLL_COUNT.MERGE_PARTIAL(respondents_hll) AS num_respondents, flavor
FROM (
SELECT
HLL_COUNT.INIT(respondent) AS respondents_hll,
flavor,
country
FROM UNNEST([
STRUCT(1 AS respondent, "Vanilla" AS flavor, "CH" AS country),
(1, "Chocolate", "CH"),
(2, "Chocolate", "US"),
(2, "Strawberry", "US")])
GROUP BY flavor, country)
GROUP BY flavor;
HLL_COUNT.EXTRACT
HLL_COUNT.EXTRACT(sketch)
Description
A scalar function that extracts a cardinality estimate of a single HLL++ sketch.
If sketch
is NULL, this function returns a cardinality estimate of 0
.
Supported input types
BYTES
Return type
INT64
Example
SELECT
flavor,
country,
HLL_COUNT.EXTRACT(respondents_hll) AS num_respondents
FROM (
SELECT
HLL_COUNT.INIT(respondent) AS respondents_hll,
flavor,
country
FROM UNNEST([
STRUCT(1 AS respondent, "Vanilla" AS flavor, "CH" AS country),
(1, "Chocolate", "CH"),
(2, "Chocolate", "US"),
(2, "Strawberry", "US")])
GROUP BY flavor, country);
+------------+---------+-----------------+
| flavor | country | num_respondents |
+------------+---------+-----------------+
| Vanilla | CH | 1 |
| Chocolate | CH | 1 |
| Chocolate | US | 1 |
| Strawberry | US | 1 |
+------------+---------+-----------------+
About the HLL++ algorithm
The HLL++ algorithm improves on the HLL algorithm by more accurately estimating very small and large cardinalities. The HLL++ algorithm includes a 64-bit hash function, sparse representation to reduce memory requirements for small cardinality estimates, and empirical bias correction for small cardinality estimates.
About sketches
A sketch is a summary of a large data stream. You can extract statistics from a sketch to estimate particular statistics of the original data, or merge sketches to summarize multiple data streams. A sketch has these features:
- It compresses raw data into a fixed-memory representation.
- It's asymptotically smaller than the input.
- It's the serialized form of an in-memory, sublinear data structure.
- It typically requires less memory than the input used to create it.
Sketches allow integration with other systems. For example, it is possible to
build sketches in external applications, like Cloud Dataflow, or
Apache Spark and consume them in BigQuery or
vice versa. Sketches also allow building intermediate aggregations for
non-additive functions like COUNT(DISTINCT)
.
Numbering functions
The following sections describe the numbering functions that BigQuery supports. Numbering functions are a subset of analytic functions. For an explanation of how analytic functions work, see Analytic Function Concepts. For a description of how numbering functions work, see the Numbering Function Concepts.
OVER
clause requirements:
PARTITION BY
: Optional.ORDER BY
: Required, except forROW_NUMBER()
.window_frame_clause
: Disallowed.
RANK
Description
Returns the ordinal (1-based) rank of each row within the ordered partition.
All peer rows receive the same rank value. The next row or set of peer rows
receives a rank value which increments by the number of peers with the previous
rank value, instead of DENSE_RANK
, which always increments by 1.
Supported Argument Types
INT64
DENSE_RANK
Description
Returns the ordinal (1-based) rank of each row within the window partition. All peer rows receive the same rank value, and the subsequent rank value is incremented by one.
Supported Argument Types
INT64
PERCENT_RANK
Description
Return the percentile rank of a row defined as (RK-1)/(NR-1), where RK is
the RANK
of the row and NR is the number of rows in the partition.
Returns 0 if NR=1.
Supported Argument Types
FLOAT64
CUME_DIST
Description
Return the relative rank of a row defined as NP/NR. NP is defined to be the number of rows that either precede or are peers with the current row. NR is the number of rows in the partition.
Supported Argument Types
FLOAT64
NTILE
NTILE(constant_integer_expression)
Description
This function divides the rows into constant_integer_expression
buckets based on row ordering and returns the 1-based bucket number that is
assigned to each row. The number of rows in the buckets can differ by at most 1.
The remainder values (the remainder of number of rows divided by buckets) are
distributed one for each bucket, starting with bucket 1. If
constant_integer_expression
evaluates to NULL, 0 or negative, an
error is provided.
Supported Argument Types
INT64
ROW_NUMBER
Description
Does not require the ORDER BY
clause. Returns the sequential
row ordinal (1-based) of each row for each ordered partition. If the
ORDER BY
clause is unspecified then the result is
non-deterministic.
Supported Argument Types
INT64
Bit functions
BigQuery supports the following bit functions.
BIT_COUNT
BIT_COUNT(expression)
Description
The input, expression
, must be an
integer or BYTES.
Returns the number of bits that are set in the input expression
.
For signed integers, this is the number of bits in two's complement form.
Return Data Type
INT64
Example
SELECT a, BIT_COUNT(a) AS a_bits, FORMAT("%T", b) as b, BIT_COUNT(b) AS b_bits
FROM UNNEST([
STRUCT(0 AS a, b'' AS b), (0, b'\x00'), (5, b'\x05'), (8, b'\x00\x08'),
(0xFFFF, b'\xFF\xFF'), (-2, b'\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFE'),
(-1, b'\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF'),
(NULL, b'\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF')
]) AS x;
+-------+--------+---------------------------------------------+--------+
| a | a_bits | b | b_bits |
+-------+--------+---------------------------------------------+--------+
| 0 | 0 | b"" | 0 |
| 0 | 0 | b"\x00" | 0 |
| 5 | 2 | b"\x05" | 2 |
| 8 | 1 | b"\x00\x08" | 1 |
| 65535 | 16 | b"\xff\xff" | 16 |
| -2 | 63 | b"\xff\xff\xff\xff\xff\xff\xff\xfe" | 63 |
| -1 | 64 | b"\xff\xff\xff\xff\xff\xff\xff\xff" | 64 |
| NULL | NULL | b"\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff" | 80 |
+-------+--------+---------------------------------------------+--------+
Mathematical functions
All mathematical functions have the following behaviors:
- They return
NULL
if any of the input parameters isNULL
. - They return
NaN
if any of the arguments isNaN
.
ABS
ABS(X)
Description
Computes absolute value. Returns an error if the argument is an integer and the output value cannot be represented as the same type; this happens only for the largest negative input value, which has no positive representation.
X | ABS(X) |
---|---|
25 | 25 |
-25 | 25 |
+inf |
+inf |
-inf |
+inf |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
OUTPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
SIGN
SIGN(X)
Description
Returns -1
, 0
, or +1
for negative, zero and positive arguments
respectively. For floating point arguments, this function does not distinguish
between positive and negative zero.
X | SIGN(X) |
---|---|
25 | +1 |
0 | 0 |
-25 | -1 |
NaN | NaN |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
OUTPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
IS_INF
IS_INF(X)
Description
Returns TRUE
if the value is positive or negative infinity.
X | IS_INF(X) |
---|---|
+inf |
TRUE |
-inf |
TRUE |
25 | FALSE |
IS_NAN
IS_NAN(X)
Description
Returns TRUE
if the value is a NaN
value.
X | IS_NAN(X) |
---|---|
NaN |
TRUE |
25 | FALSE |
IEEE_DIVIDE
IEEE_DIVIDE(X, Y)
Description
Divides X by Y; this function never fails. Returns
FLOAT64
. Unlike the division operator (/),
this function does not generate errors for division by zero or overflow.
X | Y | IEEE_DIVIDE(X, Y) |
---|---|---|
20.0 | 4.0 | 5.0 |
0.0 | 25.0 | 0.0 |
25.0 | 0.0 | +inf |
-25.0 | 0.0 | -inf |
0.0 | 0.0 | NaN |
0.0 | NaN |
NaN |
NaN |
0.0 | NaN |
+inf |
+inf |
NaN |
-inf |
-inf |
NaN |
RAND
RAND()
Description
Generates a pseudo-random value of type FLOAT64
in
the range of [0, 1), inclusive of 0 and exclusive of 1.
SQRT
SQRT(X)
Description
Computes the square root of X. Generates an error if X is less than 0.
X | SQRT(X) |
---|---|
25.0 |
5.0 |
+inf |
+inf |
X < 0 |
Error |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
OUTPUT | FLOAT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
POW
POW(X, Y)
Description
Returns the value of X raised to the power of Y. If the result underflows and is not representable, then the function returns a value of zero.
X | Y | POW(X, Y) |
---|---|---|
2.0 | 3.0 | 8.0 |
1.0 | Any value including NaN |
1.0 |
Any value including NaN |
0 | 1.0 |
-1.0 | +inf |
1.0 |
-1.0 | -inf |
1.0 |
ABS(X) < 1 | -inf |
+inf |
ABS(X) > 1 | -inf |
0.0 |
ABS(X) < 1 | +inf |
0.0 |
ABS(X) > 1 | +inf |
+inf |
-inf |
Y < 0 | 0.0 |
-inf |
Y > 0 | -inf if Y is an odd integer, +inf otherwise |
+inf |
Y < 0 | 0 |
+inf |
Y > 0 | +inf |
Finite value < 0 | Non-integer | Error |
0 | Finite value < 0 | Error |
Return Data Type
The return data type is determined by the argument types with the following table.
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
INT64 | FLOAT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
NUMERIC | NUMERIC | NUMERIC | BIGNUMERIC | FLOAT64 |
BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | FLOAT64 |
FLOAT64 | FLOAT64 | FLOAT64 | FLOAT64 | FLOAT64 |
POWER
POWER(X, Y)
Description
Synonym of POW(X, Y)
.
EXP
EXP(X)
Description
Computes e to the power of X, also called the natural exponential function. If the result underflows, this function returns a zero. Generates an error if the result overflows.
X | EXP(X) |
---|---|
0.0 | 1.0 |
+inf |
+inf |
-inf |
0.0 |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
OUTPUT | FLOAT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
LN
LN(X)
Description
Computes the natural logarithm of X. Generates an error if X is less than or equal to zero.
X | LN(X) |
---|---|
1.0 | 0.0 |
+inf |
+inf |
X < 0 |
Error |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
OUTPUT | FLOAT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
LOG
LOG(X [, Y])
Description
If only X is present, LOG
is a synonym of LN
. If Y is also present,
LOG
computes the logarithm of X to base Y.
X | Y | LOG(X, Y) |
---|---|---|
100.0 | 10.0 | 2.0 |
-inf |
Any value | NaN |
Any value | +inf |
NaN |
+inf |
0.0 < Y < 1.0 | -inf |
+inf |
Y > 1.0 | +inf |
X <= 0 | Any value | Error |
Any value | Y <= 0 | Error |
Any value | 1.0 | Error |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
INT64 | FLOAT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
NUMERIC | NUMERIC | NUMERIC | BIGNUMERIC | FLOAT64 |
BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | FLOAT64 |
FLOAT64 | FLOAT64 | FLOAT64 | FLOAT64 | FLOAT64 |
LOG10
LOG10(X)
Description
Similar to LOG
, but computes logarithm to base 10.
X | LOG10(X) |
---|---|
100.0 | 2.0 |
-inf |
NaN |
+inf |
NaN |
X <= 0 | Error |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
OUTPUT | FLOAT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
GREATEST
GREATEST(X1,...,XN)
Description
Returns the largest value among X1,...,XN according to the < comparison.
X1,...,XN | GREATEST(X1,...,XN) |
---|---|
3,5,1 | 5 |
Return Data Types
Data type of the input values.
LEAST
LEAST(X1,...,XN)
Description
Returns the smallest value among X1,...,XN according to the > comparison.
X1,...,XN | LEAST(X1,...,XN) |
---|---|
3,5,1 | 1 |
Return Data Types
Data type of the input values.
DIV
DIV(X, Y)
Description
Returns the result of integer division of X by Y. Division by zero returns an error. Division by -1 may overflow.
X | Y | DIV(X, Y) |
---|---|---|
20 | 4 | 5 |
0 | 20 | 0 |
20 | 0 | Error |
Return Data Type
The return data type is determined by the argument types with the following table.
INPUT | INT64 | NUMERIC | BIGNUMERIC |
---|---|---|---|
INT64 | INT64 | NUMERIC | BIGNUMERIC |
NUMERIC | NUMERIC | NUMERIC | BIGNUMERIC |
BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | BIGNUMERIC |
SAFE_DIVIDE
SAFE_DIVIDE(X, Y)
Description
Equivalent to the division operator (X / Y
), but returns
NULL
if an error occurs, such as a division by zero error.
X | Y | SAFE_DIVIDE(X, Y) |
---|---|---|
20 | 4 | 5 |
0 | 20 | 0 |
20 | 0 | NULL |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
INT64 | FLOAT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
NUMERIC | NUMERIC | NUMERIC | BIGNUMERIC | FLOAT64 |
BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | FLOAT64 |
FLOAT64 | FLOAT64 | FLOAT64 | FLOAT64 | FLOAT64 |
SAFE_MULTIPLY
SAFE_MULTIPLY(X, Y)
Description
Equivalent to the multiplication operator (*
), but returns
NULL
if overflow occurs.
X | Y | SAFE_MULTIPLY(X, Y) |
---|---|---|
20 | 4 | 80 |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
INT64 | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
NUMERIC | NUMERIC | NUMERIC | BIGNUMERIC | FLOAT64 |
BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | FLOAT64 |
FLOAT64 | FLOAT64 | FLOAT64 | FLOAT64 | FLOAT64 |
SAFE_NEGATE
SAFE_NEGATE(X)
Description
Equivalent to the unary minus operator (-
), but returns
NULL
if overflow occurs.
X | SAFE_NEGATE(X) |
---|---|
+1 | -1 |
-1 | +1 |
0 | 0 |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
OUTPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
SAFE_ADD
SAFE_ADD(X, Y)
Description
Equivalent to the addition operator (+
), but returns
NULL
if overflow occurs.
X | Y | SAFE_ADD(X, Y) |
---|---|---|
5 | 4 | 9 |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
INT64 | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
NUMERIC | NUMERIC | NUMERIC | BIGNUMERIC | FLOAT64 |
BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | FLOAT64 |
FLOAT64 | FLOAT64 | FLOAT64 | FLOAT64 | FLOAT64 |
SAFE_SUBTRACT
SAFE_SUBTRACT(X, Y)
Description
Returns the result of Y subtracted from X.
Equivalent to the subtraction operator (-
), but returns
NULL
if overflow occurs.
X | Y | SAFE_SUBTRACT(X, Y) |
---|---|---|
5 | 4 | 1 |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
INT64 | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
NUMERIC | NUMERIC | NUMERIC | BIGNUMERIC | FLOAT64 |
BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | FLOAT64 |
FLOAT64 | FLOAT64 | FLOAT64 | FLOAT64 | FLOAT64 |
MOD
MOD(X, Y)
Description
Modulo function: returns the remainder of the division of X by Y. Returned value has the same sign as X. An error is generated if Y is 0.
X | Y | MOD(X, Y) |
---|---|---|
25 | 12 | 1 |
25 | 0 | Error |
Return Data Type
The return data type is determined by the argument types with the following table.
INPUT | INT64 | NUMERIC | BIGNUMERIC |
---|---|---|---|
INT64 | INT64 | NUMERIC | BIGNUMERIC |
NUMERIC | NUMERIC | NUMERIC | BIGNUMERIC |
BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | BIGNUMERIC |
ROUND
ROUND(X [, N])
Description
If only X is present, ROUND
rounds X to the nearest integer. If N is present,
ROUND
rounds X to N decimal places after the decimal point. If N is negative,
ROUND
will round off digits to the left of the decimal point. Rounds halfway
cases away from zero. Generates an error if overflow occurs.
X | ROUND(X) |
---|---|
2.0 | 2.0 |
2.3 | 2.0 |
2.8 | 3.0 |
2.5 | 3.0 |
-2.3 | -2.0 |
-2.8 | -3.0 |
-2.5 | -3.0 |
0 | 0 |
+inf |
+inf |
-inf |
-inf |
NaN |
NaN |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
OUTPUT | FLOAT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
TRUNC
TRUNC(X [, N])
Description
If only X is present, TRUNC
rounds X to the nearest integer whose absolute
value is not greater than the absolute value of X. If N is also present, TRUNC
behaves like ROUND(X, N)
, but always rounds towards zero and never overflows.
X | TRUNC(X) |
---|---|
2.0 | 2.0 |
2.3 | 2.0 |
2.8 | 2.0 |
2.5 | 2.0 |
-2.3 | -2.0 |
-2.8 | -2.0 |
-2.5 | -2.0 |
0 | 0 |
+inf |
+inf |
-inf |
-inf |
NaN |
NaN |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
OUTPUT | FLOAT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
CEIL
CEIL(X)
Description
Returns the smallest integral value that is not less than X.
X | CEIL(X) |
---|---|
2.0 | 2.0 |
2.3 | 3.0 |
2.8 | 3.0 |
2.5 | 3.0 |
-2.3 | -2.0 |
-2.8 | -2.0 |
-2.5 | -2.0 |
0 | 0 |
+inf |
+inf |
-inf |
-inf |
NaN |
NaN |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
OUTPUT | FLOAT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
CEILING
CEILING(X)
Description
Synonym of CEIL(X)
FLOOR
FLOOR(X)
Description
Returns the largest integral value that is not greater than X.
X | FLOOR(X) |
---|---|
2.0 | 2.0 |
2.3 | 2.0 |
2.8 | 2.0 |
2.5 | 2.0 |
-2.3 | -3.0 |
-2.8 | -3.0 |
-2.5 | -3.0 |
0 | 0 |
+inf |
+inf |
-inf |
-inf |
NaN |
NaN |
Return Data Type
INPUT | INT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|---|
OUTPUT | FLOAT64 | NUMERIC | BIGNUMERIC | FLOAT64 |
COS
COS(X)
Description
Computes the cosine of X where X is specified in radians. Never fails.
X | COS(X) |
---|---|
+inf |
NaN |
-inf |
NaN |
NaN |
NaN |
COSH
COSH(X)
Description
Computes the hyperbolic cosine of X where X is specified in radians. Generates an error if overflow occurs.
X | COSH(X) |
---|---|
+inf |
+inf |
-inf |
+inf |
NaN |
NaN |
ACOS
ACOS(X)
Description
Computes the principal value of the inverse cosine of X. The return value is in the range [0,π]. Generates an error if X is a value outside of the range [-1, 1].
X | ACOS(X) |
---|---|
+inf |
NaN |
-inf |
NaN |
NaN |
NaN |
X < -1 | Error |
X > 1 | Error |
ACOSH
ACOSH(X)
Description
Computes the inverse hyperbolic cosine of X. Generates an error if X is a value less than 1.
X | ACOSH(X) |
---|---|
+inf |
+inf |
-inf |
NaN |
NaN |
NaN |
X < 1 | Error |
SIN
SIN(X)
Description
Computes the sine of X where X is specified in radians. Never fails.
X | SIN(X) |
---|---|
+inf |
NaN |
-inf |
NaN |
NaN |
NaN |
SINH
SINH(X)
Description
Computes the hyperbolic sine of X where X is specified in radians. Generates an error if overflow occurs.
X | SINH(X) |
---|---|
+inf |
+inf |
-inf |
-inf |
NaN |
NaN |
ASIN
ASIN(X)
Description
Computes the principal value of the inverse sine of X. The return value is in the range [-π/2,π/2]. Generates an error if X is outside of the range [-1, 1].
X | ASIN(X) |
---|---|
+inf |
NaN |
-inf |
NaN |
NaN |
NaN |
X < -1 | Error |
X > 1 | Error |
ASINH
ASINH(X)
Description
Computes the inverse hyperbolic sine of X. Does not fail.
X | ASINH(X) |
---|---|
+inf |
+inf |
-inf |
-inf |
NaN |
NaN |
TAN
TAN(X)
Description
Computes the tangent of X where X is specified in radians. Generates an error if overflow occurs.
X | TAN(X) |
---|---|
+inf |
NaN |
-inf |
NaN |
NaN |
NaN |
TANH
TANH(X)
Description
Computes the hyperbolic tangent of X where X is specified in radians. Does not fail.
X | TANH(X) |
---|---|
+inf |
1.0 |
-inf |
-1.0 |
NaN |
NaN |
ATAN
ATAN(X)
Description
Computes the principal value of the inverse tangent of X. The return value is in the range [-π/2,π/2]. Does not fail.
X | ATAN(X) |
---|---|
+inf |
π/2 |
-inf |
-π/2 |
NaN |
NaN |
ATANH
ATANH(X)
Description
Computes the inverse hyperbolic tangent of X. Generates an error if X is outside of the range [-1, 1].
X | ATANH(X) |
---|---|
+inf |
NaN |
-inf |
NaN |
NaN |
NaN |
X < -1 | Error |
X > 1 | Error |
ATAN2
ATAN2(X, Y)
Description
Calculates the principal value of the inverse tangent of X/Y using the signs of the two arguments to determine the quadrant. The return value is in the range [-π,π].
X | Y | ATAN2(X, Y) |
---|---|---|
NaN |
Any value | NaN |
Any value | NaN |
NaN |
0.0 | 0.0 | 0.0 |
Positive Finite value | -inf |
π |
Negative Finite value | -inf |
-π |
Finite value | +inf |
0.0 |
+inf |
Finite value | π/2 |
-inf |
Finite value | -π/2 |
+inf |
-inf |
¾π |
-inf |
-inf |
-¾π |
+inf |
+inf |
π/4 |
-inf |
+inf |
-π/4 |
RANGE_BUCKET
RANGE_BUCKET(point, boundaries_array)
Description
RANGE_BUCKET
scans through a sorted array and returns the 0-based position
of the point's upper bound. This can be useful if you need to group your data to
build partitions, histograms, business-defined rules, and more.
RANGE_BUCKET
follows these rules:
If the point exists in the array, returns the index of the next larger value.
RANGE_BUCKET(20, [0, 10, 20, 30, 40]) -- 3 is return value RANGE_BUCKET(20, [0, 10, 20, 20, 40, 40]) -- 4 is return value
If the point does not exist in the array, but it falls between two values, returns the index of the larger value.
RANGE_BUCKET(25, [0, 10, 20, 30, 40]) -- 3 is return value
If the point is smaller than the first value in the array, returns 0.
RANGE_BUCKET(-10, [5, 10, 20, 30, 40]) -- 0 is return value
If the point is greater than or equal to the last value in the array, returns the length of the array.
RANGE_BUCKET(80, [0, 10, 20, 30, 40]) -- 5 is return value
If the array is empty, returns 0.
RANGE_BUCKET(80, []) -- 0 is return value
If the point is
NULL
orNaN
, returnsNULL
.RANGE_BUCKET(NULL, [0, 10, 20, 30, 40]) -- NULL is return value
The data type for the point and array must be compatible.
RANGE_BUCKET('a', ['a', 'b', 'c', 'd']) -- 1 is return value RANGE_BUCKET(1.2, [1, 1.2, 1.4, 1.6]) -- 2 is return value RANGE_BUCKET(1.2, [1, 2, 4, 6]) -- execution failure
Execution failure occurs when:
The array has a
NaN
orNULL
value in it.RANGE_BUCKET(80, [NULL, 10, 20, 30, 40]) -- execution failure
The array is not sorted in ascending order.
RANGE_BUCKET(30, [10, 30, 20, 40, 50]) -- execution failure
Parameters
point
: A generic value.boundaries_array
: A generic array of values.
Return Value
INT64
Examples
In a table called students
, check to see how many records would
exist in each age_group
bucket, based on a student's age:
- age_group 0 (age < 10)
- age_group 1 (age >= 10, age < 20)
- age_group 2 (age >= 20, age < 30)
- age_group 3 (age >= 30)
WITH students AS
(
SELECT 9 AS age UNION ALL
SELECT 20 AS age UNION ALL
SELECT 25 AS age UNION ALL
SELECT 31 AS age UNION ALL
SELECT 32 AS age UNION ALL
SELECT 33 AS age
)
SELECT RANGE_BUCKET(age, [10, 20, 30]) AS age_group, COUNT(*) AS count
FROM students
GROUP BY 1
+--------------+-------+
| age_group | count |
+--------------+-------+
| 0 | 1 |
| 2 | 2 |
| 3 | 3 |
+--------------+-------+
Navigation functions
The following sections describe the navigation functions that BigQuery supports. Navigation functions are a subset of analytic functions. For an explanation of how analytic functions work, see Analytic Function Concepts. For an explanation of how navigation functions work, see Navigation Function Concepts.
FIRST_VALUE
FIRST_VALUE (value_expression [{RESPECT | IGNORE} NULLS])
Description
Returns the value of the value_expression
for the first row in the current
window frame.
This function includes NULL
values in the calculation unless IGNORE NULLS
is
present. If IGNORE NULLS
is present, the function excludes NULL
values from
the calculation.
Supported Argument Types
value_expression
can be any data type that an expression can return.
Return Data Type
Same type as value_expression
.
Examples
The following example computes the fastest time for each division.
WITH finishers AS
(SELECT 'Sophia Liu' as name,
TIMESTAMP '2016-10-18 2:51:45' as finish_time,
'F30-34' as division
UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
FORMAT_TIMESTAMP('%X', finish_time) AS finish_time,
division,
FORMAT_TIMESTAMP('%X', fastest_time) AS fastest_time,
TIMESTAMP_DIFF(finish_time, fastest_time, SECOND) AS delta_in_seconds
FROM (
SELECT name,
finish_time,
division,
FIRST_VALUE(finish_time)
OVER (PARTITION BY division ORDER BY finish_time ASC
ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS fastest_time
FROM finishers);
+-----------------+-------------+----------+--------------+------------------+
| name | finish_time | division | fastest_time | delta_in_seconds |
+-----------------+-------------+----------+--------------+------------------+
| Carly Forte | 03:08:58 | F25-29 | 03:08:58 | 0 |
| Sophia Liu | 02:51:45 | F30-34 | 02:51:45 | 0 |
| Nikki Leith | 02:59:01 | F30-34 | 02:51:45 | 436 |
| Jen Edwards | 03:06:36 | F30-34 | 02:51:45 | 891 |
| Meghan Lederer | 03:07:41 | F30-34 | 02:51:45 | 956 |
| Lauren Reasoner | 03:10:14 | F30-34 | 02:51:45 | 1109 |
| Lisa Stelzner | 02:54:11 | F35-39 | 02:54:11 | 0 |
| Lauren Matthews | 03:01:17 | F35-39 | 02:54:11 | 426 |
| Desiree Berry | 03:05:42 | F35-39 | 02:54:11 | 691 |
| Suzy Slane | 03:06:24 | F35-39 | 02:54:11 | 733 |
+-----------------+-------------+----------+--------------+------------------+
LAST_VALUE
LAST_VALUE (value_expression [{RESPECT | IGNORE} NULLS])
Description
Returns the value of the value_expression
for the last row in the current
window frame.
This function includes NULL
values in the calculation unless IGNORE NULLS
is
present. If IGNORE NULLS
is present, the function excludes NULL
values from
the calculation.
Supported Argument Types
value_expression
can be any data type that an expression can return.
Return Data Type
Same type as value_expression
.
Examples
The following example computes the slowest time for each division.
WITH finishers AS
(SELECT 'Sophia Liu' as name,
TIMESTAMP '2016-10-18 2:51:45' as finish_time,
'F30-34' as division
UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
FORMAT_TIMESTAMP('%X', finish_time) AS finish_time,
division,
FORMAT_TIMESTAMP('%X', slowest_time) AS slowest_time,
TIMESTAMP_DIFF(slowest_time, finish_time, SECOND) AS delta_in_seconds
FROM (
SELECT name,
finish_time,
division,
LAST_VALUE(finish_time)
OVER (PARTITION BY division ORDER BY finish_time ASC
ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS slowest_time
FROM finishers);
+-----------------+-------------+----------+--------------+------------------+
| name | finish_time | division | slowest_time | delta_in_seconds |
+-----------------+-------------+----------+--------------+------------------+
| Carly Forte | 03:08:58 | F25-29 | 03:08:58 | 0 |
| Sophia Liu | 02:51:45 | F30-34 | 03:10:14 | 1109 |
| Nikki Leith | 02:59:01 | F30-34 | 03:10:14 | 673 |
| Jen Edwards | 03:06:36 | F30-34 | 03:10:14 | 218 |
| Meghan Lederer | 03:07:41 | F30-34 | 03:10:14 | 153 |
| Lauren Reasoner | 03:10:14 | F30-34 | 03:10:14 | 0 |
| Lisa Stelzner | 02:54:11 | F35-39 | 03:06:24 | 733 |
| Lauren Matthews | 03:01:17 | F35-39 | 03:06:24 | 307 |
| Desiree Berry | 03:05:42 | F35-39 | 03:06:24 | 42 |
| Suzy Slane | 03:06:24 | F35-39 | 03:06:24 | 0 |
+-----------------+-------------+----------+--------------+------------------+
NTH_VALUE
NTH_VALUE (value_expression, constant_integer_expression [{RESPECT | IGNORE} NULLS])
Description
Returns the value of value_expression
at the Nth row of the current window
frame, where Nth is defined by constant_integer_expression
. Returns NULL if
there is no such row.
This function includes NULL
values in the calculation unless IGNORE NULLS
is
present. If IGNORE NULLS
is present, the function excludes NULL
values from
the calculation.
Supported Argument Types
value_expression
can be any data type that can be returned from an expression.constant_integer_expression
can be any constant expression that returns an integer.
Return Data Type
Same type as value_expression
.
Examples
WITH finishers AS
(SELECT 'Sophia Liu' as name,
TIMESTAMP '2016-10-18 2:51:45' as finish_time,
'F30-34' as division
UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
FORMAT_TIMESTAMP('%X', finish_time) AS finish_time,
division,
FORMAT_TIMESTAMP('%X', fastest_time) AS fastest_time,
FORMAT_TIMESTAMP('%X', second_fastest) AS second_fastest
FROM (
SELECT name,
finish_time,
division,finishers,
FIRST_VALUE(finish_time)
OVER w1 AS fastest_time,
NTH_VALUE(finish_time, 2)
OVER w1 as second_fastest
FROM finishers
WINDOW w1 AS (
PARTITION BY division ORDER BY finish_time ASC
ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING));
+-----------------+-------------+----------+--------------+----------------+
| name | finish_time | division | fastest_time | second_fastest |
+-----------------+-------------+----------+--------------+----------------+
| Carly Forte | 03:08:58 | F25-29 | 03:08:58 | NULL |
| Sophia Liu | 02:51:45 | F30-34 | 02:51:45 | 02:59:01 |
| Nikki Leith | 02:59:01 | F30-34 | 02:51:45 | 02:59:01 |
| Jen Edwards | 03:06:36 | F30-34 | 02:51:45 | 02:59:01 |
| Meghan Lederer | 03:07:41 | F30-34 | 02:51:45 | 02:59:01 |
| Lauren Reasoner | 03:10:14 | F30-34 | 02:51:45 | 02:59:01 |
| Lisa Stelzner | 02:54:11 | F35-39 | 02:54:11 | 03:01:17 |
| Lauren Matthews | 03:01:17 | F35-39 | 02:54:11 | 03:01:17 |
| Desiree Berry | 03:05:42 | F35-39 | 02:54:11 | 03:01:17 |
| Suzy Slane | 03:06:24 | F35-39 | 02:54:11 | 03:01:17 |
+-----------------+-------------+----------+--------------+----------------+
LEAD
LEAD (value_expression[, offset [, default_expression]])
Description
Returns the value of the value_expression
on a subsequent row. Changing the
offset
value changes which subsequent row is returned; the default value is
1
, indicating the next row in the window frame. An error occurs if offset
is
NULL or a negative value.
The optional default_expression
is used if there isn't a row in the window
frame at the specified offset. This expression must be a constant expression and
its type must be implicitly coercible to the type of value_expression
. If left
unspecified, default_expression
defaults to NULL.
Supported Argument Types
value_expression
can be any data type that can be returned from an expression.offset
must be a non-negative integer literal or parameter.default_expression
must be compatible with the value expression type.
Return Data Type
Same type as value_expression
.
Examples
The following example illustrates a basic use of the LEAD
function.
WITH finishers AS
(SELECT 'Sophia Liu' as name,
TIMESTAMP '2016-10-18 2:51:45' as finish_time,
'F30-34' as division
UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
finish_time,
division,
LEAD(name)
OVER (PARTITION BY division ORDER BY finish_time ASC) AS followed_by
FROM finishers;
+-----------------+-------------+----------+-----------------+
| name | finish_time | division | followed_by |
+-----------------+-------------+----------+-----------------+
| Carly Forte | 03:08:58 | F25-29 | NULL |
| Sophia Liu | 02:51:45 | F30-34 | Nikki Leith |
| Nikki Leith | 02:59:01 | F30-34 | Jen Edwards |
| Jen Edwards | 03:06:36 | F30-34 | Meghan Lederer |
| Meghan Lederer | 03:07:41 | F30-34 | Lauren Reasoner |
| Lauren Reasoner | 03:10:14 | F30-34 | NULL |
| Lisa Stelzner | 02:54:11 | F35-39 | Lauren Matthews |
| Lauren Matthews | 03:01:17 | F35-39 | Desiree Berry |
| Desiree Berry | 03:05:42 | F35-39 | Suzy Slane |
| Suzy Slane | 03:06:24 | F35-39 | NULL |
+-----------------+-------------+----------+-----------------+
This next example uses the optional offset
parameter.
WITH finishers AS
(SELECT 'Sophia Liu' as name,
TIMESTAMP '2016-10-18 2:51:45' as finish_time,
'F30-34' as division
UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
finish_time,
division,
LEAD(name, 2)
OVER (PARTITION BY division ORDER BY finish_time ASC) AS two_runners_back
FROM finishers;
+-----------------+-------------+----------+------------------+
| name | finish_time | division | two_runners_back |
+-----------------+-------------+----------+------------------+
| Carly Forte | 03:08:58 | F25-29 | NULL |
| Sophia Liu | 02:51:45 | F30-34 | Jen Edwards |
| Nikki Leith | 02:59:01 | F30-34 | Meghan Lederer |
| Jen Edwards | 03:06:36 | F30-34 | Lauren Reasoner |
| Meghan Lederer | 03:07:41 | F30-34 | NULL |
| Lauren Reasoner | 03:10:14 | F30-34 | NULL |
| Lisa Stelzner | 02:54:11 | F35-39 | Desiree Berry |
| Lauren Matthews | 03:01:17 | F35-39 | Suzy Slane |
| Desiree Berry | 03:05:42 | F35-39 | NULL |
| Suzy Slane | 03:06:24 | F35-39 | NULL |
+-----------------+-------------+----------+------------------+
The following example replaces NULL values with a default value.
WITH finishers AS
(SELECT 'Sophia Liu' as name,
TIMESTAMP '2016-10-18 2:51:45' as finish_time,
'F30-34' as division
UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
finish_time,
division,
LEAD(name, 2, 'Nobody')
OVER (PARTITION BY division ORDER BY finish_time ASC) AS two_runners_back
FROM finishers;
+-----------------+-------------+----------+------------------+
| name | finish_time | division | two_runners_back |
+-----------------+-------------+----------+------------------+
| Carly Forte | 03:08:58 | F25-29 | Nobody |
| Sophia Liu | 02:51:45 | F30-34 | Jen Edwards |
| Nikki Leith | 02:59:01 | F30-34 | Meghan Lederer |
| Jen Edwards | 03:06:36 | F30-34 | Lauren Reasoner |
| Meghan Lederer | 03:07:41 | F30-34 | Nobody |
| Lauren Reasoner | 03:10:14 | F30-34 | Nobody |
| Lisa Stelzner | 02:54:11 | F35-39 | Desiree Berry |
| Lauren Matthews | 03:01:17 | F35-39 | Suzy Slane |
| Desiree Berry | 03:05:42 | F35-39 | Nobody |
| Suzy Slane | 03:06:24 | F35-39 | Nobody |
+-----------------+-------------+----------+------------------+
LAG
LAG (value_expression[, offset [, default_expression]])
Description
Returns the value of the value_expression
on a preceding row. Changing the
offset
value changes which preceding row is returned; the default value is
1
, indicating the previous row in the window frame. An error occurs if
offset
is NULL or a negative value.
The optional default_expression
is used if there isn't a row in the window
frame at the specified offset. This expression must be a constant expression and
its type must be implicitly coercible to the type of value_expression
. If left
unspecified, default_expression
defaults to NULL.
Supported Argument Types
value_expression
can be any data type that can be returned from an expression.offset
must be a non-negative integer literal or parameter.default_expression
must be compatible with the value expression type.
Return Data Type
Same type as value_expression
.
Examples
The following example illustrates a basic use of the LAG
function.
WITH finishers AS
(SELECT 'Sophia Liu' as name,
TIMESTAMP '2016-10-18 2:51:45' as finish_time,
'F30-34' as division
UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
finish_time,
division,
LAG(name)
OVER (PARTITION BY division ORDER BY finish_time ASC) AS preceding_runner
FROM finishers;
+-----------------+-------------+----------+------------------+
| name | finish_time | division | preceding_runner |
+-----------------+-------------+----------+------------------+
| Carly Forte | 03:08:58 | F25-29 | NULL |
| Sophia Liu | 02:51:45 | F30-34 | NULL |
| Nikki Leith | 02:59:01 | F30-34 | Sophia Liu |
| Jen Edwards | 03:06:36 | F30-34 | Nikki Leith |
| Meghan Lederer | 03:07:41 | F30-34 | Jen Edwards |
| Lauren Reasoner | 03:10:14 | F30-34 | Meghan Lederer |
| Lisa Stelzner | 02:54:11 | F35-39 | NULL |
| Lauren Matthews | 03:01:17 | F35-39 | Lisa Stelzner |
| Desiree Berry | 03:05:42 | F35-39 | Lauren Matthews |
| Suzy Slane | 03:06:24 | F35-39 | Desiree Berry |
+-----------------+-------------+----------+------------------+
This next example uses the optional offset
parameter.
WITH finishers AS
(SELECT 'Sophia Liu' as name,
TIMESTAMP '2016-10-18 2:51:45' as finish_time,
'F30-34' as division
UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
finish_time,
division,
LAG(name, 2)
OVER (PARTITION BY division ORDER BY finish_time ASC) AS two_runners_ahead
FROM finishers;
+-----------------+-------------+----------+-------------------+
| name | finish_time | division | two_runners_ahead |
+-----------------+-------------+----------+-------------------+
| Carly Forte | 03:08:58 | F25-29 | NULL |
| Sophia Liu | 02:51:45 | F30-34 | NULL |
| Nikki Leith | 02:59:01 | F30-34 | NULL |
| Jen Edwards | 03:06:36 | F30-34 | Sophia Liu |
| Meghan Lederer | 03:07:41 | F30-34 | Nikki Leith |
| Lauren Reasoner | 03:10:14 | F30-34 | Jen Edwards |
| Lisa Stelzner | 02:54:11 | F35-39 | NULL |
| Lauren Matthews | 03:01:17 | F35-39 | NULL |
| Desiree Berry | 03:05:42 | F35-39 | Lisa Stelzner |
| Suzy Slane | 03:06:24 | F35-39 | Lauren Matthews |
+-----------------+-------------+----------+-------------------+
The following example replaces NULL values with a default value.
WITH finishers AS
(SELECT 'Sophia Liu' as name,
TIMESTAMP '2016-10-18 2:51:45' as finish_time,
'F30-34' as division
UNION ALL SELECT 'Lisa Stelzner', TIMESTAMP '2016-10-18 2:54:11', 'F35-39'
UNION ALL SELECT 'Nikki Leith', TIMESTAMP '2016-10-18 2:59:01', 'F30-34'
UNION ALL SELECT 'Lauren Matthews', TIMESTAMP '2016-10-18 3:01:17', 'F35-39'
UNION ALL SELECT 'Desiree Berry', TIMESTAMP '2016-10-18 3:05:42', 'F35-39'
UNION ALL SELECT 'Suzy Slane', TIMESTAMP '2016-10-18 3:06:24', 'F35-39'
UNION ALL SELECT 'Jen Edwards', TIMESTAMP '2016-10-18 3:06:36', 'F30-34'
UNION ALL SELECT 'Meghan Lederer', TIMESTAMP '2016-10-18 3:07:41', 'F30-34'
UNION ALL SELECT 'Carly Forte', TIMESTAMP '2016-10-18 3:08:58', 'F25-29'
UNION ALL SELECT 'Lauren Reasoner', TIMESTAMP '2016-10-18 3:10:14', 'F30-34')
SELECT name,
finish_time,
division,
LAG(name, 2, 'Nobody')
OVER (PARTITION BY division ORDER BY finish_time ASC) AS two_runners_ahead
FROM finishers;
+-----------------+-------------+----------+-------------------+
| name | finish_time | division | two_runners_ahead |
+-----------------+-------------+----------+-------------------+
| Carly Forte | 03:08:58 | F25-29 | Nobody |
| Sophia Liu | 02:51:45 | F30-34 | Nobody |
| Nikki Leith | 02:59:01 | F30-34 | Nobody |
| Jen Edwards | 03:06:36 | F30-34 | Sophia Liu |
| Meghan Lederer | 03:07:41 | F30-34 | Nikki Leith |
| Lauren Reasoner | 03:10:14 | F30-34 | Jen Edwards |
| Lisa Stelzner | 02:54:11 | F35-39 | Nobody |
| Lauren Matthews | 03:01:17 | F35-39 | Nobody |
| Desiree Berry | 03:05:42 | F35-39 | Lisa Stelzner |
| Suzy Slane | 03:06:24 | F35-39 | Lauren Matthews |
+-----------------+-------------+----------+-------------------+
PERCENTILE_CONT
PERCENTILE_CONT (value_expression, percentile [{RESPECT | IGNORE} NULLS])
Description
Computes the specified percentile value for the value_expression, with linear interpolation.
This function ignores NULL values if RESPECT NULLS
is absent. If RESPECT
NULLS
is present:
- Interpolation between two
NULL
values returnsNULL
. - Interpolation between a
NULL
value and a non-NULL
value returns the non-NULL
value.
Supported Argument Types
value_expression
andpercentile
must have one of the following types:NUMERIC
BIGNUMERIC
FLOAT64
percentile
must be a literal in the range[0, 1]
.
Return Data Type
The return data type is determined by the argument types with the following table.
INPUT | NUMERIC | BIGNUMERIC | FLOAT64 |
---|---|---|---|
NUMERIC | NUMERIC | BIGNUMERIC | FLOAT64 |
BIGNUMERIC | BIGNUMERIC | BIGNUMERIC | FLOAT64 |
FLOAT64 | FLOAT64 | FLOAT64 | FLOAT64 |
Examples
The following example computes the value for some percentiles from a column of values while ignoring nulls.
SELECT
PERCENTILE_CONT(x, 0) OVER() AS min,
PERCENTILE_CONT(x, 0.01) OVER() AS percentile1,
PERCENTILE_CONT(x, 0.5) OVER() AS median,
PERCENTILE_CONT(x, 0.9) OVER() AS percentile90,
PERCENTILE_CONT(x, 1) OVER() AS max
FROM UNNEST([0, 3, NULL, 1, 2]) AS x LIMIT 1;
+-----+-------------+--------+--------------+-----+
| min | percentile1 | median | percentile90 | max |
+-----+-------------+--------+--------------+-----+
| 0 | 0.03 | 1.5 | 2.7 | 3 |
+-----+-------------+--------+--------------+-----+
The following example computes the value for some percentiles from a column of values while respecting nulls.
SELECT
PERCENTILE_CONT(x, 0 RESPECT NULLS) OVER() AS min,
PERCENTILE_CONT(x, 0.01 RESPECT NULLS) OVER() AS percentile1,
PERCENTILE_CONT(x, 0.5 RESPECT NULLS) OVER() AS median,
PERCENTILE_CONT(x, 0.9 RESPECT NULLS) OVER() AS percentile90,
PERCENTILE_CONT(x, 1 RESPECT NULLS) OVER() AS max
FROM UNNEST([0, 3, NULL, 1, 2]) AS x LIMIT 1;
+------+-------------+--------+--------------+-----+
| min | percentile1 | median | percentile90 | max |
+------+-------------+--------+--------------+-----+
| NULL | 0 | 1 | 2.6 | 3 |
+------+-------------+--------+--------------+-----+
PERCENTILE_DISC
PERCENTILE_DISC (value_expression, percentile [{RESPECT | IGNORE} NULLS])
Description
Computes the specified percentile value for a discrete value_expression
. The
returned value is the first sorted value of value_expression
with cumulative
distribution greater than or equal to the given percentile
value.
This function ignores NULL
values unless RESPECT NULLS
is present.
Supported Argument Types
value_expression
can be any orderable type.percentile
must be a literal in the range[0, 1]
, with one of the following types:NUMERIC
BIGNUMERIC
FLOAT64
Return Data Type
Same type as value_expression
.
Examples
The following example computes the value for some percentiles from a column of values while ignoring nulls.
SELECT
x,
PERCENTILE_DISC(x, 0) OVER() AS min,
PERCENTILE_DISC(x, 0.5) OVER() AS median,
PERCENTILE_DISC(x, 1) OVER() AS max
FROM UNNEST(['c', NULL, 'b', 'a']) AS x;
+------+-----+--------+-----+
| x | min | median | max |
+------+-----+--------+-----+
| c | a | b | c |
| NULL | a | b | c |
| b | a | b | c |
| a | a | b | c |
+------+-----+--------+-----+
The following example computes the value for some percentiles from a column of values while respecting nulls.
SELECT
x,
PERCENTILE_DISC(x, 0 RESPECT NULLS) OVER() AS min,
PERCENTILE_DISC(x, 0.5 RESPECT NULLS) OVER() AS median,
PERCENTILE_DISC(x, 1 RESPECT NULLS) OVER() AS max
FROM UNNEST(['c', NULL, 'b', 'a']) AS x;
+------+------+--------+-----+
| x | min | median | max |
+------+------+--------+-----+
| c | NULL | a | c |
| NULL | NULL | a | c |
| b | NULL | a | c |
| a | NULL | a | c |
+------+------+--------+-----+
Aggregate analytic functions
The following sections describe the aggregate analytic functions that BigQuery supports. For an explanation of how analytic functions work, see Analytic Function Concepts. For an explanation of how aggregate analytic functions work, see Aggregate Analytic Function Concepts.
BigQuery supports the following aggregate functions as analytic functions:
- ANY_VALUE
- ARRAY_AGG
- AVG
- CORR
- COUNT
- COUNTIF
- COVAR_POP
- COVAR_SAMP
- MAX
- MIN
- ST_CLUSTERDBSCAN
- STDDEV_POP
- STDDEV_SAMP
- STRING_AGG
- SUM
- VAR_POP
- VAR_SAMP
OVER
clause requirements:
PARTITION BY
: Optional.ORDER BY
: Optional. Disallowed ifDISTINCT
is present.window_frame_clause
: Optional. Disallowed ifDISTINCT
is present.
Example:
COUNT(*) OVER (ROWS UNBOUNDED PRECEDING)
SUM(DISTINCT x) OVER ()
Hash functions
FARM_FINGERPRINT
FARM_FINGERPRINT(value)
Description
Computes the fingerprint of the STRING
or BYTES
input using the
Fingerprint64
function from the
open-source FarmHash library. The output
of this function for a particular input will never change.
Return type
INT64
Examples
WITH example AS (
SELECT 1 AS x, "foo" AS y, true AS z UNION ALL
SELECT 2 AS x, "apple" AS y, false AS z UNION ALL
SELECT 3 AS x, "" AS y, true AS z
)
SELECT
*,
FARM_FINGERPRINT(CONCAT(CAST(x AS STRING), y, CAST(z AS STRING)))
AS row_fingerprint
FROM example;
+---+-------+-------+----------------------+
| x | y | z | row_fingerprint |
+---+-------+-------+----------------------+
| 1 | foo | true | -1541654101129638711 |
| 2 | apple | false | 2794438866806483259 |
| 3 | | true | -4880158226897771312 |
+---+-------+-------+----------------------+
MD5
MD5(input)
Description
Computes the hash of the input using the
MD5 algorithm. The input can either be
STRING
or BYTES
. The string version treats the input as an array of bytes.
This function returns 16 bytes.
Return type
BYTES
Example
SELECT MD5("Hello World") as md5;
-- Note that the result of MD5 is of type BYTES, displayed as a base64-encoded string.
+--------------------------+
| md5 |
+--------------------------+
| sQqNsWTgdUEFt6mb5y4/5Q== |
+--------------------------+
SHA1
SHA1(input)
Description
Computes the hash of the input using the
SHA-1 algorithm. The input can either be
STRING
or BYTES
. The string version treats the input as an array of bytes.
This function returns 20 bytes.
Return type
BYTES
Example
SELECT SHA1("Hello World") as sha1;
-- Note that the result of SHA1 is of type BYTES, displayed as a base64-encoded string.
+------------------------------+
| sha1 |
+------------------------------+
| Ck1VqNd45QIvq3AZd8XYQLvEhtA= |
+------------------------------+
SHA256
SHA256(input)
Description
Computes the hash of the input using the
SHA-256 algorithm. The input can either be
STRING
or BYTES
. The string version treats the input as an array of bytes.
This function returns 32 bytes.
Return type
BYTES
Example
SELECT SHA256("Hello World") as sha256;
SHA512
SHA512(input)
Description
Computes the hash of the input using the
SHA-512 algorithm. The input can either be
STRING
or BYTES
. The string version treats the input as an array of bytes.
This function returns 64 bytes.
Return type
BYTES
Example
SELECT SHA512("Hello World") as sha512;
String functions
These string functions work on two different values:
STRING
and BYTES
data types. STRING
values must be well-formed UTF-8.
Functions that return position values, such as STRPOS,
encode those positions as INT64
. The value 1
refers to the first character (or byte), 2
refers to the second, and so on.
The value 0
indicates an invalid index. When working on STRING
types, the
returned positions refer to character positions.
All string comparisons are done byte-by-byte, without regard to Unicode canonical equivalence.
ASCII
ASCII(value)
Description
Returns the ASCII code for the first character or byte in value
. Returns
0
if value
is empty or the ASCII code is 0
for the first character
or byte.
Return type
INT64
Examples
SELECT ASCII('abcd') as A, ASCII('a') as B, ASCII('') as C, ASCII(NULL) as D;
+-------+-------+-------+-------+
| A | B | C | D |
+-------+-------+-------+-------+
| 97 | 97 | 0 | NULL |
+-------+-------+-------+-------+
BYTE_LENGTH
BYTE_LENGTH(value)
Description
Returns the length of the STRING
or BYTES
value in BYTES
,
regardless of whether the type of the value is STRING
or BYTES
.
Return type
INT64
Examples
WITH example AS
(SELECT "абвгд" AS characters, b"абвгд" AS bytes)
SELECT
characters,
BYTE_LENGTH(characters) AS string_example,
bytes,
BYTE_LENGTH(bytes) AS bytes_example
FROM example;
+------------+----------------+-------+---------------+
| characters | string_example | bytes | bytes_example |
+------------+----------------+-------+---------------+
| абвгд | 10 | абвгд | 10 |
+------------+----------------+-------+---------------+
CHAR_LENGTH
CHAR_LENGTH(value)
Description
Returns the length of the STRING
in characters.
Return type
INT64
Examples
WITH example AS
(SELECT "абвгд" AS characters)
SELECT
characters,
CHAR_LENGTH(characters) AS char_length_example
FROM example;
+------------+---------------------+
| characters | char_length_example |
+------------+---------------------+
| абвгд | 5 |
+------------+---------------------+
CHARACTER_LENGTH
CHARACTER_LENGTH(value)
Description
Synonym for CHAR_LENGTH.
Return type
INT64
Examples
WITH example AS
(SELECT "абвгд" AS characters)
SELECT
characters,
CHARACTER_LENGTH(characters) AS char_length_example
FROM example;
+------------+---------------------+
| characters | char_length_example |
+------------+---------------------+
| абвгд | 5 |
+------------+---------------------+
CHR
CHR(value)
Description
Takes a Unicode code point and returns
the character that matches the code point. Each valid code point should fall
within the range of [0, 0xD7FF] and [0xE000, 0x10FFFF]. Returns an empty string
if the code point is 0
. If an invalid Unicode code point is specified, an
error is returned.
To work with an array of Unicode code points, see
CODE_POINTS_TO_STRING
Return type
STRING
Examples
SELECT CHR(65) AS A, CHR(255) AS B, CHR(513) AS C, CHR(1024) AS D;
+-------+-------+-------+-------+
| A | B | C | D |
+-------+-------+-------+-------+
| A | ÿ | ȁ | Ѐ |
+-------+-------+-------+-------+
SELECT CHR(97) AS A, CHR(0xF9B5) AS B, CHR(0) AS C, CHR(NULL) AS D;
+-------+-------+-------+-------+
| A | B | C | D |
+-------+-------+-------+-------+
| a | 例 | | NULL |
+-------+-------+-------+-------+
CODE_POINTS_TO_BYTES
CODE_POINTS_TO_BYTES(ascii_values)
Description
Takes an array of extended ASCII
code points
(ARRAY
of INT64
) and returns BYTES
.
To convert from BYTES
to an array of code points, see
TO_CODE_POINTS.
Return type
BYTES
Examples
The following is a basic example using CODE_POINTS_TO_BYTES
.
SELECT CODE_POINTS_TO_BYTES([65, 98, 67, 100]) AS bytes;
-- Note that the result of CODE_POINTS_TO_BYTES is of type BYTES, displayed as a base64-encoded string.
-- In BYTES format, b'AbCd' is the result.
+----------+
| bytes |
+----------+
| QWJDZA== |
+----------+
The following example uses a rotate-by-13 places (ROT13) algorithm to encode a string.
SELECT CODE_POINTS_TO_BYTES(ARRAY_AGG(
(SELECT
CASE
WHEN chr BETWEEN b'a' and b'z'
THEN TO_CODE_POINTS(b'a')[offset(0)] +
MOD(code+13-TO_CODE_POINTS(b'a')[offset(0)],26)
WHEN chr BETWEEN b'A' and b'Z'
THEN TO_CODE_POINTS(b'A')[offset(0)] +
MOD(code+13-TO_CODE_POINTS(b'A')[offset(0)],26)
ELSE code
END
FROM
(SELECT code, CODE_POINTS_TO_BYTES([code]) chr)
) ORDER BY OFFSET)) AS encoded_string
FROM UNNEST(TO_CODE_POINTS(b'Test String!')) code WITH OFFSET;
-- Note that the result of CODE_POINTS_TO_BYTES is of type BYTES, displayed as a base64-encoded string.
-- In BYTES format, b'Grfg Fgevat!' is the result.
+------------------+
| encoded_string |
+------------------+
| R3JmZyBGZ2V2YXQh |
+------------------+
CODE_POINTS_TO_STRING
CODE_POINTS_TO_STRING(value)
Description
Takes an array of Unicode code points
(ARRAY
of INT64
) and
returns a STRING
. If a code point is 0, does not return a character for it
in the STRING
.
To convert from a string to an array of code points, see TO_CODE_POINTS.
Return type
STRING
Examples
The following are basic examples using CODE_POINTS_TO_STRING
.
SELECT CODE_POINTS_TO_STRING([65, 255, 513, 1024]) AS string;
+--------+
| string |
+--------+
| AÿȁЀ |
+--------+
SELECT CODE_POINTS_TO_STRING([97, 0, 0xF9B5]) AS string;
+--------+
| string |
+--------+
| a例 |
+--------+
SELECT CODE_POINTS_TO_STRING([65, 255, NULL, 1024]) AS string;
+--------+
| string |
+--------+
| NULL |
+--------+
The following example computes the frequency of letters in a set of words.
WITH Words AS (
SELECT word
FROM UNNEST(['foo', 'bar', 'baz', 'giraffe', 'llama']) AS word
)
SELECT
CODE_POINTS_TO_STRING([code_point]) AS letter,
COUNT(*) AS letter_count
FROM Words,
UNNEST(TO_CODE_POINTS(word)) AS code_point
GROUP BY 1
ORDER BY 2 DESC;
+--------+--------------+
| letter | letter_count |
+--------+--------------+
| a | 5 |
| f | 3 |
| r | 2 |
| b | 2 |
| l | 2 |
| o | 2 |
| g | 1 |
| z | 1 |
| e | 1 |
| m | 1 |
| i | 1 |
+--------+--------------+
CONCAT
CONCAT(value1[, ...])
Description
Concatenates one or more values into a single result. All values must be
BYTES
or data types that can be cast to STRING
.
The function returns NULL
if any input argument is NULL
.
Return type
STRING
or BYTES
Examples
SELECT CONCAT("T.P.", " ", "Bar") as author;
+---------------------+
| author |
+---------------------+
| T.P. Bar |
+---------------------+
SELECT CONCAT("Summer", " ", 1923) as release_date;
+---------------------+
| release_date |
+---------------------+
| Summer 1923 |
+---------------------+
With Employees AS
(SELECT
"John" AS first_name,
"Doe" AS last_name
UNION ALL
SELECT
"Jane" AS first_name,
"Smith" AS last_name
UNION ALL
SELECT
"Joe" AS first_name,
"Jackson" AS last_name)
SELECT
CONCAT(first_name, " ", last_name)
AS full_name
FROM Employees;
+---------------------+
| full_name |
+---------------------+
| John Doe |
| Jane Smith |
| Joe Jackson |
+---------------------+
ENDS_WITH
ENDS_WITH(value1, value2)
Description
Takes two STRING
or BYTES
values. Returns TRUE
if the second
value is a suffix of the first.
Return type
BOOL
Examples
WITH items AS
(SELECT "apple" as item
UNION ALL
SELECT "banana" as item
UNION ALL
SELECT "orange" as item)
SELECT
ENDS_WITH(item, "e") as example
FROM items;
+---------+
| example |
+---------+
| True |
| False |
| True |
+---------+
FORMAT
BigQuery supports a FORMAT()
function for formatting strings. This
function is similar to the C printf
function. It produces a STRING
from a
format string that contains zero or more format specifiers, along with a
variable length list of additional arguments that matches the format specifiers.
Here are some examples:
Description | Statement | Result |
---|---|---|
Simple integer | FORMAT("%d", 10) | 10 |
Integer with left blank padding | FORMAT("|%10d|", 11) | | 11| |
Integer with left zero padding | FORMAT("+%010d+", 12) | +0000000012+ |
Integer with commas | FORMAT("%'d", 123456789) | 123,456,789 |
STRING | FORMAT("-%s-", 'abcd efg') | -abcd efg- |
FLOAT64 | FORMAT("%f %E", 1.1, 2.2) | 1.100000 2.200000E+00 |
DATE | FORMAT("%t", date "2015-09-01") | 2015-09-01 |
TIMESTAMP | FORMAT("%t", timestamp "2015-09-01 12:34:56 America/Los_Angeles") | 2015‑09‑01 19:34:56+00 |
The FORMAT()
function does not provide fully customizable formatting for all
types and values, nor formatting that is sensitive to locale.
If custom formatting is necessary for a type, you must first format it using
type-specific format functions, such as FORMAT_DATE()
or FORMAT_TIMESTAMP()
.
For example:
SELECT FORMAT("date: %s!", FORMAT_DATE("%B %d, %Y", date '2015-01-02'));
Returns
date: January 02, 2015!
Syntax
The FORMAT()
syntax takes a format string and variable length list of
arguments and produces a STRING
result:
FORMAT(format_string, ...)
The format_string
expression can contain zero or more format specifiers.
Each format specifier is introduced by the %
symbol, and must map to one or
more of the remaining arguments. For the most part, this is a one-to-one
mapping, except when the *
specifier is present. For example, %.*i
maps to
two arguments—a length argument and a signed integer argument. If the
number of arguments related to the format specifiers is not the same as the
number of arguments, an error occurs.
Supported format specifiers
The FORMAT()
function format specifier follows this prototype:
%[flags][width][.precision]specifier
The supported format specifiers are identified in the following table. Deviations from printf() are identified in italics.
Specifier | Description | Examples | Types |
d or i |
Decimal integer | 392 |
INT64 |
o |
Octal | 610 |
INT64* |
x |
Hexadecimal integer | 7fa |
INT64* |
X |
Hexadecimal integer (uppercase) | 7FA |
INT64* |
f |
Decimal notation, in [-](integer part).(fractional part) for finite values, and in lowercase for non-finite values | 392.650000 inf nan |
NUMERIC BIGNUMERIC FLOAT64 |
F |
Decimal notation, in [-](integer part).(fractional part) for finite values, and in uppercase for non-finite values | 392.650000 INF NAN |
NUMERIC BIGNUMERIC FLOAT64 |
e |
Scientific notation (mantissa/exponent), lowercase | 3.926500e+02 inf nan |
NUMERIC BIGNUMERIC FLOAT64 |
E |
Scientific notation (mantissa/exponent), uppercase | 3.926500E+02 INF NAN |
NUMERIC BIGNUMERIC FLOAT64 |
g |
Either decimal notation or scientific notation, depending on the input value's exponent and the specified precision. Lowercase. See %g and %G behavior for details. | 392.65 3.9265e+07 inf nan |
NUMERIC BIGNUMERIC FLOAT64 |
G |
Either decimal notation or scientific notation, depending on the input value's exponent and the specified precision. Uppercase. See %g and %G behavior for details. |
392.65 3.9265E+07 INF NAN |
NUMERIC BIGNUMERIC FLOAT64 |
s |
String of characters | sample | STRING |
t |
Returns a printable string representing the value. Often looks
similar to casting the argument to STRING .
See %t and %T behavior.
|
sample 2014‑01‑01 |
<any> |
T |
Produces a string that is a valid BigQuery constant with a similar type to the value's type (maybe wider, or maybe string). See %t and %T behavior. |
'sample' b'bytes sample' 1234 2.3 date '2014‑01‑01' |
<any> |
% |
'%%' produces a single '%' | % | n/a |
*The specifiers %o
, %x
, and %X
raise an
error if negative values are used.
The format specifier can optionally contain the sub-specifiers identified above in the specifier prototype.
These sub-specifiers must comply with the following specifications.
Flags
Flags | Description |
- |
Left-justify within the given field width; Right justification is the default (see width sub-specifier) |
+ |
Forces to precede the result with a plus or minus sign (+
or - ) even for positive numbers. By default, only negative numbers
are preceded with a - sign |
<space> | If no sign is going to be written, a blank space is inserted before the value |
# |
|
0 |
Left-pads the number with zeroes (0) instead of spaces when padding is specified (see width sub-specifier) |
' |
Formats integers using the appropriating grouping character. For example:
This flag is only relevant for decimal, hex, and octal values. |
Flags may be specified in any order. Duplicate flags are not an error. When flags are not relevant for some element type, they are ignored.
Width
Width | Description |
<number> | Minimum number of characters to be printed. If the value to be printed is shorter than this number, the result is padded with blank spaces. The value is not truncated even if the result is larger |
* |
The width is not specified in the format string, but as an additional integer value argument preceding the argument that has to be formatted |
Precision
Precision | Description |
. <number> |
|
.* |
The precision is not specified in the format string, but as an additional integer value argument preceding the argument that has to be formatted |
%g and %G behavior
The %g
and %G
format specifiers choose either the decimal notation (like
the %f
and %F
specifiers) or the scientific notation (like the %e
and %E
specifiers), depending on the input value's exponent and the specified
precision.
Let p stand for the specified precision (defaults to 6; 1 if the specified precision is less than 1). The input value is first converted to scientific notation with precision = (p - 1). If the resulting exponent part x is less than -4 or no less than p, the scientific notation with precision = (p - 1) is used; otherwise the decimal notation with precision = (p - 1 - x) is used.
Unless #
flag is present, the trailing zeros after the decimal point
are removed, and the decimal point is also removed if there is no digit after
it.
%t and %T behavior
The %t
and %T
format specifiers are defined for all types. The
width, precision, and flags act as they do
for %s
: the width is the minimum width and the STRING
will be
padded to that size, and precision is the maximum width
of content to show and the STRING
will be truncated to that size, prior to
padding to width.
The %t
specifier is always meant to be a readable form of the value.
The %T
specifier is always a valid SQL literal of a similar type, such as a
wider numeric type.
The literal will not include casts or a type name, except for the special case
of non-finite floating point values.
The STRING
is formatted as follows:
Type | %t | %T |
NULL of any type |
NULL | NULL |
INT64 |
123 | 123 |
NUMERIC | 123.0 (always with .0) | NUMERIC "123.0" |
FLOAT64 |
123.0 (always with .0) 123e+10 inf -inf NaN
|
123.0 (always with .0) 123e+10 CAST("inf" AS <type>) CAST("-inf" AS <type>) CAST("nan" AS <type>) |
STRING | unquoted string value | quoted string literal |
BYTES |
unquoted escaped bytes e.g. abc\x01\x02 |
quoted bytes literal e.g. b"abc\x01\x02" |
DATE | 2011-02-03 | DATE "2011-02-03" |
TIMESTAMP | 2011-02-03 04:05:06+00 | TIMESTAMP "2011-02-03 04:05:06+00" |
ARRAY | [value, value, ...] where values are formatted with %t |
[value, value, ...] where values are formatted with %T |
STRUCT | (value, value, ...) where fields are formatted with %t |
(value, value, ...) where fields are formatted with %T Special cases: Zero fields: STRUCT() One field: STRUCT(value) |
Error conditions
If a format specifier is invalid, or is not compatible with the related
argument type, or the wrong number or arguments are provided, then an error is
produced. For example, the following <format_string>
expressions are invalid:
FORMAT('%s', 1)
FORMAT('%')
NULL argument handling
A NULL
format string results in a NULL
output STRING
. Any other arguments
are ignored in this case.
The function generally produces a NULL
value if a NULL
argument is present.
For example, FORMAT('%i', NULL_expression)
produces a NULL STRING
as
output.
However, there are some exceptions: if the format specifier is %t or %T
(both of which produce STRING
s that effectively match CAST and literal value
semantics), a NULL
value produces 'NULL' (without the quotes) in the result
STRING
. For example, the function:
FORMAT('00-%t-00', NULL_expression);
Returns
00-NULL-00
Additional semantic rules
FLOAT64
values can be +/-inf
or NaN
.
When an argument has one of those values, the result of the format specifiers
%f
, %F
, %e
, %E
, %g
, %G
, and %t
are inf
, -inf
, or nan
(or the same in uppercase) as appropriate. This is consistent with how
BigQuery casts these values to STRING
. For %T
,
BigQuery returns quoted strings for
FLOAT64
values that don't have non-string literal
representations.
FROM_BASE32
FROM_BASE32(string_expr)
Description
Converts the base32-encoded input string_expr
into BYTES
format. To convert
BYTES
to a base32-encoded STRING
, use TO_BASE32.
Return type
BYTES
Example
SELECT FROM_BASE32('MFRGGZDF74======') AS byte_data;
-- Note that the result of FROM_BASE32 is of type BYTES, displayed as a base64-encoded string.
+-----------+
| byte_data |
+-----------+
| YWJjZGX/ |
+-----------+
FROM_BASE64
FROM_BASE64(string_expr)
Description
Converts the base64-encoded input string_expr
into
BYTES
format. To convert
BYTES
to a base64-encoded STRING
,
use TO_BASE64.
There are several base64 encodings in common use that vary in exactly which
alphabet of 65 ASCII characters are used to encode the 64 digits and padding.
See RFC 4648 for details. This
function expects the alphabet [A-Za-z0-9+/=]
.
Return type
BYTES
Example
SELECT FROM_BASE64('/+A=') AS byte_data;
-- Note that the result of FROM_BASE64 is of type BYTES, displayed as a base64-encoded string.
+-----------+
| byte_data |
+-----------+
| /+A= |
+-----------+
To work with an encoding using a different base64 alphabet, you might need to
compose FROM_BASE64
with the REPLACE
function. For instance, the
base64url
url-safe and filename-safe encoding commonly used in web programming
uses -_=
as the last characters rather than +/=
. To decode a
base64url
-encoded string, replace +
and /
with -
and _
respectively.
SELECT FROM_BASE64(REPLACE(REPLACE("_-A=", "-", "+"), "_", "/")) AS binary;
-- Note that the result of FROM_BASE64 is of type BYTES, displayed as a base64-encoded string.
+--------+
| binary |
+--------+
| /+A= |
+--------+
FROM_HEX
FROM_HEX(string)
Description
Converts a hexadecimal-encoded STRING
into BYTES
format. Returns an error
if the input STRING
contains characters outside the range
(0..9, A..F, a..f)
. The lettercase of the characters does not matter. If the
input STRING
has an odd number of characters, the function acts as if the
input has an additional leading 0
. To convert BYTES
to a hexadecimal-encoded
STRING
, use TO_HEX.
Return type
BYTES
Example
WITH Input AS (
SELECT '00010203aaeeefff' AS hex_str UNION ALL
SELECT '0AF' UNION ALL
SELECT '666f6f626172'
)
SELECT hex_str, FROM_HEX(hex_str) AS bytes_str
FROM Input;
-- Note that the result of FROM_HEX is of type BYTES, displayed as a base64-encoded string.
+------------------+--------------+
| hex_str | bytes_str |
+------------------+--------------+
| 0AF | AAECA6ru7/8= |
| 00010203aaeeefff | AK8= |
| 666f6f626172 | Zm9vYmFy |
+------------------+--------------+
INITCAP
INITCAP(value[, delimiters])
Description
Takes a STRING
and returns it with the first character in each word in
uppercase and all other characters in lowercase. Non-alphabetic characters
remain the same.
delimiters
is an optional string argument that is used to override the default
set of characters used to separate words. If delimiters
is not specified, it
defaults to the following characters:
<whitespace> [ ] ( ) { } / | \ < > ! ? @ " ^ # $ & ~ _ , . : ; * % + -
If value
or delimiters
is NULL
, the function returns NULL
.
Return type
STRING
Examples
WITH example AS
(
SELECT "Hello World-everyone!" AS value UNION ALL
SELECT "tHe dog BARKS loudly+friendly" AS value UNION ALL
SELECT "apples&oranges;&pears" AS value UNION ALL
SELECT "καθίσματα ταινιών" AS value
)
SELECT value, INITCAP(value) AS initcap_value FROM example
+-------------------------------+-------------------------------+
| value | initcap_value |
+-------------------------------+-------------------------------+
| Hello World-everyone! | Hello World-Everyone! |
| tHe dog BARKS loudly+friendly | The Dog Barks Loudly+Friendly |
| apples&oranges;&pears | Apples&Oranges;&Pears |
| καθίσματα ταινιών | Καθίσματα Ταινιών |
+-------------------------------+-------------------------------+
WITH example AS
(
SELECT "hello WORLD!" AS value, "" AS delimiters UNION ALL
SELECT "καθίσματα ταιντιώ@ν" AS value, "τ@" AS delimiters UNION ALL
SELECT "Apples1oranges2pears" AS value, "12" AS delimiters UNION ALL
SELECT "tHisEisEaESentence" AS value, "E" AS delimiters
)
SELECT value, delimiters, INITCAP(value, delimiters) AS initcap_value FROM example;
+----------------------+------------+----------------------+
| value | delimiters | initcap_value |
+----------------------+------------+----------------------+
| hello WORLD! | | Hello world! |
| καθίσματα ταιντιώ@ν | τ@ | ΚαθίσματΑ τΑιντΙώ@Ν |
| Apples1oranges2pears | 12 | Apples1Oranges2Pears |
| tHisEisEaESentence | E | ThisEIsEAESentence |
+----------------------+------------+----------------------+
INSTR
INSTR(source_value, search_value[, position[, occurrence]])
Description
Returns the lowest 1-based index of search_value
in source_value
. 0 is
returned when no match is found. source_value
and search_value
must be the
same type, either STRING
or BYTES
.
If position
is specified, the search starts at this position in
source_value
, otherwise it starts at the beginning of source_value
. If
position
is negative, the function searches backwards from the end of
source_value
, with -1 indicating the last character. position
cannot be 0.
If occurrence
is specified, the search returns the position of a specific
instance of search_value
in source_value
, otherwise it returns the index of
the first occurrence. If occurrence
is greater than the number of matches
found, 0 is returned. For occurrence
> 1, the function searches for
overlapping occurrences, in other words, the function searches for additional
occurrences beginning with the second character in the previous occurrence.
occurrence
cannot be 0 or negative.
Return type
INT64
Examples
WITH example AS
(SELECT 'banana' as source_value, 'an' as search_value, 1 as position, 1 as
occurrence UNION ALL
SELECT 'banana' as source_value, 'an' as search_value, 1 as position, 2 as
occurrence UNION ALL
SELECT 'banana' as source_value, 'an' as search_value, 1 as position, 3 as
occurrence UNION ALL
SELECT 'banana' as source_value, 'an' as search_value, 3 as position, 1 as
occurrence UNION ALL
SELECT 'banana' as source_value, 'an' as search_value, -1 as position, 1 as
occurrence UNION ALL
SELECT 'banana' as source_value, 'an' as search_value, -3 as position, 1 as
occurrence UNION ALL
SELECT 'banana' as source_value, 'ann' as search_value, 1 as position, 1 as
occurrence UNION ALL
SELECT 'helloooo' as source_value, 'oo' as search_value, 1 as position, 1 as
occurrence UNION ALL
SELECT 'helloooo' as source_value, 'oo' as search_value, 1 as position, 2 as
occurrence
)
SELECT source_value, search_value, position, occurrence, INSTR(source_value,
search_value, position, occurrence) AS instr
FROM example;
+--------------+--------------+----------+------------+-------+
| source_value | search_value | position | occurrence | instr |
+--------------+--------------+----------+------------+-------+
| banana | an | 1 | 1 | 2 |
| banana | an | 1 | 2 | 4 |
| banana | an | 1 | 3 | 0 |
| banana | an | 3 | 1 | 4 |
| banana | an | -1 | 1 | 4 |
| banana | an | -3 | 1 | 4 |
| banana | ann | 1 | 1 | 0 |
| helloooo | oo | 1 | 1 | 5 |
| helloooo | oo | 1 | 2 | 6 |
+--------------+--------------+----------+------------+-------+
LEFT
LEFT(value, length)
Description
Returns a STRING
or BYTES
value that consists of the specified
number of leftmost characters or bytes from value
. The length
is an
INT64
that specifies the length of the returned
value. If value
is of type BYTES
, length
is the number of leftmost bytes
to return. If value
is STRING
, length
is the number of leftmost characters
to return.
If length
is 0, an empty STRING
or BYTES
value will be
returned. If length
is negative, an error will be returned. If length
exceeds the number of characters or bytes from value
, the original value
will be returned.
Return type
STRING
or BYTES
Examples
WITH examples AS
(SELECT 'apple' as example
UNION ALL
SELECT 'banana' as example
UNION ALL
SELECT 'абвгд' as example
)
SELECT example, LEFT(example, 3) AS left_example
FROM examples;
+---------+--------------+
| example | left_example |
+---------+--------------+
| apple | app |
| banana | ban |
| абвгд | абв |
+---------+--------------+
WITH examples AS
(SELECT b'apple' as example
UNION ALL
SELECT b'banana' as example
UNION ALL
SELECT b'\xab\xcd\xef\xaa\xbb' as example
)
SELECT example, LEFT(example, 3) AS left_example
FROM examples;
-- Note that the result of LEFT is of type BYTES, displayed as a base64-encoded string.
+----------+--------------+
| example | left_example |
+----------+--------------+
| YXBwbGU= | YXBw |
| YmFuYW5h | YmFu |
| q83vqrs= | q83v |
+----------+--------------+
LENGTH
LENGTH(value)
Description
Returns the length of the STRING
or BYTES
value. The returned
value is in characters for STRING
arguments and in bytes for the BYTES
argument.
Return type
INT64
Examples
WITH example AS
(SELECT "абвгд" AS characters)
SELECT
characters,
LENGTH(characters) AS string_example,
LENGTH(CAST(characters AS BYTES)) AS bytes_example
FROM example;
+------------+----------------+---------------+
| characters | string_example | bytes_example |
+------------+----------------+---------------+
| абвгд | 5 | 10 |
+------------+----------------+---------------+
LPAD
LPAD(original_value, return_length[, pattern])
Description
Returns a STRING
or BYTES
value that consists of original_value
prepended
with pattern
. The return_length
is an INT64
that
specifies the length of the returned value. If original_value
is of type
BYTES
, return_length
is the number of bytes. If original_value
is
of type STRING
, return_length
is the number of characters.
The default value of pattern
is a blank space.
Both original_value
and pattern
must be the same data type.
If return_length
is less than or equal to the original_value
length, this
function returns the original_value
value, truncated to the value of
return_length
. For example, LPAD("hello world", 7);
returns "hello w"
.
If original_value
, return_length
, or pattern
is NULL
, this function
returns NULL
.
This function returns an error if:
return_length
is negativepattern
is empty
Return type
STRING
or BYTES
Examples
SELECT t, len, FORMAT("%T", LPAD(t, len)) AS LPAD FROM UNNEST([
STRUCT('abc' AS t, 5 AS len),
('abc', 2),
('例子', 4)
]);
+------+-----+----------+
| t | len | LPAD |
|------|-----|----------|
| abc | 5 | " abc" |
| abc | 2 | "ab" |
| 例子 | 4 | " 例子" |
+------+-----+----------+
SELECT t, len, pattern, FORMAT("%T", LPAD(t, len, pattern)) AS LPAD FROM UNNEST([
STRUCT('abc' AS t, 8 AS len, 'def' AS pattern),
('abc', 5, '-'),
('例子', 5, '中文')
]);
+------+-----+---------+--------------+
| t | len | pattern | LPAD |
|------|-----|---------|--------------|
| abc | 8 | def | "defdeabc" |
| abc | 5 | - | "--abc" |
| 例子 | 5 | 中文 | "中文中例子" |
+------+-----+---------+--------------+
SELECT FORMAT("%T", t) AS t, len, FORMAT("%T", LPAD(t, len)) AS LPAD FROM UNNEST([
STRUCT(b'abc' AS t, 5 AS len),
(b'abc', 2),
(b'\xab\xcd\xef', 4)
]);
+-----------------+-----+------------------+
| t | len | LPAD |
|-----------------|-----|------------------|
| b"abc" | 5 | b" abc" |
| b"abc" | 2 | b"ab" |
| b"\xab\xcd\xef" | 4 | b" \xab\xcd\xef" |
+-----------------+-----+------------------+
SELECT
FORMAT("%T", t) AS t,
len,
FORMAT("%T", pattern) AS pattern,
FORMAT("%T", LPAD(t, len, pattern)) AS LPAD
FROM UNNEST([
STRUCT(b'abc' AS t, 8 AS len, b'def' AS pattern),
(b'abc', 5, b'-'),
(b'\xab\xcd\xef', 5, b'\x00')
]);
+-----------------+-----+---------+-------------------------+
| t | len | pattern | LPAD |
|-----------------|-----|---------|-------------------------|
| b"abc" | 8 | b"def" | b"defdeabc" |
| b"abc" | 5 | b"-" | b"--abc" |
| b"\xab\xcd\xef" | 5 | b"\x00" | b"\x00\x00\xab\xcd\xef" |
+-----------------+-----+---------+-------------------------+
LOWER
LOWER(value)
Description
For STRING
arguments, returns the original string with all alphabetic
characters in lowercase. Mapping between lowercase and uppercase is done
according to the
Unicode Character Database
without taking into account language-specific mappings.
For BYTES
arguments, the argument is treated as ASCII text, with all bytes
greater than 127 left intact.
Return type
STRING
or BYTES
Examples
WITH items AS
(SELECT
"FOO" as item
UNION ALL
SELECT
"BAR" as item
UNION ALL
SELECT
"BAZ" as item)
SELECT
LOWER(item) AS example
FROM items;
+---------+
| example |
+---------+
| foo |
| bar |
| baz |
+---------+
LTRIM
LTRIM(value1[, value2])
Description
Identical to TRIM, but only removes leading characters.
Return type
STRING
or BYTES
Examples
WITH items AS
(SELECT " apple " as item
UNION ALL
SELECT " banana " as item
UNION ALL
SELECT " orange " as item)
SELECT
CONCAT("#", LTRIM(item), "#") as example
FROM items;
+-------------+
| example |
+-------------+
| #apple # |
| #banana # |
| #orange # |
+-------------+
WITH items AS
(SELECT "***apple***" as item
UNION ALL
SELECT "***banana***" as item
UNION ALL
SELECT "***orange***" as item)
SELECT
LTRIM(item, "*") as example
FROM items;
+-----------+
| example |
+-----------+
| apple*** |
| banana*** |
| orange*** |
+-----------+
WITH items AS
(SELECT "xxxapplexxx" as item
UNION ALL
SELECT "yyybananayyy" as item
UNION ALL
SELECT "zzzorangezzz" as item
UNION ALL
SELECT "xyzpearxyz" as item)
SELECT
LTRIM(item, "xyz") as example
FROM items;
+-----------+
| example |
+-----------+
| applexxx |
| bananayyy |
| orangezzz |
| pearxyz |
+-----------+
NORMALIZE
NORMALIZE(value[, normalization_mode])
Description
Takes a string value and returns it as a normalized string.
Normalization is used to ensure that two strings are equivalent. Normalization is often used in situations in which two strings render the same on the screen but have different Unicode code points.
NORMALIZE
supports four optional normalization modes:
Value | Name | Description |
---|---|---|
NFC | Normalization Form Canonical Composition | Decomposes and recomposes characters by canonical equivalence. |
NFKC | Normalization Form Compatibility Composition | Decomposes characters by compatibility, then recomposes them by canonical equivalence. |
NFD | Normalization Form Canonical Decomposition | Decomposes characters by canonical equivalence, and multiple combining characters are arranged in a specific order. |
NFKD | Normalization Form Compatibility Decomposition | Decomposes characters by compatibility, and multiple combining characters are arranged in a specific order. |
The default normalization mode is NFC
.
Return type
STRING
Examples
SELECT a, b, a = b as normalized
FROM (SELECT NORMALIZE('\u00ea') as a, NORMALIZE('\u0065\u0302') as b)
AS normalize_example;
+---+---+------------+
| a | b | normalized |
+---+---+------------+
| ê | ê | true |
+---+---+------------+
The following example normalizes different space characters.
WITH EquivalentNames AS (
SELECT name
FROM UNNEST([
'Jane\u2004Doe',
'John\u2004Smith',
'Jane\u2005Doe',
'Jane\u2006Doe',
'John Smith']) AS name
)
SELECT
NORMALIZE(name, NFKC) AS normalized_name,
COUNT(*) AS name_count
FROM EquivalentNames
GROUP BY 1;
+-----------------+------------+
| normalized_name | name_count |
+-----------------+------------+
| John Smith | 2 |
| Jane Doe | 3 |
+-----------------+------------+
NORMALIZE_AND_CASEFOLD
NORMALIZE_AND_CASEFOLD(value[, normalization_mode])
Description
Takes a STRING
, value
, and performs the same actions as
NORMALIZE
, as well as
casefolding for
case-insensitive operations.
NORMALIZE_AND_CASEFOLD
supports four optional normalization modes:
Value | Name | Description |
---|---|---|
NFC | Normalization Form Canonical Composition | Decomposes and recomposes characters by canonical equivalence. |
NFKC | Normalization Form Compatibility Composition | Decomposes characters by compatibility, then recomposes them by canonical equivalence. |
NFD | Normalization Form Canonical Decomposition | Decomposes characters by canonical equivalence, and multiple combining characters are arranged in a specific order. |
NFKD | Normalization Form Compatibility Decomposition | Decomposes characters by compatibility, and multiple combining characters are arranged in a specific order. |
The default normalization mode is NFC
.
Return type
STRING
Example
WITH Strings AS (
SELECT '\u2168' AS a, 'IX' AS b UNION ALL
SELECT '\u0041\u030A', '\u00C5'
)
SELECT a, b,
NORMALIZE_AND_CASEFOLD(a, NFD)=NORMALIZE_AND_CASEFOLD(b, NFD) AS nfd,
NORMALIZE_AND_CASEFOLD(a, NFC)=NORMALIZE_AND_CASEFOLD(b, NFC) AS nfc,
NORMALIZE_AND_CASEFOLD(a, NFKD)=NORMALIZE_AND_CASEFOLD(b, NFKD) AS nkfd,
NORMALIZE_AND_CASEFOLD(a, NFKC)=NORMALIZE_AND_CASEFOLD(b, NFKC) AS nkfc
FROM Strings;
+---+----+-------+-------+------+------+
| a | b | nfd | nfc | nkfd | nkfc |
+---+----+-------+-------+------+------+
| Ⅸ | IX | false | false | true | true |
| Å | Å | true | true | true | true |
+---+----+-------+-------+------+------+
OCTET_LENGTH
OCTET_LENGTH(value)
Alias for BYTE_LENGTH
.
REGEXP_CONTAINS
REGEXP_CONTAINS(value, regexp)
Description
Returns TRUE
if value
is a partial match for the regular expression,
regexp
.
If the regexp
argument is invalid, the function returns an error.
You can search for a full match by using ^
(beginning of text) and $
(end of
text). Due to regular expression operator precedence, it is good practice to use
parentheses around everything between ^
and $
.
Return type
BOOL
Examples
SELECT
email,
REGEXP_CONTAINS(email, r"@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+") AS is_valid
FROM
(SELECT
["foo@example.com", "bar@example.org", "www.example.net"]
AS addresses),
UNNEST(addresses) AS email;
+-----------------+----------+
| email | is_valid |
+-----------------+----------+
| foo@example.com | true |
| bar@example.org | true |
| www.example.net | false |
+-----------------+----------+
# Performs a full match, using ^ and $. Due to regular expression operator
# precedence, it is good practice to use parentheses around everything between ^
# and $.
SELECT
email,
REGEXP_CONTAINS(email, r"^([\w.+-]+@foo\.com|[\w.+-]+@bar\.org)$")
AS valid_email_address,
REGEXP_CONTAINS(email, r"^[\w.+-]+@foo\.com|[\w.+-]+@bar\.org$")
AS without_parentheses
FROM
(SELECT
["a@foo.com", "a@foo.computer", "b@bar.org", "!b@bar.org", "c@buz.net"]
AS addresses),
UNNEST(addresses) AS email;
+----------------+---------------------+---------------------+
| email | valid_email_address | without_parentheses |
+----------------+---------------------+---------------------+
| a@foo.com | true | true |
| a@foo.computer | false | true |
| b@bar.org | true | true |
| !b@bar.org | false | true |
| c@buz.net | false | false |
+----------------+---------------------+---------------------+
REGEXP_EXTRACT
REGEXP_EXTRACT(value, regexp[, position[, occurrence]])
Description
Returns the substring in value
that matches the regular expression, regexp
.
Returns NULL
if there is no match.
If the regular expression contains a capturing group, the function returns the substring that is matched by that capturing group. If the expression does not contain a capturing group, the function returns the entire matching substring.
If position
is specified, the search starts at this
position in value
, otherwise it starts at the beginning of value
. The
position
must be a positive integer and cannot be 0. If position
is greater
than the length of value
, NULL
is returned.
If occurrence
is specified, the search returns a specific occurrence of the
regexp
in value
, otherwise returns the first match. If occurrence
is
greater than the number of matches found, NULL
is returned. For
occurrence
> 1, the function searches for additional occurrences beginning
with the character following the previous occurrence.
Returns an error if:
- The regular expression is invalid
- The regular expression has more than one capturing group
- The
position
is not a positive integer - The
occurrence
is not a positive integer
Return type
STRING
or BYTES
Examples
WITH email_addresses AS
(SELECT "foo@example.com" as email
UNION ALL
SELECT "bar@example.org" as email
UNION ALL
SELECT "baz@example.net" as email)
SELECT
REGEXP_EXTRACT(email, r"^[a-zA-Z0-9_.+-]+")
AS user_name
FROM email_addresses;
+-----------+
| user_name |
+-----------+
| foo |
| bar |
| baz |
+-----------+
WITH email_addresses AS
(SELECT "foo@example.com" as email
UNION ALL
SELECT "bar@example.org" as email
UNION ALL
SELECT "baz@example.net" as email)
SELECT
REGEXP_EXTRACT(email, r"^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.([a-zA-Z0-9-.]+$)")
AS top_level_domain
FROM email_addresses;
+------------------+
| top_level_domain |
+------------------+
| com |
| org |
| net |
+------------------+
WITH example AS
(SELECT 'Hello Helloo and Hellooo' AS value, 'H?ello+' AS regex, 1 as position,
1 AS occurrence UNION ALL
SELECT 'Hello Helloo and Hellooo', 'H?ello+', 1, 2 UNION ALL
SELECT 'Hello Helloo and Hellooo', 'H?ello+', 1, 3 UNION ALL
SELECT 'Hello Helloo and Hellooo', 'H?ello+', 1, 4 UNION ALL
SELECT 'Hello Helloo and Hellooo', 'H?ello+', 2, 1 UNION ALL
SELECT 'Hello Helloo and Hellooo', 'H?ello+', 3, 1 UNION ALL
SELECT 'Hello Helloo and Hellooo', 'H?ello+', 3, 2 UNION ALL
SELECT 'Hello Helloo and Hellooo', 'H?ello+', 3, 3 UNION ALL
SELECT 'Hello Helloo and Hellooo', 'H?ello+', 20, 1 UNION ALL
SELECT 'cats&dogs&rabbits' ,'\\w+&', 1, 2 UNION ALL
SELECT 'cats&dogs&rabbits', '\\w+&', 2, 3
)
SELECT value, regex, position, occurrence, REGEXP_EXTRACT(value, regex,
position, occurrence) AS regexp_value FROM example;
+--------------------------+---------+----------+------------+--------------+
| value | regex | position | occurrence | regexp_value |
+--------------------------+---------+----------+------------+--------------+
| Hello Helloo and Hellooo | H?ello+ | 1 | 1 | Hello |
| Hello Helloo and Hellooo | H?ello+ | 1 | 2 | Helloo |
| Hello Helloo and Hellooo | H?ello+ | 1 | 3 | Hellooo |
| Hello Helloo and Hellooo | H?ello+ | 1 | 4 | NULL |
| Hello Helloo and Hellooo | H?ello+ | 2 | 1 | ello |
| Hello Helloo and Hellooo | H?ello+ | 3 | 1 | Helloo |
| Hello Helloo and Hellooo | H?ello+ | 3 | 2 | Hellooo |
| Hello Helloo and Hellooo | H?ello+ | 3 | 3 | NULL |
| Hello Helloo and Hellooo | H?ello+ | 20 | 1 | NULL |
| cats&dogs&rabbits | \w+& | 1 | 2 | dogs& |
| cats&dogs&rabbits | \w+& | 2 | 3 | NULL |
+--------------------------+---------+----------+------------+--------------+
REGEXP_EXTRACT_ALL
REGEXP_EXTRACT_ALL(value, regexp)
Description
Returns an array of all substrings of value
that match the regular expression,
regexp
.
The REGEXP_EXTRACT_ALL
function only returns non-overlapping matches. For
example, using this function to extract ana
from banana
returns only one
substring, not two.
Return type
An ARRAY
of either STRING
s or BYTES
Examples
WITH code_markdown AS
(SELECT "Try `function(x)` or `function(y)`" as code)
SELECT
REGEXP_EXTRACT_ALL(code, "`(.+?)`") AS example
FROM code_markdown;
+----------------------------+
| example |
+----------------------------+
| [function(x), function(y)] |
+----------------------------+
REGEXP_INSTR
REGEXP_INSTR(source_value, regexp [, position[, occurrence, [occurrence_position]]])
Description
Returns the lowest 1-based index of a regular expression, regexp
, in
source_value
. Returns 0
when no match is found or the regular expression
is empty. Returns an error if the regular expression is invalid or has more than
one capturing group. source_value
and regexp
must be the same type, either
STRING
or BYTES
.
If position
is specified, the search starts at this position in
source_value
, otherwise it starts at the beginning of source_value
. If
position
is negative, the function searches backwards from the end of
source_value
, with -1 indicating the last character. position
cannot be 0.
If occurrence
is specified, the search returns the position of a specific
instance of regexp
in source_value
, otherwise it returns the index of
the first occurrence. If occurrence
is greater than the number of matches
found, 0 is returned. For occurrence
> 1, the function searches for
overlapping occurrences, in other words, the function searches for additional
occurrences beginning with the second character in the previous occurrence.
occurrence
cannot be 0 or negative.
You can optionally use occurrence_position
to specify where a position
in relation to an occurrence
starts. Your choices are:
+ 0
: Returns the beginning position of the occurrence.
+ 1
: Returns the first position following the end of the occurrence. If the
end of the occurrence is also the end of the input, one off the
end of the occurrence is returned. For example, length of a string + 1.
Return type
INT64
Examples
WITH example AS (
SELECT 'ab@gmail.com' AS source_value, '@[^.]*' AS regexp UNION ALL
SELECT 'ab@mail.com', '@[^.]*' UNION ALL
SELECT 'abc@gmail.com', '@[^.]*' UNION ALL
SELECT 'abc.com', '@[^.]*')
SELECT source_value, regexp, REGEXP_INSTR(source_value, regexp) AS instr
FROM example;
+---------------+--------+-------+
| source_value | regexp | instr |
+---------------+--------+-------+
| ab@gmail.com | @[^.]* | 3 |
| ab@mail.com | @[^.]* | 3 |
| abc@gmail.com | @[^.]* | 4 |
| abc.com | @[^.]* | 0 |
+---------------+--------+-------+
WITH example AS (
SELECT 'a@gmail.com b@gmail.com' AS source_value, '@[^.]*' AS regexp, 1 AS position UNION ALL
SELECT 'a@gmail.com b@gmail.com', '@[^.]*', 2 UNION ALL
SELECT 'a@gmail.com b@gmail.com', '@[^.]*', 3 UNION ALL
SELECT 'a@gmail.com b@gmail.com', '@[^.]*', 4)
SELECT
source_value, regexp, position,
REGEXP_INSTR(source_value, regexp, position) AS instr
FROM example;
+-------------------------+--------+----------+-------+
| source_value | regexp | position | instr |
+-------------------------+--------+----------+-------+
| a@gmail.com b@gmail.com | @[^.]* | 1 | 2 |
| a@gmail.com b@gmail.com | @[^.]* | 2 | 2 |
| a@gmail.com b@gmail.com | @[^.]* | 3 | 14 |
| a@gmail.com b@gmail.com | @[^.]* | 4 | 14 |
+-------------------------+--------+----------+-------+
WITH example AS (
SELECT 'a@gmail.com b@gmail.com c@gmail.com' AS source_value,
'@[^.]*' AS regexp, 1 AS position, 1 AS occurrence UNION ALL
SELECT 'a@gmail.com b@gmail.com c@gmail.com', '@[^.]*', 1, 2 UNION ALL
SELECT 'a@gmail.com b@gmail.com c@gmail.com', '@[^.]*', 1, 3)
SELECT
source_value, regexp, position, occurrence,
REGEXP_INSTR(source_value, regexp, position, occurrence) AS instr
FROM example;
+-------------------------------------+--------+----------+------------+-------+
| source_value | regexp | position | occurrence | instr |
+-------------------------------------+--------+----------+------------+-------+
| a@gmail.com b@gmail.com c@gmail.com | @[^.]* | 1 | 1 | 2 |
| a@gmail.com b@gmail.com c@gmail.com | @[^.]* | 1 | 2 | 14 |
| a@gmail.com b@gmail.com c@gmail.com | @[^.]* | 1 | 3 | 26 |
+-------------------------------------+--------+----------+------------+-------+
WITH example AS (
SELECT 'a@gmail.com' AS source_value, '@[^.]*' AS regexp,
1 AS position, 1 AS occurrence, 0 AS o_position UNION ALL
SELECT 'a@gmail.com', '@[^.]*', 1, 1, 1)
SELECT
source_value, regexp, position, occurrence, o_position,
REGEXP_INSTR(source_value, regexp, position, occurrence, o_position) AS instr
FROM example;
+--------------+--------+----------+------------+------------+-------+
| source_value | regexp | position | occurrence | o_position | instr |
+--------------+--------+----------+------------+------------+-------+
| a@gmail.com | @[^.]* | 1 | 1 | 0 | 2 |
| a@gmail.com | @[^.]* | 1 | 1 | 1 | 8 |
+--------------+--------+----------+------------+------------+-------+
REGEXP_REPLACE
REGEXP_REPLACE(value, regexp, replacement)
Description
Returns a STRING
where all substrings of value
that
match regular expression regexp
are replaced with replacement
.
You can use backslashed-escaped digits (\1 to \9) within the replacement
argument to insert text matching the corresponding parenthesized group in the
regexp
pattern. Use \0 to refer to the entire matching text.
The REGEXP_REPLACE
function only replaces non-overlapping matches. For
example, replacing ana
within banana
results in only one replacement, not
two.
If the regexp
argument is not a valid regular expression, this function
returns an error.
Return type
STRING
or BYTES
Examples
WITH markdown AS
(SELECT "# Heading" as heading
UNION ALL
SELECT "# Another heading" as heading)
SELECT
REGEXP_REPLACE(heading, r"^# ([a-zA-Z0-9\s]+$)", "<h1>\\1</h1>")
AS html
FROM markdown;
+--------------------------+
| html |
+--------------------------+
| <h1>Heading</h1> |
| <h1>Another heading</h1> |
+--------------------------+
REGEXP_SUBSTR
REGEXP_SUBSTR(value, regexp[, position[, occurrence]])
Description
Synonym for REGEXP_EXTRACT.
Return type
STRING
or BYTES
Examples
WITH example AS
(SELECT 'Hello World Helloo' AS value, 'H?ello+' AS regex, 1 AS position, 1 AS
occurrence
)
SELECT value, regex, position, occurrence, REGEXP_SUBSTR(value, regex,
position, occurrence) AS regexp_value FROM example;
+--------------------+---------+----------+------------+--------------+
| value | regex | position | occurrence | regexp_value |
+--------------------+---------+----------+------------+--------------+
| Hello World Helloo | H?ello+ | 1 | 1 | Hello |
+--------------------+---------+----------+------------+--------------+
REPLACE
REPLACE(original_value, from_value, to_value)
Description
Replaces all occurrences of from_value
with to_value
in original_value
.
If from_value
is empty, no replacement is made.
Return type
STRING
or BYTES
Examples
WITH desserts AS
(SELECT "apple pie" as dessert
UNION ALL
SELECT "blackberry pie" as dessert
UNION ALL
SELECT "cherry pie" as dessert)
SELECT
REPLACE (dessert, "pie", "cobbler") as example
FROM desserts;
+--------------------+
| example |
+--------------------+
| apple cobbler |
| blackberry cobbler |
| cherry cobbler |
+--------------------+
REPEAT
REPEAT(original_value, repetitions)
Description
Returns a STRING
or BYTES
value that consists of original_value
, repeated.
The repetitions
parameter specifies the number of times to repeat
original_value
. Returns NULL
if either original_value
or repetitions
are NULL
.
This function returns an error if the repetitions
value is negative.
Return type
STRING
or BYTES
Examples
SELECT t, n, REPEAT(t, n) AS REPEAT FROM UNNEST([
STRUCT('abc' AS t, 3 AS n),
('例子', 2),
('abc', null),
(null, 3)
]);
+------+------+-----------+
| t | n | REPEAT |
|------|------|-----------|
| abc | 3 | abcabcabc |
| 例子 | 2 | 例子例子 |
| abc | NULL | NULL |
| NULL | 3 | NULL |
+------+------+-----------+
REVERSE
REVERSE(value)
Description
Returns the reverse of the input STRING
or BYTES
.
Return type
STRING
or BYTES
Examples
WITH example AS (
SELECT "foo" AS sample_string, b"bar" AS sample_bytes UNION ALL
SELECT "абвгд" AS sample_string, b"123" AS sample_bytes
)
SELECT
sample_string,
REVERSE(sample_string) AS reverse_string,
sample_bytes,
REVERSE(sample_bytes) AS reverse_bytes
FROM example;
+---------------+----------------+--------------+---------------+
| sample_string | reverse_string | sample_bytes | reverse_bytes |
+---------------+----------------+--------------+---------------+
| foo | oof | bar | rab |
| абвгд | дгвба | 123 | 321 |
+---------------+----------------+--------------+---------------+
RIGHT
RIGHT(value, length)
Description
Returns a STRING
or BYTES
value that consists of the specified
number of rightmost characters or bytes from value
. The length
is an
INT64
that specifies the length of the returned
value. If value
is BYTES
, length
is the number of rightmost bytes to
return. If value
is STRING
, length
is the number of rightmost characters
to return.
If length
is 0, an empty STRING
or BYTES
value will be
returned. If length
is negative, an error will be returned. If length
exceeds the number of characters or bytes from value
, the original value
will be returned.
Return type
STRING
or BYTES
Examples
WITH examples AS
(SELECT 'apple' as example
UNION ALL
SELECT 'banana' as example
UNION ALL
SELECT 'абвгд' as example
)
SELECT example, RIGHT(example, 3) AS right_example
FROM examples;
+---------+---------------+
| example | right_example |
+---------+---------------+
| apple | ple |
| banana | ana |
| абвгд | вгд |
+---------+---------------+
WITH examples AS
(SELECT b'apple' as example
UNION ALL
SELECT b'banana' as example
UNION ALL
SELECT b'\xab\xcd\xef\xaa\xbb' as example
)
SELECT example, RIGHT(example, 3) AS right_example
FROM examples;
-- Note that the result of RIGHT is of type BYTES, displayed as a base64-encoded string.
+----------+---------------+
| example | right_example |
+----------+---------------+
| YXBwbGU= | cGxl |
| YmFuYW5h | YW5h |
| q83vqrs= | 76q7 |
+----------+---------------+
RPAD
RPAD(original_value, return_length[, pattern])
Description
Returns a STRING
or BYTES
value that consists of original_value
appended
with pattern
. The return_length
parameter is an
INT64
that specifies the length of the
returned value. If original_value
is BYTES
,
return_length
is the number of bytes. If original_value
is STRING
,
return_length
is the number of characters.
The default value of pattern
is a blank space.
Both original_value
and pattern
must be the same data type.
If return_length
is less than or equal to the original_value
length, this
function returns the original_value
value, truncated to the value of
return_length
. For example, RPAD("hello world", 7);
returns "hello w"
.
If original_value
, return_length
, or pattern
is NULL
, this function
returns NULL
.
This function returns an error if:
return_length
is negativepattern
is empty
Return type
STRING
or BYTES
Examples
SELECT t, len, FORMAT("%T", RPAD(t, len)) AS RPAD FROM UNNEST([
STRUCT('abc' AS t, 5 AS len),
('abc', 2),
('例子', 4)
]);
+------+-----+----------+
| t | len | RPAD |
|------|-----|----------|
| abc | 5 | "abc " |
| abc | 2 | "ab" |
| 例子 | 4 | "例子 " |
+------+-----+----------+
SELECT t, len, pattern, FORMAT("%T", RPAD(t, len, pattern)) AS RPAD FROM UNNEST([
STRUCT('abc' AS t, 8 AS len, 'def' AS pattern),
('abc', 5, '-'),
('例子', 5, '中文')
]);
+------+-----+---------+--------------+
| t | len | pattern | RPAD |
|------|-----|---------|--------------|
| abc | 8 | def | "abcdefde" |
| abc | 5 | - | "abc--" |
| 例子 | 5 | 中文 | "例子中文中" |
+------+-----+---------+--------------+
SELECT FORMAT("%T", t) AS t, len, FORMAT("%T", RPAD(t, len)) AS RPAD FROM UNNEST([
STRUCT(b'abc' AS t, 5 AS len),
(b'abc', 2),
(b'\xab\xcd\xef', 4)
]);
+-----------------+-----+------------------+
| t | len | RPAD |
|-----------------|-----|------------------|
| b"abc" | 5 | b"abc " |
| b"abc" | 2 | b"ab" |
| b"\xab\xcd\xef" | 4 | b"\xab\xcd\xef " |
+-----------------+-----+------------------+
SELECT
FORMAT("%T", t) AS t,
len,
FORMAT("%T", pattern) AS pattern,
FORMAT("%T", RPAD(t, len, pattern)) AS RPAD
FROM UNNEST([
STRUCT(b'abc' AS t, 8 AS len, b'def' AS pattern),
(b'abc', 5, b'-'),
(b'\xab\xcd\xef', 5, b'\x00')
]);
+-----------------+-----+---------+-------------------------+
| t | len | pattern | RPAD |
|-----------------|-----|---------|-------------------------|
| b"abc" | 8 | b"def" | b"abcdefde" |
| b"abc" | 5 | b"-" | b"abc--" |
| b"\xab\xcd\xef" | 5 | b"\x00" | b"\xab\xcd\xef\x00\x00" |
+-----------------+-----+---------+-------------------------+
RTRIM
RTRIM(value1[, value2])
Description
Identical to TRIM, but only removes trailing characters.
Return type
STRING
or BYTES
Examples
WITH items AS
(SELECT "***apple***" as item
UNION ALL
SELECT "***banana***" as item
UNION ALL
SELECT "***orange***" as item)
SELECT
RTRIM(item, "*") as example
FROM items;
+-----------+
| example |
+-----------+
| ***apple |
| ***banana |
| ***orange |
+-----------+
WITH items AS
(SELECT "applexxx" as item
UNION ALL
SELECT "bananayyy" as item
UNION ALL
SELECT "orangezzz" as item
UNION ALL
SELECT "pearxyz" as item)
SELECT
RTRIM(item, "xyz") as example
FROM items;
+---------+
| example |
+---------+
| apple |
| banana |
| orange |
| pear |
+---------+
SAFE_CONVERT_BYTES_TO_STRING
SAFE_CONVERT_BYTES_TO_STRING(value)
Description
Converts a sequence of BYTES
to a STRING
. Any invalid UTF-8 characters are
replaced with the Unicode replacement character, U+FFFD
.
Return type
STRING
Examples
The following statement returns the Unicode replacement character, �.
SELECT SAFE_CONVERT_BYTES_TO_STRING(b'\xc2') as safe_convert;
SOUNDEX
SOUNDEX(value)
Description
Returns a STRING
that represents the
Soundex code for value
.
SOUNDEX produces a phonetic representation of a string. It indexes words by sound, as pronounced in English. It is typically used to help determine whether two strings, such as the family names Levine and Lavine, or the words to and too, have similar English-language pronunciation.
The result of the SOUNDEX consists of a letter followed by 3 digits. Non-latin
characters are ignored. If the remaining string is empty after removing
non-Latin characters, an empty STRING
is returned.
Return type
STRING
Examples
WITH example AS (
SELECT 'Ashcraft' AS value UNION ALL
SELECT 'Raven' AS value UNION ALL
SELECT 'Ribbon' AS value UNION ALL
SELECT 'apple' AS value UNION ALL
SELECT 'Hello world!' AS value UNION ALL
SELECT ' H3##!@llo w00orld!' AS value UNION ALL
SELECT '#1' AS value UNION ALL
SELECT NULL AS value
)
SELECT value, SOUNDEX(value) AS soundex
FROM example;
+----------------------+---------+
| value | soundex |
+----------------------+---------+
| Ashcraft | A261 |
| Raven | R150 |
| Ribbon | R150 |
| apple | a140 |
| Hello world! | H464 |
| H3##!@llo w00orld! | H464 |
| #1 | |
| NULL | NULL |
+----------------------+---------+
SPLIT
SPLIT(value[, delimiter])
Description
Splits value
using the delimiter
argument.
For STRING
, the default delimiter is the comma ,
.
For BYTES
, you must specify a delimiter.
Splitting on an empty delimiter produces an array of UTF-8 characters for
STRING
values, and an array of BYTES
for BYTES
values.
Splitting an empty STRING
returns an
ARRAY
with a single empty
STRING
.
Return type
ARRAY
of type STRING
or
ARRAY
of type BYTES
Examples
WITH letters AS
(SELECT "" as letter_group
UNION ALL
SELECT "a" as letter_group
UNION ALL
SELECT "b c d" as letter_group)
SELECT SPLIT(letter_group, " ") as example
FROM letters;
+----------------------+
| example |
+----------------------+
| [] |
| [a] |
| [b, c, d] |
+----------------------+
STARTS_WITH
STARTS_WITH(value1, value2)
Description
Takes two STRING
or BYTES
values. Returns TRUE
if the second value is a
prefix of the first.
Return type
BOOL
Examples
WITH items AS
(SELECT "foo" as item
UNION ALL
SELECT "bar" as item
UNION ALL
SELECT "baz" as item)
SELECT
STARTS_WITH(item, "b") as example
FROM items;
+---------+
| example |
+---------+
| False |
| True |
| True |
+---------+
STRPOS
STRPOS(value1, value2)
Description
Takes two STRING
or BYTES
values. Returns the 1-based index of the first
occurrence of value2
inside value1
. Returns 0
if value2
is not found.
Return type
INT64
Examples
WITH email_addresses AS
(SELECT
"foo@example.com" AS email_address
UNION ALL
SELECT
"foobar@example.com" AS email_address
UNION ALL
SELECT
"foobarbaz@example.com" AS email_address
UNION ALL
SELECT
"quxexample.com" AS email_address)
SELECT
STRPOS(email_address, "@") AS example
FROM email_addresses;
+---------+
| example |
+---------+
| 4 |
| 7 |
| 10 |
| 0 |
+---------+
SUBSTR
SUBSTR(value, position[, length])
Description
Returns a substring of the supplied STRING
or BYTES
value. The
position
argument is an integer specifying the starting position of the
substring, with position = 1 indicating the first character or byte. The
length
argument is the maximum number of characters for STRING
arguments,
or bytes for BYTES
arguments.
If position
is negative, the function counts from the end of value
,
with -1 indicating the last character.
If position
is a position off the left end of the
STRING
(position
= 0 or position
< -LENGTH(value)
), the function
starts from position = 1. If length
exceeds the length of value
, the
function returns fewer than length
characters.
If length
is less than 0, the function returns an error.
Return type
STRING
or BYTES
Examples
WITH items AS
(SELECT "apple" as item
UNION ALL
SELECT "banana" as item
UNION ALL
SELECT "orange" as item)
SELECT
SUBSTR(item, 2) as example
FROM items;
+---------+
| example |
+---------+
| pple |
| anana |
| range |
+---------+
WITH items AS
(SELECT "apple" as item
UNION ALL
SELECT "banana" as item
UNION ALL
SELECT "orange" as item)
SELECT
SUBSTR(item, 2, 2) as example
FROM items;
+---------+
| example |
+---------+
| pp |
| an |
| ra |
+---------+
WITH items AS
(SELECT "apple" as item
UNION ALL
SELECT "banana" as item
UNION ALL
SELECT "orange" as item)
SELECT
SUBSTR(item, -2) as example
FROM items;
+---------+
| example |
+---------+
| le |
| na |
| ge |
+---------+
SUBSTRING
SUBSTRING(value, position[, length])
Alias for SUBSTR
.
TO_BASE32
TO_BASE32(bytes_expr)
Description
Converts a sequence of BYTES
into a base32-encoded STRING
. To convert a
base32-encoded STRING
into BYTES
, use FROM_BASE32.
Return type
STRING
Example
SELECT TO_BASE32(b'abcde\xFF') AS base32_string;
+------------------+
| base32_string |
+------------------+
| MFRGGZDF74====== |
+------------------+
TO_BASE64
TO_BASE64(bytes_expr)
Description
Converts a sequence of BYTES
into a base64-encoded STRING
. To convert a
base64-encoded STRING
into BYTES
, use FROM_BASE64.
There are several base64 encodings in common use that vary in exactly which
alphabet of 65 ASCII characters are used to encode the 64 digits and padding.
See RFC 4648 for details. This
function adds padding and uses the alphabet [A-Za-z0-9+/=]
.
Return type
STRING
Example
SELECT TO_BASE64(b'\377\340') AS base64_string;
+---------------+
| base64_string |
+---------------+
| /+A= |
+---------------+
To work with an encoding using a different base64 alphabet, you might need to
compose TO_BASE64
with the REPLACE
function. For instance, the
base64url
url-safe and filename-safe encoding commonly used in web programming
uses -_=
as the last characters rather than +/=
. To encode a
base64url
-encoded string, replace -
and _
with +
and /
respectively.
SELECT REPLACE(REPLACE(TO_BASE64(b"\377\340"), "+", "-"), "/", "_") as websafe_base64;
+----------------+
| websafe_base64 |
+----------------+
| _-A= |
+----------------+
TO_CODE_POINTS
TO_CODE_POINTS(value)
Description
Takes a value and returns an array of
INT64
.
- If
value
is aSTRING
, each element in the returned array represents a code point. Each code point falls within the range of [0, 0xD7FF] and [0xE000, 0x10FFFF]. - If
value
isBYTES
, each element in the array is an extended ASCII character value in the range of [0, 255].
To convert from an array of code points to a STRING
or BYTES
, see
CODE_POINTS_TO_STRING or
CODE_POINTS_TO_BYTES.
Return type
ARRAY
of INT64
Examples
The following example gets the code points for each element in an array of words.
SELECT word, TO_CODE_POINTS(word) AS code_points
FROM UNNEST(['foo', 'bar', 'baz', 'giraffe', 'llama']) AS word;
+---------+------------------------------------+
| word | code_points |
+---------+------------------------------------+
| foo | [102, 111, 111] |
| bar | [98, 97, 114] |
| baz | [98, 97, 122] |
| giraffe | [103, 105, 114, 97, 102, 102, 101] |
| llama | [108, 108, 97, 109, 97] |
+---------+------------------------------------+
The following example converts integer representations of BYTES
to their
corresponding ASCII character values.
SELECT word, TO_CODE_POINTS(word) AS bytes_value_as_integer
FROM UNNEST([b'\x00\x01\x10\xff', b'\x66\x6f\x6f']) AS word;
+------------------+------------------------+
| word | bytes_value_as_integer |
+------------------+------------------------+
| \x00\x01\x10\xff | [0, 1, 16, 255] |
| foo | [102, 111, 111] |
+------------------+------------------------+
The following example demonstrates the difference between a BYTES
result and a
STRING
result.
SELECT TO_CODE_POINTS(b'Ā') AS b_result, TO_CODE_POINTS('Ā') AS s_result;
+------------+----------+
| b_result | s_result |
+------------+----------+
| [196, 128] | [256] |
+------------+----------+
Notice that the character, Ā, is represented as a two-byte Unicode sequence. As
a result, the BYTES
version of TO_CODE_POINTS
returns an array with two
elements, while the STRING
version returns an array with a single element.
TO_HEX
TO_HEX(bytes)
Description
Converts a sequence of BYTES
into a hexadecimal STRING
. Converts each byte
in the STRING
as two hexadecimal characters in the range
(0..9, a..f)
. To convert a hexadecimal-encoded
STRING
to BYTES
, use FROM_HEX.
Return type
STRING
Example
WITH Input AS (
SELECT b'\x00\x01\x02\x03\xAA\xEE\xEF\xFF' AS byte_str UNION ALL
SELECT b'foobar'
)
SELECT byte_str, TO_HEX(byte_str) AS hex_str
FROM Input;
+----------------------------------+------------------+
| byte_string | hex_string |
+----------------------------------+------------------+
| \x00\x01\x02\x03\xaa\xee\xef\xff | 00010203aaeeefff |
| foobar | 666f6f626172 |
+----------------------------------+------------------+
TRANSLATE
TRANSLATE(expression, source_characters, target_characters)
Description
In expression
, replaces each character in source_characters
with the
corresponding character in target_characters
. All inputs must be the same
type, either STRING
or BYTES
.
- Each character in
expression
is translated at most once. - A character in
expression
that is not present insource_characters
is left unchanged inexpression
. - A character in
source_characters
without a corresponding character intarget_characters
is omitted from the result. - A duplicate character in
source_characters
results in an error.
Return type
STRING
or BYTES
Examples
WITH example AS (
SELECT 'This is a cookie' AS expression, 'sco' AS source_characters, 'zku' AS
target_characters UNION ALL
SELECT 'A coaster' AS expression, 'co' AS source_characters, 'k' as
target_characters
)
SELECT expression, source_characters, target_characters, TRANSLATE(expression,
source_characters, target_characters) AS translate
FROM example;
+------------------+-------------------+-------------------+------------------+
| expression | source_characters | target_characters | translate |
+------------------+-------------------+-------------------+------------------+
| This is a cookie | sco | zku | Thiz iz a kuukie |
| A coaster | co | k | A kaster |
+------------------+-------------------+-------------------+------------------+
TRIM
TRIM(value1[, value2])
Description
Removes all leading and trailing characters that match value2
. If
value2
is not specified, all leading and trailing whitespace characters (as
defined by the Unicode standard) are removed. If the first argument is of type
BYTES
, the second argument is required.
If value2
contains more than one character or byte, the function removes all
leading or trailing characters or bytes contained in value2
.
Return type
STRING
or BYTES
Examples
WITH items AS
(SELECT " apple " as item
UNION ALL
SELECT " banana " as item
UNION ALL
SELECT " orange " as item)
SELECT
CONCAT("#", TRIM(item), "#") as example
FROM items;
+----------+
| example |
+----------+
| #apple# |
| #banana# |
| #orange# |
+----------+
WITH items AS
(SELECT "***apple***" as item
UNION ALL
SELECT "***banana***" as item
UNION ALL
SELECT "***orange***" as item)
SELECT
TRIM(item, "*") as example
FROM items;
+---------+
| example |
+---------+
| apple |
| banana |
| orange |
+---------+
WITH items AS
(SELECT "xxxapplexxx" as item
UNION ALL
SELECT "yyybananayyy" as item
UNION ALL
SELECT "zzzorangezzz" as item
UNION ALL
SELECT "xyzpearxyz" as item)
SELECT
TRIM(item, "xyz") as example
FROM items;
+---------+
| example |
+---------+
| apple |
| banana |
| orange |
| pear |
+---------+
UNICODE
UNICODE(value)
Description
Returns the Unicode code point for the first character in
value
. Returns 0
if value
is empty, or if the resulting Unicode code
point is 0
.
Return type
INT64
Examples
SELECT UNICODE('âbcd') as A, UNICODE('â') as B, UNICODE('') as C, UNICODE(NULL) as D;
+-------+-------+-------+-------+
| A | B | C | D |
+-------+-------+-------+-------+
| 226 | 226 | 0 | NULL |
+-------+-------+-------+-------+
UPPER
UPPER(value)
Description
For STRING
arguments, returns the original string with all alphabetic
characters in uppercase. Mapping between uppercase and lowercase is done
according to the
Unicode Character Database
without taking into account language-specific mappings.
For BYTES
arguments, the argument is treated as ASCII text, with all bytes
greater than 127 left intact.
Return type
STRING
or BYTES
Examples
WITH items AS
(SELECT
"foo" as item
UNION ALL
SELECT
"bar" as item
UNION ALL
SELECT
"baz" as item)
SELECT
UPPER(item) AS example
FROM items;
+---------+
| example |
+---------+
| FOO |
| BAR |
| BAZ |
+---------+
JSON functions
BigQuery supports functions that help you retrieve data stored in JSON-formatted strings and functions that help you transform data into JSON-formatted strings.
Function overview
JSON function | Description |
---|---|
JSON_EXTRACT |
Extracts a JSON value, such as an array or object, or a JSON-formatted scalar value, such as a string, integer, or boolean. If a JSON key uses invalid JSONPath characters, you can escape those characters using single quotes and brackets. |
JSON_QUERY |
Extracts a JSON value, such as an array or object, or a JSON-formatted scalar value, such as a string, integer, or boolean. If a JSON key uses invalid JSONPath characters, you can escape those characters using double quotes. |
JSON_EXTRACT_SCALAR |
Extracts a scalar value such as a string, integer, or boolean. Removes the outermost quotes and unescapes the values. If a JSON key uses invalid JSONPath characters, you can escape those characters using single quotes and brackets. |
JSON_VALUE |
Extracts a scalar value such as a string, integer, or boolean. Removes the outermost quotes and unescapes the values. If a JSON key uses invalid JSONPath characters, you can escape those characters using double quotes. |
JSON_EXTRACT_ARRAY |
Extracts an array of JSON-formatted strings. If a JSON key uses invalid JSONPath characters, you can escape those characters using single quotes and brackets. |
TO_JSON_STRING |
Returns a JSON-formatted string representation of a value. |
JSON_EXTRACT
JSON_EXTRACT(json_string_expr, json_path)
Description
Extracts a JSON value, such as an array or object, or a JSON-formatted scalar value, such as a string, integer, or boolean. If a JSON key uses invalid JSONPath characters, you can escape those characters using single quotes and brackets.
json_string_expr
: A JSON-formatted string. For example:{"class" : {"students" : [{"name" : "Jane"}]}}
json_path
: The JSONPath. This identifies the value or values you want to obtain from the JSON-formatted string. Ifjson_path
returns a JSONnull
, this is converted into a SQLNULL
.
If you want to include non-scalar values such as arrays in the extraction,
use JSON_EXTRACT
. If you only want to extract scalar values such strings,
integers, and booleans, use JSON_EXTRACT_SCALAR
.
Return type
A JSON-formatted STRING
Examples
SELECT JSON_EXTRACT(json_text, '$') AS json_text_string
FROM UNNEST([
'{"class" : {"students" : [{"name" : "Jane"}]}}',
'{"class" : {"students" : []}}',
'{"class" : {"students" : [{"name" : "John"}, {"name": "Jamie"}]}}'
]) AS json_text;
+-----------------------------------------------------------+
| json_text_string |
+-----------------------------------------------------------+
| {"class":{"students":[{"name":"Jane"}]}} |
| {"class":{"students":[]}} |
| {"class":{"students":[{"name":"John"},{"name":"Jamie"}]}} |
+-----------------------------------------------------------+
SELECT JSON_EXTRACT(json_text, '$.class.students[0]') AS first_student
FROM UNNEST([
'{"class" : {"students" : [{"name" : "Jane"}]}}',
'{"class" : {"students" : []}}',
'{"class" : {"students" : [{"name" : "John"}, {"name": "Jamie"}]}}'
]) AS json_text;
+-----------------+
| first_student |
+-----------------+
| {"name":"Jane"} |
| NULL |
| {"name":"John"} |
+-----------------+
SELECT JSON_EXTRACT(json_text, '$.class.students[1].name') AS second_student_name
FROM UNNEST([
'{"class" : {"students" : [{"name" : "Jane"}]}}',
'{"class" : {"students" : []}}',
'{"class" : {"students" : [{"name" : "John"}, {"name" : null}]}}',
'{"class" : {"students" : [{"name" : "John"}, {"name": "Jamie"}]}}'
]) AS json_text;
+-------------------+
| second_student |
+-------------------+
| NULL |
| NULL |
| NULL |
| "Jamie" |
+-------------------+
SELECT JSON_EXTRACT(json_text, "$.class['students']") AS student_names
FROM UNNEST([
'{"class" : {"students" : [{"name" : "Jane"}]}}',
'{"class" : {"students" : []}}',
'{"class" : {"students" : [{"name" : "John"}, {"name": "Jamie"}]}}'
]) AS json_text;
+------------------------------------+
| student_names |
+------------------------------------+
| [{"name":"Jane"}] |
| [] |
| [{"name":"John"},{"name":"Jamie"}] |
+------------------------------------+
JSON_QUERY
JSON_QUERY(json_string_expr, json_path)
Description
Extracts a JSON value, such as an array or object, or a JSON-formatted scalar value, such as a string, integer, or boolean. If a JSON key uses invalid JSONPath characters, you can escape those characters using double quotes.
json_string_expr
: A JSON-formatted string. For example:{"class" : {"students" : [{"name" : "Jane"}]}}
json_path
: The JSONPath. This identifies the value or values you want to obtain from the JSON-formatted string. Ifjson_path
returns a JSONnull
, this is converted into a SQLNULL
.
If you want to include non-scalar values such as arrays in the extraction,
use JSON_QUERY
. If you only want to extract scalar values such strings,
integers, and booleans, use JSON_VALUE
.
Return type
A JSON-formatted STRING
Examples
SELECT JSON_QUERY(json_text, '$') AS json_text_string
FROM UNNEST([
'{"class" : {"students" : [{"name" : "Jane"}]}}',
'{"class" : {"students" : []}}',
'{"class" : {"students" : [{"name" : "John"}, {"name": "Jamie"}]}}'
]) AS json_text;
+-----------------------------------------------------------+
| json_text_string |
+-----------------------------------------------------------+
| {"class":{"students":[{"name":"Jane"}]}} |
| {"class":{"students":[]}} |
| {"class":{"students":[{"name":"John"},{"name":"Jamie"}]}} |
+-----------------------------------------------------------+
SELECT JSON_QUERY(json_text, '$.class.students[0]') AS first_student
FROM UNNEST([
'{"class" : {"students" : [{"name" : "Jane"}]}}',
'{"class" : {"students" : []}}',
'{"class" : {"students" : [{"name" : "John"}, {"name": "Jamie"}]}}'
]) AS json_text;
+-----------------+
| first_student |
+-----------------+
| {"name":"Jane"} |
| NULL |
| {"name":"John"} |
+-----------------+
SELECT JSON_QUERY(json_text, '$.class.students[1].name') AS second_student_name
FROM UNNEST([
'{"class" : {"students" : [{"name" : "Jane"}]}}',
'{"class" : {"students" : []}}',
'{"class" : {"students" : [{"name" : "John"}, {"name" : null}]}}',
'{"class" : {"students" : [{"name" : "John"}, {"name": "Jamie"}]}}'
]) AS json_text;
+-------------------+
| second_student |
+-------------------+
| NULL |
| NULL |
| NULL |
| "Jamie" |
+-------------------+
SELECT JSON_QUERY(json_text, '$.class."students"') AS student_names
FROM UNNEST([
'{"class" : {"students" : [{"name" : "Jane"}]}}',
'{"class" : {"students" : []}}',
'{"class" : {"students" : [{"name" : "John"}, {"name": "Jamie"}]}}'
]) AS json_text;
+------------------------------------+
| student_names |
+------------------------------------+
| [{"name":"Jane"}] |
| [] |
| [{"name":"John"},{"name":"Jamie"}] |
+------------------------------------+
JSON_EXTRACT_SCALAR
JSON_EXTRACT_SCALAR(json_string_expr, json_path)
Description
Extracts a scalar value such as a string, integer, or boolean. Removes the outermost quotes and unescapes the return values. If a JSON key uses invalid JSONPath characters, you can escape those characters using single quotes and brackets.
json_string_expr
: A JSON-formatted string. For example:{"class" : {"students" : [{"name" : "Jane"}]}}
json_path
: The JSONPath. This identifies the value or values you want to obtain from the JSON-formatted string. Ifjson_path
returns a JSONnull
, this is converted into a SQLNULL
.
If you only want to extract scalar values such strings, integers, and
booleans, use JSON_EXTRACT_SCALAR
. If you want to include non-scalar values
such as arrays in the extraction, use JSON_EXTRACT
.
Return type
STRING
Examples
The following example compares how results are returned for the JSON_EXTRACT
and JSON_EXTRACT_SCALAR
functions.
SELECT JSON_EXTRACT('{ "name" : "Jakob", "age" : "6" }', '$.name') as json_name,
JSON_EXTRACT_SCALAR('{ "name" : "Jakob", "age" : "6" }', '$.name') as scalar_name,
JSON_EXTRACT('{ "name" : "Jakob", "age" : "6" }', '$.age') as json_age,
JSON_EXTRACT_SCALAR('{ "name" : "Jakob", "age" : "6" }', '$.age') as scalar_age;
+-----------+-------------+----------+------------+
| json_name | scalar_name | json_age | scalar_age |
+-----------+-------------+----------+------------+
| "Jakob" | Jakob | "6" | 6 |
+-----------+-------------+----------+------------+
In cases where a JSON key uses invalid JSONPath characters, you can escape those
characters using single quotes and brackets, [' ']
. For example:
SELECT JSON_EXTRACT_SCALAR('{"a.b": {"c": "world"}}', "$['a.b'].c") as hello;
+-------+
| hello |
+-------+
| world |
+-------+
JSON_VALUE
JSON_VALUE(json_string_expr, json_path)
Description
Extracts a scalar value such as a string, integer, or boolean. Removes the outermost quotes and unescapes the return values. If a JSON key uses invalid JSONPath characters, you can escape those characters using double quotes.
json_string_expr
: A JSON-formatted string. For example:{"class" : {"students" : [{"name" : "Jane"}]}}
json_path
: The JSONPath. This identifies the value or values you want to obtain from the JSON-formatted string. Ifjson_path
returns a JSONnull
, this is converted into a SQLNULL
.
If you only want to extract scalar values such strings, integers, and
booleans, use JSON_VALUE
. If you want to include non-scalar values such as
arrays in the extraction, use JSON_QUERY
.
Return type
STRING
Examples
SELECT JSON_QUERY('{ "name" : "Jakob", "age" : "6" }', '$.name') as json_name,
JSON_VALUE('{ "name" : "Jakob", "age" : "6" }', '$.name') as scalar_name,
JSON_QUERY('{ "name" : "Jakob", "age" : "6" }', '$.age') as json_age,
JSON_VALUE('{ "name" : "Jakob", "age" : "6" }', '$.age') as scalar;
+-----------+-------------+----------+--------+
| json_name | scalar_name | json_age | scalar |
+-----------+-------------+----------+--------+
| "Jakob" | Jakob | "6" | 6 |
+-----------+-------------+----------+--------+
In cases where a JSON key uses invalid JSONPath characters, you can escape those characters using double quotes. For example:
SELECT JSON_VALUE('{"a.b": {"c": "world"}}', '$."a.b".c') as hello;
+-------+
| hello |
+-------+
| world |
+-------+
JSON_EXTRACT_ARRAY
JSON_EXTRACT_ARRAY(json_string_expr[, json_path])
Description
Extracts an array of JSON-formatted strings. If a JSON key uses invalid JSONPath characters, you can escape those characters using single quotes and brackets.
json_string_expr
: A JSON-formatted string. For example:{"class" : {"students" : [{"name" : "Jane"}]}}
json_path
: The JSONPath. This identifies the value or values you want to obtain from the JSON-formatted string. If this optional parameter is not provided, the JSONPath$
symbol is applied, which means that the entire JSON-formatted string is analyzed.
Return type
ARRAY<STRING>
Examples
This extracts the items in a JSON-formatted string to a string array:
SELECT JSON_EXTRACT_ARRAY('[1,2,3]') as string_array;
+--------------+
| string_array |
+--------------+
| [1, 2, 3] |
+--------------+
This extracts a string array and converts it to an integer array:
SELECT ARRAY(
SELECT CAST(integer_element as INT64)
FROM UNNEST(
JSON_EXTRACT_ARRAY('[1,2,3]','$')
) AS integer_element
) AS integer_array;
+---------------+
| integer_array |
+---------------+
| [1, 2, 3] |
+---------------+
This extracts string values in a JSON-formatted string to an array:
-- Doesn't strip the double quotes
SELECT JSON_EXTRACT_ARRAY('["apples","oranges","grapes"]', '$') as string_array;
+---------------------------------+
| string_array |
+---------------------------------+
| ["apples", "oranges", "grapes"] |
+---------------------------------+
-- Strips the double quotes
SELECT ARRAY(
SELECT JSON_EXTRACT_SCALAR(string_element, '$')
FROM UNNEST(JSON_EXTRACT_ARRAY('["apples","oranges","grapes"]','$')) AS string_element
) AS string_array;
+---------------------------+
| string_array |
+---------------------------+
| [apples, oranges, grapes] |
+---------------------------+
This extracts only the items in fruit
to an array:
SELECT JSON_EXTRACT_ARRAY(
'{"fruit":[{"apples":5,"oranges":10},{"apples":2,"oranges":4}],"vegetables":[{"lettuce":7,"kale": 8}}',
'$.fruit'
) as string_array;
+-------------------------------------------------------+
| string_array |
+-------------------------------------------------------+
| [{"apples":5,"oranges":10}, {"apples":2,"oranges":4}] |
+-------------------------------------------------------+
These are equivalent:
SELECT JSON_EXTRACT_ARRAY('{"fruits":["apples","oranges","grapes"]}','$[fruits]') as string_array;
SELECT JSON_EXTRACT_ARRAY('{"fruits":["apples","oranges","grapes"]}','$.fruits') as string_array;
-- The queries above produce the following result:
+---------------------------------+
| string_array |
+---------------------------------+
| ["apples", "oranges", "grapes"] |
+---------------------------------+
In cases where a JSON key uses invalid JSONPath characters, you can escape those
characters using single quotes and brackets, [' ']
. For example:
SELECT JSON_EXTRACT_ARRAY('{"a.b": {"c": ["world"]}}', "$['a.b'].c") as hello;
+-----------+
| hello |
+-----------+
| ["world"] |
+-----------+
The following examples explore how invalid requests and empty arrays are handled:
- If a JSONPath is invalid, an error is thrown.
- If a JSON-formatted string is invalid, the output is NULL.
- It is okay to have empty arrays in the JSON-formatted string.
-- An error is thrown if you provide an invalid JSONPath.
SELECT JSON_EXTRACT_ARRAY('["foo","bar","baz"]','INVALID_JSONPath') as result;
-- If the JSONPath does not refer to an array, NULL is returned.
SELECT JSON_EXTRACT_ARRAY('{"a":"foo"}','$.a') as result;
+--------+
| result |
+--------+
| NULL |
+--------+
-- If a key that does not exist is specified, the result is NULL.
SELECT JSON_EXTRACT_ARRAY('{"a":"foo"}','$.b') as result;
+--------+
| result |
+--------+
| NULL |
+--------+
-- Empty arrays in JSON-formatted strings are supported.
SELECT JSON_EXTRACT_ARRAY('{"a":"foo","b":[]}','$.b') as result;
+--------+
| result |
+--------+
| [] |
+--------+
TO_JSON_STRING
TO_JSON_STRING(value[, pretty_print])
Description
Returns a JSON-formatted string representation of value
. This function
supports an optional boolean parameter called pretty_print
. If pretty_print
is true
, the returned value is formatted for easy readability.
Input data type | Returned value |
---|---|
NULL of any type | null |
BOOL | true or false . |
INT64 | Same as -1 0 12345678901 9007199254740992 -9007199254740992 "9007199254740993"
|
NUMERIC, BIGNUMERIC | Same as -1 0 "9007199254740993" "123.56"
|
FLOAT64 | +/-inf and NaN are represented as
Infinity , -Infinity , and NaN ,
respectively.
Otherwise, the same as |
STRING | Quoted string value, escaped according to the JSON standard.
Specifically, " , \ , and the control characters
from U+0000 to U+001F are escaped. |
BYTES | Quoted RFC 4648 base64-escaped value. For example:
|
DATE | Quoted date. For example: "2017-03-06"
|
TIMESTAMP | Quoted ISO 8601 date-time, where T separates the date and time and Zulu/UTC represents the time zone. For example: "2017-03-06T12:34:56.789012Z"
|
DATETIME | Quoted ISO 8601 date-time, where T separates the date and time. For example: "2017-03-06T12:34:56.789012"
|
TIME | Quoted ISO 8601 time. For example: "12:34:56.789012" |
ARRAY |
Array of zero or more elements. Each element is formatted according to its type. Example without formatting: ["red", "blue", "green"] Example with formatting: [ "red", "blue", "green" ] |
STRUCT |
Object that contains zero or more key/value pairs. Each value is formatted according to its type. Example without formatting: {"colors":["red","blue"],"purchases":12,"inStock": true} Example with formatting: { "color":[ "red", "blue" ] "purchases":12, "inStock": true }
Fields with duplicate names might result in unparseable JSON. Anonymous
fields are represented with
Invalid UTF-8 field names might result in unparseable JSON. String
values are escaped according to the JSON standard. Specifically,
|
Return type
JSON string representation of the value.
Examples
Convert rows in a table to JSON.
With CoordinatesTable AS (
(SELECT 1 AS id, [10,20] AS coordinates) UNION ALL
(SELECT 2 AS id, [30,40] AS coordinates) UNION ALL
(SELECT 3 AS id, [50,60] AS coordinates))
SELECT id, coordinates, TO_JSON_STRING(t) AS json_data
FROM CoordinatesTable as t;
+----+-------------+--------------------------------+
| id | coordinates | json_data |
+----+-------------+--------------------------------+
| 1 | [10, 20] | {"id":1,"coordinates":[10,20]} |
| 2 | [30, 40] | {"id":2,"coordinates":[30,40]} |
| 3 | [50, 60] | {"id":3,"coordinates":[50,60]} |
+----+-------------+--------------------------------+
Convert rows in a table to JSON with formatting.
With CoordinatesTable AS (
(SELECT 1 AS id, [10,20] AS coordinates) UNION ALL
(SELECT 2 AS id, [30,40] AS coordinates))
SELECT id, coordinates, TO_JSON_STRING(t, true) AS json_data
FROM CoordinatesTable as t;
+----+-------------+--------------------+
| id | coordinates | json_data |
+----+-------------+--------------------+
| 1 | [10, 20] | { |
| | | "id": 1, |
| | | "coordinates": [ |
| | | 10, |
| | | 20 |
| | | ] |
| | | } |
+----+-------------+--------------------+
| 2 | [30, 40] | { |
| | | "id": 2, |
| | | "coordinates": [ |
| | | 30, |
| | | 40 |
| | | ] |
| | | } |
+----+-------------+--------------------+
JSONPath
Most JSON functions pass in a json_string_expr
and json_path
parameter. The json_string_expr
parameter passes in a JSON-formatted
string, and the json_path
parameter identifies the value or
values you want to obtain from the JSON-formatted string.
The json_string_expr
parameter must be a JSON string that is
formatted like this:
{"class" : {"students" : [{"name" : "Jane"}]}}
You construct the json_path
parameter using the
JSONPath format. As part of this format, this parameter must start
with a $
symbol, which refers to the outermost level of the JSON-formatted
string. You can identify child values using dots. If the JSON object is an
array, you can use brackets to specify the array index. If the keys contain
$
, dots, or brackets, refer to each JSON function for how to escape
them.
JSONPath | Description | Example | Result using the above json_string_expr |
---|---|---|---|
$ | Root object or element | "$" | {"class":{"students":[{"name":"Jane"}]}} |
. | Child operator | "$.class.students" | [{"name":"Jane"}] |
[] | Subscript operator | "$.class.students[0]" | {"name":"Jane"} |
A JSON functions returns NULL
if the json_path
parameter does
not match a value in json_string_expr
. If the selected value for a scalar
function is not scalar, such as an object or an array, the function
returns NULL
.
If the JSONPath is invalid, the function raises an error.
Array functions
ARRAY
ARRAY(subquery)
Description
The ARRAY
function returns an ARRAY
with one element for each row in a
subquery.
If subquery
produces a
SQL table,
the table must have exactly one column. Each element in the output ARRAY
is
the value of the single column of a row in the table.
If subquery
produces a
value table,
then each element in the output ARRAY
is the entire corresponding row of the
value table.
Constraints
- Subqueries are unordered, so the elements of the output
ARRAY
are not guaranteed to preserve any order in the source table for the subquery. However, if the subquery includes anORDER BY
clause, theARRAY
function will return anARRAY
that honors that clause. - If the subquery returns more than one column, the
ARRAY
function returns an error. - If the subquery returns an
ARRAY
typed column orARRAY
typed rows, theARRAY
function returns an error: BigQuery does not supportARRAY
s with elements of typeARRAY
. - If the subquery returns zero rows, the
ARRAY
function returns an emptyARRAY
. It never returns aNULL
ARRAY
.
Return type
ARRAY
Examples
SELECT ARRAY
(SELECT 1 UNION ALL
SELECT 2 UNION ALL
SELECT 3) AS new_array;
+-----------+
| new_array |
+-----------+
| [1, 2, 3] |
+-----------+
To construct an ARRAY
from a subquery that contains multiple
columns, change the subquery to use SELECT AS STRUCT
. Now
the ARRAY
function will return an ARRAY
of STRUCT
s. The ARRAY
will
contain one STRUCT
for each row in the subquery, and each of these STRUCT
s
will contain a field for each column in that row.
SELECT
ARRAY
(SELECT AS STRUCT 1, 2, 3
UNION ALL SELECT AS STRUCT 4, 5, 6) AS new_array;
+------------------------+
| new_array |
+------------------------+
| [{1, 2, 3}, {4, 5, 6}] |
+------------------------+
Similarly, to construct an ARRAY
from a subquery that contains
one or more ARRAY
s, change the subquery to use SELECT AS STRUCT
.
SELECT ARRAY
(SELECT AS STRUCT [1, 2, 3] UNION ALL
SELECT AS STRUCT [4, 5, 6]) AS new_array;
+----------------------------+
| new_array |
+----------------------------+
| [{[1, 2, 3]}, {[4, 5, 6]}] |
+----------------------------+
ARRAY_CONCAT
ARRAY_CONCAT(array_expression_1 [, array_expression_n])
Description
Concatenates one or more arrays with the same element type into a single array.
Return type
ARRAY
Examples
SELECT ARRAY_CONCAT([1, 2], [3, 4], [5, 6]) as count_to_six;
+--------------------------------------------------+
| count_to_six |
+--------------------------------------------------+
| [1, 2, 3, 4, 5, 6] |
+--------------------------------------------------+
ARRAY_LENGTH
ARRAY_LENGTH(array_expression)
Description
Returns the size of the array. Returns 0 for an empty array. Returns NULL
if
the array_expression
is NULL
.
Return type
INT64
Examples
WITH items AS
(SELECT ["coffee", NULL, "milk" ] as list
UNION ALL
SELECT ["cake", "pie"] as list)
SELECT ARRAY_TO_STRING(list, ', ', 'NULL'), ARRAY_LENGTH(list) AS size
FROM items
ORDER BY size DESC;
+---------------------------------+------+
| list | size |
+---------------------------------+------+
| [coffee, NULL, milk] | 3 |
| [cake, pie] | 2 |
+---------------------------------+------+
ARRAY_TO_STRING
ARRAY_TO_STRING(array_expression, delimiter[, null_text])
Description
Returns a concatenation of the elements in array_expression
as a STRING. The value for array_expression
can either be an array of STRING or
BYTES data types.
If the null_text
parameter is used, the function replaces any NULL
values in
the array with the value of null_text
.
If the null_text
parameter is not used, the function omits the NULL
value
and its preceding delimiter.
Examples
WITH items AS
(SELECT ["coffee", "tea", "milk" ] as list
UNION ALL
SELECT ["cake", "pie", NULL] as list)
SELECT ARRAY_TO_STRING(list, '--') AS text
FROM items;
+--------------------------------+
| text |
+--------------------------------+
| coffee--tea--milk |
| cake--pie |
+--------------------------------+
WITH items AS
(SELECT ["coffee", "tea", "milk" ] as list
UNION ALL
SELECT ["cake", "pie", NULL] as list)
SELECT ARRAY_TO_STRING(list, '--', 'MISSING') AS text
FROM items;
+--------------------------------+
| text |
+--------------------------------+
| coffee--tea--milk |
| cake--pie--MISSING |
+--------------------------------+
GENERATE_ARRAY
GENERATE_ARRAY(start_expression, end_expression[, step_expression])
Description
Returns an array of values. The start_expression
and end_expression
parameters determine the inclusive start and end of the array.
The GENERATE_ARRAY
function accepts the following data types as inputs:
- INT64
- NUMERIC
- BIGNUMERIC
- FLOAT64
The step_expression
parameter determines the increment used to
generate array values. The default value for this parameter is 1
.
This function returns an error if step_expression
is set to 0, or if any
input is NaN
.
If any argument is NULL
, the function will return a NULL
array.
Return Data Type
ARRAY
Examples
The following returns an array of integers, with a default step of 1.
SELECT GENERATE_ARRAY(1, 5) AS example_array;
+-----------------+
| example_array |
+-----------------+
| [1, 2, 3, 4, 5] |
+-----------------+
The following returns an array using a user-specified step size.
SELECT GENERATE_ARRAY(0, 10, 3) AS example_array;
+---------------+
| example_array |
+---------------+
| [0, 3, 6, 9] |
+---------------+
The following returns an array using a negative value, -3
for its step size.
SELECT GENERATE_ARRAY(10, 0, -3) AS example_array;
+---------------+
| example_array |
+---------------+
| [10, 7, 4, 1] |
+---------------+
The following returns an array using the same value for the start_expression
and end_expression
.
SELECT GENERATE_ARRAY(4, 4, 10) AS example_array;
+---------------+
| example_array |
+---------------+
| [4] |
+---------------+
The following returns an empty array, because the start_expression
is greater
than the end_expression
, and the step_expression
value is positive.
SELECT GENERATE_ARRAY(10, 0, 3) AS example_array;
+---------------+
| example_array |
+---------------+
| [] |
+---------------+
The following returns a NULL
array because end_expression
is NULL
.
SELECT GENERATE_ARRAY(5, NULL, 1) AS example_array;
+---------------+
| example_array |
+---------------+
| NULL |
+---------------+
The following returns multiple arrays.
SELECT GENERATE_ARRAY(start, 5) AS example_array
FROM UNNEST([3, 4, 5]) AS start;
+---------------+
| example_array |
+---------------+
| [3, 4, 5] |
| [4, 5] |
| [5] |
+---------------+
GENERATE_DATE_ARRAY
GENERATE_DATE_ARRAY(start_date, end_date[, INTERVAL INT64_expr date_part])
Description
Returns an array of dates. The start_date
and end_date
parameters determine the inclusive start and end of the array.
The GENERATE_DATE_ARRAY
function accepts the following data types as inputs:
start_date
must be a DATEend_date
must be a DATEINT64_expr
must be an INT64date_part
must be either DAY, WEEK, MONTH, QUARTER, or YEAR.
The INT64_expr
parameter determines the increment used to generate dates. The
default value for this parameter is 1 day.
This function returns an error if INT64_expr
is set to 0.
Return Data Type
An ARRAY containing 0 or more DATE values.
Examples
The following returns an array of dates, with a default step of 1.
SELECT GENERATE_DATE_ARRAY('2016-10-05', '2016-10-08') AS example;
+--------------------------------------------------+
| example |
+--------------------------------------------------+
| [2016-10-05, 2016-10-06, 2016-10-07, 2016-10-08] |
+--------------------------------------------------+
The following returns an array using a user-specified step size.
SELECT GENERATE_DATE_ARRAY(
'2016-10-05', '2016-10-09', INTERVAL 2 DAY) AS example;
+--------------------------------------+
| example |
+--------------------------------------+
| [2016-10-05, 2016-10-07, 2016-10-09] |
+--------------------------------------+
The following returns an array using a negative value, -3
for its step size.
SELECT GENERATE_DATE_ARRAY('2016-10-05',
'2016-10-01', INTERVAL -3 DAY) AS example;
+--------------------------+
| example |
+--------------------------+
| [2016-10-05, 2016-10-02] |
+--------------------------+
The following returns an array using the same value for the start_date
and
end_date
.
SELECT GENERATE_DATE_ARRAY('2016-10-05',
'2016-10-05', INTERVAL 8 DAY) AS example;
+--------------+
| example |
+--------------+
| [2016-10-05] |
+--------------+
The following returns an empty array, because the start_date
is greater
than the end_date
, and the step
value is positive.
SELECT GENERATE_DATE_ARRAY('2016-10-05',
'2016-10-01', INTERVAL 1 DAY) AS example;
+---------+
| example |
+---------+
| [] |
+---------+
The following returns a NULL
array, because one of its inputs is
NULL
.
SELECT GENERATE_DATE_ARRAY('2016-10-05', NULL) AS example;
+---------+
| example |
+---------+
| NULL |
+---------+
The following returns an array of dates, using MONTH as the date_part
interval:
SELECT GENERATE_DATE_ARRAY('2016-01-01',
'2016-12-31', INTERVAL 2 MONTH) AS example;
+--------------------------------------------------------------------------+
| example |
+--------------------------------------------------------------------------+
| [2016-01-01, 2016-03-01, 2016-05-01, 2016-07-01, 2016-09-01, 2016-11-01] |
+--------------------------------------------------------------------------+
The following uses non-constant dates to generate an array.
SELECT GENERATE_DATE_ARRAY(date_start, date_end, INTERVAL 1 WEEK) AS date_range
FROM (
SELECT DATE '2016-01-01' AS date_start, DATE '2016-01-31' AS date_end
UNION ALL SELECT DATE "2016-04-01", DATE "2016-04-30"
UNION ALL SELECT DATE "2016-07-01", DATE "2016-07-31"
UNION ALL SELECT DATE "2016-10-01", DATE "2016-10-31"
) AS items;
+--------------------------------------------------------------+
| date_range |
+--------------------------------------------------------------+
| [2016-01-01, 2016-01-08, 2016-01-15, 2016-01-22, 2016-01-29] |
| [2016-04-01, 2016-04-08, 2016-04-15, 2016-04-22, 2016-04-29] |
| [2016-07-01, 2016-07-08, 2016-07-15, 2016-07-22, 2016-07-29] |
| [2016-10-01, 2016-10-08, 2016-10-15, 2016-10-22, 2016-10-29] |
+--------------------------------------------------------------+
GENERATE_TIMESTAMP_ARRAY
GENERATE_TIMESTAMP_ARRAY(start_timestamp, end_timestamp,
INTERVAL step_expression date_part)
Description
Returns an ARRAY
of TIMESTAMPS
separated by a given interval. The
start_timestamp
and end_timestamp
parameters determine the inclusive
lower and upper bounds of the ARRAY
.
The GENERATE_TIMESTAMP_ARRAY
function accepts the following data types as
inputs:
start_timestamp
:TIMESTAMP
end_timestamp
:TIMESTAMP
step_expression
:INT64
- Allowed
date_part
values are:MICROSECOND
,MILLISECOND
,SECOND
,MINUTE
,HOUR
, orDAY
.
The step_expression
parameter determines the increment used to generate
timestamps.
Return Data Type
An ARRAY
containing 0 or more
TIMESTAMP
values.
Examples
The following example returns an ARRAY
of TIMESTAMP
s at intervals of 1 day.
SELECT GENERATE_TIMESTAMP_ARRAY('2016-10-05 00:00:00', '2016-10-07 00:00:00',
INTERVAL 1 DAY) AS timestamp_array;
+--------------------------------------------------------------------------+
| timestamp_array |
+--------------------------------------------------------------------------+
| [2016-10-05 00:00:00+00, 2016-10-06 00:00:00+00, 2016-10-07 00:00:00+00] |
+--------------------------------------------------------------------------+
The following example returns an ARRAY
of TIMESTAMP
s at intervals of 1
second.
SELECT GENERATE_TIMESTAMP_ARRAY('2016-10-05 00:00:00', '2016-10-05 00:00:02',
INTERVAL 1 SECOND) AS timestamp_array;
+--------------------------------------------------------------------------+
| timestamp_array |
+--------------------------------------------------------------------------+
| [2016-10-05 00:00:00+00, 2016-10-05 00:00:01+00, 2016-10-05 00:00:02+00] |
+--------------------------------------------------------------------------+
The following example returns an ARRAY
of TIMESTAMPS
with a negative
interval.
SELECT GENERATE_TIMESTAMP_ARRAY('2016-10-06 00:00:00', '2016-10-01 00:00:00',
INTERVAL -2 DAY) AS timestamp_array;
+--------------------------------------------------------------------------+
| timestamp_array |
+--------------------------------------------------------------------------+
| [2016-10-06 00:00:00+00, 2016-10-04 00:00:00+00, 2016-10-02 00:00:00+00] |
+--------------------------------------------------------------------------+
The following example returns an ARRAY
with a single element, because
start_timestamp
and end_timestamp
have the same value.
SELECT GENERATE_TIMESTAMP_ARRAY('2016-10-05 00:00:00', '2016-10-05 00:00:00',
INTERVAL 1 HOUR) AS timestamp_array;
+--------------------------+
| timestamp_array |
+--------------------------+
| [2016-10-05 00:00:00+00] |
+--------------------------+
The following example returns an empty ARRAY
, because start_timestamp
is
later than end_timestamp
.
SELECT GENERATE_TIMESTAMP_ARRAY('2016-10-06 00:00:00', '2016-10-05 00:00:00',
INTERVAL 1 HOUR) AS timestamp_array;
+-----------------+
| timestamp_array |
+-----------------+
| [] |
+-----------------+
The following example returns a null ARRAY
, because one of the inputs is
NULL
.
SELECT GENERATE_TIMESTAMP_ARRAY('2016-10-05 00:00:00', NULL, INTERVAL 1 HOUR)
AS timestamp_array;
+-----------------+
| timestamp_array |
+-----------------+
| NULL |
+-----------------+
The following example generates ARRAY
s of TIMESTAMP
s from columns containing
values for start_timestamp
and end_timestamp
.
SELECT GENERATE_TIMESTAMP_ARRAY(start_timestamp, end_timestamp, INTERVAL 1 HOUR)
AS timestamp_array
FROM
(SELECT
TIMESTAMP '2016-10-05 00:00:00' AS start_timestamp,
TIMESTAMP '2016-10-05 02:00:00' AS end_timestamp
UNION ALL
SELECT
TIMESTAMP '2016-10-05 12:00:00' AS start_timestamp,
TIMESTAMP '2016-10-05 14:00:00' AS end_timestamp
UNION ALL
SELECT
TIMESTAMP '2016-10-05 23:59:00' AS start_timestamp,
TIMESTAMP '2016-10-06 01:59:00' AS end_timestamp);
+--------------------------------------------------------------------------+
| timestamp_array |
+--------------------------------------------------------------------------+
| [2016-10-05 00:00:00+00, 2016-10-05 01:00:00+00, 2016-10-05 02:00:00+00] |
| [2016-10-05 12:00:00+00, 2016-10-05 13:00:00+00, 2016-10-05 14:00:00+00] |
| [2016-10-05 23:59:00+00, 2016-10-06 00:59:00+00, 2016-10-06 01:59:00+00] |
+--------------------------------------------------------------------------+
OFFSET and ORDINAL
array_expression[OFFSET(zero_based_offset)]
array_expression[ORDINAL(one_based_offset)]
Description
Accesses an ARRAY element by position and returns the
element. OFFSET
means that the numbering starts at zero, ORDINAL
means that
the numbering starts at one.
A given array can be interpreted as either 0-based or 1-based. When accessing an
array element, you must preface the array position with OFFSET
or ORDINAL
,
respectively; there is no default behavior.
Both OFFSET
and ORDINAL
generate an error if the index is out of range.
Return type
Varies depending on the elements in the ARRAY.
Examples
WITH items AS
(SELECT ["apples", "bananas", "pears", "grapes"] as list
UNION ALL
SELECT ["coffee", "tea", "milk" ] as list
UNION ALL
SELECT ["cake", "pie"] as list)
SELECT list, list[OFFSET(1)] as offset_1, list[ORDINAL(1)] as ordinal_1
FROM items;
+----------------------------------+-----------+-----------+
| list | offset_1 | ordinal_1 |
+----------------------------------+-----------+-----------+
| [apples, bananas, pears, grapes] | bananas | apples |
| [coffee, tea, milk] | tea | coffee |
| [cake, pie] | pie | cake |
+----------------------------------+-----------+-----------+
ARRAY_REVERSE
ARRAY_REVERSE(value)
Description
Returns the input ARRAY with elements in reverse order.
Return type
ARRAY
Examples
WITH example AS (
SELECT [1, 2, 3] AS arr UNION ALL
SELECT [4, 5] AS arr UNION ALL
SELECT [] AS arr
)
SELECT
arr,
ARRAY_REVERSE(arr) AS reverse_arr
FROM example;
+-----------+-------------+
| arr | reverse_arr |
+-----------+-------------+
| [1, 2, 3] | [3, 2, 1] |
| [4, 5] | [5, 4] |
| [] | [] |
+-----------+-------------+
SAFE_OFFSET and SAFE_ORDINAL
array_expression[SAFE_OFFSET(zero_based_offset)]
array_expression[SAFE_ORDINAL(one_based_offset)]
Description
Identical to OFFSET
and ORDINAL
, except returns NULL
if the index is out
of range.
Return type
Varies depending on the elements in the ARRAY.
Example
WITH items AS
(SELECT ["apples", "bananas", "pears", "grapes"] as list
UNION ALL
SELECT ["coffee", "tea", "milk" ] as list
UNION ALL
SELECT ["cake", "pie"] as list)
SELECT list,
list[SAFE_OFFSET(3)] as safe_offset_3,
list[SAFE_ORDINAL(3)] as safe_ordinal_3
FROM items;
+----------------------------------+---------------+----------------+
| list | safe_offset_3 | safe_ordinal_3 |
+----------------------------------+---------------+----------------+
| [apples, bananas, pears, grapes] | grapes | pears |
| [coffee, tea, milk] | NULL | milk |
| [cake, pie] | NULL | NULL |
+----------------------------------+---------------+----------------+
Date functions
BigQuery supports the following DATE
functions.
CURRENT_DATE
CURRENT_DATE([time_zone])
Description
Returns the current date as of the specified or default timezone. Parentheses are optional when called with no arguments.
This function supports an optional
time_zone
parameter. This parameter is a string representing the timezone to
use. If no timezone is specified, the default timezone, UTC,
is used. See Timezone definitions
for information on how to specify a time zone.
If the time_zone
parameter evaluates to NULL
, this function returns NULL
.
Return Data Type
DATE
Example
SELECT CURRENT_DATE() as the_date;
+--------------+
| the_date |
+--------------+
| 2016-12-25 |
+--------------+
When a column named current_date
is present, the column name and the function
call without parentheses are ambiguous. To ensure the function call, add
parentheses; to ensure the column name, qualify it with its
range variable. For example, the
following query will select the function in the the_date
column and the table
column in the current_date
column.
WITH t AS (SELECT 'column value' AS `current_date`)
SELECT current_date() AS the_date, t.current_date FROM t;
+------------+--------------+
| the_date | current_date |
+------------+--------------+
| 2016-12-25 | column value |
+------------+--------------+
EXTRACT
EXTRACT(part FROM date_expression)
Description
Returns the value corresponding to the specified date part. The part
must
be one of:
DAYOFWEEK
: Returns values in the range [1,7] with Sunday as the first day of the week.DAY
DAYOFYEAR
WEEK
: Returns the week number of the date in the range [0, 53]. Weeks begin with Sunday, and dates prior to the first Sunday of the year are in week 0.WEEK(<WEEKDAY>)
: Returns the week number of the date in the range [0, 53]. Weeks begin onWEEKDAY
. Dates prior to the firstWEEKDAY
of the year are in week 0. Valid values forWEEKDAY
areSUNDAY
,MONDAY
,TUESDAY
,WEDNESDAY
,THURSDAY
,FRIDAY
, andSATURDAY
.ISOWEEK
: Returns the ISO 8601 week number of thedate_expression
.ISOWEEK
s begin on Monday. Return values are in the range [1, 53]. The firstISOWEEK
of each ISO year begins on the Monday before the first Thursday of the Gregorian calendar year.MONTH
QUARTER
: Returns values in the range [1,4].YEAR
ISOYEAR
: Returns the ISO 8601 week-numbering year, which is the Gregorian calendar year containing the Thursday of the week to whichdate_expression
belongs.
Return Data Type
INT64
Examples
In the following example, EXTRACT
returns a value corresponding to the DAY
date part.
SELECT EXTRACT(DAY FROM DATE '2013-12-25') as the_day;
+---------+
| the_day |
+---------+
| 25 |
+---------+
In the following example, EXTRACT
returns values corresponding to different
date parts from a column of dates near the end of the year.
SELECT
date,
EXTRACT(ISOYEAR FROM date) AS isoyear,
EXTRACT(ISOWEEK FROM date) AS isoweek,
EXTRACT(YEAR FROM date) AS year,
EXTRACT(WEEK FROM date) AS week
FROM UNNEST(GENERATE_DATE_ARRAY('2015-12-23', '2016-01-09')) AS date
ORDER BY date;
+------------+---------+---------+------+------+
| date | isoyear | isoweek | year | week |
+------------+---------+---------+------+------+
| 2015-12-23 | 2015 | 52 | 2015 | 51 |
| 2015-12-24 | 2015 | 52 | 2015 | 51 |
| 2015-12-25 | 2015 | 52 | 2015 | 51 |
| 2015-12-26 | 2015 | 52 | 2015 | 51 |
| 2015-12-27 | 2015 | 52 | 2015 | 52 |
| 2015-12-28 | 2015 | 53 | 2015 | 52 |
| 2015-12-29 | 2015 | 53 | 2015 | 52 |
| 2015-12-30 | 2015 | 53 | 2015 | 52 |
| 2015-12-31 | 2015 | 53 | 2015 | 52 |
| 2016-01-01 | 2015 | 53 | 2016 | 0 |
| 2016-01-02 | 2015 | 53 | 2016 | 0 |
| 2016-01-03 | 2015 | 53 | 2016 | 1 |
| 2016-01-04 | 2016 | 1 | 2016 | 1 |
| 2016-01-05 | 2016 | 1 | 2016 | 1 |
| 2016-01-06 | 2016 | 1 | 2016 | 1 |
| 2016-01-07 | 2016 | 1 | 2016 | 1 |
| 2016-01-08 | 2016 | 1 | 2016 | 1 |
| 2016-01-09 | 2016 | 1 | 2016 | 1 |
+------------+---------+---------+------+------+
In the following example, date_expression
falls on a Sunday. EXTRACT
calculates the first column using weeks that begin on Sunday, and it calculates
the second column using weeks that begin on Monday.
WITH table AS (SELECT DATE('2017-11-05') AS date)
SELECT
date,
EXTRACT(WEEK(SUNDAY) FROM date) AS week_sunday,
EXTRACT(WEEK(MONDAY) FROM date) AS week_monday FROM table;
+------------+-------------+-------------+
| date | week_sunday | week_monday |
+------------+-------------+-------------+
| 2017-11-05 | 45 | 44 |
+------------+-------------+-------------+
DATE
1. DATE(year, month, day)
2. DATE(timestamp_expression[, timezone])
3. DATE(datetime_expression)
Description
- Constructs a DATE from INT64 values representing the year, month, and day.
- Extracts the DATE from a TIMESTAMP expression. It supports an optional parameter to specify a timezone. If no timezone is specified, the default timezone, UTC, is used.
- Extracts the DATE from a DATETIME expression.
Return Data Type
DATE
Example
SELECT
DATE(2016, 12, 25) as date_ymd,
DATE(DATETIME "2016-12-25 23:59:59") as date_dt,
DATE(TIMESTAMP "2016-12-25 05:30:00+07", "America/Los_Angeles") as date_tstz;
+------------+------------+------------+
| date_ymd | date_dt | date_tstz |
+------------+------------+------------+
| 2016-12-25 | 2016-12-25 | 2016-12-24 |
+------------+------------+------------+
DATE_ADD
DATE_ADD(date_expression, INTERVAL int64_expression date_part)
Description
Adds a specified time interval to a DATE.
DATE_ADD
supports the following date_part
values:
DAY
WEEK
. Equivalent to 7DAY
s.MONTH
QUARTER
YEAR
Special handling is required for MONTH, QUARTER, and YEAR parts when the date is at (or near) the last day of the month. If the resulting month has fewer days than the original date's day, then the result day is the last day of the new month.
Return Data Type
DATE
Example
SELECT DATE_ADD(DATE "2008-12-25", INTERVAL 5 DAY) as five_days_later;
+--------------------+
| five_days_later |
+--------------------+
| 2008-12-30 |
+--------------------+
DATE_SUB
DATE_SUB(date_expression, INTERVAL int64_expression date_part)
Description
Subtracts a specified time interval from a DATE.
DATE_SUB
supports the following date_part
values:
DAY
WEEK
. Equivalent to 7DAY
s.MONTH
QUARTER
YEAR
Special handling is required for MONTH, QUARTER, and YEAR parts when the date is at (or near) the last day of the month. If the resulting month has fewer days than the original date's day, then the result day is the last day of the new month.
Return Data Type
DATE
Example
SELECT DATE_SUB(DATE "2008-12-25", INTERVAL 5 DAY) as five_days_ago;
+---------------+
| five_days_ago |
+---------------+
| 2008-12-20 |
+---------------+
DATE_DIFF
DATE_DIFF(date_expression_a, date_expression_b, date_part)
Description
Returns the number of whole specified date_part
intervals between two DATE
objects
(date_expression_a
- date_expression_b
).
If the first DATE
is earlier than the second one,
the output is negative.
DATE_DIFF
supports the following date_part
values:
DAY
WEEK
This date part begins on Sunday.WEEK(<WEEKDAY>)
: This date part begins onWEEKDAY
. Valid values forWEEKDAY
areSUNDAY
,MONDAY
,TUESDAY
,WEDNESDAY
,THURSDAY
,FRIDAY
, andSATURDAY
.ISOWEEK
: Uses ISO 8601 week boundaries. ISO weeks begin on Monday.MONTH
QUARTER
YEAR
ISOYEAR
: Uses the ISO 8601 week-numbering year boundary. The ISO year boundary is the Monday of the first week whose Thursday belongs to the corresponding Gregorian calendar year.
Return Data Type
INT64
Example
SELECT DATE_DIFF(DATE '2010-07-07', DATE '2008-12-25', DAY) as days_diff;
+-----------+
| days_diff |
+-----------+
| 559 |
+-----------+
SELECT
DATE_DIFF(DATE '2017-10-15', DATE '2017-10-14', DAY) as days_diff,
DATE_DIFF(DATE '2017-10-15', DATE '2017-10-14', WEEK) as weeks_diff;
+-----------+------------+
| days_diff | weeks_diff |
+-----------+------------+
| 1 | 1 |
+-----------+------------+
The example above shows the result of DATE_DIFF
for two days in succession.
DATE_DIFF
with the date part WEEK
returns 1 because DATE_DIFF
counts the
number of date part boundaries in this range of dates. Each WEEK
begins on
Sunday, so there is one date part boundary between Saturday, 2017-10-14
and Sunday, 2017-10-15.
The following example shows the result of DATE_DIFF
for two dates in different
years. DATE_DIFF
with the date part YEAR
returns 3 because it counts the
number of Gregorian calendar year boundaries between the two dates. DATE_DIFF
with the date part ISOYEAR
returns 2 because the second date belongs to the
ISO year 2015. The first Thursday of the 2015 calendar year was 2015-01-01, so
the ISO year 2015 begins on the preceding Monday, 2014-12-29.
SELECT
DATE_DIFF('2017-12-30', '2014-12-30', YEAR) AS year_diff,
DATE_DIFF('2017-12-30', '2014-12-30', ISOYEAR) AS isoyear_diff;
+-----------+--------------+
| year_diff | isoyear_diff |
+-----------+--------------+
| 3 | 2 |
+-----------+--------------+
The following example shows the result of DATE_DIFF
for two days in
succession. The first date falls on a Monday and the second date falls on a
Sunday. DATE_DIFF
with the date part WEEK
returns 0 because this date part
uses weeks that begin on Sunday. DATE_DIFF
with the date part WEEK(MONDAY)
returns 1. DATE_DIFF
with the date part ISOWEEK
also returns 1 because
ISO weeks begin on Monday.
SELECT
DATE_DIFF('2017-12-18', '2017-12-17', WEEK) AS week_diff,
DATE_DIFF('2017-12-18', '2017-12-17', WEEK(MONDAY)) AS week_weekday_diff,
DATE_DIFF('2017-12-18', '2017-12-17', ISOWEEK) AS isoweek_diff;
+-----------+-------------------+--------------+
| week_diff | week_weekday_diff | isoweek_diff |
+-----------+-------------------+--------------+
| 0 | 1 | 1 |
+-----------+-------------------+--------------+
DATE_TRUNC
DATE_TRUNC(date_expression, date_part)
Description
Truncates the date to the specified granularity.
DATE_TRUNC
supports the following values for date_part
:
DAY
WEEK
WEEK(<WEEKDAY>)
: Truncatesdate_expression
to the preceding week boundary, where weeks begin onWEEKDAY
. Valid values forWEEKDAY
areSUNDAY
,MONDAY
,TUESDAY
,WEDNESDAY
,THURSDAY
,FRIDAY
, andSATURDAY
.ISOWEEK
: Truncatesdate_expression
to the preceding ISO 8601 week boundary.ISOWEEK
s begin on Monday. The firstISOWEEK
of each ISO year contains the first Thursday of the corresponding Gregorian calendar year. Anydate_expression
earlier than this will truncate to the preceding Monday.MONTH
QUARTER
YEAR
ISOYEAR
: Truncatesdate_expression
to the preceding ISO 8601 week-numbering year boundary. The ISO year boundary is the Monday of the first week whose Thursday belongs to the corresponding Gregorian calendar year.
Return Data Type
DATE
Examples
SELECT DATE_TRUNC(DATE '2008-12-25', MONTH) as month;
+------------+
| month |
+------------+
| 2008-12-01 |
+------------+
In the following example, the original date falls on a Sunday. Because
the date_part
is WEEK(MONDAY)
,