BigQuery API - Class Google::Cloud::Bigquery::Schema (v1.44.1)

Reference documentation and code samples for the BigQuery API class Google::Cloud::Bigquery::Schema.

Table Schema

A builder for BigQuery table schemas, passed to block arguments to Dataset#create_table and Table#schema. Supports nested and repeated fields via a nested block.

Inherits

Object

Example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.create_table "my_table"

table.schema do |schema|
  schema.string "first_name", mode: :required
  schema.record "cities_lived", mode: :repeated do |cities_lived|
    cities_lived.string "place", mode: :required
    cities_lived.integer "number_of_years", mode: :required
  end
end

Methods

.dump

def self.dump(schema, destination) -> Schema

Write a schema as JSON to a file.

The JSON schema file is the same as for the bq CLI.

Parameters

schema (Schema) — A Google::Cloud::Bigquery::Schema.
destination (IO, String) — An IO to which to write the schema, or a String containing the filename to write to.

Returns

(Schema) — The schema so that commands are chainable.

Example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
schema = Google::Cloud::Bigquery::Schema.dump(
  table.schema,
  "schema.json"
)

.load

def self.load(source) -> Schema

Load a schema from a JSON file.

The JSON schema file is the same as for the bq CLI consisting of an array of JSON objects containing the following:

name: The column name
type: The column's data type
description: (Optional) The column's description
mode: (Optional) The column's mode (if unspecified, mode defaults to NULLABLE)
fields: If type is RECORD, an array of objects defining child fields with these properties

Parameter

source (IO, String, Array<Hash>) — An IO containing the JSON schema, a String containing the JSON schema, or an Array of Hashes containing the schema details.

Returns

(Schema) — A schema.

Example

require "google/cloud/bigquery"

schema = Google::Cloud::Bigquery::Schema.load(
  File.read("schema.json")
)

#bignumeric

def bignumeric(name, description: nil, mode: :nullable, policy_tags: nil, precision: nil, scale: nil, default_value_expression: nil)

Adds a bignumeric number field to the schema. BIGNUMERIC is a decimal type with fixed precision and scale. Precision is the number of digits that the number contains. Scale is how many of these digits appear after the decimal point. It supports:

Precision: 76.76 (the 77th digit is partial) Scale: 38 Min: -5.7896044618658097711785492504343953926634992332820282019728792003956564819968E+38 Max: 5.7896044618658097711785492504343953926634992332820282019728792003956564819967E+38

This type can represent decimal fractions exactly, and is suitable for financial calculations.

Parameters

name (String) — The field name. The name must contain only letters ([A-Za-z]), numbers ([0-9]), or underscores (_), and must start with a letter or underscore. The maximum length is 128 characters.
description (String) (defaults to: nil) — A description of the field.
mode (Symbol) (defaults to: :nullable) — The field's mode. The possible values are :nullable, :required, and :repeated. The default value is :nullable.
policy_tags (Array<String>, String) (defaults to: nil) — The policy tag list or single policy tag for the field. Policy tag identifiers are of the form projects/*/locations/*/taxonomies/*/policyTags/*. At most 1 policy tag is currently allowed.
precision (Integer) (defaults to: nil) — The precision (maximum number of total digits) for the field. Acceptable values for precision must be: 1 ≤ (precision - scale) ≤ 38. Values for scale must be: 0 ≤ scale ≤ 38. If the scale value is set, the precision value must be set as well.
scale (Integer) (defaults to: nil) — The scale (maximum number of digits in the fractional part) for the field. Acceptable values for precision must be: 1 ≤ (precision - scale) ≤ 38. Values for scale must be: 0 ≤ scale ≤ 38. If the scale value is set, the precision value must be set as well.
default_value_expression (String) (defaults to: nil) —
The default value of a field using a SQL expression. It can only be set for top level fields (columns). Use a struct or array expression to specify default value for the entire struct or array. The valid SQL expressions are:
- Literals for all data types, including STRUCT and ARRAY.
- The following functions: CURRENT_TIMESTAMP CURRENT_TIME CURRENT_DATE CURRENT_DATETIME GENERATE_UUID RAND SESSION_USER ST_GEOPOINT
- Struct or array composed with the above allowed functions, for example: "[CURRENT_DATE(), DATE '2020-01-01'"]

#boolean

def boolean(name, description: nil, mode: :nullable, policy_tags: nil, default_value_expression: nil)

Adds a boolean field to the schema.

Parameters

name (String) — The field name. The name must contain only letters ([A-Za-z]), numbers ([0-9]), or underscores (_), and must start with a letter or underscore. The maximum length is 128 characters.
description (String) (defaults to: nil) — A description of the field.
mode (Symbol) (defaults to: :nullable) — The field's mode. The possible values are :nullable, :required, and :repeated. The default value is :nullable.
policy_tags (Array<String>, String) (defaults to: nil) — The policy tag list or single policy tag for the field. Policy tag identifiers are of the form projects/*/locations/*/taxonomies/*/policyTags/*. At most 1 policy tag is currently allowed.
default_value_expression (String) (defaults to: nil) —
The default value of a field using a SQL expression. It can only be set for top level fields (columns). Use a struct or array expression to specify default value for the entire struct or array. The valid SQL expressions are:
- Literals for all data types, including STRUCT and ARRAY.
- The following functions: CURRENT_TIMESTAMP CURRENT_TIME CURRENT_DATE CURRENT_DATETIME GENERATE_UUID RAND SESSION_USER ST_GEOPOINT
- Struct or array composed with the above allowed functions, for example: "[CURRENT_DATE(), DATE '2020-01-01'"]

#bytes

def bytes(name, description: nil, mode: :nullable, policy_tags: nil, max_length: nil, default_value_expression: nil)

Adds a bytes field to the schema.

Parameters

name (String) — The field name. The name must contain only letters ([A-Za-z]), numbers ([0-9]), or underscores (_), and must start with a letter or underscore. The maximum length is 128 characters.
description (String) (defaults to: nil) — A description of the field.
mode (Symbol) (defaults to: :nullable) — The field's mode. The possible values are :nullable, :required, and :repeated. The default value is :nullable.
policy_tags (Array<String>, String) (defaults to: nil) — The policy tag list or single policy tag for the field. Policy tag identifiers are of the form projects/*/locations/*/taxonomies/*/policyTags/*. At most 1 policy tag is currently allowed.
max_length (Integer) (defaults to: nil) — The maximum the maximum number of bytes in the field.
default_value_expression (String) (defaults to: nil) —
The default value of a field using a SQL expression. It can only be set for top level fields (columns). Use a struct or array expression to specify default value for the entire struct or array. The valid SQL expressions are:
- Literals for all data types, including STRUCT and ARRAY.
- The following functions: CURRENT_TIMESTAMP CURRENT_TIME CURRENT_DATE CURRENT_DATETIME GENERATE_UUID RAND SESSION_USER ST_GEOPOINT
- Struct or array composed with the above allowed functions, for example: "[CURRENT_DATE(), DATE '2020-01-01'"]

#date

def date(name, description: nil, mode: :nullable, policy_tags: nil, default_value_expression: nil)

Adds a date field to the schema.

Parameters

name (String) — The field name. The name must contain only letters ([A-Za-z]), numbers ([0-9]), or underscores (_), and must start with a letter or underscore. The maximum length is 128 characters.
description (String) (defaults to: nil) — A description of the field.
mode (Symbol) (defaults to: :nullable) — The field's mode. The possible values are :nullable, :required, and :repeated. The default value is :nullable.
policy_tags (Array<String>, String) (defaults to: nil) — The policy tag list or single policy tag for the field. Policy tag identifiers are of the form projects/*/locations/*/taxonomies/*/policyTags/*. At most 1 policy tag is currently allowed.
default_value_expression (String) (defaults to: nil) —
The default value of a field using a SQL expression. It can only be set for top level fields (columns). Use a struct or array expression to specify default value for the entire struct or array. The valid SQL expressions are:
- Literals for all data types, including STRUCT and ARRAY.
- The following functions: CURRENT_TIMESTAMP CURRENT_TIME CURRENT_DATE CURRENT_DATETIME GENERATE_UUID RAND SESSION_USER ST_GEOPOINT
- Struct or array composed with the above allowed functions, for example: "[CURRENT_DATE(), DATE '2020-01-01'"]

#datetime

def datetime(name, description: nil, mode: :nullable, policy_tags: nil, default_value_expression: nil)

Adds a datetime field to the schema.

Parameters

name (String) — The field name. The name must contain only letters ([A-Za-z]), numbers ([0-9]), or underscores (_), and must start with a letter or underscore. The maximum length is 128 characters.
description (String) (defaults to: nil) — A description of the field.
mode (Symbol) (defaults to: :nullable) — The field's mode. The possible values are :nullable, :required, and :repeated. The default value is :nullable.
policy_tags (Array<String>, String) (defaults to: nil) — The policy tag list or single policy tag for the field. Policy tag identifiers are of the form projects/*/locations/*/taxonomies/*/policyTags/*. At most 1 policy tag is currently allowed.
default_value_expression (String) (defaults to: nil) —
The default value of a field using a SQL expression. It can only be set for top level fields (columns). Use a struct or array expression to specify default value for the entire struct or array. The valid SQL expressions are:
- Literals for all data types, including STRUCT and ARRAY.
- The following functions: CURRENT_TIMESTAMP CURRENT_TIME CURRENT_DATE CURRENT_DATETIME GENERATE_UUID RAND SESSION_USER ST_GEOPOINT
- Struct or array composed with the above allowed functions, for example: "[CURRENT_DATE(), DATE '2020-01-01'"]

#dump

def dump(destination) -> Schema

Write the schema as JSON to a file.

The JSON schema file is the same as for the bq CLI.

Parameter

destination (IO, String) — An IO to which to write the schema, or a String containing the filename to write to.

Returns

(Schema) — The schema so that commands are chainable.

Example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"
table.schema.dump "schema.json"

#empty?

def empty?() -> Boolean

Whether the schema has no fields defined.

Returns

(Boolean) — true when there are no fields, false otherwise.

#field

def field(name) -> Field

Retrieve a field by name.

Yields

Returns

(Field) — A field object.

Example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

field = table.schema.field "name"
field.required? #=> true

#fields

def fields() -> Array<Field>

The fields of the table schema.

Returns

(Array<Field>) — An array of field objects.

Example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table"

schema = table.schema

schema.fields.each do |field|
  puts field.name
end

#float

def float(name, description: nil, mode: :nullable, policy_tags: nil, default_value_expression: nil)

Adds a floating-point number field to the schema.

Parameters

name (String) — The field name. The name must contain only letters ([A-Za-z]), numbers ([0-9]), or underscores (_), and must start with a letter or underscore. The maximum length is 128 characters.
description (String) (defaults to: nil) — A description of the field.
mode (Symbol) (defaults to: :nullable) — The field's mode. The possible values are :nullable, :required, and :repeated. The default value is :nullable.
policy_tags (Array<String>, String) (defaults to: nil) — The policy tag list or single policy tag for the field. Policy tag identifiers are of the form projects/*/locations/*/taxonomies/*/policyTags/*. At most 1 policy tag is currently allowed.
default_value_expression (String) (defaults to: nil) —
The default value of a field using a SQL expression. It can only be set for top level fields (columns). Use a struct or array expression to specify default value for the entire struct or array. The valid SQL expressions are:
- Literals for all data types, including STRUCT and ARRAY.
- The following functions: CURRENT_TIMESTAMP CURRENT_TIME CURRENT_DATE CURRENT_DATETIME GENERATE_UUID RAND SESSION_USER ST_GEOPOINT
- Struct or array composed with the above allowed functions, for example: "[CURRENT_DATE(), DATE '2020-01-01'"]

#geography

def geography(name, description: nil, mode: :nullable, policy_tags: nil, default_value_expression: nil)

Adds a geography field to the schema.

Parameters

name (String) — The field name. The name must contain only letters ([A-Za-z]), numbers ([0-9]), or underscores (_), and must start with a letter or underscore. The maximum length is 128 characters.
description (String) (defaults to: nil) — A description of the field.
mode (Symbol) (defaults to: :nullable) — The field's mode. The possible values are :nullable, :required, and :repeated. The default value is :nullable.
policy_tags (Array<String>, String) (defaults to: nil) — The policy tag list or single policy tag for the field. Policy tag identifiers are of the form projects/*/locations/*/taxonomies/*/policyTags/*. At most 1 policy tag is currently allowed.
default_value_expression (String) (defaults to: nil) —
The default value of a field using a SQL expression. It can only be set for top level fields (columns). Use a struct or array expression to specify default value for the entire struct or array. The valid SQL expressions are:
- Literals for all data types, including STRUCT and ARRAY.
- The following functions: CURRENT_TIMESTAMP CURRENT_TIME CURRENT_DATE CURRENT_DATETIME GENERATE_UUID RAND SESSION_USER ST_GEOPOINT
- Struct or array composed with the above allowed functions, for example: "[CURRENT_DATE(), DATE '2020-01-01'"]

#headers

def headers() -> Array<Symbol>

The names of the fields as symbols.

Returns

(Array<Symbol>) — An array of column names.

Example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.create_table "my_table"

schema = table.schema

schema.headers.each do |header|
  puts header
end

#integer

def integer(name, description: nil, mode: :nullable, policy_tags: nil, default_value_expression: nil)

Adds an integer field to the schema.

Parameters

name (String) — The field name. The name must contain only letters ([A-Za-z]), numbers ([0-9]), or underscores (_), and must start with a letter or underscore. The maximum length is 128 characters.
description (String) (defaults to: nil) — A description of the field.
mode (Symbol) (defaults to: :nullable) — The field's mode. The possible values are :nullable, :required, and :repeated. The default value is :nullable.
policy_tags (Array<String>, String) (defaults to: nil) — The policy tag list or single policy tag for the field. Policy tag identifiers are of the form projects/*/locations/*/taxonomies/*/policyTags/*. At most 1 policy tag is currently allowed.
default_value_expression (String) (defaults to: nil) —
The default value of a field using a SQL expression. It can only be set for top level fields (columns). Use a struct or array expression to specify default value for the entire struct or array. The valid SQL expressions are:
- Literals for all data types, including STRUCT and ARRAY.
- The following functions: CURRENT_TIMESTAMP CURRENT_TIME CURRENT_DATE CURRENT_DATETIME GENERATE_UUID RAND SESSION_USER ST_GEOPOINT
- Struct or array composed with the above allowed functions, for example: "[CURRENT_DATE(), DATE '2020-01-01'"]

#load

def load(source) -> Schema

Load the schema from a JSON file.

The JSON schema file is the same as for the bq CLI consisting of an array of JSON objects containing the following:

name: The column name
type: The column's data type
description: (Optional) The column's description
mode: (Optional) The column's mode (if unspecified, mode defaults to NULLABLE)
fields: If type is RECORD, an array of objects defining child fields with these properties

Parameter

source (IO, String, Array<Hash>) — An IO containing the JSON schema, a String containing the JSON schema, or an Array of Hashes containing the schema details.

Returns

(Schema) — The schema so that commands are chainable.

Example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.table "my_table" do |t|
  t.schema.load File.read("path/to/schema.json")
end

#numeric

def numeric(name, description: nil, mode: :nullable, policy_tags: nil, precision: nil, scale: nil, default_value_expression: nil)

Adds a numeric number field to the schema. NUMERIC is a decimal type with fixed precision and scale. Precision is the number of digits that the number contains. Scale is how many of these digits appear after the decimal point. It supports:

Precision: 38 Scale: 9 Min: -9.9999999999999999999999999999999999999E+28 Max: 9.9999999999999999999999999999999999999E+28

This type can represent decimal fractions exactly, and is suitable for financial calculations.

Parameters

name (String) — The field name. The name must contain only letters ([A-Za-z]), numbers ([0-9]), or underscores (_), and must start with a letter or underscore. The maximum length is 128 characters.
description (String) (defaults to: nil) — A description of the field.
mode (Symbol) (defaults to: :nullable) — The field's mode. The possible values are :nullable, :required, and :repeated. The default value is :nullable.
policy_tags (Array<String>, String) (defaults to: nil) — The policy tag list or single policy tag for the field. Policy tag identifiers are of the form projects/*/locations/*/taxonomies/*/policyTags/*. At most 1 policy tag is currently allowed.
precision (Integer) (defaults to: nil) — The precision (maximum number of total digits) for the field. Acceptable values for precision must be: 1 ≤ (precision - scale) ≤ 29. Values for scale must be: 0 ≤ scale ≤ 9. If the scale value is set, the precision value must be set as well.
scale (Integer) (defaults to: nil) — The scale (maximum number of digits in the fractional part) for the field. Acceptable values for precision must be: 1 ≤ (precision - scale) ≤ 29. Values for scale must be: 0 ≤ scale ≤ 9. If the scale value is set, the precision value must be set as well.
default_value_expression (String) (defaults to: nil) —
The default value of a field using a SQL expression. It can only be set for top level fields (columns). Use a struct or array expression to specify default value for the entire struct or array. The valid SQL expressions are:
- Literals for all data types, including STRUCT and ARRAY.
- The following functions: CURRENT_TIMESTAMP CURRENT_TIME CURRENT_DATE CURRENT_DATETIME GENERATE_UUID RAND SESSION_USER ST_GEOPOINT
- Struct or array composed with the above allowed functions, for example: "[CURRENT_DATE(), DATE '2020-01-01'"]

#param_types

def param_types() -> Hash

The types of the fields, using the same format as the optional query parameter types.

Returns

(Hash) — A hash with column names as keys, and types as values.

Example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.create_table "my_table"

schema = table.schema

schema.param_types

#record

def record(name, description: nil, mode: nil, default_value_expression: nil) { |field| ... }

Adds a record field to the schema. A block must be passed describing the nested fields of the record. For more information about nested and repeated records, see Loading denormalized, nested, and repeated data .

Parameters

name (String) — The field name. The name must contain only letters ([A-Za-z]), numbers ([0-9]), or underscores (_), and must start with a letter or underscore. The maximum length is 128 characters.
description (String) (defaults to: nil) — A description of the field.
mode (Symbol) (defaults to: nil) — The field's mode. The possible values are :nullable, :required, and :repeated. The default value is :nullable.
default_value_expression (String) (defaults to: nil) —
The default value of a field using a SQL expression. It can only be set for top level fields (columns). Use a struct or array expression to specify default value for the entire struct or array. The valid SQL expressions are:
- Literals for all data types, including STRUCT and ARRAY.
- The following functions: CURRENT_TIMESTAMP CURRENT_TIME CURRENT_DATE CURRENT_DATETIME GENERATE_UUID RAND SESSION_USER ST_GEOPOINT
- Struct or array composed with the above allowed functions, for example: "[CURRENT_DATE(), DATE '2020-01-01'"]

Yields

(field) — a block for setting the nested record's schema

Yield Parameter

field (Field) — the object accepting the nested schema

Raises

(ArgumentError)

Example

require "google/cloud/bigquery"

bigquery = Google::Cloud::Bigquery.new
dataset = bigquery.dataset "my_dataset"
table = dataset.create_table "my_table"

table.schema do |schema|
  schema.string "first_name", mode: :required
  schema.record "cities_lived", mode: :repeated do |cities_lived|
    cities_lived.string "place", mode: :required
    cities_lived.integer "number_of_years", mode: :required
  end
end

#string

def string(name, description: nil, mode: :nullable, policy_tags: nil, max_length: nil, default_value_expression: nil)

Adds a string field to the schema.

Parameters

name (String) — The field name. The name must contain only letters ([A-Za-z]), numbers ([0-9]), or underscores (_), and must start with a letter or underscore. The maximum length is 128 characters.
description (String) (defaults to: nil) — A description of the field.
mode (Symbol) (defaults to: :nullable) — The field's mode. The possible values are :nullable, :required, and :repeated. The default value is :nullable.
policy_tags (Array<String>, String) (defaults to: nil) — The policy tag list or single policy tag for the field. Policy tag identifiers are of the form projects/*/locations/*/taxonomies/*/policyTags/*. At most 1 policy tag is currently allowed.
max_length (Integer) (defaults to: nil) — The maximum UTF-8 length of strings allowed in the field.
default_value_expression (String) (defaults to: nil) —
The default value of a field using a SQL expression. It can only be set for top level fields (columns). Use a struct or array expression to specify default value for the entire struct or array. The valid SQL expressions are:
- Literals for all data types, including STRUCT and ARRAY.
- The following functions: CURRENT_TIMESTAMP CURRENT_TIME CURRENT_DATE CURRENT_DATETIME GENERATE_UUID RAND SESSION_USER ST_GEOPOINT
- Struct or array composed with the above allowed functions, for example: "[CURRENT_DATE(), DATE '2020-01-01'"]

#time

def time(name, description: nil, mode: :nullable, policy_tags: nil, default_value_expression: nil)

Adds a time field to the schema.

Parameters

name (String) — The field name. The name must contain only letters ([A-Za-z]), numbers ([0-9]), or underscores (_), and must start with a letter or underscore. The maximum length is 128 characters.
description (String) (defaults to: nil) — A description of the field.
mode (Symbol) (defaults to: :nullable) — The field's mode. The possible values are :nullable, :required, and :repeated. The default value is :nullable.
policy_tags (Array<String>, String) (defaults to: nil) — The policy tag list or single policy tag for the field. Policy tag identifiers are of the form projects/*/locations/*/taxonomies/*/policyTags/*. At most 1 policy tag is currently allowed.
default_value_expression (String) (defaults to: nil) —
The default value of a field using a SQL expression. It can only be set for top level fields (columns). Use a struct or array expression to specify default value for the entire struct or array. The valid SQL expressions are:
- Literals for all data types, including STRUCT and ARRAY.
- The following functions: CURRENT_TIMESTAMP CURRENT_TIME CURRENT_DATE CURRENT_DATETIME GENERATE_UUID RAND SESSION_USER ST_GEOPOINT
- Struct or array composed with the above allowed functions, for example: "[CURRENT_DATE(), DATE '2020-01-01'"]

#timestamp

def timestamp(name, description: nil, mode: :nullable, policy_tags: nil, default_value_expression: nil)

Adds a timestamp field to the schema.

Parameters

name (String) — The field name. The name must contain only letters ([A-Za-z]), numbers ([0-9]), or underscores (_), and must start with a letter or underscore. The maximum length is 128 characters.
description (String) (defaults to: nil) — A description of the field.
mode (Symbol) (defaults to: :nullable) — The field's mode. The possible values are :nullable, :required, and :repeated. The default value is :nullable.
policy_tags (Array<String>, String) (defaults to: nil) — The policy tag list or single policy tag for the field. Policy tag identifiers are of the form projects/*/locations/*/taxonomies/*/policyTags/*. At most 1 policy tag is currently allowed.
default_value_expression (String) (defaults to: nil) —
The default value of a field using a SQL expression. It can only be set for top level fields (columns). Use a struct or array expression to specify default value for the entire struct or array. The valid SQL expressions are:
- Literals for all data types, including STRUCT and ARRAY.
- The following functions: CURRENT_TIMESTAMP CURRENT_TIME CURRENT_DATE CURRENT_DATETIME GENERATE_UUID RAND SESSION_USER ST_GEOPOINT
- Struct or array composed with the above allowed functions, for example: "[CURRENT_DATE(), DATE '2020-01-01'"]