The CREATE MODEL statement for importing TensorFlow models

Stay organized with collections Save and categorize content based on your preferences.

CREATE MODEL statement for TensorFlow

To import an existing TensorFlow model into BigQuery from Cloud Storage, use the BigQuery ML CREATE MODEL statement with the TensorFlow model type.

For information about supported model types of each SQL statement and function, and all supported SQL statements and functions for each model type, read End-to-end user journey for each model.

CREATE MODEL syntax

{CREATE MODEL | CREATE MODEL IF NOT EXISTS | CREATE OR REPLACE MODEL}
model_name
[OPTIONS(MODEL_TYPE = 'TENSORFLOW', MODEL_PATH = string_value)];

CREATE MODEL

Creates a new BigQuery ML model in the specified dataset. For TensorFlow models, BigQuery ML imports the existing TensorFlow model and converts it to a BigQuery ML model. If the model name exists, CREATE MODEL returns an error.

CREATE MODEL IF NOT EXISTS

Creates a new imported model only if the model does not currently exist in the specified dataset.

CREATE OR REPLACE MODEL

Creates a new imported model and replaces any existing model with the same name in the specified dataset.

model_name

model_name is the name of the BigQuery ML model you're creating or replacing. The model name must be unique per dataset: no other model or table can have the same name. The model name must follow the same naming rules as a BigQuery table. A model name can contain the following:

  • Up to 1,024 characters
  • Letters of either case, numbers, and underscores

model_name is not case-sensitive.

If you do not have a default project configured, prepend the project ID to the model name in following format, including backticks:

`[PROJECT_ID].[DATASET].[MODEL]`

For example:

`myproject.mydataset.mymodel`

CREATE MODEL supports the following options:

MODEL_TYPE

Syntax

MODEL_TYPE = 'TENSORFLOW'

Description

Specifies the model type. This option is required.

MODEL_PATH

Syntax

MODEL_PATH = string_value

Description

Specifies the Cloud Storage URI of the TensorFlow model to import. This option is required for TensorFlow models.

string_value is the URI of a Cloud Storage bucket that contains the model to import.

BigQuery ML imports the model from Cloud Storage using the credentials of the user who sent the CREATE MODEL statement.

Example

MODEL_PATH = 'gs://bucket/path/to/saved_model/*'

Supported inputs

The CREATE MODEL statement supports the following data types for input label and data split columns.

Supported data types for TensorFlow model inputs and outputs

BigQuery ML converts certain TensorFlow model inputs and outputs to BigQuery ML types. Some TensorFlow model input and output types are not supported. Supported data types for the inputs and outputs of the imported TensorFlow model include:

TensorFlow types Supported BigQuery ML type
tf.int8, tf.int16, tf.int32, tf.int64, tf.uint8, tf.uint16, tf.uint32, tf.uint64 Supported INT64
tf.float16, tf.float32, tf.float64, tf.bfloat16 Supported FLOAT64
tf.complex64, tf.complex128 Unsupported N/a
tf.qint8, tf.quint8, tf.qint16, tf.quint16, tf.qint32 Unsupported N/a
tf.bool Supported BOOL
tf.string Supported STRING
tf.resource Unsupported N/a
tf.variant Unsupported N/a
SparseTensor of a supported tensorflow type Supported A NULL, the associated BigQuery ML type, or an ARRAY of the associated BigQuery ML type.
tf.train.Example containing supported tensorflow types Supported BigQuery ML automatically takes features and converts into a tf.train.Example.

The model inputs can be either dense Tensors or SparseTensors. RaggedTensors are not supported. Sparse Tensors can be passed as dense arrays and BigQuery ML automatically converts them into Sparse format to pass into TensorFlow.

If the model expects inputs in tf.train.Example format then BigQuery ML automatically determines the feature names and converts the input BigQuery columns into the model's expected format.

Limitations

CREATE MODEL statements for TensorFlow models must comply with the following rules:

  • The TensorFlow model must already exist before it can be imported into BigQuery ML.
  • Models must be stored in Cloud Storage.
  • Models are frozen at the time of model creation.
  • TensorFlow models must be in SavedModel format.
  • Currently, the following functions do not support TensorFlow models: ML.CONFUSION, ML.EVALUATE, ML.FEATURE, ML.ROC_CURVE, ML.TRAINING_INFO, and ML.WEIGHTS.
  • Models are limited to 450MB in size.
  • Models trained using a version of GraphDef below version 20 are not supported.
  • Models trained using an unreleased version of TensorFlow are not supported.
  • Only core TensorFlow operations are supported: models that use custom or tf.contrib operations are not supported.
  • RaggedTensors are not supported.

CREATE MODEL examples

The following example creates models named mymodel in mydataset in your default project.

Importing a TensorFlow model

The following example imports a TensorFlow model into BigQuery ML as a BigQuery ML model. The example assumes that there is an existing TensorFlow model located at gs://bucket/path/to/saved_model/*.

CREATE MODEL project_id.mydataset.mymodel
 OPTIONS(MODEL_TYPE='TENSORFLOW',
         MODEL_PATH="gs://bucket/path/to/saved_model/*")