The CREATE MODEL statement for importing TensorFlow models
This document describes the CREATE MODEL
statement for importing
TensorFlow models into
BigQuery.
For information about the supported SQL statements and functions for each model type, see End-to-end user journey for each model.
CREATE MODEL
syntax
{CREATE MODEL | CREATE MODEL IF NOT EXISTS | CREATE OR REPLACE MODEL} model_name OPTIONS(MODEL_TYPE = 'TENSORFLOW', MODEL_PATH = string_value [, KMS_KEY_NAME = string_value ] );
CREATE MODEL
Creates and trains a new model in the specified dataset. If the model name
exists, CREATE MODEL
returns an error.
CREATE MODEL IF NOT EXISTS
Creates and trains a new model only if the model doesn't exist in the specified dataset.
CREATE OR REPLACE MODEL
Creates and trains a model and replaces an existing model with the same name in the specified dataset.
model_name
The name of the model you're creating or replacing. The model name must be unique in the dataset: no other model or table can have the same name. The model name must follow the same naming rules as a BigQuery table. A model name can:
- Contain up to 1,024 characters
- Contain letters (upper or lower case), numbers, and underscores
model_name
is not case-sensitive.
If you don't have a default project configured, then you must prepend the project ID to the model name in the following format, including backticks:
`[PROJECT_ID].[DATASET].[MODEL]`
For example, `myproject.mydataset.mymodel`.
MODEL_TYPE
Syntax
MODEL_TYPE = 'TENSORFLOW'
Description
Specifies the model type. This option is required.
MODEL_PATH
Syntax
MODEL_PATH = string_value
Description
Specifies the Cloud Storage URI of the TensorFlow model to import. This option is required.
Arguments
A STRING
value specifying the URI of a Cloud Storage bucket that contains
the model to import.
BigQuery ML imports the model from Cloud Storage by using the
credentials of the user who runs the CREATE MODEL
statement.
Example
MODEL_PATH = 'gs://bucket/path/to/saved_model/*'
KMS_KEY_NAME
Syntax
KMS_KEY_NAME = string_value
Description
The Cloud Key Management Service customer-managed encryption key (CMEK) to use to encrypt the model.
Arguments
A STRING
value containing the fully-qualified name of the CMEK. For example,
'projects/my_project/locations/my_location/keyRings/my_ring/cryptoKeys/my_key'
Supported data types for input and output columns
BigQuery ML converts some TensorFlow model input and output columns to BigQuery ML types, and some TensorFlow types aren't supported. Supported data types for input and output columns include the following:
TensorFlow types | Supported | BigQuery ML type |
---|---|---|
tf.int8, tf.int16, tf.int32, tf.int64, tf.uint8, tf.uint16, tf.uint32, tf.uint64 |
Supported | INT64 |
tf.float16, tf.float32, tf.float64, tf.bfloat16 |
Supported | FLOAT64 |
tf.complex64, tf.complex128 |
Unsupported | N/a |
tf.qint8, tf.quint8, tf.qint16, tf.quint16, tf.qint32 |
Unsupported | N/A |
tf.bool |
Supported | BOOL |
tf.string |
Supported | STRING |
tf.resource |
Unsupported | N/A |
tf.variant |
Unsupported | N/A |
SparseTensor of a supported TensorFlow type |
Supported | A NULL , the associated BigQuery ML type, or an
ARRAY of the associated BigQuery ML type. |
tf.train.Example containing supported TensorFlow types |
Supported | BigQuery ML automatically takes features and converts into a
tf.train.Example . |
The model input columns can be either dense Tensors or SparseTensors; RaggedTensors aren't supported. You can pass SparseTensors as dense arrays and BigQuery ML automatically converts them into Sparse format to pass into TensorFlow.
If the model expects input columns in
tf.train.Example
format,
then BigQuery ML automatically determines the feature names and
converts the input BigQuery columns into the model's expected
format.
Limitations
Imported TensorFlow models have the following limitations:
- The TensorFlow model must already exist before you can import it into BigQuery ML.
- Models must be stored in Cloud Storage.
- Models are frozen at the time of model creation.
- TensorFlow models must be in SavedModel format.
- The following functions don't support TensorFlow models:
ML.CONFUSION
ML.EVALUATE
ML.FEATURE
ML.ROC_CURVE
ML.TRAINING_INFO
ML.WEIGHTS
- Models are limited to 450 MB in size.
- Models trained using a version of GraphDef earlier than version 20 aren't supported.
- Models trained using an unreleased version of TensorFlow aren't supported.
- Only core TensorFlow operations are supported; models that use
custom or
tf.contrib
operations aren't supported. - RaggedTensors aren't supported.
Example
The following example imports a TensorFlow model into
BigQuery ML as a BigQuery ML model. The example
assumes that there is an existing TensorFlow model located at
gs://bucket/path/to/saved_model/*
.
CREATE MODEL `project_id.mydataset.mymodel` OPTIONS(MODEL_TYPE='TENSORFLOW', MODEL_PATH="gs://bucket/path/to/saved_model/*")