End-to-end user journey for each model

BigQuery ML supports a variety of machine learning models and a complete machine learning flow for each model, such as feature preprocessing, model creation, hyperparameter tuning, inference, evaluation, and model export. The machine learning flow for the models are split into the following two tables:

Model creation phase

Model category Model types Model creation Feature preprocessing Hyperparameter tuning Model weights Feature & training info Tutorials
Supervised learning Linear & logistic regression create model Automatic preprocessing,
Manual preprocessing1
HP tuning2
ml.trial_info
ml.weights ml.feature_info
ml.training_info
Deep neural networks (DNN) create model N/A5 N/A
Wide & Deep networks create model N/A5 N/A
Boosted trees create model N/A5 N/A
Random forest create model N/A5 N/A
AutoML classification & regression create model N/A3 N/A3 N/A5 N/A
Unsupervised learning K-means create model Automatic preprocessing,
Manual preprocessing1
HP tuning2
ml.trial_info
ml.centroids ml.feature_info
ml.training_info
cluster bike stations
Matrix factorization create model N/A HP tuning2
ml.trial_info
ml.weights
Principal component analysis (PCA) create model Automatic preprocessing,
Manual preprocessing1
N/A ml.principal_
components
,
ml.principal_
component_info
N/A
Autoencoder create model Automatic preprocessing,
Manual preprocessing1
HP tuning2
ml.trial_info
N/A5 N/A
Time series models ARIMA_PLUS create model Automatic preprocessing auto.ARIMA4 ml.arima_ coefficients ml.feature_info
ml.training_info
ARIMA_PLUS_XREG create model Automatic preprocessing auto.ARIMA4 ml.arima_ coefficients ml.feature_info
ml.training_info
forecast a single time series
Generative AI remote models Remote model over a Vertex AI text generation model6 create model N/A N/A N/A N/A
Remote model over a Vertex AI embedding generation model6 create model N/A N/A N/A N/A
AI remote models Remote model over the Cloud Vision API create model N/A N/A N/A N/A N/A
Remote model over the Cloud Translation API create model N/A N/A N/A N/A N/A
Remote model over the Cloud Natural Language API create model N/A N/A N/A N/A N/A
Remote model over the Document AI API create model N/A N/A N/A N/A N/A
Remote model over the Speech-to-Text API
create model N/A N/A N/A N/A N/A
Remote models Remote model with a Vertex AI endpoint create model N/A N/A N/A N/A predict with remote model
Imported models TensorFlow create model N/A N/A N/A N/A predict with imported TensorFlow model
TensorFlow Lite create model N/A N/A N/A N/A N/A
Open Neural Network Exchange (ONNX) create model N/A N/A N/A N/A
XGBoost create model N/A N/A N/A N/A N/A
Transform-only models7 Transform-only create model Manual preprocessing1 N/A N/A ml.feature_info N/A
Contribution analysis models Contribution analysis
(Preview)
create model Manual preprocessing N/A N/A N/A Get data insights from a contribution analysis model

1See TRANSFORM clause for the feature engineering tutorial. For more information about the preprocessing functions, see the BQML - Feature Engineering Functions tutorial.

2See use hyperparameter tuning to improve model performance tutorial.

3Automatic feature engineering and hyperparameter tuning are embedded in the AutoML model training by default.

4The auto.ARIMA algorithm performs hyperparameter tuning for the trend module. Hyperparameter tuning is not supported for the entire modeling pipeline. See the modeling pipeline for more details.

5BigQuery ML doesn't support functions that retrieve the weights for boosted trees, random forest, DNNs, Wide-and-deep, Autoencoder, or AutoML models. To see the weights of those models, you can export an existing model from BigQuery ML to Cloud Storage and then use the XGBoost library or the TensorFlow library to visualize the tree structure for the tree models or the graph structure for the neural networks. For more information, see the EXPORT MODEL documentation and the EXPORT MODEL tutorial.

6Uses a Vertex AI foundation model or customizes it by using supervised tuning.

7This is not a typical ML model but rather an artifact that transforms raw data into features.

Model use phase

Model category Model types Evaluation Inference AI explanation Model monitoring Model export Tutorials
Supervised learning Linear & logistic regression ml.evaluate
ml.confusion_matrix1
ml.roc_curve2
ml.predict
ml.transform
ml.explain_predict3
ml.global_explain
ml.advanced_weights8
ml.describe_data
ml.validate_data_drift
ml.validate_data_skew
ml.tfdv_describe
ml.tfdv_validate
export model5
Deep neural networks (DNN) N/A
Wide & Deep networks N/A
Boosted trees ml.explain_predict3
ml.global_explain
ml.feature_importance4
N/A
Random forest N/A
AutoML classification & regression ml.predict ml.global_explain ml.describe_data
ml.validate_data_drift
ml.tfdv_describe
ml.tfdv_validate
N/A
Unsupervised learning K-means ml.evaluate ml.predict
ml.detect_anomalies
ml.transform
N/A ml.describe_data
ml.validate_data_drift
ml.validate_data_skew
ml.tfdv_describe
ml.tfdv_validate
export model5 cluster bike stations
Matrix factorization ml.recommend
ml.generate_embedding
ml.describe_data
ml.validate_data_drift
ml.tfdv_describe
ml.tfdv_validate
Principal component analysis (PCA) ml.predict
ml.generate_embedding
ml.detect_anomalies
ml.transform
ml.describe_data
ml.validate_data_drift
ml.validate_data_skew
ml.tfdv_describe
ml.tfdv_validate
N/A
Autoencoder ml.predict
ml.generate_embedding
ml.detect_anomalies
ml.reconstruction_loss
ml.transform
N/A
Time series models ARIMA_PLUS ml.evaluate
ml.arima_evaluate6
ml.holiday_info
ml.forecast
ml.detect_anomalies
ml.explain_forecast7 ml.describe_data
ml.validate_data_drift
ml.tfdv_describe
ml.tfdv_validate
N/A
ARIMA_PLUS_XREG ml.forecast
ml.detect_anomalies
ml.describe_data
ml.validate_data_drift
ml.validate_data_skew
ml.tfdv_describe
ml.tfdv_validate
multivariate forecasting
Generative AI remote models Remote model over a Vertex AI text generation model9 ml.evaluate11 (Preview) ml.generate_text N/A ml.describe_data
ml.validate_data_drift
ml.tfdv_describe
ml.tfdv_validate
N/A
Remote model over a Vertex AI embedding generation model9 N/A ml.generate_embedding N/A N/A
AI remote models Remote model over the Cloud Vision API N/A ml.annotate_image N/A ml.describe_data
ml.validate_data_drift
ml.tfdv_describe
ml.tfdv_validate
N/A N/A
Remote model over the Cloud Translation API N/A ml.translate N/A N/A N/A
Remote model over the Cloud Natural Language API N/A ml.understand_text N/A N/A N/A
Remote model over the Document AI API N/A ml.process_document N/A N/A N/A
Remote model over the Speech-to-Text API N/A ml.transcribe N/A N/A N/A
Remote models Remote model with a Vertex AI endpoint N/A ml.predict N/A ml.describe_data
ml.validate_data_drift
ml.tfdv_describe
ml.tfdv_validate
N/A predict with remote model
Imported models TensorFlow N/A ml.predict N/A ml.describe_data
ml.validate_data_drift
ml.tfdv_describe
ml.tfdv_validate
export model5 predict with imported TensorFlow model
TensorFlow Lite N/A ml.predict N/A N/A N/A
Open Neural Network Exchange (ONNX) N/A ml.predict N/A N/A
XGBoost N/A ml.predict N/A N/A N/A
Transform-only models10 Transform-only N/A ml.transform N/A N/A export model5 N/A
Contribution analysis models Contribution analysis
(Preview)
N/A ml.get_insights N/A N/A N/A Get data insights from a contribution analysis model

1ml.confusion_matrix is only applicable to classification models.

2ml.roc_curve is only applicable to binary classification models.

3ml.explain_predict is an extended version of ml.predict. For more information, see Explainable AI overview. To learn how ml.explain_predict is used, see regression tutorial and classification tutorial.

4For the difference between ml.global_explain and ml.feature_importance, see Explainable AI overview.

5See the Export a BigQuery ML model for online prediction tutorial. For more information about online serving, see the BQML - Create Model with Inline Transpose tutorial.

6For ARIMA_PLUS or ARIMA_PLUS_XREG models, ml.evaluate can take new data as input to compute forecasting metrics such as mean absolute percentage error (MAPE). In the absence of new data, ml.evaluate has an extended version ml.arima_evaluate which outputs different evaluation information.

7ml.explain_forecast is an extended version of ml.forecast. For more information, see Explainable AI overview. To learn how ml.explain_forecast is used, see the visualize results steps of the single time series forecasting and multiple time series forecasting tutorials.

8ml.advanced_weights is an extended version of ml.weights, see ml.advanced_weights for more details.

9Uses a Vertex AI foundation model or customizes it by using supervised tuning.

10This is not a typical ML model but rather an artifact that transforms raw data into features.

11Not supported for all Vertex AI LLMs. For more information, see ml.evaluate.