Generative AI overview
This document describes the generative artificial intelligence (AI) features that BigQuery ML supports. These features let you perform AI tasks in BigQuery ML by using pre-trained Vertex AI foundation models. Supported tasks include the following:
You access a Vertex AI model to perform one of these functions by creating a remote model in BigQuery ML that represents the Vertex AI model's endpoint. Once you have created a remote model over the Vertex AI model that you want to use, you access that model's capabilities by running a BigQuery ML function against the remote model.
This approach lets you use the capabilities of these Vertex AI models to analyze BigQuery data by using SQL.
Workflow
You can use remote models over Vertex AI models and remote models over Cloud AI services together with BigQuery ML functions in order to accomplish complex data analysis and generative AI tasks.
The following diagram shows some typical workflows where you might use these capabilities together:
Generative AI
You can use large language models (LLMs) to perform tasks such as text summarization and generation, or analysis of visual content. For example, you could summarize a long report, or generate text that describes visual content. You can also use vision language models (VLMs) to analyze visual content such as images and videos for tasks like visual captioning and visual Q&A.
To perform generative natural language or visual analysis tasks, you can create
a reference to a pre-trained Vertex AI
foundation model
by creating a remote model and specifying the model name
for the ENDPOINT
value. The following Vertex AI models
are supported:
gemini-pro
gemini-pro-vision
(Preview)text-bison
text-bison-32k
text-unicorn
When you create a remote model that references any version of a Vertex AI
text-bison
model, you can optionally choose to configure
supervised tuning
(Preview) at the same time.
After you create the model, you can use the
ML.GENERATE_TEXT
function
to interact with that model. For remote models based on text models,
the ML.GENERATE_TEXT
function works with text in
standard tables.
For remote models based on multimodal models, the ML.GENERATE_TEXT
function
works with visual content in
object tables. All inference
occurs in Vertex AI. The results are stored in
BigQuery.
To learn more, try
generating text with the ML.GENERATE_TEXT
function.
Embedding
You can use embedding to identify semantically similar items. For example, you can use text embedding to identify how similar two pieces of text are. If the pieces of text are semantically similar, then their respective embeddings are located near each other in the embedding vector space.
You can use BigQuery ML models to create the following types of embeddings:
- To create text embeddings, you can create a reference to one of the
Vertex AI
textembedding-gecko
ortextembedding-gecko-multilingual
text embedding foundation models by creating a remote model and specifying the LLM name for theENDPOINT
value. - To create multimodal embeddings, which embed text and images into the same
semantic space, you can create a reference to the Vertex AI
multimodalembedding
LLM by creating a remote model and specifying the LLM name for theENDPOINT
value. This feature is in preview. - To create embeddings for structured independent and identically distributed random variables (IID) data, you can use a Principal component analysis (PCA) model or an Autoencoder model. This feature is in preview.
- To create embeddings for user or item data, you can use a Matrix factorization model. This feature is in preview.
After you create the model, you can use the
ML.GENERATE_EMBEDDING
function
to interact with it. For all types of supported models, ML.GENERATE_EMBEDDING
works with data in
standard tables. For multimodal
embedding models, ML.GENERATE_EMBEDDING
also works with visual
content in object tables. For remote
models, all inference occurs in Vertex AI. For other model types,
all inference occurs in BigQuery. The results are stored in
BigQuery.
To learn more, try creating
text embeddings and
image embeddings
with the ML.GENERATE_EMBEDDING
function.
For a smaller, lightweight text embedding, try using a pretrained TensorFlow model, such as NNLM, SWIVEL, or BERT.
For information about choosing the best model for your embedding use case, see Choosing a text embedding model.
What's next
- For more information about performing inference over machine learning models, see Model inference overview.