Use this page to understand the differences between Vertex AI and BigQuery and learn how you can integrate Vertex AI with your existing BigQuery workflows. Vertex AI and BigQuery work together to meet your machine learning and MLOps use cases.
To learn more about model training differences between Vertex AI and BigQuery, see Choose a training method.
Differences between Vertex AI and BigQuery
This section covers the Vertex AI, BigQuery, and BigQuery ML services.
Vertex AI: An end-to-end AI/ML platform
Vertex AI is an AI/ML platform for both model development and governance. You can train your models in two main ways:
- AutoML: which lets you train models on image, tabular, text, and video datasets without writing code.
- Custom Training: where you can run custom training code catered to your specific use case.
You can register both AutoML and custom-trained models to the Vertex AI Model Registry. You can also import models trained outside of Vertex AI register them to Vertex AI Model Registry.
From the registry, you can manage model versions, deploy to endpoints for online and batch predictions, perform model evaluations, monitor deployments with Vertex AI Model Monitoring, and use Vertex Explainable AI.
Available languages:
- Vertex AI SDK for Python
- Client library for Java
- Client library for Node.js
BigQuery: A serverless, multicloud enterprise data warehouse
BigQuery is a fully managed enterprise data warehouse that helps you manage and analyze your data with built-in features like machine learning, geospatial analysis, and business intelligence. BigQuery tables can be queried by SQL, and data scientists who primarily use SQL can run large queries with only a few lines of code.
You can also use BigQuery as a data store that you reference when building tabular and custom models in Vertex AI. To learn more about using BigQuery as a data store, see Overview of BigQuery storage.
Available languages:
- SDKs for BigQuery. To learn more, see the BigQuery API Client Libraries.
- GoogleSQL
- Legacy SQL
To learn more, see BigQuery SQL dialects.
BigQuery ML: Machine learning directly in BigQuery
BigQuery ML is a model development service within BigQuery. With BigQuery ML, SQL users can train ML models directly in BigQuery without needing to move data or worry about the underlying training infrastructure. You can create batch predictions for BigQuery ML models to gain insights from your BigQuery data.
Available language:
- GoogleSQL
To learn more about the advantages of using BigQuery ML, see What is BigQuery ML?
Benefits of integrating BigQuery ML models in Vertex AI
Integrating BigQuery ML models in Vertex AI provides two main benefits:
Online model serving: BigQuery ML only supports batch predictions for your models. To get online predictions, you can train your models in BigQuery ML and deploy them to Vertex AI endpoints through Vertex AI Model Registry.
MLOps capabilities: Models are most beneficial when they are kept up to date through continuous training. Vertex AI offers MLOps tools that automate the monitoring and retraining of models to maintain the accuracy of predictions over time. With Vertex AI Pipelines, you can use BigQuery operators to plug any BigQuery jobs (including BigQuery ML) into an ML pipeline. With Vertex AI Model Monitoring, you can monitor your BigQuery ML predictions over time.
To learn how to register your BigQuery ML models to the Vertex AI Model Registry, see Manage BigQuery ML models with Vertex AI.
Related notebook tutorials
| What do you want to do? | Resource |
|---|---|
| Use the Vertex AI SDK for Python to train and deploy a custom tabular classification model for online prediction. | Training a TensorFlow model on BigQuery data |
| Use the Vertex AI SDK for Python to train an AutoML model for tabular regression and get online predictions from the model. | Vertex AI SDK for Python: AutoML training tabular regression model for online prediction using BigQuery |
| Use two Vertex AI Tabular Workflows pipelines to train a AutoML model using different configurations. | Tabular Workflow: AutoML Tabular Pipeline |
| Use the Vertex AI SDK for Python to train an AutoML model for tabular regression and get batch predictions from the model. | Vertex AI SDK for Python: AutoML training tabular regression model for batch prediction using BigQuery |
| Use the Vertex AI SDK to train an AutoML model for tabular forecasting and get batch predictions from the model. | Vertex AI SDK: AutoML tabular forecasting model for batch prediction |
| Train and evaluate a propensity model in BigQuery ML to predict user retention on a mobile game. | Churn prediction for game developers using Google Analytics 4 and BigQuery ML |
| Use BigQuery ML to perform pricing optimization on CDM pricing data. | Analysis of pricing optimization on CDM pricing data |
What's next
- To get started with Vertex AI see: