Overview

Large Language Models (LLMs) are powerful at solving many types of problems. However, they are constrained by the following limitations:

  • They are frozen after training, leading to stale knowledge.
  • They can't query or modify external data.

Extensions and the Vertex AI (Preview) can address these shortcomings. Extensions are connections to external APIs that process real-time data and perform real-world actions. The Vertex AI extension service registers, manages, and runs these extensions and can be linked to an application that processes user queries and communicates with an LLM.

Whenever an application receives a user query, it provides the LLM with a set of extensions and the operations that the extensions can perform. If the model determines that it should delegate query processing to an extension, it provides the application with a requested extension call and the associated parameter values. The application relays this request to the extension service, which then runs the extension. The process completes when the extension service submits the result of this operation to the application, which then relays it back to the model.

With Vertex AI Search, you can create, deploy, and manage extensions that connect LLMs to the APIs of external systems. These systems can provide LLMs with real-time data and perform data processing actions on their behalf.

To learn more about the use-cases and benefits of extensions and the Vertex AI extension service, see Use-cases and benefits.

To learn how to register, manage, and run extensions with the Vertex AI extensions service, see Register, manage, and run extensions.

Use-cases and benefits

The following are some possible extensions use cases:

  • Generate and run code.
  • Query websites and synthesize information.
  • Answer questions based on information within a collection of enterprise-specific data.
  • Query and analyze datastores.

The Vertex AI extensions service provides the following benefits:

  • Enterprise controls: Identity and Access Management (IAM) permissions and security controls.
  • Data security: Contractual agreements that your private data won't be leaked or used in model training
  • Performance agreements: Contractual agreements that the platform delivers specific features and uptimes.
  • Private extensions: Authorized users in your organization or a trusted partner can use extensions to access sensitive internal data and actions, such as searching internal knowledge bases or completing HR actions.
  • Google product integrations: Integration with Google products like Vertex AI Search, BigQuery and specialized models.

What's next