Model endpoint management reference

Preview — Model endpoint management

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

The information on this page applies only to AlloyDB Omni.

The model endpoint management feature is a preview available with AlloyDB Omni. For using AI models in production environments, see Build generative AI applications using AlloyDB AI.

This page lists the parameters for different functions provided by the google_ml_integration extension to register and manage models, and secrets with model endpoint management preview feature.

For more information about how to use the functions to register and manage models, and secrets, see Use Model endpoint management for AI models

Models

Use this reference to understand parameters for functions that let you manage models.

google_ml.create_model()

The following shows how to call the google_ml.create_model() SQL function used to register model metadata:

  CALL
    google_ml.create_model(
      model_id => 'MODEL_ID',
      model_request_url => 'REQUEST_URL',
      model_provider => 'PROVIDER_ID',
      model_type => 'MODEL_TYPE',
      model_qualified_name => 'MODEL_QUALIFIED_NAME',
      model_auth_type => 'AUTH_TYPE',
      model_auth => 'AUTH_ID',
      generate_headers_fn => 'GENERATE_HEADER_FUNCTION',
      model_in_transform_fn => 'INPUT_TRANSFORM_FUNCTION',
      model_out_transform_fn => 'OUTPUT_TRANSFORM_FUNCTION');

Parameter	Required	Description
MODEL_ID	required for all models	A unique ID for the model that you define.
REQUEST_URL	optional for models with built-in support.	The model-specific endpoint when adding other text embedding and generic models. The request URL that the function generates for built-in models refers to the AlloyDB Omni cluster's project and service account. If you want to refer to another project, then ensure that you specify the model_request_url explicitly.
PROVIDER_ID	required for models with built-in support.	The provider of the model. Set to google for Vertex AI models, open_ai for OpenAI models, and custom for other providers.
MODEL_TYPE	optional for generic models.	The model type. You can set this value to text_embedding or generic.
MODEL_QUALIFIED_NAME	required for OpenAI models; optional for other models.	The fully qualified name in case the model has multiple versions or if the model defines it—for example, textembedding-gecko@001 or textembedding-gecko@002. Since the textembedding-gecko@001 model is pre-registered with model endpoint management, you can generate embeddings using textembedding-gecko@001 as the model ID.
AUTH_TYPE	optional unless model has specific authentication requirement.	The authentication type used by the model. You can set it to either alloydb_service_agent_iam for Vertex AI models or secret_manager for other providers.
AUTH_ID	don't set for Vertex AI models; required for all other models.	The secret ID in Secret Manager.
GENERATE_HEADER_FUNCTION	optional	The function name you set to generate custom headers. The signature of this function depends on the google_ml.predict_row() function.
INPUT_TRANSFORM_FUNCTION	optional for models with built-in support.	The function to transform input of the corresponding prediction function to the model specific input.
OUTPUT_TRANSFORM_FUNCTION	optional for models with built-in support.	The function to transform model specific output to the prediction function output.

google_ml.alter_model()

The following shows how to call the google_ml.alter_model() SQL function used to update model metadata:

    CALL
    google_ml.alter_model(
      model_id => 'MODEL_ID',
      model_request_url => 'REQUEST_URL',
      model_provider => 'PROVIDER_ID',
      model_type => 'MODEL_TYPE',
      model_qualified_name => 'MODEL_QUALIFIED_NAME',
      model_auth_type => 'AUTH_TYPE',
      model_auth => 'AUTH_ID',
      generate_headers_fn => 'GENERATE_HEADER_FUNCTION',
      model_in_transform_fn => 'INPUT_TRANSFORM_FUNCTION',
      model_out_transform_fn => 'OUTPUT_TRANSFORM_FUNCTION');

Parameter	Required	Description
MODEL_ID	required for all models	A unique ID for the model that you define.
REQUEST_URL	optional for models with built-in support.	The model-specific endpoint when adding other text embedding and generic models. The request URL that the function generates for built-in models refers to the AlloyDB Omni cluster's project and service account. If you want to refer to another project, then ensure that you specify the model_request_url explicitly.
PROVIDER_ID	required for models with built-in support.	The provider of the model. Set to google for Vertex AI models, open_ai for OpenAI models, and custom for other providers.
MODEL_TYPE	optional for generic models.	The model type. You can set this value to text_embedding or generic.
MODEL_QUALIFIED_NAME	required for OpenAI models; optional for other models.	The fully qualified name in case the model has multiple versions or if the model defines it—for example, textembedding-gecko@001 or textembedding-gecko@002.
AUTH_TYPE	optional unless model has specific authentication requirement.	The authentication type used by the model. You can set it to either alloydb_service_agent_iam for Vertex AI models or secret_manager for other providers.
AUTH_ID	don't set for Vertex AI models; required for all other models.	The secret ID in Secret Manager.
GENERATE_HEADER_FUNCTION	optional	The function name you set to generate custom headers. The signature of this function depends on the google_ml.predict_row() function.
INPUT_TRANSFORM_FUNCTION	optional for models with built-in support.	The function to transform input of the corresponding prediction function to the model specific input.
OUTPUT_TRANSFORM_FUNCTION	optional for models with built-in support.	The function to transform model specific output to the prediction function output.

google_ml.drop_model()

The following shows how to call the google_ml.drop_model() SQL function used to drop a model:

  CALL google_ml.drop_model('MODEL_ID');

Parameter	Description
MODEL_ID	A unique ID for the model that you define.

google_ml.list_model()

The following shows how to call the google_ml.list_model() SQL function used to list model information:

  CALL google_ml.list_model('MODEL_ID');

Parameter	Description
MODEL_ID	A unique ID for the model that you define.

Secrets

Use this reference to understand parameters for functions that let you manage secrets.

google_ml.create_sm_secret()

The following shows how to call the google_ml.create_sm_secret() SQL function used to add the secret created in Secret Manager:

```sql
CALL
google_ml.create_sm_secret(
  secret_id => 'SECRET_ID',
  secret_path => 'projects/project-id/secrets/secret-ID/versions/VERSION_NUMBER');
```

Parameter	Description
SECRET_ID	The secret name in Secret Manager-for example, key1.
PROJECT_ID	The ID of your Google Cloud project.
VERSION_NUMBER	The version number of the secret ID.

google_ml.alter_sm_secret()

The following shows how to call the google_ml.alter_sm_secret() SQL function used to update secret information:

  CALL
    google_ml.alter_sm_secret(
      secret_id => 'SECRET_ID',
      secret_path => 'SECRET_PATH' );

Parameter	Description
SECRET_ID	The secret name in Secret Manager-for example, key1.
PROJECT_ID	The ID of your Google Cloud project.
VERSION_NUMBER	The version number of the secret ID.

google_ml.drop_sm_secret()

The following shows how to call the google_ml.drop_sm_secret() SQL function used to drop a secret:

  CALL google_ml.drop_sm_secret('SECRET_ID');

Parameter	Description
SECRET_ID	The secret name in Secret Manager-for example, key1.

Prediction functions

Use this reference to understand parameters for functions that let you generate embeddings or invoke predictions.

google_ml.embedding()

The following shows how to generate embeddings:

SELECT
  google_ml.embedding(
    model_id => 'MODEL_ID',
    contents => 'CONTENT');

Parameter	Description
MODEL_ID	A unique ID for the model that you define.
CONTENT	the text to translate into a vector embedding-for example, AlloyDB is a managed, cloud-hosted SQL database service.

google_ml.predict_row()

The following shows how to invoke predictions:

SELECT
  google_ml.predict_row(
    model_id => 'MODEL_ID',
    contents => 'CONTENT');

Parameter	Description
MODEL_ID	A unique ID for the model that you define.
CONTENT	the parameters to the prediction function, in JSON format-for example, {'inputs':'How can I replicate data from AlloyDB Omni to AlloyDB Cloud version', 'parameters':{ 'candidate_labels': ['question', 'complaint', 'feedback'] },}.

Transform functions

Use this reference to understand parameters for input and output transform functions.

Input transform function

The following shows the signature for the prediction function for text embedding models:

  CREATE OR REPLACE FUNCTION INPUT_TRANSFORM_FUNCTION(model_id VARCHAR(100), input_text TEXT) RETURNS JSON;

Parameter	Description
INPUT_TRANSFORM_FUNCTION	The function to transform input of the corresponding prediction function to the model specific input.

Output transform function

The following shows the signature for the prediction function for text embedding models:

  CREATE OR REPLACE FUNCTION OUTPUT_TRANSFORM_FUNCTION(model_id VARCHAR(100), response_json JSON) RETURNS real[];

Parameter	Description
OUTPUT_TRANSFORM_FUNCTION	The function to transform model specific output to the prediction function output.

HTTP header generation function

The following shows signature for the google_ml.embedding() prediction function.

  CREATE OR REPLACE FUNCTION GENERATE_HEADERS(model_id VARCHAR(100), input_text TEXT) RETURNS JSON;

For the google_ml.predict_row() prediction function, the signature is as follows:

CREATE OR REPLACE FUNCTION GENERATE_HEADERS(model_id TEXT, input JSON) RETURNS JSON;

Parameter	Description
GENERATE_HEADERS	The function to generate custom headers. You can also pass the authorization header generated by the header generation function while registering the model.

Request URL generation

Use the request URL generation function to infer the request URLs for the models with built-in support. The following shows the signature for this function:

CREATE OR REPLACE FUNCTION GENERATE_REQUEST_URL(provider google_ml.model_provider, model_type google_ml.MODEL_TYPE, model_qualified_name VARCHAR(100), model_region VARCHAR(100) DEFAULT NULL)

Parameter	Description
GENERATE_REQUEST_UR	The function to generate request URL generated by the extension for models with built-in support.