Try Gemini 1.5 Pro, our most advanced multimodal model in Vertex AI, and see what you can build with a 1M token context window. Try Gemini 1.5 Pro, our most advanced multimodal model in Vertex AI, and see what you can build with a 1M token context window.

Get online predictions for a forecast model

Vertex AI provides two options for projecting future values using your trained forecast model: online predictions and batch predictions.

An online prediction is a synchronous request. Use online predictions when you are making requests in response to application input or in other situations where you require timely inference.

A batch prediction request is an asynchronous request. Use batch predictions when you don't require an immediate response and want to process accumulated data by using a single request.

This page shows you how to project future values using online predictions. To learn how to project values using batch predictions, see Get batch predictions for a forecast model.

You must deploy your model to an endpoint before you can use it for predictions. An endpoint is a set of physical resources.

You can request an explanation instead of a prediction. The explanation's local feature importance values tell you how much each feature contributed to the prediction result. For a conceptual overview, see Feature attributions for forecasting.

To learn about pricing for online predictions, see Pricing for Tabular Workflows.

Before you begin

Before you can make an online prediction request, you must first train a model.

Create or select an endpoint

Use the function aiplatform.Endpoint.create() to create an endpoint. If you already have an endpoint, use the function aiplatform.Endpoint() to select it.

The following code provides an example:

# Import required modules
from google.cloud import aiplatform
from google.cloud.aiplatform import models

PROJECT_ID = "PROJECT_ID"
REGION = "REGION"

# Initialize the Vertex SDK for Python for your project.
aiplatform.init(project=PROJECT_ID, location=REGION)
endpoint = aiplatform.Endpoint.create(display_name='ENDPOINT_NAME')

Replace the following:

PROJECT_ID: Your project ID.
REGION: The region where you are using Vertex AI.
ENDPOINT_NAME: Display name for the endpoint.

Select a trained model

Use the function aiplatform.Model() to select a trained model:

# Create reference to the model trained ahead of time.
model_obj = models.Model("TRAINED_MODEL_PATH")

Replace the following:

TRAINED_MODEL_PATH: For example, projects/PROJECT_ID/locations/REGION/models/[TRAINED_MODEL_ID]

Deploy the model to the endpoint

Use the function deploy() to deploy the model to the endpoint. The following code provides an example:

deployed_model = endpoint.deploy(
    model_obj,
    machine_type='MACHINE_TYPE',
    traffic_percentage=100,
    min_replica_count='MIN_REPLICA_COUNT',
    max_replica_count='MAX_REPLICA_COUNT',
    sync=True,
    deployed_model_display_name='DEPLOYED_MODEL_NAME',
)

Replace the following:

MACHINE_TYPE: For example, n1-standard-8. Learn more about machine types.
MIN_REPLICA_COUNT: The minimum number of nodes for this deployment. The node count can be increased or decreased as required by the prediction load, up to the maximum number of nodes and never fewer than this number of nodes. This value must be greater than or equal to 1. If the min_replica_count variable is not set, the value defaults to 1.
MAX_REPLICA_COUNT: The maximum number of nodes for this deployment. The node count can be increased or decreased as required by the prediction load, up to this number of nodes and never fewer than the minimum number of nodes. If you don't set the max_replica_count variable, then the maximum number of nodes is set to the value of min_replica_count.
DEPLOYED_MODEL_NAME: A name for the DeployedModel. You can use the display name of the Model for the DeployedModel as well.

Model deployment may take approximately ten minutes.

Get online predictions

To get predictions, use the function predict() and provide one or more input instances. The following code shows an example:

predictions = endpoint.predict(instances=[{...}, {...}])

Each input instance is a Python dictionary with the same schema that the model was trained on. It must contain an available at forecast key-value pair that corresponds to the time column and an unavailable at forecast key-value pair that contains the historical values of the targeted prediction column. Vertex AI expects each input instance to belong to a single time series. The order of the key-value pairs in the instance is not important.

The input instance is subject to the following constraints:

The available at forecast key-value pairs must all have the same number of data points.
The unavailable at forecast key-value pairs must all have the same number of data points.
The available at forecast key-value pairs must have at least as many data points as the unavailable at forecast key-value pairs.

To learn more about the types of columns used in forecasting, see Feature type and availability at forecast.

The following code demonstrates a set of two input instances. The Category column contains attribute data. The Timestamp column contains data that is available at forecast. Three points are context data and two points are horizon data. The Sales column contains data that is unavailable at forecast. All three points are context data. To learn how context and horizon are used in forecasting, see Forecast horizon, context window, and forecast window.

instances=[
  {
    # Attribute
    "Category": "Electronics",
    # Available at forecast: three days of context, two days of horizon
    "Timestamp": ['2023-08-03', '2023-08-04', '2023-08-05', '2023-08-06', '2023-08-07'],
    # Unavailable at forecast: three days of context
    "Sales": [490.50, 325.25, 647.00],
  },
  {
    # Attribute
    "Category": "Food",
    # Available at forecast: three days of context, two days of horizon
    "Timestamp": ['2023-08-03', '2023-08-04', '2023-08-05', '2023-08-06', '2023-08-07'],
    # Unavailable at forecast: three days of context
    "Sales": [190.50, 395.25, 47.00],
  }
])

For each instance, Vertex AI responds with two predictions for Sales, corresponding with the two horizon timestamps ("2023-08-06" and "2023-08-07").

For optimal performance, the number of context data points and the number of horizon data points in each input instance must match the context and horizon lengths that the model was trained with. If there is a mismatch, Vertex AI pads or truncates the instance to match the model's size.

If the number of context data points in your input instance is less than or greater than the number of context data points used for model training, ensure that this number of points is consistent across all of the available at forecasting key-value pairs and all of the unavailable at forecasting key-value pairs.

For example, consider a model that was trained with four days of context data and two days of horizon data. You can make a prediction request with just three days of context data. In this case, the unavailable at forecast key-value pairs contain three values. The available at forecast key-value pairs must contain five values.

Output of online prediction

Vertex AI provides online prediction output in the value field:

{
  'value': [...]
}

The length of the prediction response depends on the horizon used in model training and on the horizon of the input instance. The length of the prediction response is the smallest of these two values.

Consider the following examples:

You train a model with context = 15 and horizon = 50. Your input instance has context = 15 and horizon = 20. The prediction response has a length of 20.
You train a model with context = 15 and horizon = 50. Your input instance has context = 15 and horizon = 100. The prediction response has a length of 50.

Online prediction output for TFT models

For models trained with Temporal Fusion Transformer (TFT), Vertex AI provides TFT interpretability tft_feature_importance in addition to predictions in the value field:

{
  "tft_feature_importance": {
    "attribute_weights": [...],
    "attribute_columns": [...],
    "context_columns": [...],
    "context_weights": [...],
    "horizon_weights": [...],
    "horizon_columns": [...]
  },
  "value": [...]
}

attribute_columns: Forecasting features which are time-invariant.
attribute_weights: The weights associated with each of the attribute_columns.
context_columns: Forecasting features whose context window values serve as inputs to the TFT Long Short-Term Memory (LSTM) Encoder.
context_weights: The feature importance weights associated with each of the context_columns for the predicted instance.
horizon_columns: Forecasting features whose forecast horizon values serve as inputs to the TFT Long Short-Term Memory (LSTM) Decoder.
horizon_weights: The feature importance weights associated with each of the horizon_columns for the predicted instance.

Online prediction output for models optimized for quantile loss

For models optimized for quantile loss, Vertex AI provides the following online prediction output:

{
  "value": [...],
  "quantile_values": [...],
  "quantile_predictions": [...]
}

value: If your set of quantiles includes the median, value is the prediction value at the median. Otherwise, value is the prediction value at the lowest quantile in the set. For example, if your set of quantiles is [0.1, 0.5, 0.9], value is the prediction for quantile 0.5. If your set of quantiles is [0.1, 0.9], value is the prediction for quantile 0.1.
quantile_values: The values of the quantiles, which are set during model training.
quantile_predictions: The prediction values associated with quantile_values.

Consider, for example, a model in which the target column is the sales value. Quantile values are defined as [0.1, 0.5, 0.9]. Vertex AI returns the following quantile predictions: [4484, 5615, 6853]. Here, the set of quantiles includes the median, so value is the prediction for quantile 0.5 (5615). The quantile predictions can be interpreted as follows:

P(sales value < 4484) = 10%
P(sales value < 5615) = 50%
P(sales value < 6853) = 90%

Online prediction output for models with probabilistic inference

If your model uses probabilistic inference, the value field contains the minimizer of the optimization objective. For example, if your optimization objective is minimize-rmse, the value field contains the mean value. If it is minimize-mae, the value field contains the median value.

If your model uses probabilistic inference with quantiles, Vertex AI provides quantile values and predictions in addition to the minimizer of the optimization objective. Quantile values are set during model training. Quantile predictions are the prediction values associated with the quantile values.

Get online explanations

To get explanations, use the function explain() and provide one or more input instances. The following code shows an example:

explanations = endpoint.explain(instances=[{...}, {...}])

The format of the input instances is the same for online predictions and online explanations. To learn more, see Get online predictions.

For a conceptual overview of feature attributions, see Feature attributions for forecasting.

Output of online explanation

The following code demonstrates how you can output the explanation results:

# Import required modules
import json
from google.protobuf import json_format

def explanation_to_dict(explanation):
  """Converts the explanation proto to a human-friendly json."""
  return json.loads(json_format.MessageToJson(explanation._pb))

for response in explanations.explanations:
  print(explanation_to_dict(response))

The explanation results have the following format:

{
  "attributions": [
    {
      "baselineOutputValue": 1.4194682836532593,
      "instanceOutputValue": 2.152980089187622,
      "featureAttributions": {
        ...
        "store_id": [
          0.007947325706481934
        ],
        ...
        "dept_id": [
          5.960464477539062e-08
        ],
        "item_id": [
          0.1100526452064514
        ],
        "date": [
          0.8525647521018982
        ],
        ...
        "sales": [
          0.0
        ]
      },
      "outputIndex": [
        2
      ],
      "approximationError": 0.01433318599207033,
      "outputName": "value"
    },
    ...
  ]
}

The number of attributions elements depends on the horizon used in model training and on the horizon of the input instance. The number of elements is the smallest of these two values.

The featureAttributions field in an attributions element contains one value for each of the columns in the input dataset. Vertex AI generates explanations for all types of features: attribute, available at forecast, and unavailable at forecast. To learn more about the fields of an attributions element, see Attribution.

Delete the endpoint

Use the functions undeploy_all() and delete() to delete your endpoint. The following code shows an example:

endpoint.undeploy_all()
endpoint.delete()

What's next

Learn about pricing for online predictions.