Online prediction logging

For AutoML tabular models, AutoML image models, and custom-trained models, you can enable or disable prediction logs during model deployment. This page explains the different types of prediction logs available, and how to enable or disable these logs.

Types of prediction logs

There are two types of prediction logs that you can use to get information from your prediction nodes:

  • Container logging, which logs the stdout and stderr streams from your prediction nodes to Cloud Logging. These logs are essential and required for debugging.

  • Access logging, which logs information like timestamp and latency for each request to Cloud Logging.

Prediction log settings

You can enable or disable online prediction logs when you deploy a model to an endpoint. To update these settings, you must undeploy your model, and then redeploy the model with your new settings.

Online prediction at a high rate of queries per second (QPS) can produce a substantial number of logs, which are subject to Cloud Logging pricing. To estimate the pricing for your online prediction logs, see Estimating your bills for logging. To reduce this cost, you can disable prediction logging.

Default log settings

You can enable or disable each type of log independently.

  • Container logging, which logs the stderr and stdout streams from your prediction nodes to Cloud Logging.

    • On the v1 service endpoint, container logging is enabled by default. You can disable it when you deploy a model to an endpoint.

    • On the v1beta1 service endpoint, container logging is not enabled by default. You can enable container logging when you deploy a model to an endpoint.

  • Access logging, which logs information like timestamp and latency for each request to Cloud Logging.

    On both the v1 and v1beta1 service endpoints, access logging is disabled by default. You can enable access logging when you deploy a model to an endpoint.

Enabling and disabling prediction logs

The following examples highlight where to modify these default settings when you deploy a model:

Console

When you deploy a model to an endpoint or create a new endpoint in the Cloud Console, you can specify which types of prediction logs to enable in the Logging step. Select the checkboxes to enable Access logging or Container logging, or clear the checkboxes to disable these logs.

To see more context about how to deploy models, read Deploying a model using the Cloud Console.

gcloud

To change the default behavior for which logs are enabled in deployed models, add flags to your gcloud command:

v1 service endpoint

Run gcloud ai endpoints deploy-model:

gcloud ai endpoints deploy-model ENDPOINT_ID\
  --region=LOCATION \
  --model=MODEL_ID \
  --display-name=DEPLOYED_MODEL_NAME \
  --machine-type=MACHINE_TYPE \
  --accelerator=count=2,type=nvidia-tesla-t4 \
  --disable-container-logging \
  --enable-access-logging

v1beta1 service endpoint

Run gcloud beta ai endpoints deploy-model:

gcloud beta ai endpoints deploy-model ENDPOINT_ID\
  --region=LOCATION \
  --model=MODEL_ID \
  --display-name=DEPLOYED_MODEL_NAME \
  --machine-type=MACHINE_TYPE \
  --accelerator=count=2,type=nvidia-tesla-t4 \
  --enable-access-logging \
  --enable-container-logging

To see more context about how to deploy models, read Deploying a model using the Vertex AI API.

REST & CMD LINE

To change the default behavior for which logs are enabled in deployed models, set the relevant fields to True:

v1 service endpoint

To disable container logging, set the disableContainerLogging field to True when deploying your model with projects.locations.endpoints.deployModel.

To enable access logging, set enableAccessLogging to True when deploying your model with projects.locations.endpoints.deployModel.

v1beta1 service endpoint

To enable container logging, set the enableContainerLogging field to True when deploying your model with projects.locations.endpoints.deployModel.

To enable access logging, set enableAccessLogging to True when deploying your model with projects.locations.endpoints.deployModel.

To see more context about how to deploy models, read Deploying a model using the Vertex AI API.

What's next