Viewing model architecture with Cloud Logging

Beta

This product is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products are available "as is" and might have limited support. For more information, see the launch stage descriptions.

This page provides information on how to use Cloud Logging to view details about an AutoML Tables model. Using Logging, you can see the final model hyperparameters as well as the hyperparameters and object values used during model training and tuning.

Note: Model architecture logs are provided as part of the Cloud Logging service. For general information about Cloud Logging, see the Cloud Logging documentation.

You can also view general logs for AutoML Tables in your project.

Logged data

The logging messages produced by AutoML Tables provide:

• Final model hyperparameters as key-value pairs.
• Hyperparameters used during tuning trials, as well as an objective value.

By default, logs are deleted after 30 days.

Before you begin

• Familiarize yourself with Cloud Logging.

Permissions required for this task

To perform this task, you must have the following permissions:

• logging.logServiceIndexes.list on the project
• logging.logServices.list on the project

Viewing training logs

Accessing model logs from the Google Cloud console

You can access the final hyperparameter logs and the tuning trial hyperparameter logs directly from AutoML Tables.

1. Go to the AutoML Tables page in the Google Cloud console.

   Go to the AutoML Tables page

2. Select the Models tab in the left navigation pane and click on the model name to open the model.

3. Select the Models tab.

   

4. To see the final hyperparameter logs, click Model.

5. To see the tuning trial hyperparameters, click Trials.

6. Expand the payload as shown below:

   

Reading model architecture logs

Activity logs are structured as described in the LogEntry type documentation.

AutoML Tables model logs have, among other fields:

• automl_tables as the value of the log_type field
• jsonPayload that contains the specific details of the log entry
• timestamp that describes when the model was created

Payload contents

The contents of a log entry are provided in JSON object format and are stored in the jsonPayload field. The jsonPayload field contains the following information.

Field	Type	Description
modelStructure	JSON	A description of the AutoML Tables model structure. This field contains one or more modelParameters fields that contain the hyperparameters of each model used. When more than one model is used, the models are combined as an ensemble to create the final model. See the hyperparameter list for details.
trainingObjectivePoint	JSON	The optimization objective used for model training. This entry includes a timestamp and an objective value at the time the log entry was recorded.

List of hyperparameters

The hyperparameter data provided in the logs differ for each type of model. The following sections describe the hyperparameters for each model type.

AdaNet models

• Model type: AdaNet
• AdaNet lambda
• Complexity type
• Initial number of layers
• Last layer type (logits or prelogits)
• Grow layer size
• Learn mixture weights (True or False)
• Learn residuals (True or False)
• Boosting iterations

• Optimizer (adagrad or adam)

  Open source Python reference

AdaNet AutoEnsemble models

• Model type: AdaNet AutoEnsembler
• Number of hidden layers
• Hidden layer size
• Dropout
• L1 regularization strength
• L2 regularization strength
• L2 shrinkage regularization strength
• Tree complexity
• Max tree depth
• Center bias (True or False)
• Growing mode (layer or tree)
• Boosting iterations

• DNN Optimizer (adagrad or adam)

  Open source Python reference

DNN Linear models

• Model type: DNNLinear
• Number of hidden layers
• Hidden layer size
• Dropout
• L1 regularization strength
• L2 regularization strength

• L2 shrinkage regularization strength

  Open source Python reference

Gradient boosted decision tree models

• Model type: GBDT
• Max tree depth
• Tree L1 regularization
• Tree L2 regularization
• Tree complexity
• Number of trees

• Center bias (True or False)

  Open source Python reference

Feedforward neural network models

• Model type: nn
• Hidden layer size
• Dropout rate
• Enable numerical embedding (True or False)
• Enable L1 (True or False)
• Enable L2 (True or False)
• Enable embedding L1 (True or False)
• Enable embedding L2 (True or False)
• Enable layerNorm (True or False)
• Enable batchNorm (True or False)
• Number of hidden layers
• Number of cross layers
• Skip connections type (dense, disable, concat, or slice_or_padding)
• Normalize numerical column (True or False)

Sample log entries

The following example shows a log entry for the final model architecture for a nn model, as indicated by the model_type parameter. If more than one model was used to create the final model, the hyperparameters for each model are returned as an entry in the modelParameters array, indexed by position (0, 1, 2, and so on).

{
 insertId: "qx7z0ifhtjpkv"
 jsonPayload: {
  @type: "type.googleapis.com/google.cloud.automl.master.TuningTrial"
  modelStructure: {
   modelParameters: [
    0: {
      Dropout rate: 0.75
      Embedding numerical embedding: "True"
      Enable L1: "False"
      Enable L2: "False"
      Enable batchNorm: "True"
      Enable embedding L1: "False"
      Enable embedding L2: "False"
      Enable layerNorm: "False"
      Hidden layer size: 16
      Normalize numerical column: "True"
      Number of cross layers: "1"
      Number of hidden layers: "2"
      Skip connections type: "dense"
      Model type: "nn"
    }
   ]
  }
  trainingObjectivePoint: {
   createTime: "2019-10-15T04:52:00Z"
   value: 0.002069325
  }
 }
 labels: {
  log_type: "automl_tables"
 }
 logName: "projects/project-id/logs/automl.googleapis.com%2Ftuning"
 receiveTimestamp: "2019-10-15T10:46:06.916277805Z"
 resource: {
  labels: {
   job_id: "109476078648557568"
   project_id: "project-id"
   region: "us-central1"
  }
  type: "cloudml_job"
 }
 severity: "INFO"
 timestamp: "2019-10-15T10:46:06.916277805Z"
}

The following example shows a log entry for model architecture during tuning.

{
 insertId: "nl7815f7p0cjb"
 jsonPayload: {
  @type: "type.googleapis.com/google.cloud.automl.master.TuningTrial"
  modelStructure: {
   modelParameters: [
    0: {
     hyperparameters: {
      Boosting iterations: "2"
      Grow layer size: "30"
      Initial number of layers: "1"
      Last layer type: "prelogits"
      Learn residuals: "False"
      Model type: "AdaNet"
      Optimizer: "adam"
      }
    }
   ]
  }
  trainingObjectivePoint: {
   createTime: "2019-10-13T20:42:25Z"
   value: 0.0000011595778
  }
 }
 labels: {
  log_type: "automl_tables"
 }
 logName: "projects/project-id/logs/automl.googleapis.com%2Ftuning"
 receiveTimestamp: "2019-10-13T21:20:08.189302731Z"
 resource: {
  labels: {
   job_id: "2804289077287845888"
   project_id: "project-id"
   region: "us-central1"
  }
  type: "cloudml_job"
 }
 severity: "INFO"
 timestamp: "2019-10-13T21:20:08.189302731Z"
}

Viewing general AutoML Tables logs

You can see your project's activity logs through the Logs Explorer in the Google Cloud console:

1. Go to the Logging page in the Google Cloud console.
2. When in the Logs Explorer, select and filter your resource type from the first drop-down menu.
3. Select automl.googleapis.com/tuning from the All logs drop-down menu to see the AutoML Tables logs.

Exporting logs

You can export your logs to BigQuery, Cloud Storage, or Pub/Sub.

Read Configuring Logs Export in the Logging documentation to learn how to export activity logs.