Reference for built-in wide and deep algorithm

This page provides detailed reference information about arguments you submit to AI Platform when running a training job using the built-in wide and deep algorithm.

Versioning

The built-in wide and deep algorithm uses TensorFlow 1.12.

Data format arguments

The following arguments are used for data formatting and automatic preprocessing:

Arguments Details
preprocessing Specify this to enable automatic preprocessing.
Types of automatic preprocessing:
  • Splits the data into validation_split and test_split percentages.
  • Fills up missing values (with mean for numerical columns).
  • Removes rows that have more than 10% column values missing.

Default: Unset
Type: Boolean flag. If set to true, enables automatic preprocessing.
training_data_path Cloud Storage path to a CSV file. The CSV file must have the following specifications:
  • CSV file must not contain a header
  • Only contain categorical or numerical columns
  • First column must be the target column
  • Blank values will be treated as missing

Required
Type: String
validation_data_path Cloud Storage path to a CSV file. The CSV file must have the same format as training_data_path.

Optional
Type: String
test_data_path Cloud Storage path to a CSV file. The CSV file must have the same format as training_data_path and validation_data_path.

Optional
Type: String
job-dir Cloud Storage path where model, checkpoints and other training artifacts reside. The following directories are created here:
  • model: This contains the trained model
  • processed_data: This contains three data files for training, testing, and validation if automatic preprocessing was enabled.
  • artifacts: This contains training preprocessing-related artifacts that help you to do client side preprocessing.
  • experiment: Checkpoints and summary related to TensorFlow model training.

Required
Type: String
PREPROCESSING PARAMETERS
(when preprocessing is set.)
validation_data_split Fraction of training data that should be used as validation data.

Default: 0.20
Type: Float
Note: validation_data_split + test_data_split <=0.40
Only specify this if you are not specifying validation_data_path.
test_data_split Fraction of training data that should be used as test data.

Default: 0.20
Type: Float
Note: validation_data_split + test_data_split <=0.40
Only specify this if you are not specifying test_data_path.

Hyperparameters

The built-in wide and deep algorithm has the following hyperparameters:

Hyperparameter Details
BASIC PARAMETERS
model_type The type of learning task.

Required
Type: String
Options: one of {classification, regression}
hidden_units Comma separated values of hidden layer sizes to use for DNN feature columns.

Default: 30,30,30
Type: String
max_steps Number of steps (batches) to run the trainer for.

Default: To 10 epochs where one epoch means one pass over the whole training dataset.
Type: Integer
Options: [1, ∞)
learning_rate A scalar used to determine gradient step in gradient descent training.

Default: 0.001
Type: Float
Options: (0, ∞)
eval_steps Number of steps (batches) to run evaluation for.
If not specified, it means run evaluation on the whole validation dataset.

Type: Integer
Options: [1, ∞)
batch_size The number of data rows to process in each training step.

Default: 100
Type: Integer
Options: [1, ∞)
eval_frequency_secs Frequency in seconds at which evaluation and checkpointing will take place.

Default: 100
Type: Integer
Options: [1, ∞)
optimizer_type An optimizer is a specific implementation of the gradient descent algorithm.

Default: 'ftrl'
Type: String
Options: one of {ftrl, adam, sgd}
FTRL optimizer parameters
(arguments for optimizer_type='ftrl')
l1_regularization_strength A type of regularization that helps remove irrelevant or barely relevant features from the model.

Default: 0
Type: Float
Options: [0, ∞)
l2_regularization_strength A type of regularization that improves generalization in linear model.

Default: 0
Type: Float
Options: [0, ∞)
l2_shrinkage_regularization_strength L2 shrinkage regularization strength

Default: 0
Type: Float
Options: [0, ∞)
Adam optimizer parameters
(arguments for optimizer_type='ftrl')
beta_1 The exponential decay rate for the first-moment estimates.

Default: 0.99
Type: Float
Options: [0, ∞)
beta_2 The exponential decay rate for the first-moment estimates.
Default: 0.999
Type: Float
Options: [0, ∞)
DNN specific parameters
use_wide If set, then categorical columns will be used in the wide part of the DNN model.

Default: Unset
Type: Boolean Flag
Options: Set or Unset
embed_categories If set, then categorical columns will be used embedded and used in the deep part of the model. Embedding size is the square root of the column cardinality.

Default: Unset
Type: Boolean Flag
Options: Set or Unset
dnn_learning_rate A scalar used to determine gradient step in gradient descent training. Specific to the DNN part of the model.

Required
Type: Float
Options: (0, ∞)
dnn_dropout The probability a given coordinate will be dropped out.

Default: 0
Type: Float
Options: [0, 1]
dnn_optimizer_type An optimizer is a specific implementation of the gradient descent algorithm. Specific to the DNN part of the model.


Default: 'ftrl'
Type: String
Options: one of {ftrl, adam, sgd}
FTRL optimizer parameters
(arguments for dnn_optimizer_type='ftrl')
dnn_l1_regularization_strength A type of regularization that helps remove irrelevant or barely relevant features from the model. Applies to FTRL DNN optimizer.

Default: 0
Type: Float
Options: [0, ∞)
dnn_l2_regularization_strength A type of regularization that improves generalization in linear model. Applies to FTRL DNN optimizer.

Default: 0
Type: Float
Options: [0, ∞)
dnn_l2_shrinkage_regularization_strength L2 shrinkage regularization strength. Applies to FTRL DNN optimizer.

Default: 0
Type: Float
Options: [0, ∞)
Adam optimizer parameters
(arguments for dnn_optimizer_type='adam')
dnn_beta_1 The exponential decay rate for the first-moment estimates. Applies to adam DNN optimizer.

Default: 0.99
Type: Float
Options: [0, ∞)
dnn_beta_2 The exponential decay rate for the first-moment estimates. Applies to adam DNN optimizer.

Default: 0.999
Type: Float
Options: [0, ∞)

Hyperparameter tuning

Hyperparameter tuning tests different hyperparameter configurations when training your model. It finds hyperparameter values that are optimal for the goal metric you choose. For each tunable argument, you can specify a range of values to restrict and focus the possibilities AI Platform can try.

Learn more about hyperparameter tuning on AI Platform.

Goal metrics

The following metrics can be optimized:

Objective Metric Direction Details
loss MINIMIZE Loss of training job
accuracy MAXIMIZE Acurracy of training (only for classification jobs)

Tunable hyperparameters

When training with the built-in wide and deep algorithm, you can tune the following hyperparameters. Start by tuning parameters with "high tunable value". These have the greatest impact on your goal metric.

Hyperparameters Type Valid values
PARAMETERS WITH HIGH TUNABLE VALUE
(greatest impact on goal metric)
learning_rate

DOUBLE

(0, ∞)
dnn_learning_rate

DOUBLE

(0, ∞)
max_steps

INTEGER

[1, ∞)
OTHER PARAMETERS
l1_reg_strength

DOUBLE

[0, ∞)
l2_reg_strength

DOUBLE

[0, ∞)
l2_shrinkage_reg_strength

DOUBLE

[0, ∞)
beta_1

DOUBLE

[0, ∞)
beta_2

DOUBLE

[0, ∞)
dnn_l1_reg_strength

DOUBLE

[0, ∞)
dnn_l2_reg_strength

DOUBLE

[0, ∞)
dnn_l2_shrinkage_reg_strength

DOUBLE

[0, ∞)
dnn_beta_1

DOUBLE

[0, ∞)
dnn_beta_2

DOUBLE

[0, ∞)
¿Te sirvió esta página? Envíanos tu opinión:

Enviar comentarios sobre…