Represents specification of a Study.
Required. Metric specs for the Study.
Required. The set of parameters to tune.
The search algorithm specified for the Study.
The observation noise level of the study. Currently only supported by the Vertex AI Vizier service. Not supported by HyperparameterTuningJob or TrainingPipeline.
Describe which measurement selection type will be used
The configuration info/options for transfer learning. Currently supported for Vertex AI Vizier service, not HyperParameterTuningJob
automated_stopping_spec
Union type
automated_stopping_spec
can be only one of the following:The automated early stopping spec using decay curve rule.
The automated early stopping spec using median rule.
Deprecated. The automated early stopping using convex stopping rule.
The automated early stopping spec using convex stopping rule.
Conditions for automated stopping of a Study. Enable automated stopping by configuring at least one condition.
JSON representation |
---|
{ "metrics": [ { object ( |
DecayCurveAutomatedStoppingSpec
The decay curve automated stopping rule builds a Gaussian Process Regressor to predict the final objective value of a Trial based on the already completed Trials and the intermediate measurements of the current Trial. Early stopping is requested for the current Trial if there is very low probability to exceed the optimal value found so far.
useElapsedDuration
boolean
True if Measurement.elapsed_duration
is used as the x-axis of each Trials Decay Curve. Otherwise, Measurement.step_count
will be used as the x-axis.
JSON representation |
---|
{ "useElapsedDuration": boolean } |
MedianAutomatedStoppingSpec
The median automated stopping rule stops a pending Trial if the Trial's best objectiveValue is strictly below the median 'performance' of all completed Trials reported up to the Trial's last measurement. Currently, 'performance' refers to the running average of the objective values reported by the Trial in each measurement.
useElapsedDuration
boolean
True if median automated stopping rule applies on Measurement.elapsed_duration
. It means that elapsedDuration field of latest measurement of current Trial is used to compute median objective value for each completed Trials.
JSON representation |
---|
{ "useElapsedDuration": boolean } |
ConvexStopConfig
Configuration for ConvexStopPolicy.
Steps used in predicting the final objective for early stopped trials. In general, it's set to be the same as the defined steps in training / tuning. When use_steps is false, this field is set to the maximum elapsed seconds.
Minimum number of steps for a trial to complete. Trials which do not have a measurement with num_steps > minNumSteps won't be considered for early stopping. It's ok to set it to 0, and a trial can be early stopped at any stage. By default, minNumSteps is set to be one-tenth of the maxNumSteps. When use_steps is false, this field is set to the minimum elapsed seconds.
The number of Trial measurements used in autoregressive model for value prediction. A trial won't be considered early stopping if has fewer measurement points.
learningRateParameterName
string
The hyper-parameter name used in the tuning job that stands for learning rate. Leave it blank if learning rate is not in a parameter in tuning. The learningRate is used to estimate the objective value of the ongoing trial.
useSeconds
boolean
This bool determines whether or not the rule is applied based on elapsed_secs or steps. If useSeconds==false, the early stopping decision is made according to the predicted objective values according to the target steps. If useSeconds==true, elapsed_secs is used instead of steps. Also, in this case, the parameters maxNumSteps and minNumSteps are overloaded to contain max_elapsed_seconds and min_elapsed_seconds.
JSON representation |
---|
{ "maxNumSteps": string, "minNumSteps": string, "autoregressiveOrder": string, "learningRateParameterName": string, "useSeconds": boolean } |
ConvexAutomatedStoppingSpec
Configuration for ConvexAutomatedStoppingSpec. When there are enough completed trials (configured by minMeasurementCount), for pending trials with enough measurements and steps, the policy first computes an overestimate of the objective value at maxNumSteps according to the slope of the incomplete objective value curve. No prediction can be made if the curve is completely flat. If the overestimation is worse than the best objective value of the completed trials, this pending trial will be early-stopped, but a last measurement will be added to the pending trial with maxNumSteps and predicted objective value from the autoregression model.
Steps used in predicting the final objective for early stopped trials. In general, it's set to be the same as the defined steps in training / tuning. If not defined, it will learn it from the completed trials. When use_steps is false, this field is set to the maximum elapsed seconds.
Minimum number of steps for a trial to complete. Trials which do not have a measurement with stepCount > minStepCount won't be considered for early stopping. It's ok to set it to 0, and a trial can be early stopped at any stage. By default, minStepCount is set to be one-tenth of the maxStepCount. When useElapsedDuration is true, this field is set to the minimum elapsed seconds.
The minimal number of measurements in a Trial. Early-stopping checks will not trigger if less than minMeasurementCount+1 completed trials or pending trials with less than minMeasurementCount measurements. If not defined, the default value is 5.
learningRateParameterName
string
The hyper-parameter name used in the tuning job that stands for learning rate. Leave it blank if learning rate is not in a parameter in tuning. The learningRate is used to estimate the objective value of the ongoing trial.
useElapsedDuration
boolean
This bool determines whether or not the rule is applied based on elapsed_secs or steps. If useElapsedDuration==false, the early stopping decision is made according to the predicted objective values according to the target steps. If useElapsedDuration==true, elapsed_secs is used instead of steps. Also, in this case, the parameters maxNumSteps and minNumSteps are overloaded to contain max_elapsed_seconds and min_elapsed_seconds.
updateAllStoppedTrials
boolean
ConvexAutomatedStoppingSpec by default only updates the trials that needs to be early stopped using a newly trained auto-regressive model. When this flag is set to True, all stopped trials from the beginning are potentially updated in terms of their finalMeasurement
. Also, note that the training logic of autoregressive models is different in this case. Enabling this option has shown better results and this may be the default option in the future.
JSON representation |
---|
{ "maxStepCount": string, "minStepCount": string, "minMeasurementCount": string, "learningRateParameterName": string, "useElapsedDuration": boolean, "updateAllStoppedTrials": boolean } |
MetricSpec
Represents a metric to optimize.
metricId
string
Required. The id of the metric. Must not contain whitespaces and must be unique amongst all MetricSpecs.
Required. The optimization goal of the metric.
Used for safe search. In the case, the metric will be a safety metric. You must provide a separate metric for objective metric.
JSON representation |
---|
{ "metricId": string, "goal": enum ( |
GoalType
The available types of optimization goals.
Enums | |
---|---|
GOAL_TYPE_UNSPECIFIED |
Goal type will default to maximize. |
MAXIMIZE |
Maximize the goal metric. |
MINIMIZE |
Minimize the goal metric. |
SafetyMetricConfig
Used in safe optimization to specify threshold levels and risk tolerance.
safetyThreshold
number
Safety threshold (boundary value between safe and unsafe). NOTE that if you leave SafetyMetricConfig unset, a default value of 0 will be used.
desiredMinSafeTrialsFraction
number
Desired minimum fraction of safe trials (over total number of trials) that should be targeted by the algorithm at any time during the study (best effort). This should be between 0.0 and 1.0 and a value of 0.0 means that there is no minimum and an algorithm proceeds without targeting any specific fraction. A value of 1.0 means that the algorithm attempts to only Suggest safe Trials.
JSON representation |
---|
{ "safetyThreshold": number, "desiredMinSafeTrialsFraction": number } |
ParameterSpec
Represents a single parameter to optimize.
parameterId
string
Required. The id of the parameter. Must not contain whitespaces and must be unique amongst all ParameterSpecs.
How the parameter should be scaled. Leave unset for CATEGORICAL
parameters.
A conditional parameter node is active if the parameter's value matches the conditional node's parent_value_condition.
If two items in conditionalParameterSpecs have the same name, they must have disjoint parent_value_condition.
parameter_value_spec
Union type
parameter_value_spec
can be only one of the following:The value spec for a 'DOUBLE' parameter.
The value spec for an 'INTEGER' parameter.
The value spec for a 'CATEGORICAL' parameter.
The value spec for a 'DISCRETE' parameter.
JSON representation |
---|
{ "parameterId": string, "scaleType": enum ( |
DoubleValueSpec
value specification for a parameter in DOUBLE
type.
minValue
number
Required. Inclusive minimum value of the parameter.
maxValue
number
Required. Inclusive maximum value of the parameter.
defaultValue
number
A default value for a DOUBLE
parameter that is assumed to be a relatively good starting point. Unset value signals that there is no offered starting point.
Currently only supported by the Vertex AI Vizier service. Not supported by HyperparameterTuningJob or TrainingPipeline.
JSON representation |
---|
{ "minValue": number, "maxValue": number, "defaultValue": number } |
IntegerValueSpec
value specification for a parameter in INTEGER
type.
Required. Inclusive minimum value of the parameter.
Required. Inclusive maximum value of the parameter.
A default value for an INTEGER
parameter that is assumed to be a relatively good starting point. Unset value signals that there is no offered starting point.
Currently only supported by the Vertex AI Vizier service. Not supported by HyperparameterTuningJob or TrainingPipeline.
JSON representation |
---|
{ "minValue": string, "maxValue": string, "defaultValue": string } |
CategoricalValueSpec
value specification for a parameter in CATEGORICAL
type.
values[]
string
Required. The list of possible categories.
defaultValue
string
A default value for a CATEGORICAL
parameter that is assumed to be a relatively good starting point. Unset value signals that there is no offered starting point.
Currently only supported by the Vertex AI Vizier service. Not supported by HyperparameterTuningJob or TrainingPipeline.
JSON representation |
---|
{ "values": [ string ], "defaultValue": string } |
DiscreteValueSpec
value specification for a parameter in DISCRETE
type.
values[]
number
Required. A list of possible values. The list should be in increasing order and at least 1e-10 apart. For instance, this parameter might have possible settings of 1.5, 2.5, and 4.0. This list should not contain more than 1,000 values.
defaultValue
number
A default value for a DISCRETE
parameter that is assumed to be a relatively good starting point. Unset value signals that there is no offered starting point. It automatically rounds to the nearest feasible discrete point.
Currently only supported by the Vertex AI Vizier service. Not supported by HyperparameterTuningJob or TrainingPipeline.
JSON representation |
---|
{ "values": [ number ], "defaultValue": number } |
ScaleType
The type of scaling that should be applied to this parameter.
Enums | |
---|---|
SCALE_TYPE_UNSPECIFIED |
By default, no scaling is applied. |
UNIT_LINEAR_SCALE |
Scales the feasible space to (0, 1) linearly. |
UNIT_LOG_SCALE |
Scales the feasible space logarithmically to (0, 1). The entire feasible space must be strictly positive. |
UNIT_REVERSE_LOG_SCALE |
Scales the feasible space "reverse" logarithmically to (0, 1). The result is that values close to the top of the feasible space are spread out more than points near the bottom. The entire feasible space must be strictly positive. |
ConditionalParameterSpec
Represents a parameter spec with condition from its parent parameter.
Required. The spec for a conditional parameter.
parent_value_condition
Union type
parent_value_condition
can be only one of the following:The spec for matching values from a parent parameter of DISCRETE
type.
The spec for matching values from a parent parameter of INTEGER
type.
The spec for matching values from a parent parameter of CATEGORICAL
type.
JSON representation |
---|
{ "parameterSpec": { object ( |
DiscreteValueCondition
Represents the spec to match discrete values from parent parameter.
values[]
number
Required. Matches values of the parent parameter of 'DISCRETE' type. All values must exist in discreteValueSpec
of parent parameter.
The Epsilon of the value matching is 1e-10.
JSON representation |
---|
{ "values": [ number ] } |
IntValueCondition
Represents the spec to match integer values from parent parameter.
Required. Matches values of the parent parameter of 'INTEGER' type. All values must lie in integerValueSpec
of parent parameter.
JSON representation |
---|
{ "values": [ string ] } |
CategoricalValueCondition
Represents the spec to match categorical values from parent parameter.
values[]
string
Required. Matches values of the parent parameter of 'CATEGORICAL' type. All values must exist in categoricalValueSpec
of parent parameter.
JSON representation |
---|
{ "values": [ string ] } |
Algorithm
The available search algorithms for the Study.
Enums | |
---|---|
ALGORITHM_UNSPECIFIED |
The default algorithm used by Vertex AI for hyperparameter tuning and Vertex AI Vizier. |
GRID_SEARCH |
Simple grid search within the feasible space. To use grid search, all parameters must be INTEGER , CATEGORICAL , or DISCRETE . |
RANDOM_SEARCH |
Simple random search within the feasible space. |
ObservationNoise
Describes the noise level of the repeated observations.
"Noisy" means that the repeated observations with the same Trial parameters may lead to different metric evaluations.
Enums | |
---|---|
OBSERVATION_NOISE_UNSPECIFIED |
The default noise level chosen by Vertex AI. |
LOW |
Vertex AI assumes that the objective function is (nearly) perfectly reproducible, and will never repeat the same Trial parameters. |
HIGH |
Vertex AI will estimate the amount of noise in metric evaluations, it may repeat the same Trial parameters more than once. |
MeasurementSelectionType
This indicates which measurement to use if/when the service automatically selects the final measurement from previously reported intermediate measurements. Choose this based on two considerations: A) Do you expect your measurements to monotonically improve? If so, choose LAST_MEASUREMENT. On the other hand, if you're in a situation where your system can "over-train" and you expect the performance to get better for a while but then start declining, choose BEST_MEASUREMENT. B) Are your measurements significantly noisy and/or irreproducible? If so, BEST_MEASUREMENT will tend to be over-optimistic, and it may be better to choose LAST_MEASUREMENT. If both or neither of (A) and (B) apply, it doesn't matter which selection type is chosen.
Enums | |
---|---|
MEASUREMENT_SELECTION_TYPE_UNSPECIFIED |
Will be treated as LAST_MEASUREMENT. |
LAST_MEASUREMENT |
Use the last measurement reported. |
BEST_MEASUREMENT |
Use the best measurement reported. |
TransferLearningConfig
This contains flag for manually disabling transfer learning for a study. The names of prior studies being used for transfer learning (if any) are also listed here.
disableTransferLearning
boolean
Flag to to manually prevent vizier from using transfer learning on a new study. Otherwise, vizier will automatically determine whether or not to use transfer learning.
priorStudyNames[]
string
Output only. Names of previously completed studies
JSON representation |
---|
{ "disableTransferLearning": boolean, "priorStudyNames": [ string ] } |
StudyStoppingConfig
The configuration (stopping conditions) for automated stopping of a Study. Conditions include trial budgets, time budgets, and convergence detection.
shouldStopAsap
boolean
If true, a Study enters STOPPING_ASAP whenever it would normally enters STOPPING state.
The bottom line is: set to true if you want to interrupt on-going evaluations of Trials as soon as the study stopping condition is met. (Please see Study.State documentation for the source of truth).
Each "stopping rule" in this proto specifies an "if" condition. Before Vizier would generate a new suggestion, it first checks each specified stopping rule, from top to bottom in this list. Note that the first few rules (e.g. minimumRuntimeConstraint, minNumTrials) will prevent other stopping rules from being evaluated until they are met. For example, setting minNumTrials=5
and always_stop_after= 1 hour
means that the Study will ONLY stop after it has 5 COMPLETED trials, even if more than an hour has passed since its creation. It follows the first applicable rule (whose "if" condition is satisfied) to make a stopping decision. If none of the specified rules are applicable, then Vizier decides that the study should not stop. If Vizier decides that the study should stop, the study enters STOPPING state (or STOPPING_ASAP if shouldStopAsap = true). IMPORTANT: The automatic study state transition happens precisely as described above; that is, deleting trials or updating StudyConfig NEVER automatically moves the study state back to ACTIVE. If you want to resume a Study that was stopped, 1) change the stopping conditions if necessary, 2) activate the study, and then 3) ask for suggestions. If the specified time or duration has not passed, do not stop the study.
If the specified time or duration has passed, stop the study.
minNumTrials
integer
If there are fewer than this many COMPLETED trials, do not stop the study.
maxNumTrials
integer
If there are more than this many trials, stop the study.
maxNumTrialsNoProgress
integer
If the objective value has not improved for this many consecutive trials, stop the study.
WARNING: Effective only for single-objective studies.
If the objective value has not improved for this much time, stop the study.
WARNING: Effective only for single-objective studies.
A duration in seconds with up to nine fractional digits, ending with 's
'. Example: "3.5s"
.
JSON representation |
---|
{ "shouldStopAsap": boolean, "minimumRuntimeConstraint": { object ( |
StudyTimeConstraint
time-based Constraint for Study
constraint
Union type
constraint
can be only one of the following:Counts the wallclock time passed since the creation of this Study.
A duration in seconds with up to nine fractional digits, ending with 's
'. Example: "3.5s"
.
Compares the wallclock time to this time. Must use UTC timezone.
A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z"
and "2014-10-02T15:01:23.045123456Z"
.
JSON representation |
---|
{ // constraint "maxDuration": string, "endTime": string // Union type } |