Vizier is a black-box optimization service that helps you tune hyperparameters in complex machine learning (ML) models. When ML models have many different hyperparameters, it can be difficult and time consuming to tune them manually. Vizier optimizes your model's output by tuning the hyperparameters for you.
Black-box optimization is the optimization of a system that meets either of the following criteria:
Doesn't have a known objective function to evaluate.
Is too costly to evaluate by using the objective function, usually due to the complexity of the system.
Additional Vizier functionality
Vizier optimizes hyperparameters of ML models, but it can also perform other optimization tasks.
You can use Vizier to effectively tune parameters in a function. For example, you can use it to determine the most effective combination of background color, font size, and link color on a news website's Subscription button. For more examples, see the use cases.
Optimize any evaluable system
Vizier works with any system that you can evaluate, including systems that cannot be expressed as a closed-form analytical function. For example, you can use Vizier to find the best neural network depth, width, and learning rate for a TensorFlow model.
How Vizier works
The following sections define terms, behavior, and available values that you can use with Vizier to optimize your ML model or function. You start by determining a study configuration.
A study configuration is the definition of the optimization problem that you are trying to solve. It includes the result you would like to optimize and the hyperparameters or parameters that affect that result.
Studies and trials
A study is the implementation of a study configuration. A study uses the study configuration's goals (metrics) and input values (hyperparameters or parameters) to conduct experiments, called trials. A trial is a specific set of input values that produce a measured outcome relative to your goals.
Vizier suggests input values to use for each trial but it does not run your trials for you.
A study continues until it reaches a set limit of trials, or you interrupt the study. A trial continues until you indicate that it is either finished or infeasible.
A measurement is the measured outcome of your trial. Each measurement can contain one or more metrics, and each trial can contain one or more measurements taken over a period of time. You can add a new measurement to the trial at any point before the trial is completed.
If you do not specify an algorithm, Vizier uses the default algorithm. The default algorithm applies Bayesian optimization to arrive at the optimal solution with a more effective search over the parameter space.
The following values are available:
ALGORITHM_UNSPECIFIED: Results in the same behavior as when you don't specify a search algorithm. Vizier uses a default algorithm, which applies Bayesian optimization to search the space of possible values, resulting in the most effective technique for your set of parameters.
GRID_SEARCH: A simple grid search within the feasible space. This option is useful if you want to specify a quantity of trials that is greater than the number of points in the feasible space. In such cases, if you do not specify a grid search, the default algorithm can generate duplicate suggestions. To use grid search, all parameters must be of type
RANDOM_SEARCH: A simple random search within the feasible space.
How Vizier differs from custom training
Vizier is an independent service for optimizing complex models with many parameters. It can be used for both ML and non-ML use cases. It can be used with Training jobs or with other systems (even multi-cloud). Hyperparameter tuning for custom training is a built-in feature that uses Vizier for training jobs. It helps determine the best hyperparameter settings for an ML model.
In the following scenarios, Vizier helps tune hyperparameters to optimize a model or tune parameters to optimize an outcome:
Optimize the learning rate, batch size, and other hyperparameters of a neural network recommendation engine.
Optimize usability of an application by testing different arrangements of user interface elements.
Minimize computing resources for a job by identifying an ideal buffer size and thread count.
Optimize the amounts of ingredients in a recipe to produce the most delicious version.
- Vizier is based on Google Vizier, a Google-internal service for performing black-box optimization. For more information about black-box optimization and Google Vizier architecture and algorithms, see the PDF Google Vizier: A Service for Black-Box Optimization.