Vertex Vizier is a black-box optimization service that helps you tune hyperparameters in complex machine learning (ML) models. When ML models have many different hyperparameters, it can be difficult and time consuming to tune them manually. Vertex Vizier optimizes your model's output by tuning the hyperparameters for you.
Black-box optimization is the optimization of a system that meets either of the following criteria:
Doesn't have a known objective function to evaluate.
Is too costly to evaluate using the objective function, usually due to the complexity of the system.
If it is easier to conduct experiments on a system than to fully understand it, that system acts as if it is a black box. If you need to optimize that system, you can use black-box optimization.
Additional Vertex Vizier functionality
Vertex Vizier was built to optimize hyperparameters of ML models but it can also perform other optimization tasks.
Vertex Vizier can be used effectively to tune parameters in a function. For example, you can use it to determine the most effective combination of background color, font size, and link color on a news website's Subscription button. See the use cases for more examples. Read about the difference between hyperparameters and parameters.
Optimizing any evaluable system
Vertex Vizier can be used with any system that you can evaluate. This includes systems that cannot be expressed as a closed-form analytical function. For example, you can use Vertex Vizier to find the best neural network depth, width, and learning rate for a TensorFlow model.
How Vertex Vizier works
The following section summarizes how to use Vertex Vizier to optimize your ML model or function. You start by determining a study configuration.
A study configuration is the definition of the optimization problem that you are trying to solve. It includes the result you would like to optimize and the hyperparameters or parameters that affect that result.
Studies and trials
A study is the implementation of a study configuration. A study uses the study configuration's goals (metrics) and input values (hyperparameters or parameters) to conduct experiments, called trials. A trial is a specific set of input values that produce a measured outcome relative to your goals.
Vertex Vizier suggests input values to use for each trial but it does not run your trials for you.
A study continues until it reaches a set limit of trials, or you interrupt the study. A trial continues until you indicate that it is either finished or infeasible.
A measurement is the measured outcome of your trial. Each measurement can contain one or more metrics, and each trial can contain one or more measurements taken over a period of time. You can add a new measurement to the trial at any point before the trial is completed.
If you do not specify an algorithm, Vertex Vizier uses the default algorithm. The default algorithm applies Bayesian optimization to arrive at the optimal solution with a more effective search over the parameter space.
Available values are:
ALGORITHM_UNSPECIFIED: Results in the same behavior as when you don't specify a search algorithm. Vertex Vizier uses a default algorithm, which applies Bayesian optimization to search the space of possible values, resulting in the most effective technique for your set of parameters.
GRID_SEARCH: A simple grid search within the feasible space. This option is particularly useful if you want to specify a quantity of trials that is greater than the number of points in the feasible space. In such cases, if you do not specify a grid search, the default algorithm may generate duplicate suggestions. To use grid search, all parameters must be of type
RANDOM_SEARCH: A simple random search within the feasible space.
How Vertex Vizier differs from custom training
Hyperparameter tuning for custom training can make it easier to determine the best hyperparameter settings for Vertex AI custom training jobs. Vertex Vizier is built to tune both hyperparameters and parameters, and can be used with any system that you can evaluate, including a Vertex AI custom training job.
Here are a few scenarios where Vertex Vizier can help tune hyperparameters to optimize a model or tune parameters to optimize an outcome:
Optimize the learning rate, batch size, and other hyperparameters of a neural network recommendation engine.
Test different arrangements of user interface elements to optimize usability of an application.
Find an ideal buffer size and thread count to minimize computing resources for a job.
Optimize the amounts of ingredients in a recipe to produce the most delicious version.
- Vertex Vizier is based on Google Vizier. Read Google Vizier: A Service for Black-Box Optimization to learn more about black-box optimization and Google Vizier's architecture and algorithms.