Using Vertex Vizier

This page describes how to make API requests to Vertex Vizier using Python. Read the Vertex Vizier overview to learn about how Vertex Vizier works.

Before you begin

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Enable the Vertex AI API.

    Enable the API

  4. Create a service account:

    1. In the Cloud Console, go to the Create service account page.

      Go to Create service account
    2. Select a project.
    3. In the Service account name field, enter a name. The Cloud Console fills in the Service account ID field based on this name.

      In the Service account description field, enter a description. For example, Service account for quickstart.

    4. Click Create.
    5. Click the Select a role field.

      Under Quick access, click Basic, then click Owner.

    6. Click Continue.
    7. Click Done to finish creating the service account.

      Do not close your browser window. You will use it in the next step.

  5. Create a service account key:

    1. In the Cloud Console, click the email address for the service account that you created.
    2. Click Keys.
    3. Click Add key, then click Create new key.
    4. Click Create. A JSON key file is downloaded to your computer.
    5. Click Close.
  6. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again.

  7. Install the Vertex SDK for Python.

Defining constants

Run the following commands, replacing REGION and PROJECT_ID with your region and project ID. Create your own study name or use the suggested values.

import json
import datetime
from google.cloud import aiplatform_v1beta1

REGION = "PROJECT_ID"
PROJECT_ID = "REGION"

# These will be automatically filled in.
STUDY_DISPLAY_NAME = '{}_study_{}'.format(PROJECT_ID.replace('-', ''), datetime.datetime.now().strftime('%Y%m%d_%H%M%S')) #@param {type: 'string'}
ENDPOINT = REGION + '-aiplatform.googleapis.com'
PARENT = 'projects/{}/locations/{}'.format(PROJECT_ID, REGION)

Constructing API requests

The following describes how to construct command line API requests to Vertex Vizier, using Python.

Creating a study

A study is a series of experiments, or trials, that help you optimize your hyperparameters or parameters.

In the following example, the goal is to maximize y = x^2 with x in the range of [-10. 10]. This example has only one parameter and uses an easily-calculated function to help demonstrate how to use Vertex Vizier.

param_x = {
    'parameter_id': 'x',
    'double_value_spec': {
        'min_value': -10.0,
        'max_value': 10.0
    }
}

metric_y = {
    'metric_id': 'y',
    'goal': 'MAXIMIZE'
}

study = {
    'display_name': STUDY_DISPLAY_NAME,
    'study_spec': {
      'algorithm': 'RANDOM_SEARCH',
      'parameters': [param_x],
      'metrics': [metric_y],
    }
}

To create the study using your study configuration, send the following request through VizierServiceClient. The returned STUDY_NAME will be used for querying the study.

vizier_client = aiplatform_v1beta1.VizierServiceClient(client_options=dict(api_endpoint=ENDPOINT))
study = vizier_client.create_study(parent=PARENT, study=study)
STUDY_NAME = study.name

Viewing your study

When your study is created, you can find the study in the Cloud Console, in the Vertex AI section, on the Experiments page.

Go to the Experiments page

Getting a study

To get a study, send the following request.

vizier_client.get_study({'name': STUDY_NAME})

Listing studies

To list studies in a specific project and region, send the following request.

vizier_client.list_studies({'parent': PARENT})

Getting suggested trials

To get a trial suggestion from Vertex Vizier, you create a request that contains a SUGGEST_COUNT and your CLIENT_ID. Then you send a request that passes this information to Vertex Vizier.

Create the request using the following commands. Change the suggestionCount to the number of suggestions that you want to get from each request.

suggest_response = vizier_client.suggest_trials({
    'parent': STUDY_NAME,
    'suggestion_count': SUGGEST_COUNT,
    'client_id': CLIENT_ID
    })

suggestTrials starts a long-running operation which works on generating the trial. The response lets you know that Vertex Vizier is working on the trial suggestions. You can use the result() function to wait for the returned result like the following.

suggest_response.result().trials

An example trial is in this format:

name: "TRIAL_ID"
state: ACTIVE
parameters {
  parameter_id: "x"
  value {
    number_value: 0.1
  }
}
start_time {
  seconds: 1618011215
}

In the example above, this trial suggests using value 0.1 for the parameter x.

Using the TRIAL_ID from the previous response, you can get the trial using the following request:

vizier_client.get_trial({
        'name': TRIAL_ID
      })

Evaluating the results

After receiving your trial suggestions, you evaluate each trial and record each result as a measurement.

For example, if the function you are trying to optimize is y = x^2, then you evaluate the function using the trial's suggested value of x. Using a suggested value of 0.1, the function evaluates to y = 0.1 * 0.1, which results in 0.01.

Adding a measurement

After evaluating your trial suggestion to get a measurement, you add this measurement to your trial.

Store your measurement and send the request using the following commands. In this example, you replace RESULT with the measurement. If the function you are optimizing is y = x^2, and the suggested value of x is 0.1, the result is 0.01.

vizier_client.add_trial_measurement({
        'trial_name': TRIAL_ID,
        'measurement': {
            'metrics': [{'metric_id': 'y', 'value':RESULT }]
        }
    })

Completing a trial

After you've added all measurements for a trial, complete the trial using the following command. Optionally, you can add a final measurement when you call completeTrial.

Without final measurement

To complete a trial without adding a final measurement, send the following request

vizier_client.complete_trial({
      'name': TRIAL_ID
  })

With final measurement

To include a final measurement when you complete the trial, construct your final measurement using the following commands, replacing RESULT with the final measurement.

vizier_client.complete_trial({
      'name': TRIAL_ID
      'final_measurement': {
        'metrics': [{'metric_id': 'y', 'value': RESULT}]
      }
  })

Listing trials

To list trials in a specific study, send the following request.

vizier_client.list_trials({
    'parent': STUDY_NAME
})

Once you complete all the pending trials, you can call suggestTrials for more suggestions and repeat the trial evaluation process.

Listing Optimal trials

list_optimal_trials returns the pareto-optimal Trials for multi-objective Study or the optimal Trials for single-objective Study. An example request is following:

vizier_client.list_optimal_trials({
    'parent': STUDY_NAME
})