Make API requests to AI Platform Vizier

Make API requests to AI Platform Vizier

Beta

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

This page describes how to make API requests to AI Platform Vizier using curl.

Before you begin

1. Read the AI Platform Vizier overview to learn about how AI Platform Vizier works.
2. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

3. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Go to project selector

4. Make sure that billing is enabled for your Google Cloud project.

5. Enable the AI Platform Training and Prediction API.

   Enable the API

6. Install the Google Cloud CLI.

7. To initialize the gcloud CLI, run the following command:

   gcloud init

8. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Go to project selector

9. Make sure that billing is enabled for your Google Cloud project.

10. Enable the AI Platform Training and Prediction API.

    Enable the API

11. Install the Google Cloud CLI.

12. To initialize the gcloud CLI, run the following command:

    gcloud init

Logging in to your Cloud Billing account

Use the following commands to log in to your Google Cloud account:

gcloud auth application-default login
gcloud auth login

Defining constants

Run the following commands, replacing USERNAME and PROJECT_ID with your username and project ID information. Create your own study name and client ID or use the suggested values.

Note: Currently, AI Platform Vizier only supports the us-central1 region.

export USER=USERNAME
export PROJECT_ID=PROJECT_ID{"</var>"}}
export REGION=us-central1

export STUDY_NAME=${USER}_vizier_study_$(date +%Y%m%d%H%M%S)
export CLIENT_ID=${USER}_client_1
export ENDPOINT=https://$REGION-ml.googleapis.com/v1

Constructing API requests

The following describes how to construct command line API requests to AI Platform Vizier, using curl.

Creating a study

A study is a series of experiments, or trials, that help you optimize your hyperparameters or parameters.

To create a study, you create a study configuration in JSON format, then send a POST request that passes your configuration to AI Platform Vizier.

The study configuration might look like the following. Learn more about algorithm options, goal types, and other options in the AI Platform Training and Prediction API documentation.

In the following example, the goal is to maximize y = x^2 with x in the range of [-10. 10]. This example has only one parameter and uses an easily-calculated function to help demonstrate how to use AI Platform Vizier.

You can learn more about how AI Platform Vizier can be used for complex optimization problems, and how to optimize multiple functions at once.

cat > /tmp/create_study.json <<EOF
{
  "studyConfig": {
    "algorithm": 0,
    "metrics": [
      {
        "goal": "MAXIMIZE",
        "metric": "y"
      }
    ],
    "parameters": [
      {
        "doubleValueSpec": {
          "maxValue": 10.0,
          "minValue": -10.0
        },
        "parameter": "x",
        "type": "DOUBLE"
      }
    ]
  }
}
EOF

To create the study using your study configuration, send the following POST request.

curl -X POST -H "Content-Type: application/json" \
  -d @/tmp/create_study.json \
  -H "Authorization: Bearer `gcloud auth print-access-token`" \
  "${ENDPOINT}/projects/${PROJECT_ID}/locations/${REGION}/studies?study_id=${STUDY_NAME}"

Getting a study

To get a study, send the following request.

curl -H "Content-Type: application/json"   \
  -H "Authorization: Bearer `gcloud auth print-access-token`"   \
  "${ENDPOINT}/projects/${PROJECT_ID}/locations/${REGION}/studies/${STUDY_NAME}"

Listing studies

To list studies in a specific project and region, send the following request.

curl -H "Content-Type: application/json"  \
  -H "Authorization: Bearer `gcloud auth print-access-token`"   \
  "${ENDPOINT}/projects/${PROJECT_ID}/locations/${REGION}/studies/"

Getting suggested trials

To get a trial suggestion from AI Platform Vizier, you create a JSON file that contains a suggestionCount and your client ID. Then you send a POST request that passes this information to AI Platform Vizier.

Create the JSON file using the following commands. Change the suggestionCount to the number of suggestions that you want to get from each request.

cat > /tmp/suggest_trial.json <<EOF
{
  "suggestionCount": 1,
  "clientId": "${CLIENT_ID}"
}
EOF

To get the suggestion or suggestions, send the following POST request.

curl -X POST -H "Content-Type: application/json" \
  -d @/tmp/suggest_trial.json \
  -H "Authorization: Bearer `gcloud auth print-access-token`" \
  "${ENDPOINT}/projects/${PROJECT_ID}/locations/${REGION}/studies/${STUDY_NAME}/trials:suggest"

suggestTrial starts a long-running operation which works on generating the the trial. The response lets you know that AI Platform Vizier is working on trial suggestions. This response will be in the following format:

{
  "name": "projects/<project>/locations/<region>/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.ml.v1.SuggestTrialsMetadata",
    "study": <study-name>,
    "createTime": <create-time>,
    "suggestionCount": <suggestion-count>
  }
}

You can use the operation ID in the previous response to poll the suggestion operation and get your trial suggestions. Use the following command:

SUGGEST_OPERATION_ID=OPERATION_ID

curl -H "Content-Type: application/json"   \
  -H "Authorization: Bearer `gcloud auth print-access-token`"   \
  "${ENDPOINT}/projects/${PROJECT_ID}/locations/${REGION}/operations/${SUGGEST_OPERATION_ID}"

If you don't get a response, the suggestTrial request might not be finished. Repeat the previous command if needed.

The response provides trial suggestions, in this format:

{
  "name": "projects/<project>/locations/<region>/operations/<operation-id>",
  "metadata": {...},
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.cloud.ml.v1.SuggestTrialsResponse",
    "trials": [
      {
        "name": "projects/<project>/locations/<region>/studies/<study-id>/trials/TRIAL_ID",
        "state": "ACTIVE",
        "parameters": [
          {
            "parameter": "x",
            "floatValue": 0.1
          }
        ],
        ...
      }
    ],
    ...
  }
}

In the example above, this trial suggests using value 0.1 for the parameter x.

Using the trial ID from the previous response, you can get the trial using the following command:

TRIAL_ID=TRIAL_ID

curl -H "Content-Type: application/json"   \
  -H "Authorization: Bearer `gcloud auth print-access-token`"   \
  "${ENDPOINT}/projects/${PROJECT_ID}/locations/${REGION}/studies/${STUDY_NAME}/trials/${TRIAL_ID}"

Evaluating the results

After receiving your trial suggestions, you evaluate each trial and record each result as a measurement.

For example, if the function you are trying to optimize is y = x^2, then you evaluate the function using the trial's suggested value of x. Using a suggested value of 0.1, the function evaluates to y = 0.1 * 0.1, which results in 0.01.

Adding a measurement

After evaluating your trial suggestion to get a measurement, you add this measurement to your trial. First, you create a JSON file that contains the metric you measured and the result. Then you send a POST request that passes this information to AI Platform Vizier.

Store your measurement and create the JSON file using the following commands. In this example, you replace RESULT with the measurement. If the function you are optimizing is y = x^2, and the suggested value of x is 0.1, the result is 0.01.

METRIC_VALUE=RESULT

cat > /tmp/add_measurement.json <<EOF
{
  "measurement": {
    "stepCount": 1,
    "metrics": [
      {
        "metric": "y",
        "value": ${METRIC_VALUE}
      }
    ]
  }
}
EOF

Note: If the evaluation is a multi-step process and can produce more than one intermediate measurement, you can call the method multiple times with different stepCount and metric values. For example, if you are training a machine learning model, you can report an intermediate measurement for each epoch.

To add this measurement to your trial, send the following POST request.

curl -X POST -H "Content-Type: application/json" \
  -d @/tmp/add_measurement.json \
  -H "Authorization: Bearer `gcloud auth print-access-token`" \
  "${ENDPOINT}/projects/${PROJECT_ID}/locations/${REGION}/studies/${STUDY_NAME}/trials/${TRIAL_ID}:addMeasurement"

Completing a trial

After you've added all measurements for a trial, complete the trial using the following command. Optionally, you can add a final measurement when you call completeTrial.

Without final measurement

To complete the trial without adding a final measurement, send the following POST request.

curl -X POST -H "Content-Type: application/json" \
  -d "" \
  -H "Authorization: Bearer `gcloud auth print-access-token`" \
  "${ENDPOINT}/projects/${PROJECT_ID}/locations/${REGION}/studies/${STUDY_NAME}/trials/${TRIAL_ID}:complete"

With final measurement

To include a final measurement when you complete the trial, first store your final measurement and create a JSON file using the following commands, replacing RESULT with the final measurement.

FINAL_METRIC_VALUE=RESULT

cat > /tmp/complete_trial.json <<EOF
{
  "finalMeasurement": {
    "stepCount": 1,
    "metrics": [
      {
        "metric": "y",
        "value": ${FINAL_METRIC_VALUE}
      }
    ]
  }
}
EOF

Send the following POST request.

curl -X POST -H "Content-Type: application/json" \
  -d @/tmp/complete_trial.json \
  -H "Authorization: Bearer `gcloud auth print-access-token`" \
  "${ENDPOINT}/projects/${PROJECT_ID}/locations/${REGION}/studies/${STUDY_NAME}/trials/${TRIAL_ID}:complete"

Listing trials

To list trials in a specific study, send the following request.

curl -H "Content-Type: application/json"   \
  -H "Authorization: Bearer `gcloud auth print-access-token`"   \
  "${ENDPOINT}/projects/${PROJECT_ID}/locations/${REGION}/studies/${STUDY_NAME}/trials/"

Once you complete all the pending trials, you can call suggestTrial for more suggestions and repeat the trial evaluation process.

What's next

For examples of how to use the API, see the following:

• Getting started: Optimizing a machine learning model.
• Getting started: Optimizing multiple objectives.