Using the Python Client Library

This tutorial describes how to use the Google API client library to easily make calls to the Cloud Machine Learning Engine REST APIs in your Python applications. The code snippets and examples in the rest of this documentation use the Python client library.

You will be creating a model in your Google Cloud Platform project in this tutorial. That is a simple task that's easily contained to a small example. You can learn more about managing models in the model and job management overview.

Objectives

This is a basic tutorial designed to familiarize you with the Python client library. When you're finished you should be able to:

  • Use application default credentials in your Python applications.
  • Get a Python representation of the Cloud ML Engine services.
  • Use that representation to create a model in your project, which should help you understand how to call the other model and job management APIs.

Costs

You will not be charged for the operations in this tutorial. Refer to the pricing page for more information.

Before you begin

Prepare your Google Cloud Platform project and your development environment for using Cloud ML Engine in your Python applications by following the setup instructions. Take special care to install the Google API Client for Python.

Importing the required modules

When you want to use the API client library to use the Cloud ML Engine REST APIs in your Python code, you must import its package and the OAuth2 package. For this tutorial (and for most standard uses of Cloud ML Engine) you only need to import specific modules from both packages: GoogleCredentials from OAuth2 and discovery from Google API Client Library. Refer to the documentation for those packages to learn about the other available modules.

Create a new Python file using your favorite editor, and add these lines:

from oauth2client.client import GoogleCredentials
from googleapiclient import discovery

Getting your application credentials

You need credentials to access the APIs for your project. When you initialized the gcloud command-line tool during setup, you entered your user identification and selected a project to work on. You'll use those credentials, called application default credentials, for this tutorial, because they are the easiest to use. Real-world applications often require more complicated auth arrangements.

Here's what to add to your code:

credentials = GoogleCredentials.get_application_default()

Building a Python representation of the API

Now that you have your application default credentials, you can get your Python representation of the REST API. The method you call is build because the API client library uses service discovery to dynamically set up connections to the services as they exist when you make the call. Call your object that encapsulates the services ml and use the credentials object you just got:

ml = discovery.build('ml','v1', credentials=credentials)

Configuring your parameters and request body

To make a call to a service, you must create the parameters and request body that will be passed to the REST API. You pass parameters as regular Python parameters to the method that represents the call. The body is a JSON resource just as you would use if calling the API with an HTTP request directly.

Take a look at the REST API for creating a model in a new browser tab, projects.models.create:

  • Notice the path parameter parent, which is the part of the URI of the request that identifies the project. If you were making the HTTP POST request directly, you would use the following URI:

    https://ml.googleapis.com/v1/projects/your_project_ID/models
    

    When using the API client library, the variable part of the URI is represented as a string-typed parameter to the API call. You'll set it to 'projects/<var>your_project_ID</var>'. Store your project in a variable to make API calls cleaner:

    projectID = 'projects/{}'.format('your_project_ID')
    

  • The body of the request is a JSON resource representing the model information. You can see in the model resource definition that it has two values for input: name and (optionally) description. You can pass a Python dictionary in the place of JSON and the API client library will perform the necessary conversion.

    Create your Python dictionary:

    requestDict = {'name': 'your-model-name',
                   'description': 'This is a machine learning model entry.'}
    

Creating your request

Making calls to APIs with the Python client library has two steps: first you create a request, then you make the call using that request.

Create the request

Use the built client objects that you created earlier (if you followed the code snippet exactly, it's called ml) as the root of the API hierarchy and specify the API you want to use. Each collection in the API path behaves like a function that returns a list of the collections and methods within it. For example, the root of all the Cloud ML Engine APIs is projects, so your call begins with ml.projects().

Use this code to form your request:

request = ml.projects().models().create(parent=projectID, body=requestDict)

Send the request

The request that you constructed in the last step exposes an execute method that you call to send the request to the service:

response = request.execute()

It's common for developers to combine this step with the last one:

response = ml.projects().models().create(parent=projectID,
                                         body=requestDict).execute()

Handle simple errors

A lot of things can go wrong when you make API calls over the Internet. It's a good idea to handle common errors. The simplest way to deal with errors is to put your request in a try block and catch likely errors. Most of the errors you're likely to get from the service are HTTP errors, which are encapsulated in the HttpError class. To catch these errors, you first need to get the errors module from the googleapiclient package. Add this statement with the rest of your imports.

from googleapiclient import errors

Then wrap your request.execute() call in a try block. Also put a print statement in the block, so that you will try to print the response only if the call succeeds:

try:
    response = request.execute()
    print(response)

Finally, add a catch block to handle HTTP errors. You can use HttpError._get_reason() to get the reason text fields from the response:

except errors.HttpError, err:
    # Something went wrong, print out some information.
    print('There was an error creating the model. Check the details:')
    print(err._get_reason())

Of course, a simple print statement might not be the right approach for your application.

Putting it all together

Here is the complete example:

from oauth2client.client import GoogleCredentials
from googleapiclient import discovery
from googleapiclient import errors

# Store your full project ID in a variable in the format the API needs.
projectID = 'projects/{}'.format('your_project_ID')

# Get application default credentials (possible only if the gcloud tool is
#  configured on your machine).
credentials = GoogleCredentials.get_application_default()

# Build a representation of the Cloud ML API.
ml = discovery.build('ml', 'v1', credentials=credentials)

# Create a dictionary with the fields from the request body.
requestDict = {'name': 'your_model_name',
               'description': 'your_model_description'}

# Create a request to call projects.models.create.
request = ml.projects().models().create(
              parent=projectID, body=requestDict)

# Make the call.
try:
    response = request.execute()
    print(response)
except errors.HttpError, err:
    # Something went wrong, print out some information.
    print('There was an error creating the model. Check the details:')
    print(err._get_reason())

Generalizing to other methods

You can use the procedure you learned here to make any of the other REST API calls. Some of the APIs require much more complicated JSON resources than creating a model does, but the principles are the same:

  1. Import oauth2client.client.GoogleCredentials, googleapiclient.discovery, and googleapiclient.errors.

  2. Get credentials appropriate to your application (the simple application default credentials you used here provide an easy auth solution while you experiment with the API).

  3. Use the discovery module to build a Python representation of the API.

  4. Use the API representation as a series of nested objects to get to the API you want and create a request. For example,

    projects().models().versions().delete(
        name='projects/myproject/models/mymodel/versions/myversion')
    

  5. Call request.execute() to send the request, handling exceptions in an appropriate way for your application.

  6. When there is a response body, you can treat it like a Python dictionary to get at the JSON objects specified in the API reference. Note that many of the objects in responses have fields that are present only in some circumstances. You should always check to avoid key errors:

    response = request.execute()

    some_value = response.get('some_key') or 'default_value'

What's next

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Cloud Machine Learning Engine (Cloud ML Engine)