Interpretable Image Classification with Prototypes

We can better explain model predictions by surfacing the most similar items from training data. Customers bring their data, and we return a classifier that not only predicts the output, but also related examples ("prototypes") that explain the decision.

Apply for access Private documentation

Intended use

Problem types: In many real-world applications of AI, a deeper understanding of how the model works is critical to improving and adopting the solution. One way to better interpret a model's behavior is to surface, in conjunction with a prediction, a handful of perceptually-relevant items from the training dataset. The comparison of predicted items to relevant, ground truth examples can help to increase trust in the model and to generate actionable insights for developers to improve it (e.g. by modifying training datasets or loss functions).

Inputs and outputs:

  • Users provide: Labeled training and evaluation image datasets.
  • Users receive: API access to a private AI model that is composed convolutional neural network based classifier with the prototype engine. Feeding prediction items into this API will return both the predicted class as well as a few prototypical images that explain the classification decision.

What data do I need?

Data and label types:

This experiment works with any image data (containing legally-allowed content) as long as they are well-labeled. Classes can include specific objects within the image (such as dog or flower), as well as characteristics of the image (such as color, shape, texture, sentiment, quality).

Data specifications:

  • Users must provide at least 1000 images for each class that needs to be trained. There is no restriction on the number of classes.
  • Each image may be assigned to only one class
  • The class labels should be integers between 0 and N-1 (where N is the number of output classes).
  • Stored in the following formats: BMP, GIF, JPEG, PNG.
  • Maximum image size: 3 MP.
  • Image labels should be provided in CSV format

What skills do I need?

As with all AI Workshop experiments, successful users are likely to have a general understanding of core AI concepts and skills in order to both deploy the experiment technology and interact with our AI researchers and engineers. We also expect familiarity with accessing Google Cloud AI Platform APIs to run the model.