This quickstart walks you through the process of annotating videos using a custom model.
Enable the API
- Visit the AutoML Video UI and click Enable API.
- After the API has been enabled, click Get Started.
Create your dataset
In the Datasets page of the AutoML Video UI, do the following:
- Click Create Dataset.
- Specify a name for this dataset and then click Create Dataset.
On the page for your dataset, provide the Cloud Storage URI of the CSV file that contains the URIs of your training data. For this quickstart, use the following CSV file. Note that the following URI intentionally doesn't include the
Also on the page for your dataset, click Continue to begin importing your data.
The import process can take a while to complete, depending on the number and length of the videos that you've provided.
Once the import process has completed, you can browse the list of videos in the dataset by clicking the Videos tab for the dataset:
To view any errors that occurred during the import process, switch to the Import tab and check the error message.
Train your model
- After your dataset has been created and processed, click the Train tab to start model training.
- Click Start Training to continue.
- In the Train new model pane, select a name for your model and click Start Training.
Training is initiated for your model. For this dataset, the training process can take up to 1 hour. The service emails you once training has completed or if any errors occur.
Once training is complete, the service automatically deploys your model.
You can click the Evaluate tab to get more details on F1, Precision, and Recall scores.
Classify a video
To make a prediction using your custom model—that is, to classify a video—do the following:
- On the Test & Use tab for the model, do the following:
- Under Test your model, type 'gs://automl-video-demo-data/hmdb_split1_test_gs_predict.csv'.
Also under Test your model, select a directory within your Cloud Storage bucket to receive the annotation results.
You may actually want to create a specific 'results' folder in your Cloud Storage bucket to hold the annotation results. By doing so, you can more easily access older predictions by loading the
video_classification.csvfile contained in the results directory.
- Click Get Predictions.
The process for getting predictions can take some time, depending on the number of videos that you want annotated.
When the process has completed, the results appear on the page for the model under Recent Predictions. To view the results, do the following:
- Under Recent Predictions in the Predictions column, click View for the prediction you want to look at.
- Under Video, select the name of the video you want to see the results for.
View the results
In the results for your video annotation, AutoML Video provides three types of information:
- Labels for the video. You can find this information under the Segment tab below the video on the results page.
- Labels for shots within the video. You can find this information under the Shot tab below the video on the results page.
- Labels for each 1-second interval within the video. You can find this information under the 1 Second Interval tab below the video on the results page.
If you want to see more labels, you can change the threshold score when you request a prediction. AutoML Video only displays the labels that are above the specified threshold.
If the prediction fails, the results in the list show a red icon on the Recent Predictions list.
If only one video in the prediction attempt failed, the prediction shows green in the Recent Predictions list. On the results page for that prediction, you can the results for the videos that AutoML Video annotated.