The Video Intelligence API can identify entities shown in video footage using the LABEL_DETECTION feature. This feature identifies objects, locations, activities, animal species, products, and more.
The analysis can be compartmentalized as follows:
- Frame level:
Entities are identified and labeled within each frame (with one frame per second sampling). - Shot level:
Shots are automatically detected within every segment (or video). Entities are then identified and labeled within each shot. - Segment level:
User-selected segments of a video can be specified for analysis by stipulating beginning and ending time offsets for the purposes of annotation (see VideoSegment). Entities are then identified and labeled within each segment. If no segments are specified, the whole video is treated as one segment.
Annotate a local file
Here is an example of performing video analysis for labels on a local file.
Looking for something more in-depth? Check out our detailed Python tutorial.
REST
Send the process request
The following shows how to send a POST
request to the
videos:annotate
method. You can configure the
LabelDetectionMode
to shot-level and/or frame-level annotations. We recommend using
SHOT_AND_FRAME_MODE
. The example uses the access token for
a service account set up for the project using the Google Cloud CLI. For
instructions on installing the Google Cloud CLI, setting up a project with a service
account, and obtaining an access token, see the
Video Intelligence quickstart.
Before using any of the request data, make the following replacements:
- BASE64_ENCODED_CONTENT: your video as base64 encoded data. See the instructions on how to convert your data to base64.
- PROJECT_NUMBER: The numeric identifier for your Google Cloud project
HTTP method and URL:
POST https://videointelligence.googleapis.com/v1/videos:annotate
Request JSON body:
{ "inputContent": "BASE64_ENCODED_CONTENT", "features": ["LABEL_DETECTION"], }
To send your request, expand one of these options:
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_NUMBER/locations/LOCATION_ID/operations/OPERATION_ID" }
If the request is successful, Video Intelligence returns the name of your operation.
Get the results
To get the results of your request, you must send a GET
request to
the projects.locations.operations
resource. The following shows how to send such a request.
Before using any of the request data, make the following replacements:
- OPERATION_NAME: the name of the operation as
returned by the Video Intelligence API. The operation name has the format
projects/PROJECT_NUMBER/locations/LOCATION_ID/operations/OPERATION_ID
- PROJECT_NUMBER: The numeric identifier for your Google Cloud project
HTTP method and URL:
GET https://videointelligence.googleapis.com/v1/OPERATION_NAME
To send your request, expand one of these options:
You should receive a JSON response similar to the following:
Go
Java
Node.js
Python
For more information on installing and using the Video Intelligence API Client Library for Python, refer to Video Intelligence API Client Libraries.Additional languages
C#: Please follow the C# setup instructions on the client libraries page and then visit the Video Intelligence reference documentation for .NET.
PHP: Please follow the PHP setup instructions on the client libraries page and then visit the Video Intelligence reference documentation for PHP.
Ruby: Please follow the Ruby setup instructions on the client libraries page and then visit the Video Intelligence reference documentation for Ruby.
Annotate a file on Cloud Storage
Here is an example of performing video analysis for labels on a file located in Cloud Storage.
REST
For more information on installing and using the Video Intelligence API Client Library for Python, refer to Video Intelligence API Client Libraries.Send the process request
The following shows how to send a POST
request to the
annotate
method. The example uses the access token for
a service account set up for the project using the Google Cloud CLI. For
instructions on installing the Google Cloud CLI, setting up a project with a service
account, and obtaining an access token, see the
Video Intelligence quickstart.
Before using any of the request data, make the following replacements:
- INPUT_URI: a Cloud Storage bucket that contains
the file you want to annotate, including the file name. Must
start with
gs://
. - PROJECT_NUMBER: The numeric identifier for your Google Cloud project
HTTP method and URL:
POST https://videointelligence.googleapis.com/v1/videos:annotate
Request JSON body:
{ "inputUri": "INPUT_URI", "features": ["LABEL_DETECTION"], }
To send your request, expand one of these options:
You should receive a JSON response similar to the following:
{ "name": "projects/PROJECT_NUMBER/locations/LOCATION_ID/operations/OPERATION_ID" }
If the request is successful, the Video Intelligence returns the name of your operation.
Get the results
To get the results of your request, you must send a GET
request to
the projects.locations.operations
resource. The following shows how to send such a request.
Before using any of the request data, make the following replacements:
- OPERATION_NAME: the name of the operation as
returned by the Video Intelligence API. The operation name has the format
projects/PROJECT_NUMBER/locations/LOCATION_ID/operations/OPERATION_ID
- PROJECT_NUMBER: The numeric identifier for your Google Cloud project
HTTP method and URL:
GET https://videointelligence.googleapis.com/v1/OPERATION_NAME
To send your request, expand one of these options:
You should receive a JSON response similar to the following:
Download annotation results
Copy the annotation from the source to the destination bucket: (see Copy files and objects)
gcloud storage cp gcs_uri gs://my-bucket
Note: If the output gcs uri is provided by the user, then the annotation is stored in that gcs uri.
Go
Java
Node.js
Python
Additional languages
C#: Please follow the C# setup instructions on the client libraries page and then visit the Video Intelligence reference documentation for .NET.
PHP: Please follow the PHP setup instructions on the client libraries page and then visit the Video Intelligence reference documentation for PHP.
Ruby: Please follow the Ruby setup instructions on the client libraries page and then visit the Video Intelligence reference documentation for Ruby.