Video Intelligence Client Libraries

This page shows how to get started with the Cloud Client Libraries for the Cloud Video Intelligence API. Read more about the client libraries for Cloud APIs, including the older Google APIs Client Libraries, in Client Libraries Explained.

Installing the client library


For more information, see Setting Up a C# Development Environment.
Install-Package -Pre Google.Cloud.VideoIntelligence.V1


go get -u


For more information, see Setting Up a Java Development Environment. If you are using Maven, add the following to your pom.xml file:
If you are using Gradle, add the following to your dependencies:
compile ''
If you are using SBT, add the following to your dependencies:
libraryDependencies += "" % "google-cloud-video-intelligence" % "0.92.0-beta"

If you're using IntelliJ or Eclipse, you can add client libraries to your project using the following IDE plugins:

The plugins provide additional functionality, such as key management for service accounts. Refer to each plugin's documentation for details.


For more information, see Setting Up a Node.js Development Environment.
npm install --save @google-cloud/video-intelligence


composer require google/cloud-videointelligence


For more information, see Setting Up a Python Development Environment.
pip install --upgrade google-cloud-videointelligence


For more information, see Setting Up a Ruby Development Environment.
gem install google-cloud-video_intelligence

Setting up authentication

To run the client library, you must first set up authentication by creating a service account and setting an environment variable. Complete the following steps to set up authentication. For more information, see the GCP authentication documentation .

GCP Console

  1. In the GCP Console, go to the Create service account key page.

    Go to the Create Service Account Key page
  2. From the Service account list, select New service account.
  3. In the Service account name field, enter a name.
  4. From the Role list, select Project > Owner.

    Note: The Role field authorizes your service account to access resources. You can view and change this field later by using the GCP Console. If you are developing a production app, specify more granular permissions than Project > Owner. For more information, see granting roles to service accounts.
  5. Click Create. A JSON file that contains your key downloads to your computer.

Command line

You can run the following commands using the Cloud SDK on your local machine, or in Cloud Shell.

  1. Create the service account. Replace [NAME] with a name for the service account.

    gcloud iam service-accounts create [NAME]
  2. Grant permissions to the service account. Replace [PROJECT_ID] with your project ID.

    gcloud projects add-iam-policy-binding [PROJECT_ID] --member "serviceAccount:[NAME]@[PROJECT_ID]" --role "roles/owner"
    Note: The Role field authorizes your service account to access resources. You can view and change this field later by using GCP Console. If you are developing a production app, specify more granular permissions than Project > Owner. For more information, see granting roles to service accounts.
  3. Generate the key file. Replace [FILE_NAME] with a name for the key file.

    gcloud iam service-accounts keys create [FILE_NAME].json --iam-account [NAME]@[PROJECT_ID]

Provide authentication credentials to your application code by setting the environment variable GOOGLE_APPLICATION_CREDENTIALS. Replace [PATH] with the file path of the JSON file that contains your service account key, and [FILE_NAME] with the filename. This variable only applies to your current shell session, so if you open a new session, set the variable again.

Linux or macOS


For example:

export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/[FILE_NAME].json"


With PowerShell:


For example:


With command prompt:


Using the client library

The following example shows how to use the client library.


using Google.Cloud.VideoIntelligence.V1;
using System;

namespace GoogleCloudSamples.VideoIntelligence
    public class QuickStart
        public static void Main(string[] args)
            var client = VideoIntelligenceServiceClient.Create();
            var request = new AnnotateVideoRequest()
                InputUri = @"gs://cloud-samples-data/video/cat.mp4",
                Features = { Feature.LabelDetection }
            var op = client.AnnotateVideo(request).PollUntilCompleted();
            foreach (var result in op.Result.AnnotationResults)
                foreach (var annotation in result.SegmentLabelAnnotations)
                    Console.WriteLine($"Video label: {annotation.Entity.Description}");
                    foreach (var entity in annotation.CategoryEntities)
                        Console.WriteLine($"Video label category: {entity.Description}");
                    foreach (var segment in annotation.Segments)
                        Console.Write("Segment location: ");
                        System.Console.WriteLine($"Confidence: {segment.Confidence}");


// Sample video_quickstart uses the Google Cloud Video Intelligence API to label a video.
package main

import (


	video ""
	videopb ""

func main() {
	ctx := context.Background()

	// Creates a client.
	client, err := video.NewClient(ctx)
	if err != nil {
		log.Fatalf("Failed to create client: %v", err)

	op, err := client.AnnotateVideo(ctx, &videopb.AnnotateVideoRequest{
		InputUri: "gs://cloud-samples-data/video/cat.mp4",
		Features: []videopb.Feature{
	if err != nil {
		log.Fatalf("Failed to start annotation job: %v", err)

	resp, err := op.Wait(ctx)
	if err != nil {
		log.Fatalf("Failed to annotate: %v", err)

	// Only one video was processed, so get the first result.
	result := resp.GetAnnotationResults()[0]

	for _, annotation := range result.SegmentLabelAnnotations {
		fmt.Printf("Description: %s\n", annotation.Entity.Description)

		for _, category := range annotation.CategoryEntities {
			fmt.Printf("\tCategory: %s\n", category.Description)

		for _, segment := range annotation.Segments {
			start, _ := ptypes.Duration(segment.Segment.StartTimeOffset)
			end, _ := ptypes.Duration(segment.Segment.EndTimeOffset)
			fmt.Printf("\tSegment: %s to %s\n", start, end)
			fmt.Printf("\tConfidence: %v\n", segment.Confidence)


import java.util.List;

public class QuickstartSample {

   * Demonstrates using the video intelligence client to detect labels in a video file.
  public static void main(String[] args) throws Exception {
    // Instantiate a video intelligence client
    try (VideoIntelligenceServiceClient client = VideoIntelligenceServiceClient.create()) {
      // The Google Cloud Storage path to the video to annotate.
      String gcsUri = "gs://demomaker/cat.mp4";

      // Create an operation that will contain the response when the operation completes.
      AnnotateVideoRequest request = AnnotateVideoRequest.newBuilder()

      OperationFuture<AnnotateVideoResponse, AnnotateVideoProgress> response =

      System.out.println("Waiting for operation to complete...");

      List<VideoAnnotationResults> results = response.get().getAnnotationResultsList();
      if (results.isEmpty()) {
        System.out.println("No labels detected in " + gcsUri);
      for (VideoAnnotationResults result : results) {
        // get video segment label annotations
        for (LabelAnnotation annotation : result.getSegmentLabelAnnotationsList()) {
              .println("Video label description : " + annotation.getEntity().getDescription());
          // categories
          for (Entity categoryEntity : annotation.getCategoryEntitiesList()) {
            System.out.println("Label Category description : " + categoryEntity.getDescription());
          // segments
          for (LabelSegment segment : annotation.getSegmentsList()) {
            double startTime = segment.getSegment().getStartTimeOffset().getSeconds()
                + segment.getSegment().getStartTimeOffset().getNanos() / 1e9;
            double endTime = segment.getSegment().getEndTimeOffset().getSeconds()
                + segment.getSegment().getEndTimeOffset().getNanos() / 1e9;
            System.out.printf("Segment location : %.3f:%.3f\n", startTime, endTime);
            System.out.println("Confidence : " + segment.getConfidence());


// Imports the Google Cloud Video Intelligence library
const videoIntelligence = require('@google-cloud/video-intelligence');

// Creates a client
const client = new videoIntelligence.VideoIntelligenceServiceClient();

// The GCS uri of the video to analyze
const gcsUri = 'gs://nodejs-docs-samples-video/quickstart_short.mp4';

// Construct request
const request = {
  inputUri: gcsUri,
  features: ['LABEL_DETECTION'],

// Execute request
const [operation] = await client.annotateVideo(request);

  'Waiting for operation to complete... (this may take a few minutes)'

const [operationResult] = await operation.promise();

// Gets annotations for video
const annotations = operationResult.annotationResults[0];

// Gets labels for video from its annotations
const labels = annotations.segmentLabelAnnotations;
labels.forEach(label => {
  console.log(`Label ${label.entity.description} occurs at:`);
  label.segments.forEach(segment => {
    segment = segment.segment;
    if (segment.startTimeOffset.seconds === undefined) {
      segment.startTimeOffset.seconds = 0;
    if (segment.startTimeOffset.nanos === undefined) {
      segment.startTimeOffset.nanos = 0;
    if (segment.endTimeOffset.seconds === undefined) {
      segment.endTimeOffset.seconds = 0;
    if (segment.endTimeOffset.nanos === undefined) {
      segment.endTimeOffset.nanos = 0;
      `\tStart: ${segment.startTimeOffset.seconds}` +
        `.${(segment.startTimeOffset.nanos / 1e6).toFixed(0)}s`
      `\tEnd: ${segment.endTimeOffset.seconds}.` +
        `${(segment.endTimeOffset.nanos / 1e6).toFixed(0)}s`


use Google\Cloud\VideoIntelligence\V1\VideoIntelligenceServiceClient;
use Google\Cloud\VideoIntelligence\V1\Feature;

# Instantiate a client.
$video = new VideoIntelligenceServiceClient();

# Execute a request.
$options = [
    'inputUri' => 'gs://cloud-samples-data/video/cat.mp4',
    'features' => [Feature::LABEL_DETECTION]
$operation = $video->annotateVideo($options);

# Wait for the request to complete.

# Print the result.
if ($operation->operationSucceeded()) {
    $results = $operation->getResult()->getAnnotationResults()[0];
    # Process video/segment level label annotations
    foreach ($results->getSegmentLabelAnnotations() as $label) {
        printf('Video label description: %s' . PHP_EOL, $label->getEntity()->getDescription());
        foreach ($label->getCategoryEntities() as $categoryEntity) {
            printf('  Category: %s' . PHP_EOL, $categoryEntity->getDescription());
        foreach ($label->getSegments() as $segment) {
            $start = $segment->getSegment()->getStartTimeOffset();
            $end = $segment->getSegment()->getEndTimeOffset();
            printf('  Segment: %ss to %ss' . PHP_EOL,
                $start->getSeconds() + $start->getNanos()/1000000000.0,
                $end->getSeconds() + $end->getNanos()/1000000000.0
            printf('  Confidence: %f' . PHP_EOL, $segment->getConfidence());
} else {


from import videointelligence

video_client = videointelligence.VideoIntelligenceServiceClient()
features = [videointelligence.enums.Feature.LABEL_DETECTION]
operation = video_client.annotate_video(
    'gs://cloud-samples-data/video/cat.mp4', features=features)
print('\nProcessing video for label annotations:')

result = operation.result(timeout=120)
print('\nFinished processing.')

# first result is retrieved because a single video was processed
segment_labels = result.annotation_results[0].segment_label_annotations
for i, segment_label in enumerate(segment_labels):
    print('Video label description: {}'.format(
    for category_entity in segment_label.category_entities:
        print('\tLabel category description: {}'.format(

    for i, segment in enumerate(segment_label.segments):
        start_time = (segment.segment.start_time_offset.seconds +
                      segment.segment.start_time_offset.nanos / 1e9)
        end_time = (segment.segment.end_time_offset.seconds +
                    segment.segment.end_time_offset.nanos / 1e9)
        positions = '{}s to {}s'.format(start_time, end_time)
        confidence = segment.confidence
        print('\tSegment {}: {}'.format(i, positions))
        print('\tConfidence: {}'.format(confidence))


require "google/cloud/video_intelligence"

video_client =
features     = [:LABEL_DETECTION]
path         = "gs://cloud-samples-data/video/cat.mp4"

# Register a callback during the method call
operation = video_client.annotate_video input_uri: path, features: features do |operation|
  raise operation.results.message? if operation.error?
  puts "Finished Processing."

  labels = operation.results.annotation_results.first.segment_label_annotations

  labels.each do |label|
    puts "Label description: #{label.entity.description}"

    label.category_entities.each do |category_entity|
      puts "Label category description: #{category_entity.description}"

    label.segments.each do |segment|
      start_time = (segment.segment.start_time_offset.seconds +
                     segment.segment.start_time_offset.nanos / 1e9)
      end_time =   (segment.segment.end_time_offset.seconds +
                     segment.segment.end_time_offset.nanos / 1e9)

      puts "Segment: #{start_time} to #{end_time}"
      puts "Confidence: #{segment.confidence}"

puts "Processing video for label annotations:"

Additional Resources

Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Video Intelligence API Documentation