L'API Video Intelligence peut identifier des entités affichées dans des séquences vidéo à l'aide de la fonctionnalité LABEL_DETECTION. Cette fonctionnalité identifie les objets, les lieux, les activités, les espèces animales, les produits, etc.
L'analyse peut être compartimentée comme suit :
- Niveau du cadre :
les entités sont identifiées et étiquetées dans chaque image (avec un échantillonnage d'une image par seconde). - Niveau de prise de vue :
Des prises de vue sont automatiquement détectées dans chaque segment (ou vidéo). Les entités sont alors identifiées et libellées dans chaque plan. - Niveau du segement :
Les segments sélectionnés par l'utilisateur peuvent être spécifiés pour l'analyse en parcourant les horodatages de début et de fin dans le cadre de l'annotation (voir VideoSegment). Les entités sont ensuite identifiées et libellées dans chaque segment. Si aucun segment n'est spécifié, l'intégralité de la vidéo est traitée comme un segment.
Annoter un fichier local
Voici un exemple d'analyse de thèmes au sein d'un fichier vidéo local.
Vous souhaitez obtenir une analyse plus approfondie ? Consultez le tutoriel détaillé en Python.
API REST et ligne de commande
Envoyer la requête de processus
Vous trouverez ci-dessous la procédure à suivre pour envoyer une requête POST à la méthode videos:annotate. Vous pouvez configurer le LabelDetectionMode sur annotations au niveau du plan et/ou du cadre. Nous vous recommandons d'utiliser SHOT_AND_FRAME_MODE. Cet exemple fait intervenir le jeton d'accès associé à un compte de service configuré pour le projet à l'aide du SDK Cloud. Pour obtenir des instructions sur l'installation du SDK Cloud, la configuration d'un projet avec un compte de service et l'obtention d'un jeton d'accès, consultez le Guide de démarrage rapide de Video Intelligence.
Avant d'utiliser les données de requête ci-dessous, effectuez les remplacements suivants :
- base64-encoded-content : votre vidéo sous forme de données encodées en base64 Consultez les instructions concernant la conversion des données en base64.
Méthode HTTP et URL :
POST https://videointelligence.googleapis.com/v1/videos:annotate
Corps JSON de la requête :
{
"inputContent": "base64-encoded-content",
"features": ["LABEL_DETECTION"],
}
Pour envoyer votre requête, développez l'une des options suivantes :
curl (Linux, macOS ou Cloud Shell)
.Enregistrez le corps de la requête dans un fichier nommé request.json, puis exécutez la commande suivante :
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://videointelligence.googleapis.com/v1/videos:annotate
PowerShell (Windows)
.Enregistrez le corps de la requête dans un fichier nommé request.json, puis exécutez la commande suivante :
$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://videointelligence.googleapis.com/v1/videos:annotate" | Select-Object -Expand Content
Vous devriez recevoir une réponse JSON de ce type :
{
"name": "projects/project-number/locations/location-id/operations/operation-id"
}
Si la requête aboutit, Video Intelligence renvoie le nom de votre opération.
Obtenir les résultats
Pour obtenir les résultats de votre requête, vous devez envoyer une requête GET à la ressource projects.locations.operations. Vous trouverez ci-dessous la procédure à suivre pour envoyer une requête de ce type.
Avant d'utiliser les données de requête ci-dessous, effectuez les remplacements suivants :
- operation-name : nom de l'opération tel qu'il est renvoyé par l'API Video Intelligence. Il est au format suivant :
projects/project-number/locations/location-id/operations/operation-id.
Méthode HTTP et URL :
GET https://videointelligence.googleapis.com/v1/operation-name
Pour envoyer votre requête, développez l'une des options suivantes :
curl (Linux, macOS ou Cloud Shell)
Exécutez la commande suivante :
curl -X GET \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
https://videointelligence.googleapis.com/v1/operation-name
PowerShell (Windows)
Exécutez la commande suivante :
$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method GET `
-Headers $headers `
-Uri "https://videointelligence.googleapis.com/v1/operation-name" | Select-Object -Expand Content
Vous devriez recevoir une réponse JSON de ce type :
Réponse
{
"name": "projects/project-number/locations/location-id/operations/operation-id",
"metadata": {
"@type": "type.googleapis.com/google.cloud.videointelligence.v1.AnnotateVideoProgress",
"annotationProgress": [
{
"progressPercent": 100,
"startTime": "2019-03-12T19:36:09.110351Z",
"updateTime": "2019-03-12T19:36:17.519069Z"
}
]
},
"done": true,
"response": {
"@type": "type.googleapis.com/google.cloud.videointelligence.v1.AnnotateVideoResponse",
"annotationResults": [
{
"segmentLabelAnnotations": [
{
"entity": {
"entityId": "/m/01prls",
"description": "land vehicle",
"languageCode": "en-US"
},
"categoryEntities": [
{
"entityId": "/m/07yv9",
"description": "vehicle",
"languageCode": "en-US"
}
],
"segments": [
{
"segment": {
"startTimeOffset": "0s",
"endTimeOffset": "38.757872s"
},
"confidence": 0.6614419
}
]
},
{
"entity": {
"entityId": "/m/039jbq",
"description": "urban area",
"languageCode": "en-US"
},
"categoryEntities": [
{
"entityId": "/m/01n32",
"description": "city",
"languageCode": "en-US"
}
],
"segments": [
{
"segment": {
"startTimeOffset": "0s",
"endTimeOffset": "38.757872s"
},
"confidence": 0.92337775
}
]
},
...
C#
public static object AnalyzeLabels(string path)
{
var client = VideoIntelligenceServiceClient.Create();
var request = new AnnotateVideoRequest()
{
InputContent = Google.Protobuf.ByteString.CopyFrom(File.ReadAllBytes(path)),
Features = { Feature.LabelDetection },
};
var op = client.AnnotateVideo(request).PollUntilCompleted();
foreach (var result in op.Result.AnnotationResults)
{
PrintLabels("Video", result.SegmentLabelAnnotations);
PrintLabels("Shot", result.ShotLabelAnnotations);
PrintLabels("Frame", result.FrameLabelAnnotations);
}
return 0;
}
static void PrintLabels(string labelName,
IEnumerable<LabelAnnotation> labelAnnotations)
{
foreach (var annotation in labelAnnotations)
{
Console.WriteLine($"{labelName} label: {annotation.Entity.Description}");
foreach (var entity in annotation.CategoryEntities)
{
Console.WriteLine($"{labelName} label category: {entity.Description}");
}
foreach (var segment in annotation.Segments)
{
Console.Write("Segment location: ");
Console.Write(segment.Segment.StartTimeOffset);
Console.Write(":");
Console.WriteLine(segment.Segment.EndTimeOffset);
System.Console.WriteLine($"Confidence: {segment.Confidence}");
}
}
}Go
func label(w io.Writer, file string) error {
ctx := context.Background()
client, err := video.NewClient(ctx)
if err != nil {
return fmt.Errorf("video.NewClient: %v", err)
}
fileBytes, err := ioutil.ReadFile(file)
if err != nil {
return err
}
op, err := client.AnnotateVideo(ctx, &videopb.AnnotateVideoRequest{
Features: []videopb.Feature{
videopb.Feature_LABEL_DETECTION,
},
InputContent: fileBytes,
})
if err != nil {
return fmt.Errorf("AnnotateVideo: %v", err)
}
resp, err := op.Wait(ctx)
if err != nil {
return fmt.Errorf("Wait: %v", err)
}
printLabels := func(labels []*videopb.LabelAnnotation) {
for _, label := range labels {
fmt.Fprintf(w, "\tDescription: %s\n", label.Entity.Description)
for _, category := range label.CategoryEntities {
fmt.Fprintf(w, "\t\tCategory: %s\n", category.Description)
}
for _, segment := range label.Segments {
start, _ := ptypes.Duration(segment.Segment.StartTimeOffset)
end, _ := ptypes.Duration(segment.Segment.EndTimeOffset)
fmt.Fprintf(w, "\t\tSegment: %s to %s\n", start, end)
}
}
}
// A single video was processed. Get the first result.
result := resp.AnnotationResults[0]
fmt.Fprintln(w, "SegmentLabelAnnotations:")
printLabels(result.SegmentLabelAnnotations)
fmt.Fprintln(w, "ShotLabelAnnotations:")
printLabels(result.ShotLabelAnnotations)
fmt.Fprintln(w, "FrameLabelAnnotations:")
printLabels(result.FrameLabelAnnotations)
return nil
}
Java
// Instantiate a com.google.cloud.videointelligence.v1.VideoIntelligenceServiceClient
try (VideoIntelligenceServiceClient client = VideoIntelligenceServiceClient.create()) {
// Read file and encode into Base64
Path path = Paths.get(filePath);
byte[] data = Files.readAllBytes(path);
AnnotateVideoRequest request = AnnotateVideoRequest.newBuilder()
.setInputContent(ByteString.copyFrom(data))
.addFeatures(Feature.LABEL_DETECTION)
.build();
// Create an operation that will contain the response when the operation completes.
OperationFuture<AnnotateVideoResponse, AnnotateVideoProgress> response =
client.annotateVideoAsync(request);
System.out.println("Waiting for operation to complete...");
for (VideoAnnotationResults results : response.get().getAnnotationResultsList()) {
// process video / segment level label annotations
System.out.println("Locations: ");
for (LabelAnnotation labelAnnotation : results.getSegmentLabelAnnotationsList()) {
System.out
.println("Video label: " + labelAnnotation.getEntity().getDescription());
// categories
for (Entity categoryEntity : labelAnnotation.getCategoryEntitiesList()) {
System.out.println("Video label category: " + categoryEntity.getDescription());
}
// segments
for (LabelSegment segment : labelAnnotation.getSegmentsList()) {
double startTime = segment.getSegment().getStartTimeOffset().getSeconds()
+ segment.getSegment().getStartTimeOffset().getNanos() / 1e9;
double endTime = segment.getSegment().getEndTimeOffset().getSeconds()
+ segment.getSegment().getEndTimeOffset().getNanos() / 1e9;
System.out.printf("Segment location: %.3f:%.2f\n", startTime, endTime);
System.out.println("Confidence: " + segment.getConfidence());
}
}
// process shot label annotations
for (LabelAnnotation labelAnnotation : results.getShotLabelAnnotationsList()) {
System.out
.println("Shot label: " + labelAnnotation.getEntity().getDescription());
// categories
for (Entity categoryEntity : labelAnnotation.getCategoryEntitiesList()) {
System.out.println("Shot label category: " + categoryEntity.getDescription());
}
// segments
for (LabelSegment segment : labelAnnotation.getSegmentsList()) {
double startTime = segment.getSegment().getStartTimeOffset().getSeconds()
+ segment.getSegment().getStartTimeOffset().getNanos() / 1e9;
double endTime = segment.getSegment().getEndTimeOffset().getSeconds()
+ segment.getSegment().getEndTimeOffset().getNanos() / 1e9;
System.out.printf("Segment location: %.3f:%.2f\n", startTime, endTime);
System.out.println("Confidence: " + segment.getConfidence());
}
}
// process frame label annotations
for (LabelAnnotation labelAnnotation : results.getFrameLabelAnnotationsList()) {
System.out
.println("Frame label: " + labelAnnotation.getEntity().getDescription());
// categories
for (Entity categoryEntity : labelAnnotation.getCategoryEntitiesList()) {
System.out.println("Frame label category: " + categoryEntity.getDescription());
}
// segments
for (LabelSegment segment : labelAnnotation.getSegmentsList()) {
double startTime = segment.getSegment().getStartTimeOffset().getSeconds()
+ segment.getSegment().getStartTimeOffset().getNanos() / 1e9;
double endTime = segment.getSegment().getEndTimeOffset().getSeconds()
+ segment.getSegment().getEndTimeOffset().getNanos() / 1e9;
System.out.printf("Segment location: %.3f:%.2f\n", startTime, endTime);
System.out.println("Confidence: " + segment.getConfidence());
}
}
}
}Node.js
// Imports the Google Cloud Video Intelligence library + Node's fs library
const video = require('@google-cloud/video-intelligence').v1;
const fs = require('fs');
const util = require('util');
// Creates a client
const client = new video.VideoIntelligenceServiceClient();
/**
* TODO(developer): Uncomment the following line before running the sample.
*/
// const path = 'Local file to analyze, e.g. ./my-file.mp4';
// Reads a local video file and converts it to base64
const readFile = util.promisify(fs.readFile);
const file = await readFile(path);
const inputContent = file.toString('base64');
// Constructs request
const request = {
inputContent: inputContent,
features: ['LABEL_DETECTION'],
};
// Detects labels in a video
const [operation] = await client.annotateVideo(request);
console.log('Waiting for operation to complete...');
const [operationResult] = await operation.promise();
// Gets annotations for video
const annotations = operationResult.annotationResults[0];
const labels = annotations.segmentLabelAnnotations;
labels.forEach(label => {
console.log(`Label ${label.entity.description} occurs at:`);
label.segments.forEach(segment => {
const time = segment.segment;
if (time.startTimeOffset.seconds === undefined) {
time.startTimeOffset.seconds = 0;
}
if (time.startTimeOffset.nanos === undefined) {
time.startTimeOffset.nanos = 0;
}
if (time.endTimeOffset.seconds === undefined) {
time.endTimeOffset.seconds = 0;
}
if (time.endTimeOffset.nanos === undefined) {
time.endTimeOffset.nanos = 0;
}
console.log(
`\tStart: ${time.startTimeOffset.seconds}` +
`.${(time.startTimeOffset.nanos / 1e6).toFixed(0)}s`
);
console.log(
`\tEnd: ${time.endTimeOffset.seconds}.` +
`${(time.endTimeOffset.nanos / 1e6).toFixed(0)}s`
);
console.log(`\tConfidence: ${segment.confidence}`);
});
});
Python
Pour en savoir plus sur l'installation et l'utilisation de la bibliothèque cliente de l'API Cloud Video Intelligence pour Python, reportez-vous aux bibliothèques clientes de l'API Cloud Video Intelligence."""Detect labels given a file path."""
video_client = videointelligence.VideoIntelligenceServiceClient()
features = [videointelligence.enums.Feature.LABEL_DETECTION]
with io.open(path, "rb") as movie:
input_content = movie.read()
operation = video_client.annotate_video(
features=features, input_content=input_content
)
print("\nProcessing video for label annotations:")
result = operation.result(timeout=90)
print("\nFinished processing.")
# Process video/segment level label annotations
segment_labels = result.annotation_results[0].segment_label_annotations
for i, segment_label in enumerate(segment_labels):
print("Video label description: {}".format(segment_label.entity.description))
for category_entity in segment_label.category_entities:
print(
"\tLabel category description: {}".format(category_entity.description)
)
for i, segment in enumerate(segment_label.segments):
start_time = (
segment.segment.start_time_offset.seconds
+ segment.segment.start_time_offset.nanos / 1e9
)
end_time = (
segment.segment.end_time_offset.seconds
+ segment.segment.end_time_offset.nanos / 1e9
)
positions = "{}s to {}s".format(start_time, end_time)
confidence = segment.confidence
print("\tSegment {}: {}".format(i, positions))
print("\tConfidence: {}".format(confidence))
print("\n")
# Process shot level label annotations
shot_labels = result.annotation_results[0].shot_label_annotations
for i, shot_label in enumerate(shot_labels):
print("Shot label description: {}".format(shot_label.entity.description))
for category_entity in shot_label.category_entities:
print(
"\tLabel category description: {}".format(category_entity.description)
)
for i, shot in enumerate(shot_label.segments):
start_time = (
shot.segment.start_time_offset.seconds
+ shot.segment.start_time_offset.nanos / 1e9
)
end_time = (
shot.segment.end_time_offset.seconds
+ shot.segment.end_time_offset.nanos / 1e9
)
positions = "{}s to {}s".format(start_time, end_time)
confidence = shot.confidence
print("\tSegment {}: {}".format(i, positions))
print("\tConfidence: {}".format(confidence))
print("\n")
# Process frame level label annotations
frame_labels = result.annotation_results[0].frame_label_annotations
for i, frame_label in enumerate(frame_labels):
print("Frame label description: {}".format(frame_label.entity.description))
for category_entity in frame_label.category_entities:
print(
"\tLabel category description: {}".format(category_entity.description)
)
# Each frame_label_annotation has many frames,
# here we print information only about the first frame.
frame = frame_label.frames[0]
time_offset = frame.time_offset.seconds + frame.time_offset.nanos / 1e9
print("\tFirst frame time offset: {}s".format(time_offset))
print("\tFirst frame confidence: {}".format(frame.confidence))
print("\n")PHP
use Google\Cloud\VideoIntelligence\V1\VideoIntelligenceServiceClient;
use Google\Cloud\VideoIntelligence\V1\Feature;
/** Uncomment and populate these variables in your code */
// $path = 'File path to a video file to analyze';
// $options = [];
# Instantiate a client.
$video = new VideoIntelligenceServiceClient();
# Read the local video file
$inputContent = file_get_contents($path);
# Execute a request.
$operation = $video->annotateVideo([
'inputContent' => $inputContent,
'features' => [Feature::LABEL_DETECTION]
]);
# Wait for the request to complete.
$operation->pollUntilComplete($options);
# Print the results.
if ($operation->operationSucceeded()) {
$results = $operation->getResult()->getAnnotationResults()[0];
# Process video/segment level label annotations
foreach ($results->getSegmentLabelAnnotations() as $label) {
printf('Video label description: %s' . PHP_EOL, $label->getEntity()->getDescription());
foreach ($label->getCategoryEntities() as $categoryEntity) {
printf(' Category: %s' . PHP_EOL, $categoryEntity->getDescription());
}
foreach ($label->getSegments() as $segment) {
$start = $segment->getSegment()->getStartTimeOffset();
$end = $segment->getSegment()->getEndTimeOffset();
printf(' Segment: %ss to %ss' . PHP_EOL,
$start->getSeconds() + $start->getNanos()/1000000000.0,
$end->getSeconds() + $end->getNanos()/1000000000.0);
printf(' Confidence: %f' . PHP_EOL, $segment->getConfidence());
}
}
print(PHP_EOL);
# Process shot level label annotations
foreach ($results->getShotLabelAnnotations() as $label) {
printf('Shot label description: %s' . PHP_EOL, $label->getEntity()->getDescription());
foreach ($label->getCategoryEntities() as $categoryEntity) {
printf(' Category: %s' . PHP_EOL, $categoryEntity->getDescription());
}
foreach ($label->getSegments() as $shot) {
$start = $shot->getSegment()->getStartTimeOffset();
$end = $shot->getSegment()->getEndTimeOffset();
printf(' Shot: %ss to %ss' . PHP_EOL,
$start->getSeconds() + $start->getNanos()/1000000000.0,
$end->getSeconds() + $end->getNanos()/1000000000.0);
printf(' Confidence: %f' . PHP_EOL, $shot->getConfidence());
}
}
print(PHP_EOL);
} else {
print_r($operation->getError());
}Ruby
# path = "Path to a local video file: path/to/file.mp4"
require "google/cloud/video_intelligence"
video = Google::Cloud::VideoIntelligence.video_intelligence_service
video_contents = File.binread path
# Register a callback during the method call
operation = video.annotate_video features: [:LABEL_DETECTION], input_content: video_contents
puts "Processing video for label annotations:"
operation.wait_until_done!
raise operation.results.message? if operation.error?
puts "Finished Processing."
labels = operation.results.annotation_results.first.segment_label_annotations
print_labels labelsAnnoter un fichier sur Cloud Storage
Voici un exemple d'analyse de thèmes sur un fichier vidéo situé sur Cloud Storage.
API REST et ligne de commande
Envoyer la requête de processus
Vous trouverez ci-dessous la procédure à suivre pour envoyer une requête POST à la méthode annotate. Cet exemple fait intervenir le jeton d'accès associé à un compte de service configuré pour le projet à l'aide du SDK Cloud. Pour obtenir des instructions sur l'installation du SDK Cloud, la configuration d'un projet avec un compte de service et l'obtention d'un jeton d'accès, consultez le Guide de démarrage rapide de Video Intelligence.
Avant d'utiliser les données de requête ci-dessous, effectuez les remplacements suivants :
- input-uri : bucket Cloud Storage contenant le fichier que vous souhaitez annoter, y compris son nom. Doit commencer par
gs://.
Méthode HTTP et URL :
POST https://videointelligence.googleapis.com/v1/videos:annotate
Corps JSON de la requête :
{
"inputUri": "input-uri",
"features": ["LABEL_DETECTION"],
}
Pour envoyer votre requête, développez l'une des options suivantes :
curl (Linux, macOS ou Cloud Shell)
.Enregistrez le corps de la requête dans un fichier nommé request.json, puis exécutez la commande suivante :
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://videointelligence.googleapis.com/v1/videos:annotate
PowerShell (Windows)
.Enregistrez le corps de la requête dans un fichier nommé request.json, puis exécutez la commande suivante :
$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://videointelligence.googleapis.com/v1/videos:annotate" | Select-Object -Expand Content
Vous devriez recevoir une réponse JSON de ce type :
{
"name": "projects/project-number/locations/location-id/operations/operation-id"
}
Si la requête aboutit, Video Intelligence renvoie le nom de votre opération.
Obtenir les résultats
Pour obtenir les résultats de votre requête, vous devez envoyer une requête GET à la ressource projects.locations.operations. Vous trouverez ci-dessous la procédure à suivre pour envoyer une requête de ce type.
Avant d'utiliser les données de requête ci-dessous, effectuez les remplacements suivants :
- operation-name : nom de l'opération tel qu'il est renvoyé par l'API Video Intelligence. Il est au format suivant :
projects/project-number/locations/location-id/operations/operation-id.
Méthode HTTP et URL :
GET https://videointelligence.googleapis.com/v1/operation-name
Pour envoyer votre requête, développez l'une des options suivantes :
curl (Linux, macOS ou Cloud Shell)
Exécutez la commande suivante :
curl -X GET \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
https://videointelligence.googleapis.com/v1/operation-name
PowerShell (Windows)
Exécutez la commande suivante :
$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method GET `
-Headers $headers `
-Uri "https://videointelligence.googleapis.com/v1/operation-name" | Select-Object -Expand Content
Vous devriez recevoir une réponse JSON de ce type :
Réponse
{
"name": "projects/project-number/locations/location-id/operations/operation-id",
"metadata": {
"@type": "type.googleapis.com/google.cloud.videointelligence.v1.AnnotateVideoProgress",
"annotationProgress": [
{
"inputUri": "input-uri",
"progressPercent": 100,
"startTime": "2019-11-13T19:25:56.206335Z",
"updateTime": "2019-11-13T19:26:16.215615Z"
}
]
},
"done": true,
"response": {
"@type": "type.googleapis.com/google.cloud.videointelligence.v1.AnnotateVideoResponse",
"annotationResults": [
{
"inputUri": "input-uri",
"segmentLabelAnnotations": [
{
"entity": {
"entityId": "/m/01vkl",
"description": "circle",
"languageCode": "en-US"
},
"categoryEntities": [
{
"entityId": "/m/016nqd",
"description": "shape",
"languageCode": "en-US"
}
],
"segments": [
{
"segment": {
"startTimeOffset": "0s",
"endTimeOffset": "16.416666s"
},
"confidence": 0.36535457
}
]
},
...
]
}
]
}
}
C#
public static object AnalyzeLabelsGcs(string uri)
{
var client = VideoIntelligenceServiceClient.Create();
var request = new AnnotateVideoRequest()
{
InputUri = uri,
Features = { Feature.LabelDetection }
};
var op = client.AnnotateVideo(request).PollUntilCompleted();
foreach (var result in op.Result.AnnotationResults)
{
PrintLabels("Video", result.SegmentLabelAnnotations);
PrintLabels("Shot", result.ShotLabelAnnotations);
PrintLabels("Frame", result.FrameLabelAnnotations);
}
return 0;
}
static void PrintLabels(string labelName,
IEnumerable<LabelAnnotation> labelAnnotations)
{
foreach (var annotation in labelAnnotations)
{
Console.WriteLine($"{labelName} label: {annotation.Entity.Description}");
foreach (var entity in annotation.CategoryEntities)
{
Console.WriteLine($"{labelName} label category: {entity.Description}");
}
foreach (var segment in annotation.Segments)
{
Console.Write("Segment location: ");
Console.Write(segment.Segment.StartTimeOffset);
Console.Write(":");
Console.WriteLine(segment.Segment.EndTimeOffset);
System.Console.WriteLine($"Confidence: {segment.Confidence}");
}
}
}Go
func labelURI(w io.Writer, file string) error {
ctx := context.Background()
client, err := video.NewClient(ctx)
if err != nil {
return fmt.Errorf("video.NewClient: %v", err)
}
op, err := client.AnnotateVideo(ctx, &videopb.AnnotateVideoRequest{
Features: []videopb.Feature{
videopb.Feature_LABEL_DETECTION,
},
InputUri: file,
})
if err != nil {
return fmt.Errorf("AnnotateVideo: %v", err)
}
resp, err := op.Wait(ctx)
if err != nil {
return fmt.Errorf("Wait: %v", err)
}
printLabels := func(labels []*videopb.LabelAnnotation) {
for _, label := range labels {
fmt.Fprintf(w, "\tDescription: %s\n", label.Entity.Description)
for _, category := range label.CategoryEntities {
fmt.Fprintf(w, "\t\tCategory: %s\n", category.Description)
}
for _, segment := range label.Segments {
start, _ := ptypes.Duration(segment.Segment.StartTimeOffset)
end, _ := ptypes.Duration(segment.Segment.EndTimeOffset)
fmt.Fprintf(w, "\t\tSegment: %s to %s\n", start, end)
}
}
}
// A single video was processed. Get the first result.
result := resp.AnnotationResults[0]
fmt.Fprintln(w, "SegmentLabelAnnotations:")
printLabels(result.SegmentLabelAnnotations)
fmt.Fprintln(w, "ShotLabelAnnotations:")
printLabels(result.ShotLabelAnnotations)
fmt.Fprintln(w, "FrameLabelAnnotations:")
printLabels(result.FrameLabelAnnotations)
return nil
}
Java
// Instantiate a com.google.cloud.videointelligence.v1.VideoIntelligenceServiceClient
try (VideoIntelligenceServiceClient client = VideoIntelligenceServiceClient.create()) {
// Provide path to file hosted on GCS as "gs://bucket-name/..."
AnnotateVideoRequest request = AnnotateVideoRequest.newBuilder()
.setInputUri(gcsUri)
.addFeatures(Feature.LABEL_DETECTION)
.build();
// Create an operation that will contain the response when the operation completes.
OperationFuture<AnnotateVideoResponse, AnnotateVideoProgress> response =
client.annotateVideoAsync(request);
System.out.println("Waiting for operation to complete...");
for (VideoAnnotationResults results : response.get().getAnnotationResultsList()) {
// process video / segment level label annotations
System.out.println("Locations: ");
for (LabelAnnotation labelAnnotation : results.getSegmentLabelAnnotationsList()) {
System.out
.println("Video label: " + labelAnnotation.getEntity().getDescription());
// categories
for (Entity categoryEntity : labelAnnotation.getCategoryEntitiesList()) {
System.out.println("Video label category: " + categoryEntity.getDescription());
}
// segments
for (LabelSegment segment : labelAnnotation.getSegmentsList()) {
double startTime = segment.getSegment().getStartTimeOffset().getSeconds()
+ segment.getSegment().getStartTimeOffset().getNanos() / 1e9;
double endTime = segment.getSegment().getEndTimeOffset().getSeconds()
+ segment.getSegment().getEndTimeOffset().getNanos() / 1e9;
System.out.printf("Segment location: %.3f:%.3f\n", startTime, endTime);
System.out.println("Confidence: " + segment.getConfidence());
}
}
// process shot label annotations
for (LabelAnnotation labelAnnotation : results.getShotLabelAnnotationsList()) {
System.out
.println("Shot label: " + labelAnnotation.getEntity().getDescription());
// categories
for (Entity categoryEntity : labelAnnotation.getCategoryEntitiesList()) {
System.out.println("Shot label category: " + categoryEntity.getDescription());
}
// segments
for (LabelSegment segment : labelAnnotation.getSegmentsList()) {
double startTime = segment.getSegment().getStartTimeOffset().getSeconds()
+ segment.getSegment().getStartTimeOffset().getNanos() / 1e9;
double endTime = segment.getSegment().getEndTimeOffset().getSeconds()
+ segment.getSegment().getEndTimeOffset().getNanos() / 1e9;
System.out.printf("Segment location: %.3f:%.3f\n", startTime, endTime);
System.out.println("Confidence: " + segment.getConfidence());
}
}
// process frame label annotations
for (LabelAnnotation labelAnnotation : results.getFrameLabelAnnotationsList()) {
System.out
.println("Frame label: " + labelAnnotation.getEntity().getDescription());
// categories
for (Entity categoryEntity : labelAnnotation.getCategoryEntitiesList()) {
System.out.println("Frame label category: " + categoryEntity.getDescription());
}
// segments
for (LabelSegment segment : labelAnnotation.getSegmentsList()) {
double startTime = segment.getSegment().getStartTimeOffset().getSeconds()
+ segment.getSegment().getStartTimeOffset().getNanos() / 1e9;
double endTime = segment.getSegment().getEndTimeOffset().getSeconds()
+ segment.getSegment().getEndTimeOffset().getNanos() / 1e9;
System.out.printf("Segment location: %.3f:%.2f\n", startTime, endTime);
System.out.println("Confidence: " + segment.getConfidence());
}
}
}
}Node.js
// Imports the Google Cloud Video Intelligence library
const video = require('@google-cloud/video-intelligence').v1;
// Creates a client
const client = new video.VideoIntelligenceServiceClient();
/**
* TODO(developer): Uncomment the following line before running the sample.
*/
// const gcsUri = 'GCS URI of the video to analyze, e.g. gs://my-bucket/my-video.mp4';
const request = {
inputUri: gcsUri,
features: ['LABEL_DETECTION'],
};
// Detects labels in a video
const [operation] = await client.annotateVideo(request);
console.log('Waiting for operation to complete...');
const [operationResult] = await operation.promise();
// Gets annotations for video
const annotations = operationResult.annotationResults[0];
const labels = annotations.segmentLabelAnnotations;
labels.forEach(label => {
console.log(`Label ${label.entity.description} occurs at:`);
label.segments.forEach(segment => {
const time = segment.segment;
if (time.startTimeOffset.seconds === undefined) {
time.startTimeOffset.seconds = 0;
}
if (time.startTimeOffset.nanos === undefined) {
time.startTimeOffset.nanos = 0;
}
if (time.endTimeOffset.seconds === undefined) {
time.endTimeOffset.seconds = 0;
}
if (time.endTimeOffset.nanos === undefined) {
time.endTimeOffset.nanos = 0;
}
console.log(
`\tStart: ${time.startTimeOffset.seconds}` +
`.${(time.startTimeOffset.nanos / 1e6).toFixed(0)}s`
);
console.log(
`\tEnd: ${time.endTimeOffset.seconds}.` +
`${(time.endTimeOffset.nanos / 1e6).toFixed(0)}s`
);
console.log(`\tConfidence: ${segment.confidence}`);
});
});Python
""" Detects labels given a GCS path. """
video_client = videointelligence.VideoIntelligenceServiceClient()
features = [videointelligence.enums.Feature.LABEL_DETECTION]
mode = videointelligence.enums.LabelDetectionMode.SHOT_AND_FRAME_MODE
config = videointelligence.types.LabelDetectionConfig(label_detection_mode=mode)
context = videointelligence.types.VideoContext(label_detection_config=config)
operation = video_client.annotate_video(
input_uri=path, features=features, video_context=context
)
print("\nProcessing video for label annotations:")
result = operation.result(timeout=180)
print("\nFinished processing.")
# Process video/segment level label annotations
segment_labels = result.annotation_results[0].segment_label_annotations
for i, segment_label in enumerate(segment_labels):
print("Video label description: {}".format(segment_label.entity.description))
for category_entity in segment_label.category_entities:
print(
"\tLabel category description: {}".format(category_entity.description)
)
for i, segment in enumerate(segment_label.segments):
start_time = (
segment.segment.start_time_offset.seconds
+ segment.segment.start_time_offset.nanos / 1e9
)
end_time = (
segment.segment.end_time_offset.seconds
+ segment.segment.end_time_offset.nanos / 1e9
)
positions = "{}s to {}s".format(start_time, end_time)
confidence = segment.confidence
print("\tSegment {}: {}".format(i, positions))
print("\tConfidence: {}".format(confidence))
print("\n")
# Process shot level label annotations
shot_labels = result.annotation_results[0].shot_label_annotations
for i, shot_label in enumerate(shot_labels):
print("Shot label description: {}".format(shot_label.entity.description))
for category_entity in shot_label.category_entities:
print(
"\tLabel category description: {}".format(category_entity.description)
)
for i, shot in enumerate(shot_label.segments):
start_time = (
shot.segment.start_time_offset.seconds
+ shot.segment.start_time_offset.nanos / 1e9
)
end_time = (
shot.segment.end_time_offset.seconds
+ shot.segment.end_time_offset.nanos / 1e9
)
positions = "{}s to {}s".format(start_time, end_time)
confidence = shot.confidence
print("\tSegment {}: {}".format(i, positions))
print("\tConfidence: {}".format(confidence))
print("\n")
# Process frame level label annotations
frame_labels = result.annotation_results[0].frame_label_annotations
for i, frame_label in enumerate(frame_labels):
print("Frame label description: {}".format(frame_label.entity.description))
for category_entity in frame_label.category_entities:
print(
"\tLabel category description: {}".format(category_entity.description)
)
# Each frame_label_annotation has many frames,
# here we print information only about the first frame.
frame = frame_label.frames[0]
time_offset = frame.time_offset.seconds + frame.time_offset.nanos / 1e9
print("\tFirst frame time offset: {}s".format(time_offset))
print("\tFirst frame confidence: {}".format(frame.confidence))
print("\n")PHP
use Google\Cloud\VideoIntelligence\V1\VideoIntelligenceServiceClient;
use Google\Cloud\VideoIntelligence\V1\Feature;
/** Uncomment and populate these variables in your code */
// $uri = 'The cloud storage object to analyze (gs://your-bucket-name/your-object-name)';
// $options = [];
# Instantiate a client.
$video = new VideoIntelligenceServiceClient();
# Execute a request.
$operation = $video->annotateVideo([
'inputUri' => $uri,
'features' => [Feature::LABEL_DETECTION]
]);
# Wait for the request to complete.
$operation->pollUntilComplete($options);
# Print the results.
if ($operation->operationSucceeded()) {
$results = $operation->getResult()->getAnnotationResults()[0];
# Process video/segment level label annotations
foreach ($results->getSegmentLabelAnnotations() as $label) {
printf('Video label description: %s' . PHP_EOL, $label->getEntity()->getDescription());
foreach ($label->getCategoryEntities() as $categoryEntity) {
printf(' Category: %s' . PHP_EOL, $categoryEntity->getDescription());
}
foreach ($label->getSegments() as $segment) {
$start = $segment->getSegment()->getStartTimeOffset();
$end = $segment->getSegment()->getEndTimeOffset();
printf(' Segment: %ss to %ss' . PHP_EOL,
$start->getSeconds() + $start->getNanos()/1000000000.0,
$end->getSeconds() + $end->getNanos()/1000000000.0);
printf(' Confidence: %f' . PHP_EOL, $segment->getConfidence());
}
}
print(PHP_EOL);
# Process shot level label annotations
foreach ($results->getShotLabelAnnotations() as $label) {
printf('Shot label description: %s' . PHP_EOL, $label->getEntity()->getDescription());
foreach ($label->getCategoryEntities() as $categoryEntity) {
printf(' Category: %s' . PHP_EOL, $categoryEntity->getDescription());
}
foreach ($label->getSegments() as $shot) {
$start = $shot->getSegment()->getStartTimeOffset();
$end = $shot->getSegment()->getEndTimeOffset();
printf(' Shot: %ss to %ss' . PHP_EOL,
$start->getSeconds() + $start->getNanos()/1000000000.0,
$end->getSeconds() + $end->getNanos()/1000000000.0);
printf(' Confidence: %f' . PHP_EOL, $shot->getConfidence());
}
}
print(PHP_EOL);
} else {
print_r($operation->getError());
}Ruby
# path = "Path to a video file on Google Cloud Storage: gs://bucket/video.mp4"
require "google/cloud/video_intelligence"
video = Google::Cloud::VideoIntelligence.video_intelligence_service
# Register a callback during the method call
operation = video.annotate_video features: [:LABEL_DETECTION], input_uri: path
puts "Processing video for label annotations:"
operation.wait_until_done!
raise operation.results.message? if operation.error?
puts "Finished Processing."
labels = operation.results.annotation_results.first.segment_label_annotations
print_labels labels