Manage media assets and video clips

The Vision Warehouse API allows you to manage Vision Warehouse resources using the command line.

A connected warehouse (corpus) in a data-ingesting deployed app has one or more media objects (for example, video resources). These media objects (asset resources) contain metadata and resource annotations. Use the following commands to manage these media objects.

List video assets in a warehouse

REST

To list assets in a corpus, send a GET request by using the projects.locations.corpora.assets.list method.

Before using any of the request data, make the following replacements:

PROJECT_NUMBER: Your Google Cloud project number.
LOCATION_ID: The region where you are using Vertex AI Vision. For example: us-central1, europe-west2, or asia-south1 (not an exhaustive list).
CORPUS_ID: The ID of your target corpus.

HTTP method and URL:

GET https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets

To send your request, choose one of these options:

curl

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login, or by using Cloud Shell, which automatically logs you into the gcloud CLI. You can check the currently active account by running gcloud auth list.

Execute the following command:

curl -X GET \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets"

PowerShell

Execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method GET `
    -Headers $headers `
    -Uri "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "assets": [
    {
      "name": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID1",
      "ttl": "86400s"
    },
    {
      "name": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID2",
      "ttl": "86400s"
    }
  ]
}

Get a video asset

REST

To get details about an asset, send a GET request using the projects.locations.corpora.assets.get method.

Before using any of the request data, make the following replacements:

PROJECT_NUMBER: Your Google Cloud project number.
LOCATION_ID: The region where you are using Vertex AI Vision. For example: us-central1, europe-west2, or asia-south1 (not an exhaustive list).
CORPUS_ID: The ID of your target corpus.
ASSET_ID: The ID of your target asset.

HTTP method and URL:

GET https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID

To send your request, choose one of these options:

curl

Execute the following command:

curl -X GET \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID"

PowerShell

Execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method GET `
    -Headers $headers `
    -Uri "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "name": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID",
  "ttl": "86400s"
}

Get a video clip asset using temporal partitions

After you have created a warehouse and stored video asset resources in the warehouse you can get specific video clips. You can send a ClipAssetRequest to get media data over a time range you define.

When sending a ClipAssetRequest you must define the following:

temporal_partition -The date range for the desired clips.

The following example considers a warehouse containing an asset with security camera footage for early September 2021 which has not exceeded its time-to-live (TTL).

To get a clip of the period ranging from 9/1/2021 2:51:59 GMT to 9/1/2021 2:52:30 GMT you first need to convert the start and end time to google.protobuf.Timestamp, which is expressed in duration since Unix epoch. After you have converted start and finish times, you can send a request similar to the following:

REST

To get details about a video clip, send a GET request by using the projects.locations.corpora.assets.clip method.

Before using any of the request data, make the following replacements:

PROJECT_NUMBER: Your Google Cloud project number.
LOCATION_ID: The region where you are using Vertex AI Vision. For example: us-central1, europe-west2, or asia-south1 (not an exhaustive list).
CORPUS_ID: The ID of your target corpus.
ASSET_ID: The ID of your target asset.
START_TIME: Start time of the partition, expressed in duration since Unix epoch (for example, 1630464719).
END_TIME: End time of the partition, expressed in duration since Unix epoch (for example, 1630464750).

HTTP method and URL:

POST https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID:clip

Request JSON body:

{
  "temporal_partition": {
    "start_time": {
      "seconds": START_TIME
    },
    "end_time": {
      "seconds": END_TIME
    }
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "Content-Type: application/json; charset=utf-8" \
    -d @request.json \
    "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID:clip"

PowerShell

Save the request body in a file named request.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID:clip" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "time_indexed_uris" : {
    "temporal_partition" : {
      "start_time" : {
        "seconds": 1630464719
      },
      "end_time" : {
        "seconds": 1630464729
      }
    },
    "uri": "[...]"
  }
}

A time_indexed_uris includes a URI with a link to the clip retrieved and a temporal_partition detailing the start time and end time of the clip. You can access the link provided in the uri field for up to 12 hours.

The clip returned will only display a short clip of the first content available after start_time, but before end_time. This content may extend beyond the temporal bounds in the request. For example, in the sample request the clip returned only covers a 10 second clip that occurs after 1630464719, even though there may still be more content that occurs after 1630464729, and before the end_time (1630464750).

The clip returned can not be the exact length you requested. This behavior is expected and can be due to multiple reasons:

The time range requested exceeded the data size limit of 100MB.
The time range requested contains data which has exceeded its time-to-live (TTL).
The time range requested has no data.

Get video clip assets using HTTP Live Streaming

After you have created a warehouse and ingested video data (warehouse assets) in the warehouse, you can request a URI to an HTTP Live Streaming (HLS) video stream manifest for a series of specific segments of time. You can send a GenerateHlsUriRequest to get media data over a series of time ranges you define.

When you send a GenerateHlsUriRequest you must define the following:

temporal_partitions - A collection of date ranges for the desired content you want to stream. The total duration should be less than 24h.

The following example uses a warehouse asset with security camera footage for September 2021 which has not exceeded its time-to-live (TTL). The sample request body specifies two video time periods:

9/1/2021 2:51:59 GMT to 9/1/2021 2:52:30 GMT
9/1/2021 4:59:00 GMT to 9/1/2021 5:05:30 GMT

You first need to convert the start and end times to a google.protobuf.Timestamp format, which is expressed in duration since Unix epoch. After you have converted these times to Unix times, you can send a request similar to the following:

REST & CMD LINE

Before using any of the request data, make the following replacements:

PROJECT_NUMBER: Your Google Cloud project number.
LOCATION_ID: The region where you are using Vertex AI Vision. For example: us-central1, europe-west2, or asia-south1 (not an exhaustive list).
CORPUS_ID: The ID of your target corpus.
ASSET_ID: The ID of your target asset.
START_TIME: Start time of the partition, expressed in duration since Unix epoch (for example, 1630464719).
END_TIME: End time of the partition, expressed in duration since Unix epoch (for example, 1630464750).

HTTP method and URL:

POST https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID:generateHlsUri

Request JSON body:

{
  "temporal_partitions": {
    "start_time": {
      "seconds": 1630464719
    },
    "end_time": {
      "seconds": 1630464750
    }
  },
  "temporal_partitions": {
    "start_time": {
      "seconds": 1630472340
    },
    "end_time": {
      "seconds": 1630472730
    }
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "Content-Type: application/json; charset=utf-8" \
    -d @request.json \
    "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID:generateHlsUri"

PowerShell

Save the request body in a file named request.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID:generateHlsUri" | Select-Object -Expand Content

The uri in the response has a link to download the HLS manifest corresponding to the requested times and each temporal_partitions details the start and end time of the clips included in the stream. The link provided in the uri field can be accessed for up to 12 hours.

A response may include multiple temporal_partitions objects. Each temporal_partitions object represents an individual clip within the requested time range. Clips are broken up if the source media does not contain data for a time within the requested range, or if the request includes non-consecutive date ranges. In this example multiple temporal_partitions are returned because the request asked for data at two different times on 9/1/2021.

The stream returned may not have the exact length you requested. This is expected and can be due to the following reasons:

The time range requested contains data which has exceeded its time-to-live
The time range requested has no data

Response:

{
  "uri": "[...]",
  "temporal_partitions": {
    "start_time": {
      "seconds": 1630464719
    },
    "end_time": {
      "seconds": 1630464750
    }
  },
  "temporal_partitions": {
    "start_time": {
      "seconds": 1630472340
    },
    "end_time": {
      "seconds": 1630472730
    }
  }
}

Delete media assets

The following sample deletes a warehouse asset, making any annotations related to it inaccessible and removed from search results.

REST & CMD LINE

The following code sample deletes assets in a warehouse using the projects.locations.corpora.assets.delete method.

Before using any of the request data, make the following replacements:

PROJECT_NUMBER: Your Google Cloud project number.
LOCATION_ID: The region where you are using Vertex AI Vision. For example: us-central1, europe-west2, or asia-south1 (not an exhaustive list).
CORPUS_ID: The ID of your target corpus.
ASSET_ID: The ID of your target asset.

HTTP method and URL:

DELETE https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID

To send your request, choose one of these options:

curl

Execute the following command:

curl -X DELETE \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID"

PowerShell

Execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method DELETE `
    -Headers $headers `
    -Uri "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID" | Select-Object -Expand Content

The response of this call returns a long-running operation ID.

{
  "name": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID/operations/OPERATION_ID"
}

Get the operation status

To get the status of the long-running operation, call the projects.locations.operations.get method.

Before using any of the request data, make the following replacements:

PROJECT_NUMBER: Your Google Cloud project number.
LOCATION_ID: The region where you are using Vertex AI Vision. For example: us-central1, europe-west2, or asia-south1 (not an exhaustive list).
OPERATION_ID: The operation ID that is returned when you start a long-running operation. For example:
- projects/123456/locations/us-central1/[...]/operations/OPERATION_ID

HTTP method and URL:

GET https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/warehouseoperations/OPERATION_ID

To send your request, choose one of these options:

curl

Execute the following command:

curl -X GET \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/warehouseoperations/OPERATION_ID"

PowerShell

Execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method GET `
    -Headers $headers `
    -Uri "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/warehouseoperations/OPERATION_ID" | Select-Object -Expand Content

The response is empty if there are no errors. At this point the asset doesn't appear if you call GetAsset or ListAssets.