Counting Discrete Actions in Videos

With this experiment, customers can count how many times a specific action occurs in one or more video clips. Customers bring a video clip with one cycle of the action, as well as (unlabeled) clips where this action (may) occur. For each clip they receive a count of the number of times the action occurs.

Apply for access Private documentation

Intended use

Inputs and outputs:

  • Users provide:
    • Training videos in a folder that contain one period of the same repetitive action
    • Testing videos containing the same action being performed repeatedly
  • Users receive:
    • JSON file with counts for all frames in testing videos in the folder

Industries and functions:

Use-case may include but are not limited to following industries: sports/exercise analytics, retail, entertainment. A typical use-case would be to count the number of times (and when) a particular action occurs like how many jumping jacks someone did in a video, how many times a shelf on an aisle had to be restocked, etc.

Technical challenges:

It is quite difficult to label many videos with periodic motion to count the number of times an action has happened. Our approach to counting activities does not require explicit labeling but can be used to count with some examples of what one period of that motion looks like.

As part of the application to participate in this experiment, we will ask you about your use case, data types, and/or other relevant questions to ensure that the experiment is a good fit for you.

What data do I need?


  • Training videos that contain one period of an action.
  • Testing videos that contain many periods of the same action.

Data specifications:

  • All videos should be of the same action or at least related to each other.
  • Training video lengths (with the single discrete action) should not exceed 2 minutes. Testing videos (where actions are counted) can be longer.

Meta-data specifications:

  • Video format: MP4

What skills do I need?

As with all AI Workshop experiments, successful users are likely to be savvy with core AI concepts and skills in order to both deploy the experiment technology and interact with our AI researchers and engineers.

In particular, users of this experiment should:

  • Be able to edit videos and provide data as specified
  • Be able to use the output labels to solve a meaningful business challenge