Detecting people

Stay organized with collections Save and categorize content based on your preferences.

Video Intelligence can detect the presence of humans in a video file and track the bounding box of individual people across the video or video segment.

Person detection

To detect people in a video, send a video annotation request to Video Intelligence, and include the PERSON_DETECTION flag. The annotation request returns details about each person detected in the video, including video segment locations where the person is detected. The annotation request can also provide bounding boxes too.

Additionally, person detection can detect the location of specific body parts as "landmarks," such as nose, left_shoulder, or right_shoulder. Furthermore, person detection can detect other characteristics including clothing color, and clothing type. Specific characteristics that person detection can recognize are provided in the following table.

Characteristic or landmark Possible values
Body parts nose, left_eye, right_eye, left_ear, right_ear, left_shoulder, right_shoulder, left_elbow, right_elbow, left_wrist, right_wrist, left_hip, right_hip, left_knee, right_knee, left_ankle, right_ankle
Upper clothing color UpperCloth:Black, UpperCloth:Blue, UpperCloth:Brown, UpperCloth:Gray, UpperCloth:Green, UpperCloth:MultiColor, UpperCloth:Orange, UpperCloth:Purple, UpperCloth:Red, UpperCloth:White, UpperCloth:Yellow
Upper clothing type UpperCloth:Coat, UpperCloth:Dress, UpperCloth:Jacket, UpperCloth:Shirt, UpperCloth:Suit, UpperCloth:Sweater, UpperCloth:T-Shirt, UpperCloth:TankTop
Upper clothing pattern UpperCloth:Floral, UpperCloth:Graphics, UpperCloth:Plaid, UpperCloth:Plain, UpperCloth:Spotted, UpperCloth:Striped
Sleeve length UpperCloth:LongSleeve, UpperCloth:ShortSleeve, UpperCloth:NoSleeve
Lower clothing type LowerCloth:LongPants, LowerCloth:ShortPants, LowerCloth:LongSkirt, LowerCloth:ShortSkirt

Check out the Video Intelligence API visualizer to see this feature in action.

For an example, see Person detection.