MyView is a system for the capture, processing, and playback of multiple streams of context aware video data. Three components of this system are being developed in parallel: a communication and processing architecture to support multi-camera capture and both online and offline processing services, human tracking and identification as an example service, and an interface for browsing in the resulting context aware video space. MyView is currently in the prototype stage and does simple human tracking in a lab environment. It is intended for initial deployment at the 2010 Olympics in Vancouver, where it will be used to record hockey games.

Communication and Processing Architecture

The MyView system is intended to be used with anywhere from a few to hundreds of cameras. To facilitate interoperability between different types of cameras and processing units, devices on the network are required to advertise the services that they offer. Clients can then connect to a specific device like a processing unit, and register to receive information such as tracking data, or an image stream. The communication and services layer is built with Python, which allows for fast and flexible prototyping of high level services that interface with native application code.

Currently all of the image processing is done online, but in the future there will be an offline processing component as well. The offline processing will utilize available resources to refine the accuracy of metadata gathered from the video streams. This process will be non-realtime and employ better quality versions of the computer vision algorithms used during online processing. Offline processing will also allow for better correlation of data from different camera views, which is difficult to do within the constraints of real-time video processing.

Human Tracking and Identification

The prototype MyView system offers a human tracking service. Using the service in capture mode allows for real-time recording of multi-camera image streams, which include metadata giving the identification and location of the people in each stream. The data can later be reviewed using the MyView Video Browser. In the first prototype system users were required to have a pulse coded IR marker on them so that they could be uniquely identified. The marker only needed to be active to initialize the system, and could be turned off once the user was identified.

The current prototype has moved away from using active markers for identification. Instead face detection is used. Users can initialize their face into a face database and then when the system is run they will be tracked and identified appropriately. Face detection is done in the background, and whenever a match is found a full body colour histogram of the user is built. The histogram is used for per frame human tracking, and is updated as often as possible.

The MyView Video Browser

We are developing a video browser for the exploration and viewing of an augmented video space using a mobile device. The augmented video space includes video and metadata captured by the MyView system, as well as additional content based on the MyView metadata.

In our first phases of development we have been exploring new models of video space structure and corresponding interfaces for the following types of navigation in the video space:

Spatial: with sources from multiple cameras, a user may change viewing position, angle, and zoom.

Personalized: a user may browse and view video according to preferences and interests, supported by automated video clip selecting functions.

Content-based: new object and event recognition as well as video annotation and hyper-linking allow for content-based viewing and browsing of complex video spaces.

Contributors

Active:

Dr. Gregor Miller, ECE, UBC
Dr. Sidney Fels, ECE and MAGIC, UBC
Dr. Jim Little, CS, UBC
Dr. David Lowe, CS, UBC
Michael Ilich, ECE, UBC
Kenji Okuma, CS, UBC
Ankur Gupta, CS, UBC
Zoltan Foley-Fisher, ECE, UBC
Robin Roy, ECE, UBC

Alumni:

Amir Afrah, ECE, UBC
Dr. Matthias Finke, MAGIC, UBC
Morgan Hibbert, ECE, UBC
Ryleigh Kostash, ECE, UBC
Clement Leung, ECE, UBC
Thomas Bauer
Wesley Chan, CS, UBC
Donovan Parks, ECE, UBC
Samir Gupta, ECE, UBC
Kiky Tangerine, ECE, UBC
Evelyn Tsai, ECE, UBC
Onn Tai Yong, ECE, UBC
Amy Wei You, ECE, UBC
Dr. Sung-Bae Cho, Yonsei University, Korea
Will Motz, ECE, UBC
Chris Eagleston, ECE, UBC
Walker Eagleston, ECE, UBC
Lan Wu, CS, UBC
Meghan Deutscher, ECE, UBC
Tricia Pang, ECE, UBC
Justine Lu, ECE, UBC
Erik Kremers, Technical University of Eindhoven
Hao Jiang, Boston College
Jing Chen, Hunan University, China
Troy Therrien, ECE, UBC
Dr. Chris Zhang, ECE, UBC
Changsong Shen, ECE, UBC

Affiliates:

Steve Oldridge, ECE, UBC
Dr. Rodger Lea, MAGIC, UBC

Funding

We gratefully acknowledge our funding support from:

• Bell University Labs (BUL)
• Natural Sciences and Engineering Research Council (NSERC)

Contact Information