Video Recognition Systems
(Perceptual Vision Technology)
Technical Website

Description

This site is dedicated to developing Perceptual Vision Technology, also known as Video Recognition Technology.  Several research areas are explored to make this technology work: Computer Vision and Image Processing, Machine Learning and Pattern Recognition, Neurobiology. Three important application domains demand the development of this technology.

Domain 1: Security, Surveillance and Monitoring. 
Domain 2: Visually-enabled computer-human interaction (inc. hand-free user interfaces) 
Domain 3: Visually-enhanced communication and Intelligent video processing (inc. augmentation and annotation of video) 

Below is description of Licensable Technologies and Foundational Research projects developed in our group for each of these  application domains.  The links to detailed descriptions, online demonstration programs, user documentation, software downloads and licensing opportunities are provided.

Current Projects:

1. ACE Surveillance™ : Automated extraction and Annotation of Critical Evidence snapshots from Surveillance Video. A new generation surveillance system capable of intelligent processing and managing the surveillance video data. 
   [ Status: Ready to use and for licensing. Demos and download available ]
   Subprojects & Keywords:
   • Automated abstraction and summarization of surveillance video
   • Automated detection and tracking of objects in video
   • Automated extracted of Critical Evidence Snapshots from surveillance video
   • Zoom-on-the-evidence browsing and search of archived 
     
     
2. Face Memorization & Recognition from/in Video 
   [ Status: Ongoing Foundational Research with licensing opportunities. Demos available ]
   Subprojects & Keywords:
   • Memorization and recognition of faces from low-resolution video using associative memory: accumulation of evidence over time in learning and  recognition 
   • Face tracking for face recognition: automatic "best shot" selection & waiting for the evidence
   • Canonical face model suitable for recognition in video
   • Video-based registration of faces
   • Low-resolution  video-based face dataset for testing and tuning FRiV techniques
   • Performance evaluation of  face recognition in video
   • Face annotation in TV shows and surveillance video, tele-conferences
   • Soft biometrics of face recognition in video
   • On-fly memorization and recognition of computer users
     
     
3. Video Processing and Recognition for Security and Biometrics: Many governmental and private companies that use video data for security purposes provide a variety of highly demanded applications for video processing and recognition. This research is aimed at offering solutions to these applications. 
   Partners: CBSA, RCMP, CPRS, DRDC, DND 
   [ Status: Ongoing Foundational Research with licensing opportunities ]
   Subprojects:
   • Surveillance applications
   • Remote video-based biometrics
   • People tracking and backtracking
   • Multiple-people tracking
   • Multi-camera tracking
   • Memorization-assisted tracking
   • Activity annotation and recognition
     
     
4. Nouse™ Perceptual Vision User Interfaces  
   [ Status: Ongoing application-driven research with licensing opportunities. Several Publications. Demos available ]
   Partner & Licensee: SCO Ottawa Health Center.
   Subprojects & Keywords:
   • User's face detection and tracking robust to head motion
   • User's nose sub-pixel precision tracking
   • Blink Detection
   • Facial Gestures Recognition
   • User recognition
   • Vision-based "virtual" Onscreen Keyboard & Hands-free Typing
   • Hands-free Operating System and computer control
     
     
5. Perceptually-enhanced Video Communication 
   ( for e-conferences, e-learning, e-presence, distant or online learning)
   [ Status: Ongoing application-driven research within the Broadband Visual Communication NRC-IIT Strategic Initiative]
   
   1. Automated tele-operator
      [ Status: Client-requested application-driven. Demos available ]
      Description (Goal and the Means):
      • From a number of on-line or offline video streams capturing an event (such as hockey game), the one showing the best  view of it is  automatically selected. 
      • Replaces manual video operator job (e.g. when broadcasting or taping a hockey game on a single recorder or output stream  using several video cameras mounted around a hockey area).

   2. Video recognition filters for distant learning and visual communication
      [ Status: Ongoing ]
      Subprojects & Keywords: 
      • Visual attention driven auto-zooming and auto-focus switching (inc. hand-raising recognition, instructor recognition)
      • Annotation-assisted visual aids for communication and teaching 

1. Pianist Hands-Fingers Detection and Tracking
   [  Status: Finished. Published. Demos available ]
   Partner & Licensee: U. of Ottawa's Piano Pedagogy Lab.
   Subprojects: 
   • Piano keys detection and rectification
   • Video-midi calibration
   • Hand detection and tracking: foreground, skin segmentation, multiple-object tracking & deformable objects tracking
   • Finger detection and tracking
   • Video annotation of piano playing with and without midi
     
2. StereoTracking Project: 
   Projection Video Toolkit is combined with Nouse convex-shape tracking to yield 3D face tracking. Can also be used for tracking any object in 3D using two webcams. 
   [ Status: Finished. Last updated: 2/2/2002 ]
   Collaborator: Gerhard Roth.

Libraries:

• Video Processing and Recognition Library: We are constantly in search for the techniques that would allow us to process video as efficiently as it is processed in living vision systems. Both biologically-driven techniques: such as those used for efficient visual attention, temporal processing of low-resolution visual data, data accumulation and neuro- associative memorization and recognition, as well as computational techniques: such as those based on statistical analysis of video images and mathematical operations on them, are considered.  The result is a constantly updated  library that is used in all Perceptual Vision projects of our group. 
  [ Status: Ongoing Foundational Research]
  Includes:
  • Video capture from single or multiple cameras or video files & display 
  • Colour analysis of video: robust skin detection using combination of skin models and statistics, general purpose colour tracking and multiple colours tracking
  • Motion analysis of video: statistics-driven background detection and foreground tracking, linear and non-linear first-order and second-order change detection.
  • Intensity analysis of video: convex-shape detection and tracking, sub-pixel precision feature tracking, texture- and edge- based tracking.
  • Face analyzer: eye detection, face rectification, binary facial features extraction,  12-pixel-between-the-eyes canonical face model  suitable for storing from and recognizing in video sequences
  • Multiple-object tracking, multiple-camera tracking, recognition-assisted tracking
  • Tracking-assisted memorization and recognition

• Associative Neural Network Library: This is a free for research and development library based on the doctoral work of Dr. Dmitry Gorodnichy and his colleagues. It is used in memorization and recognition parts of Perceptual Vision projects. 
  [ Status: Finished. Free license available]
  Keywords:
  • Models of Associative Memory
  • Error correction, generalization, artificial intelligence
  • Non-linear processing of data, Synaptic plasticity for data storage, Distributed collective decision making

Technology description:

• The Presentations website contains publications, power-point presentations, links to the organized events and media reviews of the NRC-IIT Perceptual Vision technology.

Contact

Dr. Dmitry Gorodnichy, Former Project Leader
Video Recognition Systems
Computational Video Group
NRC Institute for Information Technology
1200 Montreal Road, Building M-50
Ottawa, ON  K1A 0R6  Canada
Fax: (613) 952-7151

Email: 
Dmitry.Gorodnichy AT nrc-cnrc.gc.ca (dg @ videorecognition.com, dg @ ivim.ca)

Copyright (R) 2007-2008
Last updated: 14-III-2008