People Tracking for Visual Surveillance

1 Introduction

Here at the University of Reading, research is done on People Tracking in camera images for automatic visual surveillance systems. The research is being carried out within the European Framework V project ADVISOR in which we are developing an integrated visual surveillance and behaviour analysis system. The People Tracker used for the project is based on the Leeds People Tracker which was developed by Adam Baumberg.

Contents of this page

1 Introduction

2 Main Results

3 Tracking Algorithm

 3.1 Overview of the 4 Detection / Tracking Modules

 3.2 Details of the Tracking Algorithm

 3.3 Example Image of Tracked People

4 Software Engineering Aspects

5 Special Features of our System

6 Relevant Publications

 6.1 Image Processing Aspects

 6.2 Software Engineering Aspects

 6.3 System Integration and Testing

7 Acknowledgements

8 Source Code of the Reading People Tracker

2 Main Results

The main research results are the following:

• I have introduced a new modular system for tracking people and other objects in video sequences. The main contribution is a powerful and scalable structure which has been achieved by a complete re-design and much refactoring of the original Leeds People Tracker, and the addition of new functionality, resulting in
  • high robustness of the tracker to image noise and occlusion through the use of redundant, different type trackers running in parallel, and the use of multiple tracking hypotheses (see ECCV 2002 paper below).
  • the ability to track multiple objects in an arbitrary number of cameras, writing out tracking results in XML format.
  • scalability and much improved maintainability of the people tracker (see ICSM 2002 paper below).
  • new documentation for maintaining the People Tracker, including a well-defined and documented software maintenance process (see JSME article for a case study on processes).
• Validation of the new tracking system to track people and vehicles (see PETS 2001 paper below for the validation of an earlier version of the tracker).
• Integration of the tracker as a subsystem of ADVISOR, interfacing with other subsystems through Ethernet.
• Examination of whether and to what extend colour filtering methods can help to improve motion detection and local edge search.
• The tracker has been ported from an SGI platform to a PC running GNU/Linux to make economic system integration feasible. Also, the source code now adheres to the ISO/IEC 9899:1999(E) C++ standard, as well as IEEE POSIX 1003.1c-1995 extensions for multi-threading, making it easily portable. While the code is being maintained under GNU/Linux, it also compiles under Windows 2000.

3 Tracking Algorithm

3.1 Overview of the 4 Detection / Tracking Modules

1. A Motion Detector detects moving pixels in the image. It models the background as an image with no people in it. Simply subtracting it pixelwise from of the current video image and thresholding the result yields the binary Motion Image. Regions (bounding boxes) with detected moving blobs are then extracted and written out as the output from this module. Main features: simple background image subtraction image filtering (spatial median filter, dilation) depending on available CPU time temporal inclusion of static objects into the background background modelling using a speed-optimised median filter static regions incorporated into background (multi-layer background).

- =

2. A Region Tracker tracks these moving regions (i.e. bounding boxes) over time. This includes region splitting and merging using predictions from the previous frame. Main features: region splitting and merging using predictions adjust bounding box from Active Shape Tracker results identify static regions for background integration.

3. A Head Detector makes rapid guesses of head positions in all detected moving regions. Main features: works in binary motion image looks for peaks in detected moving regions vertical pixel histogram with low-pass filter optimised for speed not accuracy

4. An Active Shape Tracker uses a deformable model for the 2D outline shape of a walking pedestrian to detect and track people. The initialisation of contour shapes (called profiles) is done from the output by the Region Tracker and the Head Detector. Main features: local edge search for shape fitting initialisation of shapes from Region Tracker, Head Detector and own predictions occlusion reasoning

• interaction between modules to avoid non- or mis-detection
• independent prediction in the two tracking modules, for greater robustness
• multiple hypotheses during tracking to recover from lost or mixed-up tracks
• all modules have camera calibration data available for their use
• through the use of software engineering principles for the software design, it is scalable and extensible (new modules...) as well as highly maintainable and portable.

3.2 Details of the Tracking Algorithm

3.3 Example Image of Tracked People

4 Software Engineering Aspects

The People Tracker was completely re-engineered, yielding a new design. Now, the software is highly maintainable and portable, and a software process for all maintenence work is well defined and documented. Extensibility and scalability were kept in mind while designing the new tracker.

The source code adheres to the ISO/IEC 9899:1999(E) C++ standard, as well as IEEE POSIX 1003.1c-1995 extensions for multi-threading, making it easily portable. While the code is being maintained under GNU/Linux, it also compiles under Windows 2000.

Together with members of our department's Applied Software Engineering group we have examined the software processes in our own work on the People Tracker and the way these have influenced the maintainability of the code (see JSME paper below). The re-engineering phase was more closely examined in a case study (see ICSM 2002 paper below).

5 Special Features of our System

• next-to-market, integrated product
• combination of multiple algorithms for robustness
• redundancy helps offset bad image quality
• scalable, extensible design
• highly maintainable and portable implementation
• other trackers (e.g. W⁴ from UMD) achieve similar tracking performance by different means.

6 Relevant Publications (refereed)

Please note: The files available here are provided for personal use in order to achieve timely dissemination of research results. You are kindly requested to check and observe the copyright of each file before using it. Re-publication of the files provided here requires the explicit permission of the author.

6.1 Image Processing Aspects

• My PhD thesis on the "Design and Implementation of People Tracking Algorithms for Visual Surveillance Applications":
  Nils T Siebel: "Design and Implementation of People Tracking Algorithms for Visual Surveillance Applications". PhD thesis, Department of Computer Science, The University of Reading, Reading, UK, March 2003.
  Available for download as compressed PostScript (14 MB) and PDF (5 MB) for one-sided (simplex) printing as well as compressed PostScript (14 MB) and PDF (5 MB) for two-sided (duplex) printing.
• Conference article on the "Fusion of Multiple Tracking Algorithms for Robust People Tracking":
  Nils T Siebel, Steve Maybank: "Fusion of Multiple Tracking Algorithms for Robust People Tracking". In Proceedings of the 7th European Conference on Computer Vision (ECCV 2002), København, Denmark, May 2002, volume IV, pages 373--387, Springer Verlag. Available for download as compressed PostScript (6 MB) or PDF (4 MB). Also available: poster, scaled to A4 (landscape), as compressed PostScript (2 MB) and PNG image format (1 MB).
• Conference article on "Real-Time Tracking of Pedestrians and Vehicles":
  Nils T Siebel, Steve Maybank: "Real-Time Tracking of Pedestrians and Vehicles". In 2nd IEEE Workshop on Performance Evaluation of Tracking and Surveillance (PETS'2001), Hawaii, December 2001. Available for download as compressed PostScript (10 MB) or PDF (6 MB).
• Book chapter "On the Use of Colour Filtering in an Integrated Real-Time People Tracking System":
  Nils T Siebel, Steve Maybank: "On the Use of Colour Filtering in an Integrated Real-Time People Tracking System". In Paolo Remagnino and Graeme A Jones and Nikos Paragios and Carlo S Regazzoni: Video Based Surveillance Systems: Computer Vision and Distributed Processing, chapter 13, pages 159--166, Kluwer Academic Publishers, 2002, ISBN 0-7923-7632-3.
• Conference article on "The Application of Colour Filtering to Real-Time Person Tracking":
  Nils T Siebel, Steve Maybank: "The Application of Colour Filtering to Real-Time Person Tracking". In Proceedings of the 2nd European Workshop on Advanced Video-Based Surveillance Systems (AVBS'2001), Kingston upon Thames (UK), pages 227--234, September 2001. An updated version of this article is available for download as compressed PostScript (2 MB) or PDF (504 KB).

6.2 Software Engineering Aspects

• Journal article on "Latitudinal and Longitudinal Process Diversity":
  Nils T Siebel, Stephen Cook, Manoranjan Satpathy, Daniel Rodríguez: "Latitudinal and Longitudinal Process Diversity". In Journal of Software Maintenance and Evolution (JSME aka SMR), Volume 15, Issue 1, pages 9--25 , January/February 2003, Online ISSN: 1532-0618, Print ISSN: 1532-060X, DOI: 10.1002/smr.264).
  Available for download as PDF (344 KB).
  External Reference: Zentralblatt review.
• Conference article on the "Maintenance of Object Oriented Systems through Re-engineering: A Case Study":
  Manoranjan Satpathy, Nils T Siebel, Daniel Rodríguez: "Maintenance of Object Oriented Systems through Re-engineering: A Case Study". In Proceedings of the IEEE International Conference on Software Maintenance (ICSM 2002), Montréal, pages 540--549, October 2002, ISBN 0-76951819-2, ISSN 1063-6773.
  Available for download as compressed PostScript (1 MB) or PDF (1.1 MB).

6.3 System Integration and Testing

• Conference article on "The ADVISOR Visual Surveillance System":
  Nils T Siebel, Stephen J Maybank: "The ADVISOR Visual Surveillance System". In Proceedings of the ECCV Workshop on Applications of Computer Vision, Prague, May 2004. To appear.

7 Acknowledgements

• This work was supported by the European Union, grant ADVISOR (IST-1999-11287)
• Thanks to Sergio Velastin (Kingston University) and London Underground Ltd. for providing the original video images.

8 Source Code of the Reading People Tracker

The Reading People Tracker which has been developed within this project is maintained by Nils T Siebel who is now in the Cognitive Systems Group of the Christian-Albrechts-University of Kiel, Germany. Please have a look at the Reading People Tracker's new home page for new releases and more documentation.

Top of this page