16.7.1.2 Evaluation, Surveillance Systems

Chapter Contents (Back)
Evaluation, Surveillance.

LHI Surveillance Dataset,
Annotated surveillance images. Online2008
HTML Version. Dataset, Segmentation. Subset of larger dataset. See also Lotus Hill Institute. BibRef 0800

CLEAR: Classification of Events, Activities and Relationships,
MTPH07
WWW Version. Dataset, Activity Recogniton. BibRef 0700

i-LIDS: Bag and vehicle detection challenge,
Online2007 BibRef 0700 AVSBS07
HTML Version. Dataset, Activity Recogniton. Data used at Advanced Video and Signal Based Surveillance, 2007. BibRef

PETS 2006 Benchmark Data,
Online2006 BibRef 0600 PETS06
HTML Version. Dataset, Activity Recogniton. Data used at International Workshop on Performance Evaluation of Tracking and Surveillance 2006. BibRef

PETS 2001 Benchmark Data,
Online2001 BibRef 0100 PETS01
WWW Version. Dataset, Activity Recogniton. Data used at International Workshop on Performance Evaluation of Tracking and Surveillance 2001. BibRef

OTCBVS Benchmark Dataset Collection,
OTCBVS072007
WWW Version. Dataset, Activity Recogniton. Beyound the Visual Spectrum (IR especially). Data for various OTCBVS workshops. BibRef 0700

Boyd, J.E.[Jeffrey E.], Meloche, J.[Jean],
Evaluation of statistical and multiple-hypothesis tracking for video traffic surveillance,
MVA(13), No. 5-6, 2003, pp. 344-351.
HTML Version. 0304
BibRef

Boyd, J.E., Meloche, J., Vardi, Y.,
Statistical Tracking in Video Traffic Surveillance,
ICCV99(163-168).
IEEE DOI Link BibRef 9900

Oberti, F.[Franco], Stringa, E.[Elena], Vernazza, G.[Gianni],
Performance Evaluation Criterion for Characterizing Video-Surveillance Systems,
RealTimeImg(7), No. 5, October 2001, pp. 457-471.
WWW Version. 0110
BibRef

Oberti, F.[Franco], Teschioni, A.[Andrea], Regazzoni, C.S.[Carlo S.],
ROC Curves for Performance Evaluation of Video Sequences Processing Systems for Surveillance Applications,
ICIP99(II:949-953).
IEEE Abstract. BibRef 9900

Fisher, R.B.[Robert B.],
CAVIAR Test Case Scenarios,
Online BookOctober 2004.
WWW Version. Dataset, Video. From the EC funded CAVIAR project (Context Aware Vision using Image-based Active Recognition). The sequences are labelled (in XML) with both the tracked persons and a semantic description of their activities. 81 video sequences comprising about 90K frames. These sequences include indoor plaza and shopping center observations of individuals and small groups of people walking, browsing, window shopping, fighting, meeting, leaving packages behind, collapsing, entering and exiting shops, etc. BibRef 0410

Optic Flow Data,
Edinburgh2007. Smoothed flow sequences for the Waverly train station scene.
WWW Version. Dataset, Video. Behavior, pedestrian analysis. BibRef 0700

BEHAVE Interactions Test Case Scenarios,
Edinburgh2007. Two views of various scenarios of people acting out various interactions.
WWW Version. Dataset, Video. Behavior, pedestrian analysis. Includes ground truth bounding boxes for much of the data. BibRef 0700

Sigal, L.[Leonid], Balan, A.O.[Alexandru O.], Black, M.J.[Michael J.],
HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion,
IJCV(87), No. 1-2, March 2010, pp. xx-yy.
Springer DOI Link 1001
Dataset, Human Motion. BibRef
Earlier: A1, A3, Only:
HumanEva: Synchronized Video and Motion Capture Dataset for Evaluation of Articulated Human Motion,
BrownTechnical Report CS-06-08, September 2006.
HTML Version. For the dataset:
HTML Version. Calibrated video sequences synchronized with motion capture data. BibRef

Sanderson, C.[Conrad], Bigdeli, A.[Abbas], Shan, T.[Ting], Chen, S.K.[Shao-Kang], Berglund, E.[Erik], Lovell, B.C.[Brian C.],
Intelligent CCTV for Mass Transport Security: Challenges and Opportunities for Video and Face Processing,
ELCVIA(6), No. 3, December 2007, pp. 30-41.
WWW Version. 0801
See also Experimental Analysis of Face Recognition on Still and CCTV Images. BibRef

Norouznezhad, E., Bigdeli, A., Postula, A., Lovell, B.C.,
A high resolution smart camera with GigE Vision extension for surveillance applications,
ICDSC08(1-8).
IEEE DOI Link 0809
BibRef

Lazarevic-McManus, N., Renno, J.R., Makris, D., Jones, G.A.,
An object-based comparative methodology for motion detection based on the F-Measure,
CVIU(111), No. 1, July 2008, pp. 74-85.
WWW Version. 0711
BibRef
Earlier: A1, A2, A4, Only:
Performance evaluation in visual surveillance using the F-measure,
VSSN06(45-52).
WWW Version. 0701
Visual surveillance; Motion detection; Performance evaluation; ROC analysis; F-Measure BibRef

Altun, K.[Kerem], Barshan, B.[Billur], Tuncel, O.[Orkun],
Comparative study on classifying human activities with miniature inertial and magnetic sensors,
PR(43), No. 10, October 2010, pp. 3605-3620.
Elsevier DOI Link
WWW Version. 1007
Inertial sensors; Gyroscope; Accelerometer; Magnetometer; Activity recognition and classification; Feature extraction; Feature reduction; Bayesian decision making; Rule-based algorithm; Decision tree; Least-squares method; k-Nearest neighbor; Dynamic time warping; Support vector machines; Artificial neural networks BibRef

Venetianer, P.L.[Peter L.], Deng, H.L.[Hong-Li],
Performance evaluation of an intelligent video surveillance system: A case study,
CVIU(114), No. 11, November 2010, pp. 1292-1302.
Elsevier DOI Link
WWW Version. 1011
Performance evaluation; Intelligent video surveillance; Embedded vision BibRef

Sasse, M.A.[M. Angela],
Not Seeing the Crime for the Cameras?,
CACM(53), No. 2, February 2010, pp. 22-25.
WWW Version. 1101
Why it is difficult - but essential - to monitor the effectiveness of security technologies. BibRef

Fernández Llorca, D.[David], Parra, I.[Ignacio], Sotelo, M.Á.[Miguel Ángel], and Lacey, G.[Gerard],
A vision-based system for automatic hand washing quality assessment,
MVA(22), No. 2, March 2011, pp. 219-234.
WWW Version. 1103
BibRef

Fernández Llorca, D.[David], Vilarino, F., Zhou, J., Lacey, G.,
A multi-class SVM classifier for automatic hand washing quality assessment,
BMVC07(xx-yy).
PDF Version. 0709
BibRef

Zhou, J.[Jiang], Vilarino, F.[Fernando], Lacey, G.[Gerard], Li, X.C.[Xu-Chun],
Statistical analysis of ground truth in human labeled data,
IMVIP07(211-211).
IEEE DOI Link 0709
Analysis of human analysis of hand washing videos. BibRef


Oh, S.M.[Sang-Min], Hoogs, A.[Anthony], Perera, A.[Amitha], Cuntoor, N.[Naresh], Chen, C.C.[Chia-Chih], Lee, J.T.[Jong Taek], Mukherjee, S.[Saurajit], Aggarwal, J.K., Lee, H.[Hyungtae], Davis, L.S.[Larry S.], Swears, E.[Eran], Wang, X.Y.[Xiao-Yang], Ji, Q.A.[Qi-Ang], Reddy, K.[Kishore], Shah, M.[Mubarak], Vondrick, C.[Carl], Pirsiavash, H.[Hamed], Ramanan, D.[Deva], Yuen, J.[Jenny], Torralba, A.[Antonio], Song, B.[Bi], Fong, A.[Anesco], Roy-Chowdhury, A.[Amit], Desai, M.[Mita],
A large-scale benchmark dataset for event recognition in surveillance video,
CVPR11(3153-3160).
IEEE DOI Link 1106
BibRef
And: AVSBS11(527-528).
IEEE DOI Link 1111
Dataset, Action Recognition. Dataset, Event Recognition. BibRef

Israel, S.A.[Steven A.],
Evaluation of ISR technologies for counter insurgency warfare,
AIPR10(1-5).
IEEE DOI Link 1010
BibRef

Ryoo, M.S., Chen, C.C.[Chia-Chih], Aggarwal, J.K., Roy-Chowdhury, A.[Amit],
An Overview of Contest on Semantic Description of Human Activities (SDHA) 2010,
ICPR-Contests10(270-285).
Springer DOI Link 1008
BibRef

Singh, S., Velastin, S.A., Ragheb, H.,
MuHAVi: A Multicamera Human Action Video Dataset for the Evaluation of Action Recognition Methods,
AVSS10(48-55).
IEEE DOI Link 1009
BibRef

Desurmont, X., Carincotte, C., Bremond, F.,
Intelligent Video Systems: A Review of Performance Evaluation Metrics that Use Mapping Procedures,
AVSS10(127-134).
IEEE DOI Link 1009
BibRef

Kuhn, W.[Werner],
A Functional Ontology of Observation and Measurement,
GS09(26-43).
Springer DOI Link 0912
BibRef

Rose, T.[Travis], Fiscus, J.[Jonathan], Over, P.[Paul], Garofolo, J.[John], Michel, M.[Martial],
The TRECVid 2008 Event Detection evaluation,
WACV09(1-8).
IEEE DOI Link 0912
BibRef

Ferryman, J.M., Ellis, A.,
PETS2010: Dataset and Challenge,
AVSS10(143-150).
IEEE DOI Link 1009
BibRef

Ellis, A., Ferryman, J.M.,
PETS2010 and PETS2009 Evaluation of Results Using Individual Ground Truthed Single Views,
AVSS10(135-142).
IEEE DOI Link 1009
BibRef

Ellis, A., Shahrokni, A., Ferryman, J.M.,
PETS2009 and Winter-PETS 2009 results: A combined evaluation,
PETS-Winter09(1-8).
IEEE DOI Link 0912
BibRef

Ferryman, J.M., Shahrokni, A.,
PETS2009: Dataset and challenge,
PETS-Winter09(1-6).
IEEE DOI Link 0912
BibRef

Kovesi, P.[Peter],
Video Surveillance: Legally Blind?,
DICTA09(204-211).
IEEE DOI Link 0912
Surveillance cameras are not very good. BibRef

Wang, H.[Heng], Ullah, M.M.[Muhammad Muneeb], Klaser, A.[Alexander], Laptev, I.[Ivan], Schmid, C.[Cordelia],
Evaluation of local spatio-temporal features for action recognition,
BMVC09(xx-yy).
PDF Version. 0909
BibRef

Sulman, N.[Noah], Sanocki, T.A.[Thomas A.], Goldgof, D.[Dmitry], Kasturi, R.[Rangachar],
How effective is human video surveillance performance?,
ICPR08(1-3).
IEEE DOI Link 0812
BibRef

Vezzani, R.[Roberto], Cucchiara, R.[Rita],
Annotation Collection and Online Performance Evaluation for Video Surveillance: The ViSOR Project,
AVSBS08(227-234).
IEEE DOI Link 0809
BibRef

Roth, D., Koller-Meier, E., Rowe, D., Moeslund, T.B., Van Gool, L.J.,
Event-Based Tracking Evaluation Metric,
Motion08(1-8).
IEEE DOI Link 0801
BibRef

Nghiem, A.T., Bremond, F., Thonnat, M., Valentin, V.,
ETISEO, performance evaluation for video surveillance systems,
AVSBS07(476-481).
IEEE DOI Link 0709
BibRef

Garofolo, J.[John],
Directions in automatic video analysis evaluations at NIST,
AVSBS07(6-6).
IEEE DOI Link 0709
Evaluation, Video Analysis. BibRef

Taylor, G.R.[Geoffrey R.], Chosak, A.J.[Andrew J.], Brewer, P.C.[Paul C.],
OVVV: Using Virtual Worlds to Design and Evaluate Surveillance Systems,
VS07(1-8).
IEEE DOI Link 0706
BibRef

Negre, A., Tran, H., Gourier, N., Hall, D., Lux, A., Crowley, J.L.,
Comparative Study of People Detection in Surveillance Scenes,
SSPR06(100-108).
Springer DOI Link 0608
BibRef

Nghiem, A.T., Bremond, F., Thonnat, M., Ma, R.,
A New Evaluation Approach for Video Processing Algorithms,
Motion07(15-15).
IEEE DOI Link 0702
To avoid dataset dependence. The maximum difficulty level of the videos at which the algorithm is performing good enough is defined as the upper bound of the algorithm capacity for handling the problem. BibRef

Tsuchiya, M.[Masamitsu], Fujiyoshi, H.[Hironobu],
Evaluating Feature Importance for Object Classification in Visual Surveillance,
ICPR06(II: 978-981).
IEEE DOI Link 0609
BibRef

Lienhart, R.[Rainer],
Algorithm competition,
VSSN05(53-54).
WWW Version. 0511
Evaluation discussion. For surveillance applications. BibRef

List, T., Bins, J., Vazquez, J., Fisher, R.B.,
Performance evaluating the evaluator,
PETS05(129-136).
IEEE DOI Link 0602
How to compare to human results. BibRef

Annesley, J., Orwell, J., Renno, J.R.,
Evaluation of MPEG7 color descriptors for visual surveillance retrieval,
PETS05(105-112).
IEEE DOI Link 0602
BibRef

Muller-Schneiders, S., Jager, T., Loos, H.S., Niem, W.,
Performance evaluation of a real time video surveillance system,
PETS05(137-143).
IEEE DOI Link 0602
BibRef

Ziliani, F., Velastin, S.A., Porikli, F.M., Marcenaro, L., Kelliher, T., Cavallaro, A., Bruneaut, P.,
Performance Evaluation of Event Detection Solutions: The CREDS Experience,
AVSBS05(201-206).
IEEE DOI Link 0602
BibRef

Zang, Q.[Qi], Klette, R.[Reinhard],
Object Classification and Tracking in Video Surveillance,
CAIP03(198-205).
WWW Version. 0311
BibRef

Zang, Q.[Qi], Klette, R.[Reinhard],
Evaluation of an Adaptive Composite Gaussian Model in Video Surveillance,
CAIP03(165-172).
WWW Version. 0311
BibRef

Jaynes, C., Webb, S., Steele, R.M., Xiong, Q.,
An Open Development Environment for Evaluation of Video Surveillance Systems,
PETS02(32-39). 0207
BibRef

Ferryman, J.M.,
Performance Evaluation of Tracking and Surveillance,
EEMCV01(xx-yy). 0110
BibRef

Ellis, T.,
Performance Metrics and Methods for Tracking in Surveillance,
PETS02(26-31). 0207
BibRef

Chapter on Motion -- Feature-Based, Long Range, Motion and Structure Estimates, Tracking, Surveillance, Activities continues in
Vehicle Motion Understanding and Analysis .


Last update:Feb 8, 2012 at 11:25:05