21.3.4 Lipreading, Lip Reading, Lip Tracking

Language Independent Lip Reading,
2007.
HTML Version. Dataset, Lip Reading.

Mase, K.,
Recognition of Facial Expression from Optical Flow,
IEICE(E74-xx), No. 10, 1991, pp. 3474-3483. BibRef 9100

Mase, K., and Pentland, A.P.,
Automatic Lipreading by Computer,
IEICE(J73-D-II), No. 6, June 1990, pp. 796-803. BibRef 9006
Earlier:
Lip Reading: Automatic Visual Recognition of Spoken Words,
OSAMV89(1565-1570). BibRef

Murase, H., Sakai, R.,
Moving Object Recognition in Eigenspace Representation: Gait Analysis and Lip Reading,
PRL(17), No. 2, February 8 1996, pp. 155-162. BibRef 9602

Zhou, Y.[Yong],
Low bit rate audio-visual communication having improved face and lip region detection,
US_Patent5,596,362, Jan 21, 1997
WWW Version. BibRef 9701

Luettin, J., Thacker, N.A.,
Speechreading Using Probabilistic Models,
CVIU(65), No. 2, February 1997, pp. 163-178. 9704

WWW Version. BibRef

Luettin, J., Thacker, N.A., Beet, S.W.,
Locating and Tracking Facial Speech Features,
ICPR96(I: 652-656).
IEEE DOI Link 9608
BibRef
And:
Learning to Recognise Talking Faces,
ICPR96(IV: 55-59).
IEEE DOI Link 9608
(Univ. of Sheffield, UK) BibRef

Yu, K., Jiang, X.Y., Bunke, H.,
Lipreading: A Classifier Combination Approach,
PRL(18), No. 11-13, November 1997, pp. 1421-1426. 9806
BibRef

Goldschen, A.J.[Alan J.], (MITRE), Petajan, E.D.[Eric D.], (ATT), and Garcia, O.N.[Oscar N.], (Wright State University),
Continuous Automatic Speech Recognition by Lipreading,
MBR97(Chapter 14). BibRef 9700

Nan, L.[Li], Dettmer, S.[Shawn], and Shah, M.[Mubarak],
Visually Recognizing Speech Using Eigen Sequences,
MBR97(Chapter 15), UCF. BibRef 9700

Graf, H.P.[Hans Peter],
Method for locating a subject's lips in a facial image,
US_Patent5,805,745, September 8, 1998.
WWW Version. BibRef 9809

Petajan, E.D., Graf, H.P.,
Robust face feature analysis for automatic speechreading and character animation,
AFGR96(357-362).
IEEE DOI Link 9610
BibRef

Yu, K.[Keren], Jiang, X.Y.[Xiao-Yi], Bunke, H.[Horst],
Lipreading using signal analysis over time,
SP(77), No. 2, 1 September 1999, pp. 195-208. BibRef 9909
Earlier:
Lipreading using Fourier transform over time,
CAIP97(472-479).
WWW Version. 9709
BibRef

Mak, M.W., Allen, W.G.,
A lip-tracking system based on morphological processing and block matching techniques,
SP:IC(6), No. 4, August 1994, pp. 335-348.
WWW Version. BibRef 9408

Lepsøy, S.[Skjalg], Curinga, S.[Sergio],
Conversion of articulatory parameters into active shape model coefficients for lip motion representation and synthesis,
SP:IC(13), No. 3, September 1998, pp. 209-225.
WWW Version. BibRef 9809

Chan, S., Ngo, C.W., Lai, K.F.,
Motion tracking of human mouth by generalized deformable models,
PRL(20), No. 8, August 1999, pp. 879-887. BibRef 9908

Oliver, N.[Nuria], Pentland, A.P.[Alex P.], Bérard, F.[François],
LAFTER: a real-time face and lips tracker with facial expression recognition,
PR(33), No. 8, August 2000, pp. 1369-1382.
WWW Version. 0005
BibRef
Earlier:
LAFTER: Lips and Face Real Time Tracker,
CVPR97(123-129).
IEEE Abstract. IEEE Top Reference.
WWW Version. 9704
With demo. BibRef

Jebara, T.S., Pentland, A.P.,
Parameterized Structure from Motion for 3D Adaptive Feedback Tracking of Faces,
CVPR97(144-150).
IEEE Abstract. IEEE Top Reference.
WWW Version. 9704
BibRef
And: Vismod--401, 1996.
HTML Version. MIT. Color. Symmetries. BibRef

Chiou, G.I., Hwang, J.N.,
Lipreading from Color Video,
IP(6), No. 8, August 1997, pp. 1192-1195.
IEEE DOI Link 9708
BibRef
Earlier:
Lipreading from Color Motion Video,
ICASSP96(XX) Dept of EE. University of Seattle. BibRef

Matthews, I.[Iain], Cootes, T.F.[Timothy F.], Bangham, J.A.[J. Andrew], Cox, S.J.[Stephen J.], Harvey, R.W.[Richard W.],
Extraction of Visual Features for Lipreading,
PAMI(24), No. 2, February 2002, pp. 198-213.
IEEE Abstract. IEEE Top Reference.
WWW Version. 0202
Evaluation, Lip Reading. Compare 3 methods for evaluation. BibRef

Matthews, I., Bangham, J.A., Harvey, R.W., Cox, S.J.,
A Comparison of Active Shape Model and Scale Decomposition Based Features for Visual Speech Recognition,
ECCV98(II: 514).
WWW Version. BibRef 9800
Earlier: A3, A1, A2, A4:
Lip Reading from Scale-Space Measurements,
CVPR97(582-587).
IEEE Abstract. IEEE Top Reference.
WWW Version. 9704
BibRef

Daubias, P.[Philippe], Deléglise, P.[Paul],
Statistical Lip-Appearance Models Trained Automatically Using Audio Information,
JASP(2002), No. 11, November 2002, pp. 1202.
HTML Version. 0304
BibRef

Zhang, X.Z.[Xiao-Zheng], Broun, C.C.[Charles C.], Mersereau, R.M.[Russell M.], Clements, M.A.[Mark A.],
Automatic Speechreading with Applications to Human-Computer Interfaces,
JASP(2002), No. 11, November 2002, pp. 1228.
HTML Version. 0304
BibRef

Zhang, X.Z.[Xiao-Zheng], Mersereau, R.M.[Russell M.],
Lip Feature Extraction Towards an Automatic Speechreading System,
ICIP00(Vol III: 226-229).
IEEE Abstract. IEEE Top Reference. 0008
BibRef

Gordan, M.[Mihaela], Kotropoulos, C.[Constantine], Pitas, I.[Ioannis],
A Support Vector Machine-Based Dynamic Network for Visual Speech Recognition Applications,
JASP(2002), No. 11, November 2002, pp. 1248.
HTML Version. 0304
BibRef
Earlier:
Application of support vector machines classifiers to visual speech recognition,
ICIP02(III: 129-132).
IEEE Abstract. IEEE Top Reference. 0210
BibRef

Siatras, S., Nikolaidis, N.[Nikos], Krinidis, M.[Michail], Pitas, I.[Ioannis],
Visual Lip Activity Detection and Speaker Detection Using Mouth Region Intensities,
CirSysVideo(19), No. 1, January 2009, pp. 133-137.
IEEE DOI Link 0902
BibRef

Luthon, F., Caplier, A., Lievin, M.,
Spatiotemporal MRF approach to video segmentation: Application to motion detection and lip segmentation,
SP(76), No. 1, 1 July 1999, pp. 61-80. BibRef 9907

Caplier, A.[Alice], Luthon, F.[Franck],
A new spatiotemporal approach for image analysis. Application to motion detection,
CAIP95(246-253).
Springer DOI Link 9509
BibRef

Lievin, M., Luthon, F.,
Nonlinear Color Space and Spatiotemporal MRF for Hierarchical Segmentation of Face Features in Video,
IP(13), No. 1, January 2004, pp. 63-71.
IEEE DOI Link 0402
BibRef

Lievin, M., Luthon, F.,
A Hierarchical Segmentation Algorithm for Face Analysis Application for Lipreading,
ICME00(TP8). 0007
BibRef
Earlier:
Lip features automatic extraction,
ICIP98(III: 168-172).
IEEE DOI Link 9810
BibRef

Luthon, F.[Franck], and Lievin, M.,
Lip Motion Automatic Detection,
SCIA97(xx-yy) 9705

HTML Version. BibRef

Cetingul, H.E., Yemez, Y., Erzin, E., Tekalp, A.M.,
Discriminative Analysis of Lip Motion Features for Speaker Identification and Speech-Reading,
IP(15), No. 10, October 2006, pp. 2879-2891.
IEEE DOI Link 0609
BibRef
Earlier:
Discriminative lip-motion features for biometric speaker identification,
ICIP04(III: 2023-2026).
IEEE DOI Link 0505
BibRef

Lafon, S.[Stephane], Keller, Y., Coifman, R.R.[Ronald R.],
Data Fusion and Multicue Data Matching by Diffusion Maps,
PAMI(28), No. 11, November 2006, pp. 1784-1797.
IEEE DOI Link 0609
Laplace-Beltrami approach for computing density invariant embeddings. Second, a refinement of the Nyström extension algorithm. Finally, a multicue data matching scheme based on nonlinear spectral graphs alignment. Apply to lipreading. BibRef

Bayro-Corrochano, E.[Eduardo], Trujillo, N.[Noel], Naranjo, M.[Michel],
Quaternion Fourier Descriptors for the Preprocessing and Recognition of Spoken Words Using Images of Spatiotemporal Representations,
JMIV(28), No. 2, June 2007, pp. 179-190.
Springer DOI Link 0710
BibRef

Yau, W.C.[Wai Chee], Kumar, D.K.[Dinesh Kant], Arjunan, S.P.[Sridhar Poosapadi],
Visual Speech Recognition Using Dynamic Features And Support Vector Machines,
IJIG(8), No. 3, July 2008, pp. 419-437. 0807
BibRef
Earlier:
Visual Speech Recognition Method Using Translation, Scale and Rotation Invariant Features,
AVSBS06(63-63).
IEEE DOI Link 0611
BibRef

Yau, W.C.[Wai Chee], Kumar, D.K.[Dinesh Kant], Chinnadurai, T.[Tharangini],
Lip-Reading Technique Using Spatio-Temporal Templates and Support Vector Machines,
CIARP08(610-617).
Springer DOI Link 0809
BibRef

Yau, W.C.[Wai Chee], Kumar, D.K.[Dinesh Kant], Weghorn, H.[Hans],
Visual Speech Recognition Using Motion Features and Hidden Markov Models,
CAIP07(832-839).
Springer DOI Link 0708
BibRef

Seymour, R.[Rowan], Stewart, D.[Darryl], Ming, J.[Ji],
Comparison of Image Transform-Based Features for Visual Speech Recognition in Clean and Corrupted Videos,
JIVP(2008), No. 2008, pp. xx-yy.
WWW Version. 0804
BibRef

Borgstrom, B.J., Alwan, A.[Abeer],
A Low-Complexity Parabolic Lip Contour Model With Speaker Normalization for High-Level Feature Extraction in Noise-Robust Audiovisual Speech Recognition,
SMC-A(38), No. 6, November 2008, pp. 1273-1280.
IEEE DOI Link 0811
BibRef

Wang, S.L., Liew, A.W.C., Lau, W.H., Leung, S.H.,
An Automatic Lipreading System for Spoken Digits With Limited Training Data,
CirSysVideo(18), No. 12, December 2008, pp. 1760-1765.
IEEE DOI Link 0812
BibRef
Earlier: A1, A3, A4, Only:
Automatic Lipreading with Limited Training Data,
ICPR06(III: 881-884).
WWW Version. 0609
BibRef

Zhao, G., Barnard, M., Pietikäinen, M.[Matti],
Lipreading With Local Spatiotemporal Descriptors,
MultMed(11), No. 7, November 2009, pp. 1254-1265.
IEEE DOI Link 0911
BibRef

Yang, M.T.[Mau-Tsuen], You, Z.W.[Zhen-Wei], Shih, Y.C.[Ya-Chun],
Lip contour extraction for language learning in VEC3D,
MVA(21), No. 1, November 2009, pp. xx-yy.
Springer DOI Link 0911
BibRef

Gastelum, A., Krueger, M., Marquez, J., Gimel'farb, G.L., Delmas, P.,
Automatic 3D lip shape segmentation and modelling,
IVCNZ08(1-6).
IEEE DOI Link 0811
BibRef

Saitoh, T.[Takeshi], Morishita, K.[Kazutoshi], Konishi, R.[Ryosuke],
Analysis of efficient lip reading method for various languages,
ICPR08(1-4).
IEEE DOI Link 0812
BibRef

Saitoh, T.[Takeshi], Konishi, R.[Ryosuke],
Lip Reading Based on Sampled Active Contour Model,
ICIAR05(507-515).
Springer DOI Link 0509
BibRef

Terry, L.H.[Louis H.], Shiell, D.J.[Derek J.], Katsaggelos, A.K.[Aggelos K.],
Vector quantization with memory and multi-labeling for isolated video-only automatic speech recognition,
ICIP08(1320-1323).
IEEE DOI Link 0810
BibRef

Wu, G.Y.[Guan-Yong], Zhu, J.[Jie],
Modular BDPCA based visual feature representation for lip-reading,
ICIP08(1328-1331).
IEEE DOI Link 0810
BibRef

de la Cuesta, A.G., Zhang, J.G.[Jian-Guo], Miller, P.,
Biometric Identification Using Motion History Images of a Speaker's Lip Movements,
IMVIP08(83-88).
IEEE DOI Link 0809
BibRef

Pachoud, S.[Samuel], Gong, S.G.[Shao-Gang], Cavallaro, A.[Andrea],
Macro-cuboïd based probabilistic matching for lip-reading digits,
CVPR08(1-8).
IEEE DOI Link 0806
BibRef

Kumar, K.[Kshitiz], Chen, T.H.[Tsu-Han], and Stern, R.M.[Richard M.],
Profile View Lip Reading,
ICASSP07(IV: 429-432).
PDF Version. Intro to profile view lip reading, compares to frontal, and combines with audio for comprenhensive system. BibRef 0700

Faraj, M.I.[Maycel Isaac], Bigun, J.[Josef],
Speaker and Digit Recognition by Audio-Visual Lip Biometrics,
ICB07(1016-1024).
Springer DOI Link 0708
BibRef

Faraj, M.I.[Maycel Isaac], Bigun, J.[Josef],
Lip Biometrics for Digit Recognition,
CAIP07(360-365).
Springer DOI Link 0708
BibRef
Earlier:
Motion Features from Lip Movement for Person Authentication,
ICPR06(III: 1059-1062).
WWW Version. 0609
BibRef
And:
Person Verification by Lip-Motion,
Biometrics06(37).
IEEE DOI Link 0609
BibRef

Fu, Y.[Yun], Zhou, X.[Xi], Liu, M.[Ming], Hasegawa-Johnson, M.[Mark], Huang, T.S.[Thomas S.],
Lipreading by Locality Discriminant Graph,
ICIP07(III: 325-328).
IEEE DOI Link 0709
BibRef

Gómez, J.B.[Juan B.], Hernández, J.E.[Jorge E.], Prieto, F.[Flavio], Redarce, T.[Tanneguy],
Real-Time Robot Manipulation Using Mouth Gestures in Facial Video Sequences,
BVAI07(224-233).
Springer DOI Link 0710
BibRef

Yu, D.H.[Da-Hai], Ghita, O.[Ovidiu], Sutherland, A.[Alistair], Whelan, P.F.[Paul F.],
A Novel Visual Speech Representation and HMM Classification for Visual Speech Recognition,
PSIVT09(398-409).
Springer DOI Link 0901
BibRef
Earlier:
A New Manifold Representation for Visual Speech Recognition,
IMVIP07(210-210).
IEEE DOI Link 0709
BibRef
And: CAIP07(374-382).
Springer DOI Link 0708
BibRef

Kricke, R.[Ralph], Gernoth, T.[Thorsten], Grigat, R.R.[Rolf-Rainer],
Local binary patterns for lip motion analysis,
ICIP08(1472-1475).
IEEE DOI Link 0810
BibRef

Shafait, F., Kricke, R., Shdaifat, I., Grigat, R.R.,
Real Time Lip Motion Analysis for a Person Authentication System using Near Infrared Illumination,
ICIP06(1957-1960). 0610

IEEE DOI Link BibRef

Kumatani, K.[Kenichi], Stiefelhagen, R.[Rainer],
Mouth Region Localization Method Based on Gaussian Mixture Model,
IWICPAS06(115-124).
Springer DOI Link 0608
BibRef

Ichino, M., Sakano, H., Komatsu, N.,
Multimodal Biometrics of Lip Movements and Voice using Kernel Fisher Discriminant Analysis,
ICARCV06(1-6).
IEEE DOI Link 0612
BibRef

Tsunekawa, T.[Takuya], Hotta, K.[Kazuhiro], Takahashi, H.[Haruhisa],
Lipreading Using Recurrent Neural Prediction Model,
ICIAR04(II: 405-412).
WWW Version. 0409
BibRef

Yin, P.[Pei], Essa, I.A., Rehg, J.M.,
Asymmetrically boosted HMM for speech reading,
CVPR04(II: 755-761).
IEEE Abstract. IEEE Top Reference. 0408
BibRef
Earlier:
Boosted audio-visual HMM for speech reading,
AMFG03(68-73).
IEEE Abstract. IEEE Top Reference. 0311
BibRef

Yao, H.X.[Hong-Xun], Gao, W.[Wen], Shan, W.[Wei], Xu, M.H.[Ming-Hui],
Visual Features Extracting and Selecting for Lipreading,
AVBPA03(251-259).
HTML Version. 0310
BibRef

Chindaro, S.[Samuel], Deravi, F.[Farzin],
Directional Properties of Colour Co-occurrence Features for Lip Location and Segmentation,
AVBPA01(84).
HTML Version. 0310
BibRef

Auckenthaler, R., Brand, J.D., Mason, J.S., Deravi, F., Chibelushi, C.C.,
Lip Signatures for Automatic Person Recognition,
AVBPA99(xx-yy). BibRef 9900

Brand, J.D., Mason, J.S., Colomb, S.[Sylvain],
Visual Speech: A Physiological or Behavioural Biometric?,
AVBPA01(157).
HTML Version. 0310
BibRef

Roach, M.J., Brand, J.D., Mason, J.S.D.,
Acoustic and Facial Features for Speaker Recognition,
ICPR00(Vol III: 258-261).
IEEE DOI Link
IEEE DOI Link
HTML Version. 0009
BibRef

Lucey, S.[Simon],
An Evaluation of Visual Speech Features for the Tasks of Speech and Speaker Recognition,
AVBPA03(260-267).
HTML Version. 0310
BibRef

Kalberer, G.A.[Gregor A.], Müller, P.[Pascal], Van Gool, L.J.[Luc J.],
Biological Motion of Speech,
BMCV02(199 ff.).
HTML Version. 0303
People are sensitive to lip motion in speech. Analyze detailed motion of the face. BibRef

Delmas, P., Eveno, N., Lievin, M.,
Towards robust lip tracking,
ICPR02(II: 528-531).
IEEE DOI Link 0211
BibRef

Uda, K., Tagawa, N., Minagawa, A., Moriya, T.,
Effectiveness evaluation of word characteristics obtained from 3D image information for lipreading,
CIAP01(296-301).
IEEE Top Reference. 0210
BibRef

Murakami, H., Baba, H., Noma, T.,
MLSLib: A Lip Sync Library for Multi Agents and Languages,
WSCG02(295).
PDF Version.
HTML Version. 0209
BibRef

Seguier, R., Cladel, N., Foucher, C., Mercier, D.,
Lipreading with Spiking Neurons: One Pass Learning,
WSCG02(397).
WWW Version.
HTML Version. 0209
BibRef

Mujal, M., Kirlin, R.L.,
Compression enhancement of video motion of mouth region using joint audio and video coding,
Southwest02(82-86).
IEEE Top Reference. 0208
BibRef

Arya, A., Hamidzadeh, B.,
Talking Face: Using Facial Feature Detection and Image Transformations for Visual Speech,
ICIP01(III: 943-946).
IEEE Abstract. IEEE Top Reference. 0108
BibRef

Potamianos, G., Neti, C.,
Improved ROI and Within Frame Discriminant Features for Lipreading,
ICIP01(III: 250-253).
IEEE Abstract. IEEE Top Reference. 0108
BibRef

Kshirsagar, S.[Sumedha], Magnenat-Thalmann, N.[Nadia],
Lip Synchronization Using Linear Predictive Analysis,
ICME00(TP8). 0007
BibRef

Caplier, A., Delmas, P., Lam, D.,
Robust Initialisation for Lips Edges Detection,
SCIA99(Image Analysis). BibRef 9900

Vanegas, O.[Oscar], Tokuda, K.[Keiichi], Kitamura, T.[Tadashi],
Location Normalization of HMM-Based Lip Reading: Experiments for the M2VTS Database,
ICIP99(II:343-347).
IEEE Abstract. IEEE Top Reference. BibRef 9900

Gao, L.[Lei], Mukaigawa, Y., Ohta, Y.,
Synthesis of Facial Images with Lip Motion from Several Real Views,
AFGR98(181-186).
IEEE DOI Link BibRef 9800

Kumar, V.P.[Vinay P.], Oren, M.[Mike], Osuna, E.[Edgar], Poggio, T.[Tomaso],
Real Time Analysis and Tracking of Mouths for Expression Recognition,
DARPA98(151-155). BibRef 9800

Kumar, V.P.[Vinay P.], Poggio, T.[Tomaso],
Recognizing Expressions by Direct Estimation of the Parameters of a Pixel Morphable Model,
BMCV02(519 ff.).
HTML Version. 0303
BibRef

Kumar, V.P.[Vinay P.], Poggio, T.[Tomaso],
Learning-Based Approach to Estimation of Morphable Model Parameters,
MIT AI Memo-1696, September, 2000. This paper describes a method for estimating the parameters of a linear morphable model (LMM) that models mouth images.
WWW Version. 0105
BibRef

Kumar, V.P.[Vinay P.],
Towards Man-Machine Interfaces: Combining Top-down Constraints with Bottom-up Learning in Facial Analysis,
MIT AI-TR-2002-008, September 2002.
WWW Version. BibRef 0209

Kumar, V.P.[Vinay P.], Poggio, T.[Tomaso],
Learning-Based Approach to Real Time Tracking and Analysis of Faces,
AFGR00(96-101).
IEEE DOI Link 0003
BibRef

Yu, K., Jiang, X., Bunke, H.,
Automatic Lipreading of Sentences Combining Hidden Markov Models and Grammars,
AVBPA99(xx-yy). BibRef 9900

Baig, A.R., Seguier, R., Vaucher, G.,
Image sequence analysis using a spatio-temporal coding for automatic lipreading,
CIAP99(544-549).
IEEE DOI Link 9909
BibRef

Sridharan, S.[Sridha], Wark, T.J.[Timothy J.], Chandran, V.,
An Approach to Statistical Lip Modelling for Speaker Identification via Chromatic Feature Extraction,
ICPR98(Vol I: 123-125).
IEEE DOI Link 9808
BibRef

Wark, T.J., Sridharan, S., Chandran, V.,
Robust Speaker Verification via Asynchronous Fusion of Speech and Lip Information,
AVBPA99(xx-yy). BibRef 9900

Potamianos, G., Graf, H.P., Cosatto, E.,
An image transform approach for HMM based automatic lipreading,
ICIP98(III: 173-177).
IEEE DOI Link 9810
BibRef

Jung, J.Y.[Jae Y.], Kim, M.H.[Moon H.],
Motion Estimation of Lips in Pronouncing Korean Vowels Based on Fuzzy Constraint Line Clustering,
ICIP96(III: 507-510).
IEEE DOI Link BibRef 9600

Bregler, C.[Christopher], and Omohundro, S.M.[Stephen M.],
Learning Visual Models for Lipreading,
MBR97(Chapter 13), Berkeley, and NEC. BibRef 9700

Bregler, C.[Christopher], Covell, M.[Michele], and Slaney, M.[Malcolm],
Video Rewrite: Driving Visual Speech with Audio,
SIGGraph-97(xx-yy).
WWW Version. BibRef 9700

Bregler, C.[Christoph], Omohundro, S.[Stephen],
Nonlinear Manifold Learning for Visual Speech Recognition,
ICCV95(494-499).
IEEE DOI Link
WWW Version. BibRef 9500

Stork, D.G., Hennecke, M.E.,
Speechreading: an overview of image processing, feature extraction, sensory integration and pattern recognition techniques,
AFGR96(xvi-xxvi).
IEEE DOI Link 9610
BibRef

Chapter on Face Recognition, Detection, Tracking, Gesture Recognition, Fingerprints, Biometrics continues in
Combined Audio Visual Recognition .