Talking face: using facial feature detection and image transformations for visual speech

TitleTalking face: using facial feature detection and image transformations for visual speech
Publication TypeConference Paper
Year of Publication2001
AuthorsArya, A., and B. Hamidzadeh
Conference NameImage Processing, 2001. Proceedings. 2001 International Conference on
Pagination943 -946 vol.3
Keywordscomputational complexity, computer animation, customizable concatenative text-to-speech, face recognition, facial feature detection, feature extraction, image database requirements, image frames, image morphing, image sequences, image transformations, moving head applications, optical flow-based view morphing, personalized visual speech generation system, phonemes, speech synthesis, talking face, talking person, viewpoint, visual presentation, visual speech

Visual presentation of a talking person requires the generation of image frames showing the speaker in various views while pronouncing various phonemes. The existing approaches, mostly use either a complex 3D geometric model to reconstruct a desired image or a set of 2D images for each viewpoint, to select from. We propose a new system which utilizes facial feature detection and image-based transformation to create any talking frame using only one given image from the desired viewpoint and a set of reference images from one standard view. The proposed approach, together with optical flow-based view morphing and a customizable concatenative text-to-speech, makes a personalized visual speech generation system which can be used for moving/talking head applications where an optimal trade-of between computational complexity and image database requirements is necessary


a place of mind, The University of British Columbia

Electrical and Computer Engineering
2332 Main Mall
Vancouver, BC Canada V6T 1Z4
Tel +1.604.822.2872
Fax +1.604.822.5949

Emergency Procedures | Accessibility | Contact UBC | © Copyright 2020 The University of British Columbia