Glove-Talk: a neural network interface between a data-glove and a speech synthesizer

TitleGlove-Talk: a neural network interface between a data-glove and a speech synthesizer
Publication TypeJournal Article
Year of Publication1993
AuthorsFels, S. S., and G. E. HINTON
JournalNeural Networks, IEEE Transactions on
Pagination2 -8
Date Publishedjan.
Keywordsbackpropagation, backpropagation learning, DECtalk, Glove-Talk, hand movements to speech mapping, hand-gesture, handicapped aids, high bandwidth interface, interactive devices, learning systems, neural nets, neural network interface, peripheral interfaces, speech synthesis, speech synthesizer, vocabulary, VPL Data-Glove

To illustrate the potential of multilayer neural networks for adaptive interfaces, a VPL Data-Glove connected to a DECtalk speech synthesizer via five neural networks was used to implement a hand-gesture to speech system. Using minor variations of the standard backpropagation learning procedure, the complex mapping of hand movements to speech is learned using data obtained from a single `speaker' in a simple training phase. With a 203 gesture-to-word vocabulary, the wrong word is produced less than 1% of the time, and no word is produced about 5% of the time. Adaptive control of the speaking rate and word stress is also available. The training times and final performance speed are improved by using small, separate networks for each naturally defined subtask. The system demonstrates that neural networks can be used to develop the complex mappings required in a high bandwidth interface that adapts to the individual user


a place of mind, The University of British Columbia

Electrical and Computer Engineering
2332 Main Mall
Vancouver, BC Canada V6T 1Z4
Tel +1.604.822.2872
Fax +1.604.822.5949

Emergency Procedures | Accessibility | Contact UBC | © Copyright 2021 The University of British Columbia