Development of a Low-Cost, Noninvasive, Portable Visual Speech Recognition Program.

Development of a Low-Cost, Noninvasive, Portable Visual Speech Recognition Program.

Ann Otol Rhinol Laryngol. 2016 May 19;

Authors: Kohlberg GD, Gal YK, Lalwani AK

Abstract
OBJECTIVES: Loss of speech following tracheostomy and laryngectomy severely limits communication to simple gestures and facial expressions that are largely ineffective. To facilitate communication in these patients, we seek to develop a low-cost, noninvasive, portable, and simple visual speech recognition program (VSRP) to convert articulatory facial movements into speech.
METHODS: A Microsoft Kinect-based VSRP was developed to capture spatial coordinates of lip movements and translate them into speech. The articulatory speech movements associated with 12 sentences were used to train an artificial neural network classifier. The accuracy of the classifier was then evaluated on a separate, previously unseen set of articulatory speech movements.
RESULTS: The VSRP was successfully implemented and tested in 5 subjects. It achieved an accuracy rate of 77.2% (65.0%-87.6% for the 5 speakers) on a 12-sentence data set. The mean time to classify an individual sentence was 2.03 milliseconds (1.91-2.16).
CONCLUSION: We have demonstrated the feasibility of a low-cost, noninvasive, portable VSRP based on Kinect to accurately predict speech from articulation movements in clinically trivial time. This VSRP could be used as a novel communication device for aphonic patients.

PMID: 27208007 [PubMed - as supplied by publisher]

from #AlexandrosSfakianakis via Alexandros G.Sfakianakis on Inoreader http://ift.tt/1YNW6Ha
via IFTTT