Development of a novel intra-oral wearable device with an inaudible speech computer interface for users with cervical spinal cord injuries

About This Grant

PROJECT SUMMARY/ABSTRACT Quadriplegia is a significantly limiting condition that can result from various causes, such as cervical spinal cord injury (cSCI), amyotrophic lateral sclerosis, or stroke. Quadriplegics suffer numerous restrictions in everyday life, including difficulty using digital devices, which typically require high degrees of hand dexterity for cursor control and typing. The inability to easily interact with digital devices can impede quality of life and limit access to edu- cation, socialization, entertainment, and career opportunities. The current state-of-the-art assistive technology available to quadriplegics for text input is voice-based speech-to-text. However, speech-to-text can have errors that are laborious to correct without the use of hands, resulting in slow typing rates. In addition, speech-to-text systems are often not socially acceptable in quiet environments and are not functional in noisy environments due to interference from background noise. Speech-to-text systems can also deprive quadriplegics of privacy, with negative impacts on mental health and autonomy. Moreover, speech-to-text cannot be reliably used by those with distorted voice or restricted use of their voice, which is a common occurrence in cSCI. Augmental has developed the MouthPad^—a fully wireless intraoral device that allows users with hand impairments to control the cursor of a laptop, smartphone, and other devices via a tongue-based trackpad and head-based motion sensing platform. In this Phase I proposal, we aim to determine the feasibility of incorporating inaudible speech recognition (whisper speech) to the MouthPad^ system to improve the accessibility of language input to digital devices. Specifically, whisper speech is considered speech that is too quiet to be heard by a person next to the speaker and is defined as a quiet breathy voice arising from gestures within the mouth. Incorporating whisper speech into the MouthPad^ will enable a fast, reliable, private, and always available keyboard functionality for those living with severe hand impairments. To achieve this goal, the following specific aims will be pursued: 1) characterize and optimize English phoneme detection capability, 2) develop keyword recognition capability, and 3) develop natural speech transcription capability. In Aim 1, the MouthPad^ capacitive touch sensing capabilities will be modified and tested to improve phoneme detection. Data from newly developed sensors will be used to train classification machine learning (ML) models to identify the spoken/whispered phoneme, which will quantify the sensor combination best suited to whisper speech. In Aim 2, a set of about 50 keywords for digital device interactions will be manually designed based on which phonemes are most robustly detectable by the ML models developed in Aim 1, and a keyword recognition system will be developed. In Aim 3, the MouthPad^ sensors and ML models developed in Aim 1 will be used in conjunction with large vocabulary phonetic language ML models to decode natural-speed whisper speech into text sentences. Together with the existing MouthPad^ cursor con- trol and gestural input system, successful completion of this SBIR Phase I project will provide hand-impaired individuals with a complete, high-performance equivalent to the mouse and keyboard for digital device use.