AI-Based Speech Enhancement for Hearing Aids

About This Grant

The greatest challenge to auditory communication is background noise, especially in the complex acoustic environments that are experienced in daily life. This project proposes to develop a deep learning speech enhancement (SE) algorithm that is robust to everyday environmental interference, improves speech intelligibility for people with hearing loss and can is small enough to be embedded in chips used in hearing aids. Taking on this challenge is inspired by the high-level performance of a small deep learning SE technology developed at OmniSpeech that was embedded in headsets. This headset went to market on October 16, 2023 and has received positive reviews. Unlike traditional signal processing algorithms that are only effective at improving sound quality and comfort in noisy complex environments, large deep learning SE models can improve speech intelligibility and preserve environmental sound recognition. This difference is due to the ability of deep learning SE to (a) significantly reduce babble noise without leaving annoying artifacts (also called musical noise), (b) remove unwanted transients (e.g., jackhammer) and (c) reduce environmental sounds overlapping with speech even when the background sounds are dynamic and loud (e.g., babble, sirens, jackhammer). We aim to obtain comparable results with a small deep learning SE model. Such a success will be a major advantage over other SE models because of our model’s small size and thus, realistic potential to be incorporated in hearing aids and other personal assistive listening devices for listeners with hearing impairment. We will use the Zilany auditory- nerve (AN) models developed for normal-hearing and hearing-impaired ears, as well as correlation metrics, to select a small deep neural network (DNN) model that results in noise suppression and best restores important spectral and temporal characteristics of the signals to yield responses in impaired ears that are similar to those of normal ears. The AN models will be used to inform the adaptation and refinement of our existing deep learning SE to benefit listeners with hearing loss. Two versions of our algorithm will be developed targeting two levels of aggressiveness in reducing unwanted noise. Low aggressiveness will maintain the quality and recognizability of environmental sounds by limiting the amount of noise reduction. High aggressiveness will maximize speech intelligibility by limiting the availability of some environmental sounds. We will evaluate the benefit of the two versions of our SE technology in listeners with mild to moderate sensorineural hearing loss as indexed by improvement in speech intelligibility and listening effort. The enhanced signal will be amplified using industry standard hearing aid gain individualized for each listener. For this experiment, OmniSpeech will team up with the Hearing Technology Lab directed by Eric Hoover at the University of Maryland. The overall goal of this Phase I project is a proof-of-concept test of a viable deep learning SE technology. Phase II will build on the clinical trial of Phase I and leverage the experience OmniSpeech is gaining with bringing speech technology to market in embedded devices.