By Dave DeFusco
Imagine someone talking in a video conference while a piece of music is playing in the background. Besides being distracting, the music makes it hard for you to understand the speaker when you鈥檙e listening afterward to the recording.
Dr. Youshan Zhang, assistant professor of computer science and artificial intelligence, and Jialu Li of Cornell University have created a novel noise removal method that could benefit the hearing impaired and improve the listening experience for audiophiles everywhere.
An example of speech denoising. At left, original speech with noise and, at right, denoised speech audio.In their paper, 鈥,鈥 the researchers described how they created a deep visual audio denoising (DVAD) model using a dataset of 15,300 bird sounds鈥攙arying in length from 1 second to 15 seconds鈥攖hat strips out the background noise, in this case natural sounds like wind and rain, to produce clean bird sounds.
The researchers presented their model in January at the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) conference in Hawaii. Dr. Zhang said the model is robust enough to apply to human speech, especially to background noise that is particularly damaging to speech intelligibility for people with difficulty hearing.
鈥淥ur DVAD model can first denoise the background noise and then increase the volume of the low voice,鈥 he said.
Professor Youshan Zhang in Hawaii at the WACV conference.In a novel twist, the researchers turned the audio of the bird sounds into a series of images; used a photo editing tool that eliminates the original background of an image without compromising its integrity; created a segmentation model to edit out the noisy parts of the image; and then applied an algorithm to produce the 鈥渄enoised,鈥 or clean bird sounds.
鈥淭o the best of our knowledge, we are the first to transfer audio denoising into an image segmentation problem,鈥 said Dr. Zhang. 鈥淏y removing the noise area in the audio image, we can realize the purpose of audio denoising.鈥
Background noise removal is the ability to enhance a noisy speech signal by isolating the dominant sound. It鈥檚 used in audio and video editing software, video conferencing platforms and noise-canceling headphones. It鈥檚 a fast-evolving technology, with artificial intelligence bringing a whole new domain of approaches to improve the task.
鈥淓xtensive experimental results demonstrate that our proposed model achieves state-of-the-art performance,鈥 said Dr. Zhang. 鈥淲e also show that our method can be easily generalized to speech denoising, audio separation, audio enhancement and noise estimation.鈥
