Skip to main navigationSkip to main content
The University of Southampton

Research project: Brain inspired speech enhancer

Currently Active: 

Today’s speech enhancement systems can reduce noise and increase speech quality, but crucially they are not good at improving speech intelligibility in challenging environments. Better speech enhancement would have an impact on hearing aid and cochlear implant users, but also on all other speech communication platforms (mobile phones, television, radio) as well as for automatic speech recognition. My aim in the next 10 years is to create a ‘brain inspired speech enhancer’ (BISE); a system - informed by the understanding of the human auditory system - that is able to identify sound sources and to enhance speech perception. BISE will be used in everyday applications to facilitate communication in many situations. Using BISE tomorrow will be as normal as wearing glasses today.

Despite a clear demand for better solutions to communicate in noise, no breakthrough technology has yet emerged. Sound processing in the brain is still more successful than signal processing in silicone. A system that works as well or better than a human could lead to the next revolution in human communication. I expect that systems for assisted hearing with a broader general acceptance can only emerge from products that are used routinely by people with normal hearing. I believe that only a better understanding of the human auditory system and in consequence bio-inspired algorithms to process sound will provide the necessary breakthrough.

In order to help develop this system, I aim to create the BISE framework to understand and simulate human auditory perception better. BISE will provide a modelling link between auditory brain processing to human perception of complex sounds I envisage BISE to be an open platform in which aspects of auditory perception and signal processing are combined to allow an unprecedented ability to understand sound processing in the normal and impaired ear, as well as being able to rapidly develop and evaluate novel signal processing algorithms to enhance speech intelligibility. BISE combines ‘simple’ aspects that are important for auditory object identification (pitch, space, modulation, size, etc.) with contextual information of higher statistical order. BISE can be conceptualized on two levels: at the low level it provides a new way of understanding neuronal sound processing in the auditory system based on a generalization of the standard time frequency matrix which is so often used in auditory research. I am working to extend the classic representation to a new level on which sound representation is not based on energy (such as the ideal binary mask and most other noise reduction schemes), but on contextual information and, crucially, intelligibility. This system can highlight aspects of speech which are relevant for intelligibility such as frequency channel integrating, temporal modulated or regularly repeated information. On a functional level, BISE will be useful to understand the auditory system and to simulate psychophysical auditory responses and it will also enable the development of a variety of novel applications like speech enhancement systems in noise, or systemic evaluation of hearing aids which has eluded research and industry so far.

Current projects that are undertaken in my lab include:

  • Student project: Sound localization robot
  • Student project: dual perception of pitch: spectral vs. temporal pitch
  • PhD project Jinqiu Sang: Sparse coding in Hearing aids and simulating hearing loss (funded by EU AUDIS);
  • PhD project Lisa Spencer: localization of transient signals (funded by DSTL)
  • PhD project Niamh O’Meara: Auditory object perception (funded by DSTL and Toyota);
  • PhD project Konstanina Iliadi: Acoustic speaker identification (funded by DSTL),
  • PostDoc project Hongmei Hu: Sparse coding for noise reduction (funded by EU AUDIS)
  • PostDoc project Jessica Monaghan: Sound representation based on fractional derivatives (funded by Google).

Related research groups

Acoustics Group
Share this research project Share this on Facebook Share this on Twitter Share this on Weibo
Privacy Settings