Recently Naomi Harte, professor in the Electrical Engineering of Trinity College Dublin, recieved a SFI Frontiers Award. The project is ttiled as “SpeechSpace - a framework for understanding and modelling multimodal interactions”.
Minister for Further and Higher Education, Research, Innovation and Science Patrick O’Donovan, TD, who made the announcement, said the awards will support the development of “world-class research” in areas of science, technology, engineering and mathematics. The award is part of the 28 awards. They are of 4-5 years’ duration and will support 124 research positions including 58 postdoctoral positions, 53 PhD students and 13 research assistants and other positions. This programme has been funded in collaboration with SEAI.
The Project details of Naomi is below,
Speech is multimodal in nature. When humans have a conversation, we use much more than just words. We seamlessly signal and monitor a multitude of visual cues including eye gaze, facial expression, hand gestures and head nods. In parallel, we interpret linguistic information and prosody. Modern-day speech technology only uses the audio element of speech, and this leaves performance and applications fundamentally limited. The goal of this project is to develop a unified multimodal framework for modelling and analysing real-world speech-based interaction. SpeechSpace will transform the existing audio-only theory of hypospeech and hyperspeech to instead offer a fully multimodal framework of how humans exploit all the modalities of speech to ensure successful communication. SpeechSpace will furthermore develop new methods to incorporate multimodality into deep learning architectures that move away from an audio-dominated mindset. Approaches grounded in a fundamental understanding of human conversations are vital to unlock next generation speech technology improvements that are inclusive and do not rely on massive datasets. By exploiting the multimodality of speech in new ways, speech technology will become agile in deployment and reactive to dynamic conversations in changing environments
Over the coming months, Sigmedia will be actively recruiting new Researchers for working on this.
Relevant Links:
