Korin Richmond, Zhen-Hua Ling, Junichi Yamagishi, Benigno Uria (2013). On the Evaluation of Inversion Mapping Performance in the Acoustic Domain. Interspeech, Lyon, France.

Abstract  The two measures typically used to assess the performance of an inversion mapping method, where the aim is to estimate what articulator movements gave rise to a given acoustic signal, are root mean squared (RMS) error and correlation. In this paper, we investigate whether “task-based” evaluation using an articulatory-controllable HMM-based speech synthesis system can give useful additional information to complement these measures. To assess the usefulness of this evaluation approach, we use articulator trajectories estimated by a range of different inversion mapping methods as input to the synthesiser, and measure their performance in the acoustic domain in terms of RMS error of the generated acoustic parameters and with a listening test involving 30 participants. We then compare these results with the standard RMS error and correlation measures calculated in the articulatory domain. Interestingly, in the acoustic evaluation we observe one method performs with no statistically significant difference from measured articulatory data, and cases where statistically significant differences between methods exist which are not reflected in the results of the two standard measures. From our results, we conclude such task-based evaluation can indeed provide interesting extra information, and gives a useful way to compare inversion methods.


Warning: strtotime(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in /home/laslab/3w/listening-talker.org/libraries/joomla/utilities/date.php on line 56

Warning: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in /home/laslab/3w/listening-talker.org/libraries/joomla/utilities/date.php on line 198

Warning: Creating default object from empty value in /home/laslab/3w/listening-talker.org/modules/mod_latestnews/helper.php on line 109

Warning: Creating default object from empty value in /home/laslab/3w/listening-talker.org/modules/mod_latestnews/helper.php on line 109

Warning: Creating default object from empty value in /home/laslab/3w/listening-talker.org/modules/mod_latestnews/helper.php on line 109

Warning: Creating default object from empty value in /home/laslab/3w/listening-talker.org/modules/mod_latestnews/helper.php on line 109

Warning: Creating default object from empty value in /home/laslab/3w/listening-talker.org/modules/mod_latestnews/helper.php on line 109

EU Flag EU Flag 
Warning: strtotime(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in /home/laslab/3w/listening-talker.org/libraries/joomla/utilities/date.php on line 56

Warning: strftime(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in /home/laslab/3w/listening-talker.org/libraries/joomla/utilities/date.php on line 250
Copyright © 2017. The Listening Talker - LISTA  |  Technical Support