Multi-stream ASR: Oracle Test and Embedded Training

Abstract

Multi-stream based automatic speech recognition (ASR) systems outperform their single stream counterparts, especially in the case of noisy speech. However, the main issues in multi-stream systems are to know a) Which streams to be combined, and b) How to combine them. In order to address these issues, we have investigated an `Oracle' test, which can tell us whether two streams are complimentary. Moreover, the Oracle test justifies our previously proposed inverse entropy method for weighting various streams. We have carried out experiments on two multi-stream systems and results indicate that in clean speech around 80% of the time Oracle selected the stream which had the minimum entropy. In this paper, we have also presented an embedded iterative training for multi-stream systems. The results of the recognition experiments on Numbers95 database showed that we can improve the performance significantly by multi-stream iterative training, not only for clean speech but also for various noise conditions.