Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Reports, Documentation, and Standards
  4. An information theoretic measure of sequence recognition performance
 
report

An information theoretic measure of sequence recognition performance

Morris, Andrew
2002

Sequence recognition performance is often summarised first in terms of the number of hits (H), substitutions (S), deletions (D) and insertions (I), and then as a single statistic by the "word error rate" WER = 100(S D I)/(H S D). While in common use, WER has two disadvantages as a performance measure. One is that it has no upper bound, so it doesn't tell you how good a system is, only that one is better than another. The other is that it is not D/I symmetric, although deletions and insertions are equally disadvantageous. At low error rates these limitations can be ignored. However, for the high error rates which can occur during tests for speech recognition in noise the WER measure starts to misbehave, giving far more weight to insertions than to deletions and regularly "exceeding 100%". Here we derive an alternative summary statistic for sequence recognition accuracy: WIP = H^2/(H S D)(H S I). The WIP (word information preserved) measure results from an approximation to the proportion of the information about the true sequence which is preserved in the recognised sequence. It has comparable simplicity to WER but neither of its disadvantages.

  • Files
  • Details
  • Metrics
Type
report
Author(s)
Morris, Andrew
Date Issued

2002

Publisher

IDIAP

Subjects

mutual information

•

contingency tables

•

speech

•

likelihood ratio

•

word error rate

URL

URL

http://publications.idiap.ch/downloads/reports/2002/com02-03.pdf
Written at

EPFL

EPFL units
LIDIAP  
Available on Infoscience
March 10, 2006
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/228133
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés