Audio-visual synchronization recovery in multimedia content

• Presented at: International Conference on Acoustics, Speech and Signal Processing (ICASSP), Prague, Czech Republic, May 22-27, 2011
• Published in: Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 2280-2283
• Series: International Conference on Acoustics Speech and Signal Processing ICASSP
• Ieee Service Center, 445 Hoes Lane, Po Box 1331, Piscataway, Nj 08855-1331 Usa, 2011

This paper proposes a method recovering audio-visual synchronization of multimedia content. It exploits the correlation between the acoustic and the visual signals in order to estimate the audio-visual drift existing in the content. By shifting the audio signal relative to the visual signal, the estimation of the drift is obtained by searching for the shift producing the maximal audio-visual correlation. We consider two correlation measures, namely, mutual information and canonical correlation, and compare their performance. Experimental results demonstrate that the method using the canonical correlation is effective in recovering the audio-visual synchronization for both speech and non-speech sequences.

Keywords: Audio-visual synchronization, mutual information, canonical correlation, multimedia

Reference

• EPFL-CONF-162356
• View record in Web of Science

Record created on 2011-01-18, modified on 2012-03-20