Audio-based nonlinear video diffusion

We propose a novel non-linear video diffusion approach which is able to focus on parts of a video sequence that are relevant for applications in audio-visual analysis. The diffusion process is controlled by a diffusion coefficient based on an estimate of the synchrony between video motion and audio energy at each point of the video volume. Thus, regions whose motion is not coherent with the soundtrack are iteratively smoothed. The discretization of the proposed continuous diffusion formulation is carefully studied and its stability demonstrated. Our approach is tested in challenging situations involving sequence degradation and distracting video motion. Results show that in all cases our method is able to keep the focus of attention on the sound sources.

Published in:
Proc. of IEEE International Conference on Acoustics, Speech and Signal Processing
Presented at:
IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP), Dallas, March 14-19, 2010

Note: The status of this file is: Anyone

 Record created 2009-12-15, last modified 2020-04-20

Download fulltext

Rate this document:

Rate this document:
(Not yet reviewed)