000087061 001__ 87061
000087061 005__ 20190416220421.0
000087061 037__ $$aREP_WORK
000087061 245__ $$aMultimodal Analysis Using Redundant Parametric Decompositions
000087061 269__ $$a2004
000087061 260__ $$c2004$$aEcublens
000087061 336__ $$aReports
000087061 520__ $$aIn this work we explore the potentialities of a representational framework based on Matching Pursuit (MP) for the decomposition of audio-visual signals over redundant dictionaries. It is relatively easy for a human to correctly interpret a scene consisting on a combination of acoustic and visual stimuli and to take profit of both the information to experience a richer perception of the world. On the contrary, computer systems have considerable difficulties when having to deal with multimodal signals, and the information that each component contains about the others is usually discarded. This is basically due to the complexity of the dependencies that exist between audio and video signals and to the signals representations that are considered when attempting to mix them in multimodal fusion systems. Redundant decompositions describe audio-visual sequences in an extremely concise fashion, preserving good representational properties thanks to the use of redundant, well designed, dictionaries. This allows us to overcome two typical problems of multimodal fusion algorithms, that are the high dimensionality of the considered signals and the limitations of classical representation techniques, like pixel-based measures (for the video) or Fourier-like transforms (for the audio), that take into account only marginally the physics of the problem. The experimental results we obtain by making use of MP decompositions over redundant codebooks are encouraging and make us believe that such a research direction would allow to open a new way through multimodal signal representation.
000087061 6531_ $$aLTS2
000087061 700__ $$0241005$$g150417$$aMonaci, G.
000087061 700__ $$0241064$$g134131$$aDivorra Escoda, O.
000087061 700__ $$aVandergheynst, P.$$g120906$$0240428
000087061 8564_ $$uhttps://infoscience.epfl.ch/record/87061/files/Monaci2004_1150.pdf$$zn/a$$s542980
000087061 909C0 $$xU10380$$0252392$$pLTS2
000087061 909CO $$ooai:infoscience.tind.io:87061$$qGLOBAL_SET$$pSTI$$preport
000087061 937__ $$aEPFL-REPORT-87061
000087061 970__ $$aMonaci2004_1150/LTS
000087061 973__ $$sPUBLISHED$$aEPFL
000087061 980__ $$aREPORT