Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Audio-Visual Synchronisation for Speaker Diarisation
 
conference paper

Audio-Visual Synchronisation for Speaker Diarisation

Garau, Giulia
•
Dielmann, Alfred
•
Bourlard, Hervé  
2010
Interspeech 2010
International Conference on Speech and Language Processing, Interspeech

The role of audio–visual speech synchrony for speaker diarisation is investigated on the multiparty meeting domain. We measured both mutual information and canonical correlation on different sets of audio and video features. As acoustic features we considered energy and MFCCs. As visual features we experimented both with motion intensity features, computed on the whole image, and Kanade Lucas Tomasi motion estimation. Thanks to KLT we decomposed the motion in its horizontal and vertical components. The vertical component was found to be more reliable for speech synchrony estimation. The mutual information between acoustic energy and KLT vertical motion of skin pixels, not only resulted in a 20% relative improvement over a MFCC only diarisation system, but also outperformed visual features such as motion intensities and head poses.

  • Details
  • Metrics
Type
conference paper
DOI
10.21437/Interspeech.2010-704
Author(s)
Garau, Giulia
Dielmann, Alfred
Bourlard, Hervé  
Date Issued

2010

Published in
Interspeech 2010
Start page

2654

End page

2657

Subjects

audio–visual speech synchrony

•

canonical correlation analysis

•

multimodal speaker diarisation

•

multiparty meetings

•

mutual information

Written at

EPFL

EPFL units
LIDIAP  
Event nameEvent place
International Conference on Speech and Language Processing, Interspeech

Makuhari, Japan

Available on Infoscience
August 26, 2010
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/52445
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés