Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Associating Audio-Visual Activity Cues in a Dominance Estimation Framework
 
Loading...
Thumbnail Image
conference paper not in proceedings

Associating Audio-Visual Activity Cues in a Dominance Estimation Framework

Hung, Hayley
•
Huang, Yan
•
Yeo, Chuohao
Show more
2008
First IEEE Workshop on CVPR for Human Communicative Behavior Analysis

We address the problem of both estimating the dominant person in a meeting from a single audio source and identifying them visually in a multi-camera setting. We use a speaker diarization algorithm to perform speaker segmentation and clustering, representing when they spoke. Using a greedy ordered audio-visual association algorithm, we investigate using the speaker clusters to find the corresponding person in one of the video channels. The difficulty of the problem is that firstly the speaker diarization output is noisy (e.g. for participants who speak little) and often produces an unequal number of clusters to true participants. Secondly, personal visual activity from natural upper torso motion, which can include highly deformable pose changes and perspective distortion, is computed through computationally efficient coarse features. Our results using almost 2 hours of audio-visual data from 4-participant meetings show a strong correlation between the estimated speaker diarization and visual activity features, enabling the identification of the most dominant person as a pair of audio-visual channels.

  • Files
  • Details
  • Metrics
Loading...
Thumbnail Image
Name

Hung_CVPR2008_2008.pdf

Access type

openaccess

Size

210.01 KB

Format

Adobe PDF

Checksum (MD5)

681f341a3c6b4f51f98fbcf37a03b62e

Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés