Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. Write What YouWant: Applying Text-to-Video Retrieval to Audiovisual Archives
 
research article

Write What YouWant: Applying Text-to-Video Retrieval to Audiovisual Archives

Yang, Yuchen  
December 1, 2023
Acm Journal On Computing And Cultural Heritage

Audiovisual (AV) archives, as an essential reservoir of our cultural assets, are suffering from the issue of accessibility. The complex nature of the medium itself made processing and interaction an open challenge still in the field of computer vision, multimodal learning, and human-computer interaction, as well as in culture and heritage. In recent years, with the raising of video retrieval tasks, methods in retrieving video content with natural language (text-to-video retrieval) gained quite some attention and have reached a performance level where real-world application is on the horizon. Appealing as it may sound, such methods focus on retrieving videos using plain visual-focused descriptions of what has happened in the video and finding videos such as instructions. It is too early to say such methods would be the new paradigms for accessing and encoding complex video content into high-dimensional data, but they are indeed innovative attempts and foundations to build future exploratory interfaces for AV archives (e.g., allow users to write stories and retrieve related snippets in the archive, or encoding video content at high-level for visualisation). This work filled the application gap by examining such text-tovideo retrieval methods from an implementation point of view and proposed and verified a classifier-enhanced workflow to allow better results when dealing with in-situ queries that might have been different from the training dataset. Such a workflow is then applied to the real-world archive from Television Suisse Romande (RTS) to create a demo. At last, a humancentred evaluation is conducted to understand whether the text-to-video retrieval methods improve the overall experience of accessing AV archives.

  • Details
  • Metrics
Type
research article
DOI
10.1145/3627167
Web of Science ID

WOS:001153007700019

Author(s)
Yang, Yuchen  
Date Issued

2023-12-01

Publisher

Assoc Computing Machinery

Published in
Acm Journal On Computing And Cultural Heritage
Volume

16

Issue

4

Start page

81

Subjects

Technology

•

Audiovisual Archive

•

Computational Archival Science

•

Experimental Museology

•

Text-Tovideo Retrieval

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
EMPLUS  
Available on Infoscience
February 23, 2024
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/205424
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés