Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Student works
  4. Semi-supervised Clustering of Visual Signatures of Artworks
 
master thesis

Semi-supervised Clustering of Visual Signatures of Artworks

Schaerf, Ludovica
July 1, 2022

The project Replica, about six years ago, paved the way for computational studies of visual patterns in art history. Simultaneously, it created the possibility for art historians to trace the propagation of patterns throughout the history of art. During the project, an image retrieval network was set up to discover artistic patterns given an input image. Despite successfully serving monographic needs and targeted search attempts, the network does not propose spontaneous discoveries. In this thesis, we eliminate the middleman of the input image, creating clusters of artworks sharing a common pattern propagation. The clusters are integrated further with a 2D coordinate-based visualisation, which provides an organic view of the evolution of the patterns in art history. In this effort, we demonstrate the effectiveness of fine-tuning deep learning models on a set of visual connections using a compound Hinge loss and ResNeXt architecture. Moreover, we show that clustering the trained visual signatures with OPTICS yields remarkable precision. We emphasise the importance of the semi-supervised learning of the clusters, proving the qualitative and quantitative improvement over generic clustering methods. Furthermore, we close the loop of the semi-supervised clustering through the annotation of the new findings in the clusters proposed, and retraining thereof. In total, we add over 700 new images to the set of slightly over 1800 existing visual connections. We find, in addition, examples of cross-domain, architectural, design and sketch-based patterns, which were previously outside the scope of the known visual connections.

  • Details
  • Metrics
Type
master thesis
Author(s)
Schaerf, Ludovica
Advisors
Kaplan, Frédéric  
•
Guhennec, Paul  
Date Issued

2022-07-01

Subjects

visual patterns

•

semi-supervised

•

clustering

•

triplet learning

•

digital art history

RelationURL/DOI

HasMetadata

https://infoscience.epfl.ch/record/261212?ln=en

Cites

https://infoscience.epfl.ch/record/220638?ln=en

Cites

https://infoscience.epfl.ch/record/223771?ln=en
Available on Infoscience
July 3, 2022
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/188817
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés