Inferring Document Similarity from Hyper-links

Assessing semantic similarity between text documents is a crucial aspect in Information Retrieval systems. In this paper, we propose a technique to derive a similarity measure from hyper-link information. As linked documents are generally semantically closer than unlinked documents, we use a training corpus with hyper-links to infer a function $a,b \to sim(a,b)$ that assigns a higher value to linked documents than to unlinked ones. Two sets of experiments on different corpora show that this function compares favorably with {\em OKAPI} matching on document retrieval tasks.

Related material