Offline and online evaluation of news recommender systems at

We report on the live evaluation of various news recom- mender systems conducted on the website We demonstrate that there is a major diffierence between offine and online accuracy evaluations. In an offine setting, rec- ommending most popular stories is the best strategy, while in a live environment this strategy is the poorest. For online setting, context-tree recommender systems which profile the users in real-time improve the click-through rate by up to 35%. The visit length also increases by a factor of 2.5. Our experience holds important lessons for the evaluation of rec- ommender systems with offine data as well as for the use of the click-through rate as a performance indicator. Copyright © 2014 ACM.

Published in:
Proceedings of the 8th ACM Conference on Recommender systems - RecSys '14, 169-176
Presented at:
the 8th ACM Conference, Foster City, Silicon Valley, California, USA, 06-10 October 2014
New York, New York, USA, ACM Press

 Record created 2015-08-07, last modified 2018-09-13

Download fulltext

Rate this document:

Rate this document:
(Not yet reviewed)