Predicting Online Performance of News Recommender Systems Through Richer Evaluation Metrics
We investigate how metrics that can be measured offline can be used to predict the online performance of recommender systems, thus avoiding costly A-B testing. In addition to accuracy metrics, we combine diversity, coverage, and serendipity metrics to create a new performance model. Using the model, we quantify the tradeoff between different metrics and propose to use it to tune the parameters of recommender algorithms without the need for online testing. Another application for the model is a self-adjusting algorithm blend that optimizes a recommender's parameters over time. We evaluate our findings on data and experiments from news websites.
p179.pdf
openaccess
899.13 KB
Adobe PDF
d2bae00cabdb8e6039fcdf5582767f68