Nguyen, Quoc Viet HungHuynh, Huu VietNguyen, Thanh TamWeidlich, MatthiasYin, HongzhiZhou, Xiaofang2018-02-212018-02-212018-02-21201810.1109/ICDE.2018.00232https://infoscience.epfl.ch/handle/20.500.14299/144947WOS:000492836500224Crowdsourcing has been widely established as a means to enable human computation at large-scale, in particular for tasks that require manual labelling of large sets of data items. Answers obtained from heterogeneous crowd workers are aggregated to obtain a robust result. However, existing methods for answer aggregation are designed for \emph{discrete} tasks, where answers are given as a single label per item. In this paper, we consider \emph{partial-agreement} tasks that are common in many applications such as image tagging and document annotation, where items are assigned sets of labels. Going beyond the state-of-the-art, we propose a novel Bayesian nonparametric model to aggregate the partial-agreement answers in a generic way. This model enables us to compute the consensus of partially-sound and partially-complete worker answers, while taking into account mutual relations in labels and different answer sets. An evaluation of our method using real-world datasets reveals that it consistently outperforms the state-of-the-art in terms of precision, recall, and scalability.Computer Science, Information SystemsComputer Science, Theory & MethodsCrowdsourcingNonparametric ModelsBayesian ModelsAnswer Aggregationtext::conference output::conference proceedings::conference paper