Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Near-Optimally Teaching the Crowd to Classify
 
conference paper

Near-Optimally Teaching the Crowd to Classify

Singla, Adish
•
Bogunovic, Ilija  
•
Bartok, Gabor
Show more
2014
Proceedings of The 31st International Conference on Machine Learning
The 31st International Conference on Machine Learning (ICML)

How should we present training examples to learners to teach them classification rules? This is a natural problem when training workers for crowdsourcing labeling tasks, and is also moti- vated by challenges in data-driven online educa- tion. We propose a natural stochastic model of the learners, modeling them as randomly switching among hypotheses based on observed feedback. We then develop STRICT, an efficient algorithm for selecting examples to teach to workers. Our solution greedily maximizes a submodular surro- gate objective function in order to select examples to show to the learners. We prove that our strategy is competitive with the optimal teaching policy. Moreover, for the special case of linear separators, we prove that an exponential reduction in error probability can be achieved. Our experiments on simulated workers as well as three real image annotation tasks on Amazon Mechanical Turk show the effectiveness of our teaching algorithm.

  • Files
  • Details
  • Metrics
Type
conference paper
Author(s)
Singla, Adish
Bogunovic, Ilija  
Bartok, Gabor
Karbasi, Amin
Krause, Andreas
Date Issued

2014

Published in
Proceedings of The 31st International Conference on Machine Learning
Start page

154

End page

162

URL

URL

http://jmlr.csail.mit.edu/proceedings/papers/v32/singla14.html
Editorial or Peer reviewed

REVIEWED

Written at

OTHER

EPFL units
IEL  
Event nameEvent placeEvent date
The 31st International Conference on Machine Learning (ICML)

Beijing

June 21-26, 2014

Available on Infoscience
March 3, 2015
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/112080
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés