000229882 001__ 229882
000229882 005__ 20190125164339.0
000229882 037__ $$aREP_WORK
000229882 245__ $$aEvidence for eligibility traces in human learning
000229882 269__ $$a2017
000229882 260__ $$barXiv$$c2017
000229882 336__ $$aWorking Papers
000229882 520__ $$aWhether we prepare a coffee or navigate to a shop: in many tasks we make multiple decisions before reaching a goal. Learning such state-action sequences from sparse reward raises the problem of credit-assignment: which actions out of a long sequence should be reinforced? One solution provided by reinforcement learning (RL) theory is the eligibility trace (ET); a decaying memory of the state-action history. Here we investigate behaviorally and neurally whether humans utilize an ET when learning a multi-step decision making task. We implemented three versions of a novel task using visual, acoustic, and spatial cues. Eleven subjects performed all three conditions while we recorded their pupil diameter. We considered model-based and model-free (with and without ET) algorithms to explain human learning. Using the Akaike Information Criterion (AIC) we find that model-free learning with ET explains the human behavior best in all three conditions. Cross-validation confirms this behavioral result. We then compare pupil dilation in early and late learning and observe differences that are consistent with an ET contribution. In particular, we find significant changes in pupil response to non-goal states after just a single reward in all three experimental conditions. In this research we introduce a novel paradigm to study the ET in human learning in a multi-step sequential decision making task. The analysis of the behavioral and pupil data provides evidence that humans utilize an eligibility trace to solve the credit-assignment problem when learning from sparse and delayed reward.
000229882 6531_ $$aeligibility trace
000229882 6531_ $$ahuman learning
000229882 6531_ $$asequential decision making
000229882 6531_ $$apupillometry
000229882 700__ $$0248039$$g137533$$aLehmann, Marco
000229882 700__ $$0248976$$g225472$$aXu, He
000229882 700__ $$0248972$$g214589$$aLiakoni, Vasiliki
000229882 700__ $$0243629$$g164642$$aHerzog, Michael
000229882 700__ $$0240007$$g111732$$aGerstner, Wulfram
000229882 700__ $$0245595$$g218128$$aPreuschoff, Kerstin
000229882 909C0 $$0252006$$pLCN
000229882 909C0 $$0252249$$pLPSY$$xU10987
000229882 909CO $$pworking$$pSV$$pIC$$ooai:infoscience.tind.io:229882
000229882 917Z8 $$x137533
000229882 917Z8 $$x137533
000229882 937__ $$aEPFL-WORKING-229882
000229882 973__ $$aEPFL
000229882 980__ $$aWORKING