Research Article
Full Access
Ventral–striatal/nucleus–accumbens sensitivity to prediction errors during classification learning
P. F. Rodriguez,
A.R. Aron,
R.A. Poldrack,
P. F. Rodriguez
Department of Cognitive Science, University of California, Irvine, California
Search for more papers by this authorA.R. Aron
Department of Psychology, University of California, Los Angeles, California
Search for more papers by this authorCorresponding Author
R.A. Poldrack
Department of Psychology, University of California, Los Angeles, California
Department of Psychology, Franz Hall, Box 951563, University of California, Los Angeles, CA 90065Search for more papers by this authorP. F. Rodriguez,
A.R. Aron,
R.A. Poldrack,
P. F. Rodriguez
Department of Cognitive Science, University of California, Irvine, California
Search for more papers by this authorA.R. Aron
Department of Psychology, University of California, Los Angeles, California
Search for more papers by this authorCorresponding Author
R.A. Poldrack
Department of Psychology, University of California, Los Angeles, California
Department of Psychology, Franz Hall, Box 951563, University of California, Los Angeles, CA 90065Search for more papers by this authorAbstract
A prominent theory in neuroscience suggests reward learning is driven by the discrepancy between a subject's expectation of an outcome and the actual outcome itself. Furthermore, it is postulated that midbrain dopamine neurons relay this mismatch to target regions including the ventral striatum. Using functional MRI (fMRI), we tested striatal responses to prediction errors for probabilistic classification learning with purely cognitive feedback. We used a version of the Rescorla-Wagner model to generate prediction errors for each subject and then entered these in a parametric analysis of fMRI activity. Activation in ventral striatum/nucleus-accumbens (Nacc) increased parametrically with prediction error for negative feedback. This result extends recent neuroimaging findings in reward learning by showing that learning with cognitive feedback also depends on the same circuitry and dopaminergic signaling mechanisms. Hum Brain Mapp, 2005. © 2005 Wiley-Liss, Inc.
REFERENCES
- Aron AR, Shohamy D, Clark J, Myers C, Gluck MA, Poldrack RA (2004): Human midbrain sensitivity to cognitive feedback and uncertainty during classification learning. J Neurophysiol 92: 1144–1152. Epub 2004 Mar 10.
- Becerra L, Breiter HC, Wise R, Gonzalez RG, Borsook D (2001): Reward circuitry activation by noxious thermal stimuli. Neuron 32: 927–946.
- Beninger RJ, Wasserman J, Zanibbi K, Charbonneau D, Mangels J, Beninger BV (2003): Typical and atypical antipsychotic medications differentially affect two nondeclarative memory tasks in schizophrenic patients: a double dissociation. Schizophr Res 61: 281–292.
- Berns GS, McClure SM, Pagnoni G, Montague PR (2001): Predictability modulates human brain response to reward. J Neurosci 21: 2793–2798.
- Breiter HC, Aharon I, Kahneman D, Dale A, Shizgal P (2001): Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30: 619–639.
- Cardinal RN, Pennicott DR, Sugathapala CL, Robbins TW, Everitt BJ (2001): Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science 292: 2499–2501. Epub 2001 May 24.
- Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA (2000): Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84: 3072–3077.
- Duvernoy HM (1999): The human brain: surface, blood supply, three-dimensional sectional anatomy. New York: Springer.
- Elliott R, Friston KJ, Dolan RJ (2000): Dissociable neural responses in human reward systems. J Neurosci 20: 6159–6165.
- Fletcher PC, Anderson JM, Shanks DR, Honey R, Carpenter TA, Donovan T, Papadakis N, Bullmore ET (2001): Responses of human frontal cortex to surprising events are predicted by formal associative learning theory. Nat Neurosci 4: 1043–1048.
- Gluck MA (1991): Stimulus-generalization and representation in adaptive network models of category learning. Psychol Sci 2: 50–55.
- Gluck MA, Bower GH (1988): Evaluating an adaptive network model of human learning. J Mem Lang 27: 166–195.
- Haruno M, Kuroda T, Doya K, Toyama K, Kimura M, Samejima K, Imamizu H, Kawato M (2004): A neural correlate of reward-based behavioral learning in caudate nucleus: a functional magnetic resonance imaging study of a stochastic decision task. J Neurosci 24: 1660–1665.
- Hollerman JR, Schultz W (1998): Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1: 304–309.
- Holroyd CB, Coles MG (2002): The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol Rev 109: 679–709.
- Horvitz JC (2000): Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96: 651–656.
- Jensen J, McIntosh AR, Crawley AP, Mikulis DJ, Remington G, Kapur S (2003): Direct activation of the ventral striatum in anticipation of aversive stimuli. Neuron 40: 1251–1257.
- Knowlton BJ, Squire LR, Gluck MA (1994): Probabilistic category learning in amnesia. Learn Mem 1: 106–120.
- Knowlton BJ, Mangels JA, Squire LR (1996): A neostriatal habit learning system in humans. Science 273: 1399–1402.
- Knutson B, Adams CM, Fong GW, Hommer D (2001a): Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci 21: RC159.
- Knutson B, Fong GW, Adams CM, Varner JL, Hommer D (2001b): Dissociation of reward anticipation and outcome with event-related fMRI. Neuroreport 12: 3683–3687.
- Logothetis N (2003): The underpinnings of the BOLD functional magnetic resonance imaging signal. J Neurosci 23: 3963–3971.
- McClure SM, Berns GS, Montague PR (2003): Temporal prediction errors in a passive learning task activate human striatum. Neuron 38: 339–346.
- O'Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ (2003): Temporal difference models and reward-related learning in the human brain. Neuron 38: 329–337.
- O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004): Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304: 452–454.
- Pagnoni G, Zink CF, Montague PR, Berns GS (2002): Activity in human ventral striatum locked to errors of reward prediction. Nat Neurosci 5: 97–98.
- Poldrack RA, Clark J, Pare-Blagoev EJ, Shohamy D, Moyano JC, Myers C, Gluck MA (2001): Interactive memory systems in the human brain. Nature 414: 546–550.
- Rescorla R, Wagner A (1972): A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: A Black, W Prokasy, editors. Classical conditioning II: current research and theory. New York: Appleton Century Crofts. p 64–99.
- Schoenbaum G, Setlow B (2003): Lesions of nucleus accumbens disrupt learning about aversive outcomes. J Neurosci 23: 9833–9841.
- Schultz W (2002): Getting formal with dopamine and reward. Neuron 36: 241–263.
- Schultz W, Apicella P, Scarnati E, Ljungberg T (1992): Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12: 4595–4610.
- Setlow B, Schoenbaum G, Gallagher M (2003): Neural encoding in ventral striatum during olfactory discrimination learning. Neuron 38: 625–636.
- Seymour B, O'Doherty JP, Dayan P, Koltzenburg M, Jones AK, Dolan RJ, Friston KJ, Frackowiak RS (2004): Temporal difference models describe higher-order learning in humans. Nature 429: 664–667.
- Shohamy D, Myers CE, Grossman S, Sage J, Gluck MA, Poldrack RA (2004): Cortico-striatal contributions to feedback-based learning: converging data from neuroimaging and neuropsychology. Brain 127: 851–859. Epub 2004 Mar 10.
- Tanaka SC, Doya K, Okada G, Ueda K, Okamoto Y, Yamawaki S (2004): Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat Neurosci 7: 887–893. Epub 2004 Jul 4.
- Tricomi EM, Delgado MR, Fiez JA (2004): Modulation of caudate activity by action contingency. Neuron 41: 281–292.
- Ullsperger M, von Cramon DY (2003): Error monitoring using external feedback: specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging. J Neurosci 23: 4308–4314.
- Ungless MA, Magill PJ, Bolam JP (2004): Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 303: 2040–2042.
- Volz KG, Schubotz RI, Von Cramon DY (2003): Predicting events of varying probability: uncertainty investigated by fMRI. Neuroimage 19: 271–280.
- Zink CF, Pagnoni G, Martin ME, Dhamala M, Berns GS (2003): Human striatal response to salient nonrewarding stimuli. J Neurosci 23: 8092–8097.
- Zink CF, Pagnoni G, Martin-Skurski ME, Chappelow JC, Berns GS (2004): Human striatal responses to monetary reward depend on saliency. Neuron 42: 509–517.
