A neuro-computational model showing the effects of ventral striatum lesion on the computation of reward prediction error in VTA

Pramod S Kaushik (1) , Frédéric Alexandre (1)

1 Mnemosyne - Mnemonic Synergy

One of the earliest attempts to understand how animals learn involved pairing an unconditioned stimulus (US) with a cue or conditioned stimulus (CS) and observing that animals start responding to the CS after some point in time. This is the basis of pavlovian learning, a fundamental learning mechanism in animals, which has been addressed by several models of neural networks. We have developed a model focusing on the mechanism of reward prediction error within pavlovian learning and studied the effects of Ventral Striatum (VS) lesions to illustrate a fundamental dissociation of magnitude and timing replicating experimental studies. The paradigm used to evaluate the model is a simple CS-US associative learning task and considers also how the expectation cancels out the dopamine peak at the time of the reward. The trial duration is 500 time steps with each time step corresponding to 1ms. The stimulus is presented at the 10th time step and is kept switched on till the arrival of the reward at the 400th time step (400ms). The reward and the stimulus have by default a magnitude of 1. The number of trials for the entire conditioning to happen was 16 trials. Virtual lesions of VS to VTA GABA was made by disconnecting the link between them. Two experiments were conducted where the time and magnitude were varied. In the first experiment, a reward magnitude of 2 is given instead of 1. The reward prediction error in VTA still shows a firing of 1 indicating the magnitude is conserved even after the VS lesion. In the second experiment, an early reward is delivered which doesn’t show firing compared to the control scenario where firing exists thus replicating the studies done by Takahashi(2016). The results show that there exists fundamental dissociation of magnitude and temporal when calculating reward prediction error (RPE) in the VTA. We propose this is achieved through magnitude being computed in the PPN FT neurons and time being computed in the ventral striatum (VS) respectively. The implications of this model bring into light new interpretations of dopamine firing extending to state prediction errors and creating a sensory representation before learning the reward fully.

Domaines: Réseau de neurones [cs.NE]

Pramod S Kaushik, Frédéric Alexandre. A neuro-computational model showing the effects of ventral striatum lesion on the computation of reward prediction error in VTA. NeuroFrance, the international conference of the french society of Neuroscience, May 2019, Marseille, France. 2019. ⟨hal-02388198⟩