Neural circuit mechanisms of dynamic learning rates

About This Grant

Project Summary Biological accounts of reinforcement learning posit that dopamine encodes reward prediction errors (RPEs), which are multiplied by a learning rate to update state or action values. The learning rate is often assumed to be constant, but studies in humans, monkeys, rats, and mice, have found behavioral evidence for dynamic learning rates. In volatile environments, dynamic learning rates allow animals to learn faster when the world is changing, and more slowly when the world is stable. While dopamine is thought to instantiate RPEs, we recently found that dopamine release in the ventral striatum did not reflect learning rates, suggesting that dopamine-independent mechanisms determine the rate of error-driven learning. Moreover, we present strong preliminary data showing that inactivation of the orbitofrontal cortex (OFC) eliminates dynamic learning rates behaviorally, and that OFC neurons that project to the ventral striatum seem to encode the learning rate in their firing rates. In this proposal, we will determine how OFC projections to the ventral striatum dictate the rate of error-driven learning at behavioral and neural levels. This proposal will use a novel behavioral paradigm in rats, in which reward statistics vary over latent blocks of trials. We previously found strong behavioral signatures of dynamic learning rates in rats performing this task. High-throughput behavioral training will generate dozens of trained subjects for experiments in parallel, accelerating the rate of research progress. We will use optogenetics and electrophysiology to record from and manipulate OFC neurons that project to the ventral striatum, to determine if this projection pathway dictates behavioral learning rates (Aim 1). We will use electrophysiology and optogenetics to relate behavioral learning rates and activation of OFC neurons that project to the ventral striatum to trial-by-trial changes in evoked spiking in the striatum (Aim 2). We will use optical methods to measure dopamine release in the striatum and activation of OFC axon terminals, while simultaneously recording action potentials from the ventral striatum, to relate endogenous fluctuations in coincident dopamine and OFC inputs to trial-by-trial plasticity of evoked spiking (Aim 3). These experiments will test key predictions of “three-factor” plasticity rules in behaving animals. These experiments will address a major open question, which is how specific output pathways from OFC interact with downstream circuits to coordinate value-based decisions and learning. Neuromodulatory systems including dopamine are implicated in myriad neuropsychiatric disorders including schizophrenia and depression. A greater understanding of the circuit mechanisms by which they coordinate different aspects of behavior and interact holds promise for revealing novel therapeutic targets for these disorders.