Preprints/Submitted/In press

RL or not RL? Parsing the processes that support human reward-based learning. [PsyArXiv]
Collins (preprint)

Latent Learning Progress Drives Autonomous Goal Selection in Human Reinforcement Learning [PsyArXiv]
Molinaro, Colas, Oudeyer and Collins (preprint)

Latent Variable Sequence Identification for Cognitive Models with Neural Bayes Estimation [arXiv]
Pan, Li, Thompson and Collins (preprint)

Dual Effects of Dual-Tasking on Instrumental Learning [PsyArXiv]
Ham, McDougle and Collins (preprint)

Episodic memory contributions to working memory-supported reinforcement learning [PsyArXiv]
Hong, Zou, Yoo and Collins (preprint)

Nucleus accumbens dopamine release reflects Bayesian inference during instrumental learning [Biorxiv]
Qu, Tai, Hall, Tu, Eckstein, Mischanchuk, Lin, Chase, MacAskill, Collins, Gershman, and Wilbrecht (preprint)

Strategic Processes in Sensorimotor Learning: Reasoning, Refinement, and Retrieval [PsyArXiv]
Tsay, Kim, McDougle, Taylor, Haith, Avraham, Krakauer, Collins, and Ivry (preprint)

Published journal articles

Reinforcement Learning [OECS]
Anne GE Collins (2024)
Open Encyclopedia of Cognitive Science

An algorithmic account for how humans efficiently learn, transfer, and compose hierarchically structured decision policies [Cognition] [Data]
Li and Collins (2024)
Cognition

Artificial neural networks for model identification and parameter estimation in computational cognitive models [PLoS]
Rmus, Pan, Xia and Collins (2024) PLoS Computational Biology

Dynamic noise estimation: A generalized method for modeling noise fluctuations in decision-making [PDF]
[Data]
Li Jing-Jing, Shi, Li Lexin, and Collins (2024)
Journal of Mathematical Psychology

A goal-centric outlook on learning [TCS]
Molinaro and Collins (2023)
Trends in Cognitive Sciences

Lowered inter-stimulus discriminability hurts incremental contributions to learning [Cognitive, Affective, & Behavioral Neuroscience] [Data]
Yoo, Keglovits, and Collins (2023)
Cognitive, Affective, & Behavioral Neuroscience

A generalized method for dynamic noise inference in modeling sequential decision-making [PDF]
Li Jing-Jing, Shi, Li Lexin, and Collins (2023)
Proceedings of the 45th Annual Conference of the Cognitive Science Society

Human hacks and bugs in the recruitment of reward systems for goal achievement [PDF]
Molinaro and Collins (2023)
Proceedings of the 45th Annual Conference of the Cognitive Science Society

Intrinsic rewards explain context-sensitive valuation in reinforcement learning [PLOS][Data]
Molinaro and Collins (2023)
PLOS Biology

Age-related differences in prefrontal glutamate are associated with increased working memory decay that gives the appearance of learning deficits [elife]
Rmus, He, Baribault, Walsh, Festa, Collins & Nassar (2023)
eLife

Neural index of reinforcement learning predicts improved stimulus-response retention under high working memory load [PDF]
Rac-Lubashevsky, Cremer, Collins, Frank, and Schwabe (2023)
Journal of Neuroscience

Troubleshooting Bayesian cognitive models [PDF]
Baribault and Collins (2023)
Psychological Methods

Choice Type Impacts Human Reinforcement Learning [PDF] [Data]
Rmus, Zou, Collins (2023)
Journal of Cognitive Neuroscience

The interpretation of computational model parameters depends on the context [PDF] [Data]
Eckstein, Master, Xia, Dahl, Wilbrecht, and Collins (2022)
eLife

Reinforcement Learning and Bayesian Inference Provide Complementary Models for the Unique Advantage of Adolescents in Stochastic Reversal [PDF] [Data]
Eckstein, Master, Dahl, Wilbrecht, and Collins (2022)
Developmental Cognitive Neuroscience

Activation, but not inhibition, of the indirect pathway disrupts choice rejection in a freely moving, multiple-choice foraging task [PDF]
Delevich, Hoshal, Zhou, Zhang, Vedula, Lin, Chase, Collins, Wilbrecht (2022)
Cell Reports

Credit assignment in hierarchical option transfer [PDF]
Li, Xia, Dong, and Collins (2022)
Proceedings of Cog Sci 2022

Three systems interact in one-shot reinforcement learning [PDF]
Zou and Collins (2022)
Proceedings of Cog Sci 2022

Impulsivity relates to multi-trial choice strategy in probabilistic reversal learning [PDF] [Data]
Zou, Muñoz Lopez, Johnson, and Collins (2022)
Frontiers in Psychiatry

Event segmentation reveals working memory forgetting rate [PDF] [RLWM Data]
Jafarpour, Buffalo, Knight, and Collins (2022)
iScience

How working memory and reinforcement learning are intertwined: a cognitive, neural, and computational perspective [PDF]
Yoo and Collins (2022)
Journal of Cognitive Neuroscience

Advances in modeling learning and decision-making in neuroscience [PDF]
Collins and Shenhav (2021)
Neuropsychopharmacology

How the Mind Creates Structure: Hierarchical Learning of Action Sequences [PDF]
Eckstein and Collins (2021)
Proceedings of Cog Sci 2021

Modeling Changes in Probabilistic Reinforcement Learning during Adolescence [PDF] [Data]
Xia, Master, Eckstein, Baribault, Dahl, Wilbrecht, Collins (2021)
PLoS Computational Biology

Context is key for learning motor skills [PDF]
Collins and McDougle (2021)
Nature

Executive Function Assigns Value to Novel Goal-Congruent Outcomes [PDF]
McDougle, Ballard, Baribault, Bishop, and Collins (2021)
Cerebral Cortex

What do reinforcement learning models measure? Interpreting model parameters in cognition and neuroscience [PDF]
Eckstein, Wilbrecht, and Collins (2021)
Current Opinion in Behavioral Sciences

Proof-of-Mechanism Study of the PDE10 Inhibitor RG7203 in Patients with Schizophrenia and Negative Symptoms [PDF]
Umbricht D, Abt M, Tamburri P, Chatham C, Holiga Š, Frank MJ, Collins AGE, Walling DP, Mofsen R, Gruener D, Gertsik L, Sevigny J, Keswani S & Dukart J (2021)
Biological Psychiatry Global Open Science

Temporal and state abstractions for efficient learning, transfer and composition in humans [PDF] [SI] [Data]
Xia and Collins (2021)
Psychological Review

Dopaminergic activity and exercise behavior in anorexia nervosa [PDF]
Gorrell, Collins, Le Grange, Yang (2020)
OBM Neurobiol

Beyond simple dichotomies in reinforcement learning. [PDF]
Collins and Cockburn (2020)
Nature Reviews Neuroscience

Modeling the influence of working memory, reinforcement, and action uncertainty on reaction time and choice during instrumental learning [PDF]
McDougle and Collins (2020)
Psychonomic Bulletin & Review

The Role of Executive Function in Shaping Reinforcement Learning [PDF]
Rmus, McDougle, and Collins (2020)
Current Opinion in Behavioral Sciences

Learning under uncertainty changes during adolescence [PDF]
Xia, Master, Eckstein, Wilbrecht, and Collins (2020)
Proceedings of Cog Sci 2020

What is a Choice in Reinforcement Learning? [PDF]
Rmus and Collins (2020)
Proceedings of Cog Sci 2020

Computational Evidence for Hierarchically-Structured Reinforcement Learning in Humans [PDF] [Data]
Eckstein and Collins (2020)
PNAS

Computational Mechanisms of Effort and Reward Decisions in Patients With Depression and Their Association With Relapse After Antidepressant Discontinuation [PDF]
Berwian, Wenzel, Collins, Seifritz, Stephan, Walter, Huys (2020)
JAMA Psychiatry

Disentangling the systems contributing to changes in learning during adolescence [PDF]
Master, Eckstein, Gotlieb, Dahl, Wilbrecht, and Collins (2020)
Developmental Cognitive Neuroscience

Ten simple rules for the computational modeling of behavioral data [PDF] [example code]
Wilson and Collins (2019)
eLIFE

Reinforcement learning: bringing together computation and cognition [PDF]
Collins (2019)
Current Opinion in Behavioral Sciences

Cross-Task Contributions of Frontobasal Ganglia Circuitry in Response Inhibition and Conflict-Induced Slowing [PDF]
Jahfari, Ridderinkhof, Collins, Knapen, Waldorp, Frank (2019)
Cerebral Cortex

Chapter 5: Learning Structures Through Reinforcement [PDF]
Collins, AGE (2018)
Goal-Directed Decision Making

Sequential control underlies robust ramping dynamics in the rostrolateral prefrontal cortex [PDF]
Desrochers, Collins, and Badre (2018)
Journal of Neuroscience

The tortoise and the hare: interactions between reinforcement learning and working memory [PDF] [summary]
Collins, AGE (2018)
Journal of Cognitive Neuroscience

Within and across-trial dynamics of human EEG reveal cooperative interplay between reinforcement learning and working memory [PDF] [Data]
Collins, AGE and Frank, MJ (2018)
PNAS

Interactions Among Working Memory, Reinforcement Learning, and Effort in Value-Based Choice: A New Paradigm and Selective Deficits in Schizophrenia [PDF]
Collins, AGE, Albrecht, MA, Waltz, JA, Gold, JM and Frank, MJ (2017)
Biological Psychiatry

Prefrontal Cortex in Control: Broadening the Scope to Identify Mechanisms [PDF]
Alexander, WH, Brown, JW, Collins, AGE, Hayden, BY, and Vassena, E (2017)
Journal of Cognitive Neuroscience

Working memory load strengthens reward prediction errors [PDF] [SI] [Data]
Collins, AGE, Ciullo, B, Frank, MJ, and Badre, D. (2017)
Journal of Neuroscience

The cost of structure learning [PDF]
Collins, AGE (2017)
Journal of Cognitive Neuroscience

Stimulus discriminability may bias value-based probabilistic learning [PDF]
Schutte, I, Slagter, HA, Collins, AGE, Frank, MJ, and Kenemans, JL (2017)
PLoS One

Role of Prefrontal Cortex in Learning and Generalizing Hierarchical Rules in 8-Month-Old Infants [PDF]
Werchan, DM, Collins, AGE, Frank, MJ, and Amso, D (2016)
The Journal of Neuroscience

Probabilistic Reinforcement Learning in Patients with Schizophrenia: Relationships to Anhedonia and Avolition [PDF]
Dowd, EC, Frank, MJ, Collins, AGE, Gold, JM, and Barch, DM (2016)
Biological Psychiatry

Neural signature of hierarchically structured expectations predicts clustering and transfer of rule sets in reinforcement learning [PDF]
Collins, AGE, Frank, MJ (2016)
Cognition

Motor demands constrain cognitive rule structures [PDF]
Collins, AGE, and Frank, MJ (2016)
Plos Computational Biology

Surprise! dopamine signals mix action, value and error [PDF]
Collins, AGE, and Frank, MJ (2015)
Nature Neuroscience

8-Month-Old infants spontaneously learn and generalize hierarchical rules [PDF]
Werchan, DM, Collins, AGE, Frank, MJ, and Amso, D (2015)
Psychological Science

Working memory contributions to reinforcement learning impairments in schizophrenia [PDF] [Data]
Collins, AGE, Brown, J, Gold, J, Waltz, J, and Frank, MJ (2014)
Journal of Neuroscience

A reinforcement learning mechanism responsible for the valuation of free choice [PDF]
Cockburn, J, Collins, AGE, and Frank, MJ (2014)
Neuron

Opponent Actor Learning (OpAL): modeling interactive effect of striatal dopamine on reinforcement learning and choice incentive [PDF]
Collins, AGE and Frank, MJ. (2014)
Psychological Review

Foundations of human reasoning in the prefrontal cortex [PDF]
Donoso, M, Collins, AGE, Koechlin, E. (2014)
Science

Human EEG uncovers latent generalizable rule structure during learning [PDF]
Collins, AGE, Cavanagh, JF, and Frank, MJ (2014)
Journal of Neuroscience

Cognitive control over learning: creating, clustering and generalizing task-set structure [PDF]
Collins, AGE and Frank, MJ. (2013)
Psychological Review

How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational and neurogenetic analysis [PDF] [Data]
Collins, AGE and Frank, MJ. (2012)
European Journal of Neuroscience

Negative symptoms and the failure to represent the expected reward value of actions: behavioral and computational modeling evidence [PDF] [Data]
Gold, JM; Waltz, JA; Matveeva, TM; Kasanova, Z; Strauss, G; Herbener, E; Collins, AGE; Frank, MJ (2012)
Arch Gen Psychiatry

Reasoning, learning, and creativity: frontal lobe function and human decision-making [PDF]
Collins A, Koechlin E (2012)
PLoS Biology

A computational theory of prefrontal executive control
Collins, A. and Koechlin, E. (2009)
Frontiers in Systems Neuroscience

Computational models and code

This is a link to the neural network model simulated in Collins & Frank, 2013, Psychological Review. It is a model of task-set structure learning in hierarchical corticostriatal circuits. It runs on the emergent neural simulator (Aisa et al., 2008). Details about emergent and previous neural networks on which this network builds can be found on Michael Frank’s basal ganglia projects webpage, here.

This network includes a two-stage cascaded basal ganglia loop circuit enabling hierarchical control of action selection and learning by generating task-set structure, generalizable to novel situations. The model selects among four different motor actions, and at the higher level, three possible task-sets, and simultaneously learns to create (or re-use) abstract task-sets while also learning the particular response mappings given the selected task-set, using pure reinforcement learning.

This matlab script can be used for more detailed analysis of model output showing transfer, and here is an example mat file. Similarly, for more detailed analysis of a case in which there is incentive to clustering task-sets around context during initial learning, please use this matlab script.

The computations of this model were linked to those of a higher level “C-TS” (context task-set) model based on a non-parametric Bayesian approach to clustering task-sets using a Chinese Restaurant Process. Here is a single zip file including simulations from the C-TS model in matlab.