Bayesian reinforcement learning models reveal how great-tailed grackles improve their behavioral flexibility in serial reversal learning experiments
Environments can change suddenly and unpredictably and animals might benefit from being able to flexibly adapt their behavior through learning new associations. Serial (repeated) reversal learning experiments have long been used to investigate differences in behavioral flexibility among individuals...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Peer Community In
2024-09-01
|
| Series: | Peer Community Journal |
| Subjects: | |
| Online Access: | https://peercommunityjournal.org/articles/10.24072/pcjournal.456/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1825206400328925184 |
|---|---|
| author | Lukas, Dieter McCune, Kelsey Blaisdell, Aaron Johnson-Ulrich, Zoe MacPherson, Maggie Seitz, Benjamin Sevchik, August Logan, Corina |
| author_facet | Lukas, Dieter McCune, Kelsey Blaisdell, Aaron Johnson-Ulrich, Zoe MacPherson, Maggie Seitz, Benjamin Sevchik, August Logan, Corina |
| author_sort | Lukas, Dieter |
| collection | DOAJ |
| description | Environments can change suddenly and unpredictably and animals might benefit from being able to flexibly adapt their behavior through learning new associations. Serial (repeated) reversal learning experiments have long been used to investigate differences in behavioral flexibility among individuals and species. In these experiments, individuals initially learn that a reward is associated with a specific cue before the reward is reversed back and forth between cues, forcing individuals to reverse their learned associations. Cues are reliably associated with a reward, but the association between the reward and the cue frequently changes. Here, we apply and expand newly developed Bayesian reinforcement learning models to gain additional insights into how individuals might dynamically modulate their behavioral flexibility if they experience serial reversals. We derive mathematical predictions that, during serial reversal learning experiments, individuals will gain the most rewards if they 1) increase their *rate of updating associations* between cues and the reward to quickly change to a new option after a reversal, and 2) decrease their *sensitivity* to their learned association to explore the alternative option after a reversal. We reanalyzed reversal learning data from 19 wild-caught great-tailed grackles (Quiscalus mexicanus), eight of whom participated in serial reversal learning experiment, and found that these predictions were supported. Their estimated association-updating rate was more than twice as high at the end of the serial reversal learning experiment than at the beginning, and their estimated sensitivities to their learned associations declined by about a third. The changes in behavioral flexibility that grackles showed in their experience of the serial reversals also influenced their behavior in a subsequent experiment, where individuals with more extreme rates or sensitivities solved more options on a multi-option puzzle box. Our findings offer new insights into how individuals react to uncertainty and changes in their environment, in particular, showing how they can modulate their behavioral flexibility in response to their past experiences. |
| format | Article |
| id | doaj-art-68e5278a8f5943cd83bc0ac592fea4ee |
| institution | Kabale University |
| issn | 2804-3871 |
| language | English |
| publishDate | 2024-09-01 |
| publisher | Peer Community In |
| record_format | Article |
| series | Peer Community Journal |
| spelling | doaj-art-68e5278a8f5943cd83bc0ac592fea4ee2025-02-07T10:17:17ZengPeer Community InPeer Community Journal2804-38712024-09-01410.24072/pcjournal.45610.24072/pcjournal.456Bayesian reinforcement learning models reveal how great-tailed grackles improve their behavioral flexibility in serial reversal learning experiments Lukas, Dieter0https://orcid.org/0000-0002-7141-3545McCune, Kelsey1https://orcid.org/0000-0003-0951-0827Blaisdell, Aaron2https://orcid.org/0000-0002-6063-1010Johnson-Ulrich, Zoe3https://orcid.org/0000-0001-6000-5880MacPherson, Maggie4https://orcid.org/0000-0002-9522-5688Seitz, Benjamin5https://orcid.org/0000-0001-8046-7011Sevchik, August6Logan, Corina7https://orcid.org/0000-0002-5944-906XDepartment of Human Behavior, Ecology and Culture, Max Planck Institute for Evolutionary Anthropology, Leipzig, GermanyInstitute for Social, Behavioral and Economic Research, University of California Santa Barbara, Santa Barbara, USA; College of Forestry, Wildlife and Environment, Auburn University, Auburn, USA Department of Psychology & Brain Research Institute, University of California Los Angeles, Los Angeles, USAInstitute for Social, Behavioral and Economic Research, University of California Santa Barbara, Santa Barbara, USAInstitute for Social, Behavioral and Economic Research, University of California Santa Barbara, Santa Barbara, USADepartment of Psychology & Brain Research Institute, University of California Los Angeles, Los Angeles, USABarrett, The Honors College, Arizona State University, Phoenix, USA Department of Human Behavior, Ecology and Culture, Max Planck Institute for Evolutionary Anthropology, Leipzig, Germany; Neurosciences Research Institute, University of California Santa Barbara, Santa Barbara, USAEnvironments can change suddenly and unpredictably and animals might benefit from being able to flexibly adapt their behavior through learning new associations. Serial (repeated) reversal learning experiments have long been used to investigate differences in behavioral flexibility among individuals and species. In these experiments, individuals initially learn that a reward is associated with a specific cue before the reward is reversed back and forth between cues, forcing individuals to reverse their learned associations. Cues are reliably associated with a reward, but the association between the reward and the cue frequently changes. Here, we apply and expand newly developed Bayesian reinforcement learning models to gain additional insights into how individuals might dynamically modulate their behavioral flexibility if they experience serial reversals. We derive mathematical predictions that, during serial reversal learning experiments, individuals will gain the most rewards if they 1) increase their *rate of updating associations* between cues and the reward to quickly change to a new option after a reversal, and 2) decrease their *sensitivity* to their learned association to explore the alternative option after a reversal. We reanalyzed reversal learning data from 19 wild-caught great-tailed grackles (Quiscalus mexicanus), eight of whom participated in serial reversal learning experiment, and found that these predictions were supported. Their estimated association-updating rate was more than twice as high at the end of the serial reversal learning experiment than at the beginning, and their estimated sensitivities to their learned associations declined by about a third. The changes in behavioral flexibility that grackles showed in their experience of the serial reversals also influenced their behavior in a subsequent experiment, where individuals with more extreme rates or sensitivities solved more options on a multi-option puzzle box. Our findings offer new insights into how individuals react to uncertainty and changes in their environment, in particular, showing how they can modulate their behavioral flexibility in response to their past experiences.https://peercommunityjournal.org/articles/10.24072/pcjournal.456/Behavioral flexibility, comparative cognition, grackle, innovativeness, multi-access box, problem solving, reversal learning |
| spellingShingle | Lukas, Dieter McCune, Kelsey Blaisdell, Aaron Johnson-Ulrich, Zoe MacPherson, Maggie Seitz, Benjamin Sevchik, August Logan, Corina Bayesian reinforcement learning models reveal how great-tailed grackles improve their behavioral flexibility in serial reversal learning experiments Peer Community Journal Behavioral flexibility, comparative cognition, grackle, innovativeness, multi-access box, problem solving, reversal learning |
| title | Bayesian reinforcement learning models reveal how great-tailed grackles improve their behavioral flexibility in serial reversal learning experiments
|
| title_full | Bayesian reinforcement learning models reveal how great-tailed grackles improve their behavioral flexibility in serial reversal learning experiments
|
| title_fullStr | Bayesian reinforcement learning models reveal how great-tailed grackles improve their behavioral flexibility in serial reversal learning experiments
|
| title_full_unstemmed | Bayesian reinforcement learning models reveal how great-tailed grackles improve their behavioral flexibility in serial reversal learning experiments
|
| title_short | Bayesian reinforcement learning models reveal how great-tailed grackles improve their behavioral flexibility in serial reversal learning experiments
|
| title_sort | bayesian reinforcement learning models reveal how great tailed grackles improve their behavioral flexibility in serial reversal learning experiments |
| topic | Behavioral flexibility, comparative cognition, grackle, innovativeness, multi-access box, problem solving, reversal learning |
| url | https://peercommunityjournal.org/articles/10.24072/pcjournal.456/ |
| work_keys_str_mv | AT lukasdieter bayesianreinforcementlearningmodelsrevealhowgreattailedgracklesimprovetheirbehavioralflexibilityinserialreversallearningexperiments AT mccunekelsey bayesianreinforcementlearningmodelsrevealhowgreattailedgracklesimprovetheirbehavioralflexibilityinserialreversallearningexperiments AT blaisdellaaron bayesianreinforcementlearningmodelsrevealhowgreattailedgracklesimprovetheirbehavioralflexibilityinserialreversallearningexperiments AT johnsonulrichzoe bayesianreinforcementlearningmodelsrevealhowgreattailedgracklesimprovetheirbehavioralflexibilityinserialreversallearningexperiments AT macphersonmaggie bayesianreinforcementlearningmodelsrevealhowgreattailedgracklesimprovetheirbehavioralflexibilityinserialreversallearningexperiments AT seitzbenjamin bayesianreinforcementlearningmodelsrevealhowgreattailedgracklesimprovetheirbehavioralflexibilityinserialreversallearningexperiments AT sevchikaugust bayesianreinforcementlearningmodelsrevealhowgreattailedgracklesimprovetheirbehavioralflexibilityinserialreversallearningexperiments AT logancorina bayesianreinforcementlearningmodelsrevealhowgreattailedgracklesimprovetheirbehavioralflexibilityinserialreversallearningexperiments |