Complex behavior from intrinsic motivation to occupy future action-state path space

Mostra el registre complet Registre parcial de l'ítem

• dc.contributor.author Ramirez-Ruiz, Jorge
• dc.contributor.author Grytskyy, Dmytro
• dc.contributor.author Mastrogiuseppe, Chiara
• dc.contributor.author Habib, Yamen
• dc.contributor.author Moreno-Bote, Ruben
• dc.date.accessioned 2025-06-16T09:18:39Z
• dc.date.available 2025-06-16T09:18:39Z
• dc.date.issued 2024
• dc.description.abstract Most theories of behavior posit that agents tend to maximize some form of reward or utility. However, animals very often move with curiosity and seem to be motivated in a reward-free manner. Here we abandon the idea of reward maximization and propose that the goal of behavior is maximizing occupancy of future paths of actions and states. According to this maximum occupancy principle, rewards are the means to occupy path space, not the goal per se; goal-directedness simply emerges as rational ways of searching for resources so that movement, understood amply, never ends. We find that action-state path entropy is the only measure consistent with additivity and other intuitive properties of expected future action-state path occupancy. We provide analytical expressions that relate the optimal policy and state-value function and prove convergence of our value iteration algorithm. Using discrete and continuous state tasks, including a high-dimensional controller, we show that complex behaviors such as “dancing”, hide-and-seek, and a basic form of altruistic behavior naturally result from the intrinsic motivation to occupy path space. All in all, we present a theory of behavior that generates both variability and goal-directedness in the absence of reward maximization.
• dc.description.sponsorship This work is supported by the Howard Hughes Medical Institute (HHMI, ref 55008742), ICREA Academia 2022 and MINECO (Spain; BFU2017-85936-P) to R.M.-B., MINECO/ESF (Spain; PRE2018-084757) to J.R.-R, and AGAUR-FI ajuts from Generalitat de Catalunya/ESF (2024 FI-B3 00020) to C.M and (2023 FI-1 00245) to Y.H.
• dc.format.mimetype application/pdf
• dc.identifier.citation Ramírez-Ruiz J, Grytskyy D, Mastrogiuseppe C, Habib Y, Moreno-Bote R. Complex behavior from intrinsic motivation to occupy future action-state path space. Nat Commun. 2024;15:6368. DOI: 10.1038/s41467-024-49711-1
• dc.identifier.doi http://dx.doi.org/10.1038/s41467-024-49711-1
• dc.identifier.issn 2041-1723
• dc.identifier.uri http://hdl.handle.net/10230/70695
• dc.language.iso eng
• dc.publisher Nature Research
• dc.relation.ispartof Nature Communications. 2024;15:6368
• dc.relation.projectID info:eu-repo/grantAgreement/ES/1PE/BFU2017-85936-P
• dc.rights © The Author(s) 2024. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
• dc.rights.accessRights info:eu-repo/semantics/openAccess
• dc.rights.uri http://creativecommons.org/licenses/by/4.0/
• dc.title Complex behavior from intrinsic motivation to occupy future action-state path space
• dc.type info:eu-repo/semantics/article
• dc.type.version info:eu-repo/semantics/publishedVersion