Scrutinizing the predictive power of large language models for brain function
Mostra el registre complet Registre parcial de l'ítem
- dc.contributor.author Yang, Ni
- dc.date.accessioned 2025-03-07T16:30:50Z
- dc.date.available 2025-03-07T16:30:50Z
- dc.date.issued 2024
- dc.description Treball de fi de màster en Lingüística Teòrica i Aplicada. Director: Dr. Wolfram Hinzen
- dc.description.abstract Based on predictive coding and hierarchical processing as a commonality between large language models (LLMs) and the brain, many studies have linked the two by regressing brain activity on LLMs’ representations. However, increasing evidence has revealed problems in this new line of research. To address this issue, we attempted to replicate a pioneering study (Kumar et al., 2022) on an independent fMRI dataset with several methodological adaptations. Results showed overall low correlation scores and sparse predictions across the cortex. Contrary to the reference study, representation’s performances across most ROIs did not differ significantly. However, in areas where significant differences were observed, fastText consistently outperformed BERT. Additionally, the layer-wise performance of embeddings and transformations showed no consistent patterns. Our findings challenge the existint assumptions regarding the predictive power of LLMs for brain function and highlight potential issues in the current methodologies to map predictions from LLMs onto brain activations.
- dc.identifier.uri http://hdl.handle.net/10230/69856
- dc.language.iso eng
- dc.rights Llicència Creative Commons, Reconeixement-NoComercial-SenseObraDerivada 4.0 Internacional
- dc.rights.accessRights info:eu-repo/semantics/openAccess
- dc.rights.uri https://creativecommons.org/licenses/by-nc-nd/4.0/deed.ca
- dc.subject.keyword LLMs
- dc.subject.keyword Bertology
- dc.subject.keyword Computational linguistics
- dc.subject.keyword Neurolinguistics
- dc.subject.keyword Natural language processing
- dc.subject.keyword Transformers
- dc.subject.keyword Predictive coding
- dc.title Scrutinizing the predictive power of large language models for brain function
- dc.type info:eu-repo/semantics/masterThesis