Scrutinizing the predictive power of large language models for brain function

Abstract

Based on predictive coding and hierarchical processing as a commonality between large language models (LLMs) and the brain, many studies have linked the two by regressing brain activity on LLMs’ representations. However, increasing evidence has revealed problems in this new line of research. To address this issue, we attempted to replicate a pioneering study (Kumar et al., 2022) on an independent fMRI dataset with several methodological adaptations. Results showed overall low correlation scores and sparse predictions across the cortex. Contrary to the reference study, representation’s performances across most ROIs did not differ significantly. However, in areas where significant differences were observed, fastText consistently outperformed BERT. Additionally, the layer-wise performance of embeddings and transformations showed no consistent patterns. Our findings challenge the existint assumptions regarding the predictive power of LLMs for brain function and highlight potential issues in the current methodologies to map predictions from LLMs onto brain activations.