Positioning political texts with large language models by asking and averaging

Mostra el registre complet Registre parcial de l'ítem

• dc.contributor.author Le Mens, Gaël
• dc.contributor.author Gallego, Aina
• dc.date.accessioned 2025-05-20T12:34:13Z
• dc.date.available 2025-05-20T12:34:13Z
• dc.date.issued 2025
• dc.date.updated 2025-05-20T12:34:13Z
• dc.description Data de publicació electrònica: 27-01-2025
• dc.description.abstract We use instruction-tuned large language models (LLMs) like GPT-4, Llama 3, MiXtral, or Aya to position political texts within policy and ideological spaces. We ask an LLM where a tweet or a sentence of a political text stands on the focal dimension and take the average of the LLM responses to position political actors such as US Senators, or longer texts such as UK party manifestos or EU policy speeches given in 10 different languages. The correlations between the position estimates obtained with the best LLMs and benchmarks based on text coding by experts, crowdworkers, or roll call votes exceed.90. This approach is generally more accurate than the positions obtained with supervised classifiers trained on large amounts of research data. Using instruction-tuned LLMs to position texts in policy and ideological spaces is fast, cost-efficient, reliable, and reproducible (in the case of open LLMs) even if the texts are short and written in different languages. We conclude with cautionary notes about the need for empirical validation.
• dc.description.sponsorship This research was funded by ERC Consolidator Grant 772268 from the European Commission to G.L.M, ICREA Academia grants to A.G and G.L.M, grants PID2021-123111OB-I00 (A.G.) and PID2022-137908NB-I00 (G.L.M.) funded by MICIN/AEI/10.13039/501100011033 and by 'ERDF/UE A way of making Europe', and the Severo Ochoa Programme for Centres of Excellence in R&D (Barcelona School of Economics CEX2019-000915-S) funded by MCIN/AEI/10.13039/501100011033.
• dc.format.mimetype application/pdf
• dc.identifier.citation Le Mens G, Gallego A. Positioning political texts with large language models by asking and averaging. Polit Anal. 2025 Jan 27. DOI: 10.1017/pan.2024.29
• dc.identifier.doi http://dx.doi.org/10.1017/pan.2024.29
• dc.identifier.issn 1047-1987
• dc.identifier.uri http://hdl.handle.net/10230/70440
• dc.language.iso eng
• dc.publisher Cambridge University Press
• dc.relation.ispartof Political Analysis. 2025 Jan 27
• dc.relation.projectID info:eu-repo/grantAgreement/ES/3PE/PID2021-123111OB-I00
• dc.relation.projectID info:eu-repo/grantAgreement/ES/3PE/PID2022-137908NB-I00
• dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/772268
• dc.rights © The Author(s), 2025. Published by Cambridge University Press on behalf of The Society for Political Methodology. This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
• dc.rights.accessRights info:eu-repo/semantics/openAccess
• dc.rights.uri http://creativecommons.org/licenses/by/4.0/
• dc.subject.keyword LLM
• dc.subject.keyword Ideology
• dc.subject.keyword Scaling
• dc.subject.keyword Text as data
• dc.title Positioning political texts with large language models by asking and averaging
• dc.type info:eu-repo/semantics/article
• dc.type.version info:eu-repo/semantics/publishedVersion