Learning to prompt in the classroom to understand AI limits: a pilot study

Theophilou, Emily; Koyutürk, Cansu; Yavari, Mona; Bursić, Sathya; Donabauer, Gregor; Telari, Alessia; Testa, Alessia; Boiano, Raffaele; Hernández-Leo, Davinia; Ruskov, Martin; Taibi, Davide; Gabbiadini, Alessandro; Ognibene, Dimitri

Learning to prompt in the classroom to understand AI limits: a pilot study

Mostra el registre complet Registre parcial de l'ítem

dc.contributor.author Theophilou, Emily
dc.contributor.author Koyutürk, Cansu
dc.contributor.author Yavari, Mona
dc.contributor.author Bursić, Sathya
dc.contributor.author Donabauer, Gregor
dc.contributor.author Telari, Alessia
dc.contributor.author Testa, Alessia
dc.contributor.author Boiano, Raffaele
dc.contributor.author Hernández-Leo, Davinia
dc.contributor.author Ruskov, Martin
dc.contributor.author Taibi, Davide
dc.contributor.author Gabbiadini, Alessandro
dc.contributor.author Ognibene, Dimitri
dc.date.accessioned 2025-01-10T07:44:27Z
dc.date.available 2025-01-10T07:44:27Z
dc.date.issued 2023
dc.description.abstract Artificial intelligence’s (AI) progress holds great promise in tackling pressing societal concerns such as health and climate. Large Language Models (LLM) and the derived chatbots, like ChatGPT, have highly improved the natural language processing capabilities of AI systems allowing them to process an unprecedented amount of unstructured data. However, the ensuing excitement has led to negative sentiments, even as AI methods demonstrate remarkable contributions (e.g. in health and genetics). A key factor contributing to this sentiment is the misleading perception that LLMs can effortlessly provide solutions across domains, ignoring their limitations such as hallucinations and reasoning constraints. Acknowledging AI fallibility is crucial to address the impact of dogmatic overconfidence in possibly erroneous suggestions generated by LLMs. At the same time, it can reduce fear and other negative attitudes toward AI. This necessitates comprehensive AI literacy interventions that educate the public about LLM constraints and effective usage techniques, i.e. prompting strategies. With this aim, a pilot educational intervention was performed in a high school with 21 students. It involved presenting high-level concepts about intelligence, AI, and LLMs, followed by practical exercises involving ChatGPT in creating natural educational conversations and applying established prompting strategies. Encouraging preliminary results emerged, including high appreciation of the activity, improved interaction quality with the LLM, reduced negative AI sentiments, and a better grasp of limitations, specifically unreliability, limited understanding of commands leading to unsatisfactory responses, and limited presentation flexibility. Our aim is to explore AI acceptance factors and refine this approach for more controlled future studies.
dc.description.sponsorship This work has been partially funded by the Volkswagen Foundation (COURAGE project, no. 95567). TIDE-UPF also acknowledges the support by AEI/10.13039/501100011033 (PID2020-112584RB-C33, MDM-2015-0502) and by ICREA under the ICREA Academia programme (D. Hernández-Leo, Serra Hunter) and the Department of Research and Universities of the Government of Catalonia (SGR 00930). The authors thank Marco Marelli for the useful discussions on pragmatic linguistic skills.
dc.format.mimetype application/pdf
dc.identifier.citation Theophilou E, Koyutürk C, Yavari M, Bursic S, Donabauer G, Telari A, et al. Learning to prompt in the classroom to understand AI limits: a pilot study. In: Basili R, Lembo D, Limongelli C, Orlandini A, editors. AIxIA 2023 - Advances in Artificial Intelligence. XXIInd International Conference of the Italian Association for Artificial Intelligence; Nov 6-9; Rome, Italy. Cham: Springer; 2023. p. 481-96. DOI: 10.1007/978-3-031-47546-7_33
dc.identifier.doi https://doi.org/10.1007/978-3-031-47546-7_33
dc.identifier.uri http://hdl.handle.net/10230/69044
dc.language.iso eng
dc.publisher Springer
dc.relation.projectID info:eu-repo/grantAgreement/ES/2PE/PID2020-112584RB-C33
dc.rights.accessRights info:eu-repo/semantics/openAccess
dc.subject.keyword ChatGPT
dc.subject.keyword Large language models
dc.subject.keyword HCI
dc.subject.keyword Prompting
dc.subject.keyword AI attitude
dc.subject.keyword AI limitations
dc.subject.keyword AI literacy
dc.title Learning to prompt in the classroom to understand AI limits: a pilot study
dc.type info:eu-repo/semantics/conferenceObject
dc.type.version info:eu-repo/semantics/acceptedVersion

Col·leccions

Congressos (Departament de Tecnologies de la Informació i les Comunicacions)