A comparative user study of human predictions in algorithm-supported recidivism risk assessment

Mostra el registre complet Registre parcial de l'ítem

• dc.contributor.author Portela, Manuel
• dc.contributor.author Castillo, Carlos
• dc.contributor.author Tolan, Songül
• dc.contributor.author Karimi-Haghighi, Marzieh
• dc.contributor.author Andrés Pueyo, Antonio
• dc.date.accessioned 2024-06-25T06:04:33Z
• dc.date.available 2024-06-25T06:04:33Z
• dc.date.issued 2024
• dc.description Data de publicació electrònica: 15-03-2024
• dc.description.abstract In this paper, we study the effects of using an algorithm-based risk assessment instrument (RAI) to support the prediction of risk of violent recidivism upon release. The instrument we used is a machine learning version of RiskCanvi used by the Justice Department of Catalonia, Spain. It was hypothesized that people can improve their performance on defining the risk of recidivism when assisted with a RAI. Also, that professionals can perform better than non-experts on the domain. Participants had to predict whether a person who has been released from prison will commit a new crime leading to re-incarceration, within the next two years. This user study is done with (1) general participants from diverse backgrounds recruited through a crowdsourcing platform, (2) targeted participants who are students and practitioners of data science, criminology, or social work and professionals who work with RisCanvi. We also run focus groups with participants of the targeted study, including people who use RisCanvi in a professional capacity, to interpret the quantitative results. Among other findings, we observe that algorithmic support systematically leads to more accurate predictions from all participants, but that statistically significant gains are only seen in the performance of targeted participants with respect to that of crowdsourced participants. Among other comments, professional participants indicate that they would not foresee using a fully-automated system in criminal risk assessment, but do consider it valuable for training, standardization, and to fine-tune or double-check their predictions on particularly difficult cases. We found that the revised prediction by using a RAI increases the performance of all groups, while professionals show a better performance in general. And, a RAI can be considered for extending professional capacities and skills along their careers.
• dc.description.sponsorship Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work has been partially supported by the HUMAINT programme (Human Behaviour and Machine Intelligence), Centre for Advanced Studies, Joint Research Centre, European Commission through the Expect contract CT-EX2019D347180. Additionally, it received the support from the EU-funded project “SoBigData++” (grant agreement 871042).
• dc.format.mimetype application/pdf
• dc.identifier.citation Portela M, Castillo C, Tolan S, Karimi-Haghighi M, Andrés A. A comparative user study of human predictions in algorithm-supported recidivism risk assessment. Artif Intell Law. 2024 Mar 15. DOI: 10.1007/s10506-024-09393-y
• dc.identifier.doi http://dx.doi.org/10.1007/s10506-024-09393-y
• dc.identifier.issn 1572-8382
• dc.identifier.uri http://hdl.handle.net/10230/60556
• dc.language.iso eng
• dc.publisher Springer
• dc.relation.ispartof Artif Intell Law. 2024 Mar 15
• dc.relation.projectID info:eu-repo/grantAgreement/EC/H2020/871042
• dc.rights This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
• dc.rights.accessRights info:eu-repo/semantics/openAccess
• dc.rights.uri http://creativecommons.org/licenses/by/4.0/
• dc.subject.keyword Recidivism
• dc.subject.keyword Automated decision-making
• dc.subject.keyword Risk assessment instrument
• dc.subject.keyword Human oversight
• dc.subject.keyword Machine learning
• dc.title A comparative user study of human predictions in algorithm-supported recidivism risk assessment
• dc.type info:eu-repo/semantics/article
• dc.type.version info:eu-repo/semantics/publishedVersion