General Bayesian loss function selection and the use of improper models

Mostra el registre complet Registre parcial de l'ítem

• dc.contributor.author Jewson, Jack
• dc.contributor.author Rossell Ribera, David
• dc.date.accessioned 2023-07-26T07:15:44Z
• dc.date.available 2023-07-26T07:15:44Z
• dc.date.issued 2022
• dc.description.abstract Statisticians often face the choice between using probability models or a paradigm defined by minimising a loss function. Both approaches are useful and, if the loss can be re-cast into a proper probability model, there are many tools to decide which model or loss is more appropriate for the observed data, in the sense of explaining the data's nature. However, when the loss leads to an improper model, there are no principled ways to guide this choice. We address this task by combining the Hyvärinen score, which naturally targets infinitesimal relative probabilities, and general Bayesian updating, which provides a unifying framework for inference on losses and models. Specifically we propose the ℋ-score, a general Bayesian selection criterion and prove that it consistently selects the (possibly improper) model closest to the data-generating truth in Fisher's divergence. We also prove that an associated ℋ-posterior consistently learns optimal hyper-parameters featuring in loss functions, including a challenging tempering parameter in generalised Bayesian inference. As salient examples, we consider robust regression and non-parametric density estimation where popular loss functions define improper models for the data and hence cannot be dealt with using standard model selection tools. These examples illustrate advantages in robustness-efficiency trade-offs and enable Bayesian inference for kernel density estimation, opening a new avenue for Bayesian non-parametrics.
• dc.description.sponsorship Jack Jewson and David Rossell were partially funded by the Ayudas Fundación BBVA a Equipos de Investigación Cientifica 2017 and Proyectos de Investigación Científica en Matemáticas 2021, and Government of Spain's Plan Nacional PGC2018-101643-B-I00 grants. David Rossell was also partially funded by the Europa Excelencia grant EUR2020-112096 and Ramón y Cajal Fellowship RYC-2015-18544.
• dc.format.mimetype application/pdf
• dc.identifier.citation Jewson J, Rossell D. General Bayesian loss function selection and the use of improper models. J R Stat Soc Series B Stat Methodol. 2022;84(5):1640-65. DOI: 10.1111/rssb.12553
• dc.identifier.doi http://dx.doi.org/10.1111/rssb.12553
• dc.identifier.issn 1369-7412
• dc.identifier.uri http://hdl.handle.net/10230/57676
• dc.language.iso eng
• dc.publisher Oxford University Press
• dc.relation.ispartof Journal of the Royal Statistical Society Series B: Statistical Methodology. 2022;84(5):1640-65.
• dc.relation.projectID info:eu-repo/grantAgreement/ES/2PE/PGC2018-101643-B-I00
• dc.relation.projectID info:eu-repo/grantAgreement/ES/2PE/EUR2020-112096
• dc.relation.projectID info:eu-repo/grantAgreement/ES/1PE/RYC-2015-18544
• dc.rights © 2022 The Authors.Journal of the Royal Statistical Society: Series B (Statistical Methodology) published by John Wiley & Sons Ltd on behalf of Royal Statistical Society. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.
• dc.rights.accessRights info:eu-repo/semantics/openAccess
• dc.rights.uri http://creativecommons.org/licenses/by/4.0/
• dc.subject.keyword general Bayes
• dc.subject.keyword Hyvärinen score
• dc.subject.keyword improper models
• dc.subject.keyword kernel density estimation
• dc.subject.keyword loss functions
• dc.subject.keyword robust regression
• dc.title General Bayesian loss function selection and the use of improper models
• dc.type info:eu-repo/semantics/article
• dc.type.version info:eu-repo/semantics/publishedVersion