Assessing the validity of survey measures for news exposure through digital footprints: evidence from Spain and the UK

Abstract

This paper assesses the performance of three commonly used types of questions – open-ended, check-all and forced choice – for capturing retrospective online news exposure, combining both survey and web-tracking data. It examines the performance of these different survey questions considering both systematic and random error in two unexplored non-US contexts: Spain and the UK. Results show that the check-all question produces on average the most accurate – i.e. less biased – estimates of observed exposure. Some motivational and cognitive factors underlying bias in self-reports are explored. Findings reveal that the characteristics of outlets are associated with systematic error. Finally, we find that media systems matter for accuracy – where media fragmentation is high (Spain), accuracy is low across all questions; where it is low (UK), accuracy is high across all questions. In the final section, we highlight the methodological and theoretical contributions of our study and provide some recommendations.