An experiment comparing grids and item‐by‐item formats in web surveys completed through PC and smartphones

Revilla, Melanie; Toninelli, Daniele; Ochoa, Carlos

An experiment comparing grids and item‐by‐item formats in web surveys completed through PC and smartphones

License

Citation

Revilla, M, Toninelli, D, Ochoa, C. An experiment comparing grids and item‐by‐item formats in web surveys completed through PC and smartphones. 2015. handle: http://hdl.handle.net/10230/30798

This citation was generated automatically.

Abstract

Purpose: Some respondents already complete web surveys via mobile devices, even when this is unintended. However, these devices vary at several levels from PCs (e.g. screen size). Therefore, it is necessary to study the comparability of the data collected across devices. In particular, we expect differences when grids questions are used due to the device characteristics (lower visibility on mobile devices) and because in questionnaires that optimized to be completed through smartphones, grids are automatically split up into an itemby-item format. Design/methodology/approach: This paper reports the results of a two-waves experiment conducted in Spain with the online fieldwork company Netquest in February-March 2015. In each wave, respondents were randomly assigned to three groups: PCs, smartphones notoptimized, or smartphones optimized. Findings: Similar levels of interitem correlations are observed in all groups. Longer completion times are found in the case of grids for the smartphone respondents. Less nondifferentiation is observed in the PC group for some questions. For smartphones, the nondifferentiation is reduced by splitting grids into separate questions. In general, the results suggest that both the device used and splitting the grids affect the results. Practical implications: Using the item-by-item format for both smartphones and PCs seems to be the most appropriate way to improve comparability. Originality/value: Most online panels are still using grids in the surveys proposed to their participants. This research shows that this practice creates problem of comparability of the data when smartphones are used.