Comparing questions with agree/disagree response options to questions with construct-specific response options

Abstract

Although Agree/Disagree (A/D) rating scales are hugely popular in the social sciences, a large body of research conducted during more than five decades has documented the bias that results from acquiescence in responses to these items. This may be a reason to prefer questions with Item Specific (IS) response options, but remarkably little research has explored whether responses to A/D rating scale questions are indeed of lower quality than responses to questions with IS response options. Using a research design that combines the advantages of a random assignment between-subjects experiment and the multitrait-multimethod approach in the context of representative sample surveys, we found that responses to A/D rating scale questions indeed had much lower quality than responses to comparable questions offering IS response options. These results attest to the superiority of questions with IS response options.