Comparing questions with agree/disagree response options to questions with construct-specific response options

Saris, Willem E.; Revilla, Melanie; Krosnick, Jon A.; Shaeffer, Eric M.

doi:http://dx.doi.org/10.18148/srm/2010.v4i1.2682

Comparing questions with agree/disagree response options to questions with construct-specific response options

Citation

Saris WE, Revilla M, Krosnick JA, Shaeffer EM. Comparing questions with agree/disagree response options to questions with construct-specific response options. Survey Research Methods. 2010;4(1):61-79. DOI: 10.18148/srm/2010.v4i1.2682

Abstract

Although Agree/Disagree (A/D) rating scales are hugely popular in the social sciences, a large body of research conducted during more than five decades has documented the bias that results from acquiescence in responses to these items. This may be a reason to prefer questions with Item Specific (IS) response options, but remarkably little research has explored whether responses to A/D rating scale questions are indeed of lower quality than responses to questions with IS response options. Using a research design that combines the advantages of a random assignment between-subjects experiment and the multitrait-multimethod approach in the context of representative sample surveys, we found that responses to A/D rating scale questions indeed had much lower quality than responses to comparable questions offering IS response options. These results attest to the superiority of questions with IS response options.