The split-ballot multitrait-multimethod approach: Implementation and problems

Abstract

Saris, Satorra and Coenders (2004) proposed a new approach to estimate the quality of survey questions, combining the advantages of two existing approaches: the multitrait-multimethod (MTMM) and the split-ballot (SB) ones. Implemented in practice, this new approach led to frequent problems of non-convergence and improper solutions. This paper uses Monte Carlo simulations to understand why the SB-MTMM is working well in some cases but not in others. The number of SB groups is a crucial element: the 3-group design is performing better. However, the 2-group design can also perform well: the analyses suggest that the interaction between the absolute values of the correlations between the traits and the relative values of the different correlations between traits play an important role.