Portillo-Van Diest, AnaMortier, PhilippeBallester, LauraAmigo, FrancoCarrasco, PaulaFalcó, RaquelGili, MargalidaKiekens, GlennMachancoses, Francisco H.Piqueras, José AntonioRebagliato, MarisaRoca, MiquelRodríguez Jimenez, TiscarAlonso Caballero, JordiVilagut Saiz, Gemma, 1975-2025-02-042025-02-042024Portillo-Van Diest A, Mortier P, Ballester L, Amigo F, Carrasco P, Falcó R, et al. Ecological momentary assessment of mental health problems among university students: Data quality evaluation study. J Med Internet Res. 2024 Dec 10;26:e55712. DOI: 10.2196/557121439-4456http://hdl.handle.net/10230/69478Background: The use of ecological momentary assessment (EMA) designs has been on the rise in mental health epidemiology. However, there is a lack of knowledge of the determinants of participation in and compliance with EMA studies, reliability of measures, and underreporting of methodological details and data quality indicators. Objective: This study aims to evaluate the quality of EMA data in a large sample of university students by estimating participation rate and mean compliance, identifying predictors of individual-level participation and compliance, evaluating between- and within-person reliability of measures of negative and positive affect, and identifying potential careless responding. Methods: A total of 1259 university students were invited to participate in a 15-day EMA study on mental health problems. Logistic and Poisson regressions were used to investigate the associations between sociodemographic factors, lifetime adverse experiences, stressful events in the previous 12 months, and mental disorder screens and EMA participation and compliance. Multilevel reliability and intraclass correlation coefficients were obtained for positive and negative affect measures. Careless responders were identified based on low compliance or individual reliability coefficients. Results: Of those invited, 62.1% (782/1259) participated in the EMA study, with a mean compliance of 76.9% (SD 27.7%). Participation was higher among female individuals (odds ratio [OR] 1.41, 95% CI 1.06-1.87) and lower among those aged ≥30 years (OR 0.20, 95% CI 0.08-0.43 vs those aged 18-21 years) and those who had experienced the death of a friend or family member in the previous 12 months (OR 0.73, 95% CI 0.57-0.94) or had a suicide attempt in the previous 12 months (OR 0.26, 95% CI 0.10-0.64). Compliance was particularly low among those exposed to sexual abuse before the age of 18 years (exponential of β=0.87) or to sexual assault or rape in the previous year (exponential of β=0.80) and among those with 12-month positive alcohol use disorder screens (exponential of β=0.89). Between-person reliability of negative and positive affect was strong (RkRn>0.97), whereas within-person reliability was fair to moderate (Rcn>0.43). Of all answered assessments, 0.86% (291/33,626) were flagged as careless responses because the response time per item was <1 second or the participants gave the same response to all items. Of the participants, 17.5% (137/782) could be considered careless responders due to low compliance (<25/56, 45%) or very low to null individual reliability (raw Cronbach α<0.11) for either negative or positive affect. Conclusions: Data quality assessments should be carried out in EMA studies in a standardized manner to provide robust conclusions to advance the field. Future EMA research should implement strategies to mitigate nonresponse bias as well as conduct sensitivity analyses to assess possible exclusion of careless responders.application/pdfeng© Ana Portillo-Van Diest, Philippe Mortier, Laura Ballester, Franco Amigo, Paula Carrasco, Raquel Falcó, Margalida Gili, Glenn Kiekens, Francisco H Machancoses, Jose A Piqueras, Marisa Rebagliato, Miquel Roca, Tíscar Rodríguez-Jiménez, Jordi Alonso, Gemma Vilagut. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 10.12.2024. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.info:eu-repo/semantics/articlehttp://dx.doi.org/10.2196/55712ComplianceEcological momentary assessmentExperience sampling methodMental healthMobile phoneParticipationReliabilitySensitivity analysisUniversity studentsinfo:eu-repo/semantics/openAccess