The seriousness check is a technique that can be used in online research (also called Internet-based research, Web-based research, Web-based experiments) to improve data quality. Nowadays, many scientific studies with human participants are conducted online and are accessible to a large diversity of participants. Nonetheless, many people just want to look at the different pages of the questionnaire, instead of giving carefully chosen answers to the questions.[1] The seriousness check addresses this problem: In this approach the respondents are asked about the seriousness of their participation or for a probability estimate that they will complete the entire study or experiment.[2][3][4] Thus, by using the seriousness check irrelevant data entries can be easily identified and be excluded from the data analysis.
Seriousness checks can be implemented both before [4] and after [6] participation in the study. However, it has been shown that the seriousness check is a good predictor of dropout rates when implemented in the first page of the experiment.[1]
There are Web-based tools, e.g. WEXTOR,[7] that implement the seriousness check by default.
Several studies have shown that performing a seriousness check at the start of a study best predicts motivation and dropout probability.[1][8] It was observed that of those answering “I would like to look at the pages only" around 75% will drop, while of those answering “I would like to seriously participate now" only ca. 10-15% will drop during the study. Overall, about 30-50% of visitors will fail the seriousness check, i.e. answer “I would like to look at the pages only".[1] Moreover, it was found that emphasizing seriousness increased information seeking in participants and the time they spent on the study.[9] Following up on this, it was shown that motivation and self-reported seriousness significantly predict several data quality indicators [10]
^ abcdReips, U.-D. (2009). Internet experiments: Methods, guidelines, metadata. Human Vision and Electronic Imaging XIV, Proc. SPIE, 7240, 724008. https://doi.org/10.1117/12.823416
^Musch, J., & Klauer, K. C. (2002). Psychological experimenting on the World Wide Web: Investigating content effects in syllogistic reasoning. In B. Batinic, U.-D. Reips, & M. Bosnjak (Eds.), Online social sciences (pp. 181–212). Hogrefe & Huber Publishers.
^Reips, U.-D. (2007). The methodology of Internet-based experiments. In A. Joinson, K. McKenna, T. Postmes, & U.-D. Reips (Eds.), The Oxford Handbook of Internet Psychology (pp. 373-390). Oxford University Press.
^Reips, U.-D., & Krantz, J. H. (2010). Conducting true experiments on the Web. In S. D. Gosling & J. A. Johnson (Eds.), Advanced methods for conducting online behavioral research (p. 193–216). American Psychological Association. https://doi.org/10.1037/12076-01
^Aust, F., Diedenhofen, B., Ullrich, S., & Musch, J. (2013). Seriousness checks are useful to improve data validity in online research. Behavior Research Methods, 45(2), 527–535. https://doi.org/10.3758/s13428-012-0265-2
^Reips, U.-D. (2008). How Internet-mediated research changes science. In A. Barak (Ed.), Psychological aspects of cyberspace: Theory, research, applications (p. 268–294). Cambridge University Press. https://doi.org/10.1017/CBO9780511813740.013
