DOI: https://doi.org/10.62229/rrfaxvi-1/3
Abstract: This text critically examines whether psychometric scales represent a robust measurement choice when studying conspiracy theories: a key philosophical and methodological gap in the literature on conspiracy theories. I call into question whether such scales have content validity, predictive validity and whether studies employing these instruments manifest external validity, respectively. These issues manifest differently across the two types of scales examined. The adequate development of applied scales is unfeasible because it is impossible to objectively define an ideal combination of items that fully captures the conspiratorial themes they aim to measure. Applied scales will, then, always have limited content validity, which will not only impair our ability to understand whether they really measure the construct in question but will also prevent us from using them in a standardized way. While generic scales may seem superior to applied scales in that they allow for standardized measures, they seem to suffer from the same problem due to the theoretically limitless number of dimensions needed to fully capture conspiratorial ideation. Consequently, the degree to which the predictions made on the basis of these scales are valid (i.e. predictive validity) and their generalizability (i.e. external validity) becomes unclear. In this text, I argue that the employment of psychometric scales does not represent a robust method of measuring conspiracy theories. This situation raises concerns regarding the current state of the literature, since these instruments are widely used in this research area. Given the discussed shortcomings, I propose a novel approach to measurement, one that involves indirect assessment of conspiracy theories. Moreover, a better alternative to existing measures is considered, namely discourse analysis.
- DANIEL-RADU IORDACHE, University of Bucharest Daniel-Radu Iordache is a graduate of the “Mind the Brain” master’s program in cognitive science within the Faculty of Philosophy at the University of Bucharest
Works Cited
Achimescu, V., Sultănescu, D., & Sultănescu, D. C. (2021). The path from distrusting Western actors to conspiracy beliefs and noncompliance with public health guidance during the COVID-19 crisis. Journal of Elections, Public Opinion and Parties, 31(sup1), 299–310. https://doi.org/10.1080/17457289.2021.1924746
Ahmed, W., Vidal-Alaball, J., Downing, J., & López Seguí, F. (2020). COVID-19 and the 5G Conspiracy Theory: Social Network Analysis of Twitter Data. Journal of medical Internet research, 22(5), e19458. https://doi.org/10.2196/19458
APA Dictionary of Psychology. (n.d.-a). https://dictionary.apa.org/psychological-scale
APA Dictionary of Psychology. (n.d.-b). https://dictionary.apa.org/standardization
APA Dictionary of Psychology. (n.d.-c). https://dictionary.apa.org/longitudinal-design
Bergmann, E., & Butter, M. (2020). Conspiracy Theory and Populism. In M. Butter & P. Knight (Eds.), Routledge Handbook of Conspiracy Theories (1st ed., pp. 330–343). Routledge. https://doi.org/10.4324/9780429452734-3_6
Bessi, A., Coletto, M., Davidescu, G. A., Scala, A., Caldarelli, G., & Quattrociocchi, W. (2015). Science vs Conspiracy: Collective Narratives in the Age of Misinformation. PLOS ONE, 10(2), e0118093. https://doi.org/10.1371/journal.pone.0118093
Brotherton, R., & Eser, S. (2015). Bored to fears: Boredom proneness, paranoia, and conspiracy theories. Personality and Individual Differences, 5.G=
Brotherton, R., French, C. C., & Pickering, A. D. (2013). Measuring Belief in Conspiracy Theories: The Generic Conspiracist Beliefs Scale. Frontiers in Psychology, 4. https://doi.org/10.3389/fpsyg.2013.00279
Bruder, M., Haffke, P., Neave, N., Nouripanah, N., & Imhoff, R. (2013). Measuring Individual Differences in Generic Beliefs in Conspiracy Theories Across Cultures: Conspiracy Mentality Questionnaire. Frontiers in Psychology, 4. https://doi.org/10.3389/fpsyg.2013.00225
Bruns, A., Harrington, S., & Hurcombe, E. (2020). ‘Corona? 5G? or both?’: The dynamics of COVID-19/5G conspiracy theories on Facebook. Media International Australia, 18. https://doi.org/10.1177/1329878X2094611
Buchanan, T. (2020). Why do people spread false information online? The effects of message and viewer characteristics on self-reported likelihood of sharing social media disinformation. PLOS ONE, 33. https://doi.org/10.1371/journal.pone.0239666
Buchanan, T., & Kempley, J. (2021). Individual differences in sharing false political information on social media: Direct and indirect effects of cognitive-perceptual schizotypy and psychopathy. Personality and Individual Differences, 11. https://doi.org/10.1016/j.paid.2021.111071
Buturoiu, R., Udrea, G., Oprea, D.-A., & Corbu, N. (2021). Who Believes in Conspiracy Theories about the COVID-19 Pandemic in Romania? An Analysis of Conspiracy Theories Believers’ Profiles. Societies, 11(4), 138. https://doi.org/10.3390/soc11040138
Campion-Vincent, V. (2015). Remarks on conspiracy theory entrepreneurs. Diogenes, 62(3–4), 64–70. https://doi.org/10.1177/0392192120945606
Chan, H.-W., Chiu, C. P.-Y., Zuo, S., Wang, X., Liu, L., & Hong, Y. (2021). Not-so-straightforward links between believing in COVID-19-related conspiracy theories and engaging in disease-preventive behaviours. Humanities and Social Sciences Communications, 8(1), 104. https://doi.org/10.1057/s41599-021-00781-2
Cicchetti, D. V. (1994). Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment Instruments in Psychology. Psychological Assessment, 6(4), 284–290. https://doi.org/10.1037/1040-3590.6.4.284
Dale, D. (2020, September 2). Fact check: A guide to 9 conspiracy theories Trump is currently pushing | CNN politics. CNN. https://edition.cnn.com/2020/09/02/politics/fact-check-trump-conspiracy-theories-biden-covid-thugs-plane/index.html
Devaux, M., & Sassi, F. (2016). Social disparities in hazardous alcohol use: Self-report bias may lead to incorrect estimates. The European Journal of Public Health, 26(1), 129–134. https://doi.org/10.1093/eurpub/ckv190
Douglas, K. M., & Sutton, R. M. (2015). Climate change: Why the conspiracy theories are dangerous. Bulletin of the Atomic Scientists, 71(2), 98–106. https://doi.org/10.1177/0096340215571908
Douglas, K. M., Sutton, R. M., & Cichocka, A. (2017). The Psychology of Conspiracy Theories. Current Directions in Psychological Science, 26(6), 538-542. https://doi.org/10.1177/0963721417718261
Douglas, K. M., & Sutton, R. M. (2018). Why conspiracy theories matter: A social psychological analysis. European Review of Social Psychology, 29(1), 256–298. https://doi.org/10.1080/10463283.2018.1537428
Douglas, K. M., Uscinski, J. E., Sutton, R. M., Cichocka, A., Nefes, T., Ang, C. S., & Deravi, F. (2019). Understanding Conspiracy Theories. Political Psychology, 40(S1), 3–35. https://doi.org/10.1111/pops.12568
Duetz, J. C. M. (2022). Conspiracy Theories are Not Beliefs. Erkenntnis. https://doi.org/10.1007/s10670-022-00620-z
Duetz, J. C. M. (2023). What Does It Mean for a Conspiracy Theory to be a “Theory”? Social Epistemology, 37(4), 438–453. https://doi.org/10.1080/738552930
Enders, A. M., Uscinski, J. E., Klofstad, C. A., Seelig, M. I., Wuchty, S., Murthi, M. N., Premaratne, K., & Funchion, J. R. (2021). Do conspiracy beliefs form a belief system? Examining the structure and organization of conspiracy beliefs. Journal of Social and Political Psychology, 9(1), 255–271. https://doi.org/10.5964/jspp.5649
Findley, M. G., Kikuta, K., & Denly, M. (2021). External Validity. In Annual Review of Political Science (Vol. 24, Issue Volume 24, 2021, pp. 365–393). Annual Reviews. https://doi.org/10.1146/annurev-polisci-041719-102556
Galesic, M., & Bosnjak, M. (2009). Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey. Public Opinion Quarterly, 73(2), 349–360. https://doi.org/10.1093/poq/nfp031
Gligorić, V., Silva, M. M., Eker, S., Hoek, N., Nieuwenhuijzen, E., Popova, U., & Zeighami, G. (2021). The usual suspects: How psychological motives and thinking styles predict the endorsement of well‐known and COVID ‐19 conspiracy beliefs. Applied Cognitive Psychology, 35(5), 1171–1181. https://doi.org/10.1002/acp.3844
Goreis, A., & Voracek, M. (2019). A Systematic Review and Meta-Analysis of Psychological Research on Conspiracy Beliefs: Field Characteristics, Measurement Instruments, and Associations With Personality Traits. Frontiers in psychology, 10, 205. https://doi.org/10.3389/fpsyg.2019.00205
Griffin, A. (2022, December 20). Princess Diana conspiracy theories: Eight reasons people believe her death in Paris wasn’t all it seems. The Independent. https://www.independent.co.uk/life-style/royal-family/princess-diana-death-conspiracy-theories-b2248362.html
Here’s how you can spot fake news online. (2022, May 20). World Economic Forum. https://www.weforum.org/agenda/2017/12/heres-how-you-can-spot-fake-news-online/
Herzog, A. R., & Bachman, J. G. (1981). Effects of Questionnaire Length on Response Quality. Public Opinion Quarterly, 45(4), 549. https://doi.org/10.1086/268687
Imhoff, R., & Lamberty, P. K. (2017). Too special to be duped: Need for uniqueness motivates conspiracy beliefs. European Journal of Social Psychology, 47(6), 724–734. https://doi.org/10.1002/ejsp.2265
Imhoff, R., & Lamberty, P. (2020). A Bioweapon or a Hoax? The Link Between Distinct Conspiracy Beliefs About the Coronavirus Disease (COVID-19) Outbreak and Pandemic Behavior. Social Psychological and Personality Science, 11(8), 1110–1118. https://doi.org/10.1177/1948550620934692
IONOS editorial team. (2020, July 27). What is fake news? Definition, types, and how to detect them. IONOS Digital Guide. https://www.ionos.com/digitalguide/online-marketing/social-media/what-is-fake-news/
Jolley, D., Douglas, K. M., Leite, A. C., & Schrader, T. (2019). Belief in conspiracy theories and intentions to engage in everyday crime. British Journal of Social Psychology, 58(3), 534-549. https://doi.org/10.1111/bjso.12311
Jolley, D., Douglas, K. M., & Sutton, R. M. (2018). Blaming a Few Bad Apples to Save a Threatened Barrel: The System-Justifying Function of Conspiracy Theories. Political Psychology, 39(2), 465–478. http://www.jstor.org/stable/45094751
Kay, C. S. (2021). Actors of the most fiendish character: Explaining the associations between the Dark Tetrad and conspiracist ideation. Personality and Individual Differences, 171, 110543. https://doi.org/10.1016/j.paid.2020.110543
Kruglanski, A. W., Molinario, E., Ellenberg, M., & Di Cicco, G. (2022). Terrorism and conspiracy theories: A view from the 3N model of radicalization. Current Opinion in Psychology, 47, 101396. https://doi.org/10.1016/j.copsyc.2022.101396
Lobato, E. J. C., Powell, M., Padilla, L. M. K., & Holbrook, C. (2020). Factors Predicting Willingness to Share COVID-19 Misinformation. Frontiers in Psychology, 11, 566108. https://doi.org/10.3389/fpsyg.2020.566108
McCrae, R. R., Kurtz, J. E., Yamagata, S., & Terracciano, A. (2011). Internal Consistency, Retest Reliability, and Their Implications for Personality Scale Validity. Personality and Social Psychology Review, 15(1), 28–50. https://doi.org/10.1177/1088868310366253
Miller, J. M. (2020a). Do COVID-19 Conspiracy Theory Beliefs Form a Monological Belief System? Canadian Journal of Political Science, 53(2), 319–326. https://doi.org/10.1017/S0008423920000517
Miller, J. (2020b). Psychological, Political, and Situational Factors Combine to Boost COVID-19 Conspiracy Theory Beliefs. Canadian Journal of Political Science, 53(2), 327-334. doi:10.1017/S000842392000058X
Newsome, S., Day, A. L., & Catano, V. M. (2000). Assessing the predictive validity of emotional intelligence. Personality and Individual Differences, 29(6), 1005–1016. https://doi.org/10.1016/S0191-8869(99)00250-0
Oleksy, T., Wnuk, A., Maison, D., & Łyś, A. (2021). Content matters. Different predictors and social consequences of general and government-related conspiracy theories on COVID-19. Personality and Individual Differences, 168, 110289. https://doi.org/10.1016/j.paid.2020.110289
Paola, G. (2020). The Importance of Using Valid and Reliable Measures in Psychology and Psychiatry.
Pappas, S., & Radford, B. (2023, July 11). 20 of the best conspiracy theories. livescience.com. https://www.livescience.com/11375-top-ten-conspiracy-theories.html
Paulhus, D. L. (2014). Toward a taxonomy of dark personalities. Current Directions in Psychological Science, 23(6), 421–426. https://doi.org/10.1177/0963721414547737
Pennycook, G., McPhetres, J., Zhang, Y., Lu, J. G., & Rand, D. G. (2020). Fighting COVID-19 Misinformation on Social Media: Experimental Evidence for a Scalable Accuracy-Nudge Intervention. Psychological science, 31(7), 770–780. https://doi.org/10.1177/0956797620939054
Pierre, J. M. (2020). Mistrust and misinformation: A two-component, socio-epistemic model of belief in conspiracy theories. Journal of Social and Political Psychology, 8(2), 617–641. https://doi.org/10.5964/jspp.v8i2.1362
Pirro, A. L., & Taggart, P. (2023). Populists in power and conspiracy theories. Party Politics, 29(3), 413-423. https://doi.org/10.1177/13540688221077071
Political Stability by country 2024. (n.d.). https://worldpopulationreview.com/country-rankings/political-stability-by-country
Psychological testing | Definition, Types, Examples, Importance, & Facts. (2022). Encyclopedia Britannica. Retrieved June 12, 2024, from https://www.britannica.com/science/psychological-testing/Other-characteristics
Quine, W.V.O. & Ullian, J.S. (1978/2021). The Web of Belief. McGraw-Hill Education. Translated in Romanian as Ţesătura opiniilor, transl. M. Dumitru. Iaşi: Polirom.
Google ScholarResearch Guides: Fake News and Information Literacy: What is Fake News? (n.d.). https://researchguides.uoregon.edu/fakenews/issues/defining
Robinson, M. A. (2018). Using multi-item psychometric scales for research and practice in human resource management. Hum Resour Manage. 57:739 750. https://doi.org/10.1002/hrm.21852
Schaffner, B. F., & Luks, S. (2018). Misinformation or Expressive Responding? What an Inauguration Crown Can Tell Us about the source of Political Misinformation in Surveys. Public Opinion Quarterly, 82(1), 135–147. https://doi.org/10.1093/poq/nfx042
Schwarz, N. (1999). How the Questions Shape the Answers. American Psychologist.
Simion, M. (2023). Resistance to evidence and the duty to believe. Philosophy and Phenomenological Research, 108(1), 203–216. https://doi.org/10.1111/phpr.12964
Stoica, C. A., & Umbreș, R. (2021). Suspicious minds in times of crisis: Determinants of Romanians’ beliefs in COVID-19 conspiracy theories. European Societies, 23(sup1), S246–S261. https://doi.org/10.1080/14616696.2020.1823450
Sutton, R. M., & Douglas, K. M. (2022). Agreeing to disagree: Reports of the popularity of Covid-19 conspiracy theories are greatly exaggerated. Psychological Medicine, 52(4), 791–793. https://doi.org/10.1017/S0033291720002780
Swami, V., Chamorro‐Premuzic, T., & Furnham, A. (2010). Unanswered questions: A preliminary investigation of personality and individual difference predictors of 9/11 conspiracist beliefs. Applied Cognitive Psychology, 24(6), 749–761. https://doi.org/10.1002/acp.1583
Swami, V., Barron, D., Weis, L., Voracek, M., Stieger, S., & Furnham, A. (2017). An examination of the factorial and convergent validity of four measures of conspiracist ideation, with recommendations for researchers. PLOS ONE, 12(2), e0172617. https://doi.org/10.1371/journal.pone.0172617
Trafimow, D. (2018). An a priori solution to the replication crisis. Philosophical Psychology, 31(8), 1188–1214. https://doi.org/10.1080/09515089.2018.1490707
van Mulukom, V., Pummerer, L. J., Alper, S., Bai, H., Čavojová, V., Farias, J., Kay, C. S., Lazarevic, L. B., Lobato, E. J. C., Marinthe, G., Pavela Banai, I., Šrol, J., & Žeželj, I. (2022). Antecedents and consequences of COVID-19 conspiracy beliefs: A systematic review. Social Science & Medicine, 301, 114912. https://doi.org/10.1016/j.socscimed.2022.114912
van Prooijen, J.-W., & Douglas, K. M. (2017). Conspiracy theories as part of history: The role of societal crisis situations. Memory Studies, 10(3), 323–333. https://doi.org/10.1177/1750698017701615
van Prooijen, J.-W., & van Vugt, M. (2018). Conspiracy Theories: Evolved Functions and Psychological Mechanisms. Perspectives on Psychological Science, 13(6), 770–788. https://doi.org/10.1177/1745691618774270
Wong, A., Ho, S., Olusanya, O., Antonini, M. V., & Lyness, D. (2021). The use of social media and online communications in times of pandemic COVID-19. Journal of the Intensive Care Society, 6.
Yaffe, J. (2019). From the Editor—Do We Have a Replication Crisis in Social Work Research? Journal of Social Work Education, 55(1), 1–4. https://doi.org/10.1080/10437797.2019.1594399
Zapata, C. (2024a, April 16). The Watergate Scandal – Timeline, summary & Deep throat | HISTORY. https://www.history.com/topics/1970s/watergate
Zapata, C. (2024b April 16). Assassination of John F. Kennedy – Facts, investigation, photos | HISTORY. HISTORY. https://www.history.com/topics/us-presidents/jfk-assassination
Zeng, J. (2021). Theoretical typology of deceptive content (Conspiracy Theories). DOCA – Database of Variables for Content Analysis. https://doi.org/10.34778/5g