2.1 Sample

Respondents were recruited through Qualtrics, an online survey platform that maintains a database of several million U.S. residents who have volunteered to participate in periodic survey-based research. Participants who completed the survey were financially compensated with the amount that was agreed upon with their original panel source. The only inclusion criterion was to be 18 years or older; no other exclusion criteria were employed. Participants who did not complete the survey in full were excluded. Of 5219 panel members who accessed the survey landing page, 2550 (48.9%) subsequently completed the survey. This survey study was reviewed and approved by the Institutional Review Board of Ben-Gurion University of the Negev. Data were collected in July 2022.

2.2 Measures

Psychotic experiences (predictor). Psychotic experiences were assessed using an abbreviated version of the WHO psychosis screen. Respondents were asked about: (1) A feeling that something strange and unexplainable was going on that other people would find hard to believe? [i.e., delusional mood] (2) A feeling that people were too interested in you or that there was a plot to harm you? [i.e., persecutory delusion] (3) A feeling that your thoughts were being directly interfered or controlled by another person, or your mind was being taken over by strange forces? [i.e., thought control] (4) An experience of seeing visions or hearing voices that others could not see or hear when you were not half asleep, dreaming, or under the influence of alcohol or drugs? [i.e., hallucination]. Having at least one of these experiences constituted having a lifetime psychotic experience. Respondents who endorsed at least one psychotic experience were also asked “Have any of these experiences caused you distress?” Using this item, we were able to create a variable indicating non-distressing PE versus distressing PE.

Mental health treatment (outcome). Being in mental health treatment was assessed using the single (yes/no) item: Are you currently receiving professional mental health treatment (e.g., therapy or counseling)?

Probable mental illness (outcome). The Kessler Distress Scale (Kessler et al., 2003) is a six-item scale assessing the presence of non-specific psychological distress in the last 30 days (mostly depression and anxiety symptoms, e.g., feeling hopeless, nervous, worthless). Items were rated on a 5-point Likert scale ranging from 0 (none of the time) to 4 (all of the time) and were summed into a scale ranging from 0 to 24. This variable was dichotomized, with 13 and higher indicating probable mental illness (Kessler et al., 2003, 2005). Cronbach's alpha in the present study was excellent (α = 0.92).

Loneliness (outcome). Loneliness was measured using a three-item UCLA scale, which elicited information about experiences of loneliness (feeling left out, feeling isolated, lacking companionship). Response options included, hardly ever, some of the time, and often, and items were summed (range 3–9) and dichotomized, with a score six or higher as indicating loneliness (Hughes et al., 2004). Cronbach's alpha was good (α = 0.81).

Hazardous alcohol use (outcome). Hazardous alcohol use was measured using the Alcohol Use Disorder Identification Test—Consumption (AUDIT-C; Bush et al., 1998), which elicits information about the frequency (how often, ranging from never to four or more times a week) and quantity (e.g., how many drinks on a typical day) of alcohol use and frequency of heavy drinking over the past year. Each item is rated on a 5-point Likert scale ranging from 0 to 4, and items are summed so that 12 is the maximum score. In men, a score of 4 or higher indicated hazardous alcohol use, while in women, a score of 3 or higher indicated hazardous alcohol use (Bush et al., 1998). These norms cut-offs were based on cis-gendered individuals.

Suicidal thoughts and behaviors (outcome). Two binary items asked participants whether they have ever seriously thought about attempting suicide and whether they have ever attempted suicide. Participants who marked “yes” on either item were also asked whether the suicidal thought or behavior occurred over past 12 months. In this study, we focused on suicidal thoughts and behaviors that occurred over the past 12 months.

Socio-demographic covariates. Sociodemographic information was collected through self-report items, including age (continuous), gender (man, woman, other), race/ethnicity (White, Black, Hispanic, Other), and education level (high school or less, some college, 4-year college or more). We adjusted for education level because both PE and mental health problems have been linked to socioeconomic status (Oh, Nagendra, et al., 2021). However, we acknowledge that socioeconomic status (e.g., education level) can be both a cause (social causation) and a consequence (social drift) of PE.

2.3 Analysis

We created a categorical variable as follows: No PE, non-distressing PE, and distressing PE. With this categorical variable, we examined the associations between distressing PE and a range of mental health outcomes (being in mental health treatment, loneliness, hazardous drinking, probable mental illness, suicidal ideation, and suicide attempt), with individuals who did not have any PE serving as the reference group. We adjusted all models for age, gender, race/ethnicity, and education level. We present all findings as adjusted odds ratios with 95% confidence intervals.

Sensitivity analyses. We restricted the sample to those who reported 12-month PE, and then used multivariable logistic regression to examine the relations between distressing PE and mental health outcomes, using individuals with non-distressing PE as the reference group. We repeated this analysis using lifetime PE.

4.1 Main findings

In this study, we explored whether there was evidence that including an item on distress associated with PE might enhance the sensitivity of the measure. Nearly half of the non-probability sample reported any lifetime psychotic experience. Although online survey samples tend to have high rates of psychopathology (Ophir et al., 2020), these rates still seem unlikely to represent true prevalence of PE. Specifically, we found that the most common type of lifetime psychotic experience was delusional mood, followed by persecutory ideation. Having a “feeling that something strange and unexplainable was going on that other people would find hard to believe” and a “feeling that people were too interested in you or that there was a plot to harm you?” could reflect paranoia, but could also capture normative experiences, particularly in unstable or hostile environments. We found that these items drove up the lifetime prevalence of PE. The literature acknowledges the potential to misidentify pathological versus nonpathological paranoia (Sen & Chowdhury, 2006; Whaley, 2001), some of which may emerge from a legitimate distrust of psychiatric services (Metzl, 2010; Metzl & Hansen, 2014).

We explored potential modifications of the abbreviated WHO CIDI 3.0 Psychosis Screen by adding a single item asking whether any of the PE caused distress. We found that both non-distressing and distressing PE were associated with being in mental health treatment, being lonely, having probable mental illness, and having suicidal thoughts and behavior, with greater odds for distressing PE, when compared with those who did not endorse PE. Findings comport with prior studies that have highlighted the clinical value of asking about distress (Karcher et al., 2022). The only outcome that was not significantly associated with distressing PE was hazardous drinking, which was only significantly associated with non-distressing PE. The reason for this is unclear and may speak to the complex set of factors that drive hazardous drinking, including social mechanisms (Sudhinaraset et al., 2016).

Distressing PE may be more indicative of psychopathology than non-distressing PE, potentially because the distressing PE may contain negative content. Laroi and colleagues (Larøi et al., 2019) offer some insight into this explanation by identifying different factors that may drive negative content in auditory-verbal hallucinations, including adverse life experiences and the presence of social and physical threats. The negative content of PE may directly cause more distress; however, it is also possible that the negative content may derive from a toxic, invalidating, or deprived environment, which would strongly shape risk for psychopathology. Laroi and colleagues also identify culture as a potential driver of negative content; potentially distress associated with PE may be conditional on how the larger socio-cultural environment understands and reacts to PE (e.g., normalizing the experiences vs. pathologizing the experiences). Finally, Laroi and colleagues also note that negative content of hallucinations may be related to emotion regulation strategies. Conceivably, people who have distressing PE may be less capable of managing the emotions in general, and difficulty with emotion regulation appears to relate to a range of psychopathology more broadly.

4.2 Limitations

Our findings should be interpreted bearing in mind several limitations. First, in the absence of an experimental design, we could not make any causal inferences. Large studies have suggested that the associations between PE and mental health problems can be bi-directional (McGrath et al., 2016). It is possible that PE precede mental health problems, or vice versa, and assessing temporality and order of events can be complicated.

Second, in terms of sampling, we used a non-probability sample recruited online, which may have been biased to exclude individuals who do not have reliable access to the Internet or computers, and/or biased to include self-selected individuals interested in mental health. The sample was majority White and women, and was only administered in English, which would have excluded individuals with limited English proficiency (e.g., recent immigrants). There was a relatively high prevalence of psychopathology, which reflects a sampling bias where people with mental health problems may be selecting into these online panel survey studies. Additionally, it is possible that people who are unemployed or have low income may have been drawn to the incentives provided by the study. Nonetheless, the findings of this study appear to comport with other surveys conducted in the general population in other settings (Karcher et al., 2022), and future research should use probability sampling to minimize bias and ensure that findings are representative of the larger population.

Third, in terms of measurement, the WHO CIDI psychosis screen can benefit from modification. For example, information about the onset, frequency, and duration of experiences may elucidate the clinical significance of the PE. Further, only hallucinations specified that they should not occur in the context of sleep or substance use; however, it is possible that other delusions may have occurred while under the influence of drugs. Additionally, we only included a single item about whether any of the lifetime experiences caused any distress, and so future studies should use more elaborate and precise items of distress. Along these lines, it is possible that adding a distress item may overlap with the probable mental health measure (Kessler-6), and so disentangling non-specific psychological distress from the distress caused by PE is an important area of future research.

Finally, based on the WHS data, there appears to be inter-cultural variability in prevalence of PE across countries using the WHO CIDI 3.0 psychosis screen. It is unclear the extent to which the variability may be reduced or further exacerbated by adding a distress item, since the meaning of distress itself may be different across languages and cultures. However, this would also set an agenda to move the field toward better defining distress across populations.

4.3 Implications

While the abbreviated WHO CIDI psychosis screen can expeditiously assess the presence of PE, it may also yield a high percentage of false positives as evidenced by the relatively high prevalence estimates when compared with large probability-based studies. An ongoing discussion around PE centers on how to reduce these false-positives and avoid pathologizing socially and culturally sanctioned experiences. Toward this end, our findings support the importance of this discussion as they further demonstrate the high prevalence of PE in online samples of adults in the general US population. In short, our findings suggest that modifications to the abbreviated WHO CIDI psychosis screen may make the measure more sensitive to clinically significant PE. First, we found a high prevalence of delusional mood and persecutory delusion, making these items potential culprits of false positives. It may be useful to add a question stem to impress upon respondents that PE should refer only to perceptual abnormalities and ideations that are unusual or bizarre, in accordance with a prior study (Kessler et al., 2005). Another idea would be to rephrase the delusional mood and persecutory ideation items entirely, or add examples parenthetically, though this requires more psychometric and instrumentation work. Second, and perhaps most importantly, the measure could include a single item on whether PE caused any distress, which is more strongly associated with mental health outcomes. Adding this single item could improve the measure's sensitivity, increasing the odds of capturing individuals who have clinically significant psychopathology.