Psychosis risk screening: a systematic review

Schizophrenia Research 158 (2014) 11–18

Contents lists available at ScienceDirect

Schizophrenia Research journal homepage: www.elsevier.com/locate/schres

Psychosis risk screening: A systematic review Emily Kline, Jason Schiffman ⁎ Department of Psychology, University of Maryland, Baltimore County, Baltimore, MD 21250, United States

a r t i c l e

i n f o

Article history: Received 15 April 2014 Received in revised form 13 June 2014 Accepted 16 June 2014 Available online 14 July 2014 Keywords: Psychosis Attenuated symptoms Clinical high-risk Prodrome Schizophrenia Screening

a b s t r a c t Despite the wealth of evidence linking duration of untreated psychosis to critical illness outcomes, most clinicians do not utilize any formal evaluation tools to identify attenuated or emerging psychotic symptoms. Given the costs associated with training and administration, interview-based assessments such as the Structured Interview for Psychosis Risk Syndromes (SIPS) are not likely to be widely adopted for clinical use. The ability to identify high-risk individuals through low-cost, brief methods is essential to the success of scalable prevention efforts. The aim of this article is to present a comprehensive review of the use of self-report forms as psychosis risk “screeners.” A literature search revealed 34 investigations in which authors used a self-report questionnaire as a first-step screener in a clinical high-risk assessment protocol. Information about each screener, including reported psychometric data, is presented within the review. Psychosis risk screeners have been used in diverse samples with the goals of validating assessments, screening populations for clinical referral, recruiting samples of interest for research participation, and estimating symptom prevalence and severity. Screeners focusing on attenuated psychotic experiences appear to measure a reliable construct with variable prevalence in help-seeking and general population samples. Administration of screeners to help-seeking populations can identify enriched samples with substantially elevated likelihood of meeting CHR criteria and transitioning to psychosis over time. More research is needed, however, to establish reliable norms and screening thresholds, as score elevations indicating a likely high-risk respondent appear to be unreliable across populations and settings. © 2014 Elsevier B.V. All rights reserved.

1. Introduction Duration of untreated psychosis has received considerable attention as a potentially modifiable prognostic factor impacting a variety of meaningful outcomes for individuals with schizophrenia and other psychotic spectrum disorders. Identification and intervention earlier in the course of illness appears to maximize treatment effectiveness and quality of life (Marshall et al., 2005). The imperative to reduce duration of untreated psychosis, as well as findings that symptoms of a psychosis ‘prodrome’ may appear months or years before onset of florid symptoms, has led to intensive research efforts regarding the possibility of identifying and treating illness during a pre-psychotic or ‘clinical high-risk’ (CHR) phase. Interview-based measures such as the Structured Interview for Psychosis Risk Syndromes (SIPS; Miller et al., 2003) and Comprehensive Assessment of At-Risk Mental States (CAARMS; Yung et al., 2005) have established a common set of risk criteria among researchers embarking on related but unique programs of research.

⁎ Corresponding author at: Department of Psychology, University of Maryland, Baltimore County, 1000 Hilltop Circle, Baltimore, MD 21250, United States. Tel.: +1 410 455 1535; fax: +1 410 455 1055. E-mail address: [email protected] (J. Schiffman).

http://dx.doi.org/10.1016/j.schres.2014.06.036 0920-9964/© 2014 Elsevier B.V. All rights reserved.

For several reasons, however, these measures are not well suited for contexts beyond specialty settings. Interviews targeting CHR status are typically lengthy, and clinicians must receive training to become familiar with the constructs, rating scales, and diagnostic criteria (McGlashan et al., 2010). The development of brief, easyto-use instruments that can be ‘exported’ for clinical use is a crucial step toward establishing and disseminating evidence-based care for individuals most vulnerable to psychosis. Brief self-report questionnaires have the potential to screen populations of interest and may ultimately aid in the detection of far more CHR individuals than would be possible through clinician- or selfreferrals to specialized programs, offering a potential solution to the challenge of sample ascertainment for CHR research programs. Although interview-based assessments appear to reliably identify a group with distinctive clinical impairment and at substantially increased risk for developing a psychotic illness, the majority of individuals meeting SIPS- or CAARMS-based CHR criteria are not expected to develop a psychotic disorder (Fusar-Poli et al., 2012). Refining the CHR construct by clarifying its characteristic symptoms and predictive relation to future psychopathology represents a broader goal that will inform and enhance treatment options for this population. Preliminary evidence suggests that screening provides a valid and efficient means of identifying and recruiting highrisk samples; samples recruited through screening may be even more likely to be ‘truly’ prodromal (i.e., have higher likelihood of transition)

12

E. Kline, J. Schiffman / Schizophrenia Research 158 (2014) 11–18

relative to samples identified through more idiosyncratic referral processes (Rietdijk et al., 2012; van der Gaag et al., 2012). Within the past decade, researchers have developed several brief self-report instruments to assess risk for developing psychosis (Kline et al., 2012). In several of these measures, item content focuses on symptoms associated with the attenuated symptom construct such as unusual perceptions and sensations, difficulty concentrating, affective changes, superstitious beliefs, or abnormally suspicious thoughts (e.g., Heinimaa et al., 2003; Ord et al., 2004; Loewy et al., 2005). These questionnaires differ from previous iterations of self-report forms designed to assess psychotic-spectrum experiences (e.g., Wisconsin Schizotypy Scales, Schizotypal Personality Questionnaire; Raine, 1991; Kwapil et al., 2008) in that the more recent questionnaires intend to predict CHR status rather than measuring ‘schizotypy’ as a trait construct. To this end, measures have been validated against the SIPS, CAARMS, schizotypy-focused interviews, and modified versions of the Structured Clinical Interview for DSM Disorder (SCID) and Kiddie Schedule for Affective Disorders and Schizophrenia (KSADS), with an emphasis on establishing clinical utility (i.e., predicting psychosis risk states and/or transitions to frank psychosis), rather than construct specificity per se (Liu et al., 2013; Cicero et al., in press). The use of various “gold-standard” measures of the psychosis risk construct across validation studies also reflects the heterogeneous settings, cultures, target populations, and aims of researchers employing CHR screening questionnaires. The aim of the current investigation is to conduct a systematic review of psychosis risk screening efforts to date, with the goal of consolidating available information about screening measures and strategies used in the field. By consolidating this information, we hope to identify successful approaches to psychosis risk screening as well as the limitations of current approaches and areas in need of further investigation.

2. Methods 2.1. Search method A systematic literature search was conducted using the PubMed and psycINFO databases to identify all articles published to date that used a self-report symptom questionnaire to assess putative risk for psychosis. No limitation regarding participants' age, gender, nationality, or clinical symptom presentation was applied. The last date on which a database search was conducted was June 8, 2014. The following search terms were selected and subsequently combined with the Boolean term “and”: (1) “prodrome” or “prodromal” or “psychosis risk” or “clinical high risk” or “ultra high risk” or “attenuated psychosis” or “early initial prodromal state” or “minor psychotic symptoms”; (2) “screener” or “screen” or “screening” or “self-report”; (3) “psychosis” or “schizophrenia”.1 Reference lists of included articles were also reviewed to identify additional relevant studies that may have been missed by the literature search.

2.2. Eligibility criteria and study selection Articles were included in the systematic review if they met the following criteria: available in English; reported original research; reported the name of the screener used by investigators; and provided data on either the proportion of the screened sample scoring above reported measure cutoffs, or the proportion of the screened sample meeting interview-based CHR criteria. Studies that reported only the covariance of screener scores with other self-report measures (e.g., Müller et al., 2009, 2010) were not included in the review.

1 Hyphenation of terms such as “clinical high-risk” (as opposed to “clinical high risk”) did not impact search results.

2.3. Data extraction For each included article, the following features of the study were noted and compiled: screening tool selected; nation in which research was conducted; number of participants screened; method of participant selection; whether screen results were compared to a more in-depth evaluation (and if so, type of evaluation used); threshold used for ‘positive’ screen result; sensitivity and specificity of screener to interviewbased diagnosis; and conclusions related to screen results. Both authors independently read and extracted information from each study. Inconsistencies were minor and resolved through consensus discussion. 3. Results The search term yielded a total of 121 studies on PsycINFO and 131 on PubMed. Reviewing the reference lists revealed additional articles that were scanned for potential inclusion. Thirty-five articles met the criteria listed in Section 2.2 for inclusion in this systematic review. Included investigations can generally be grouped into three categories. Measure validation studies involve explicit comparison of screener results to a gold standard assessment of psychosis risk status (e.g., Loewy et al., 2005). A second category involves the “real world” use of screeners to detect symptomatic individuals for clinical referral or to recruit samples of interest to CHR research (e.g., Quijada et al., 2010). A third category of research comprises studies in which questionnaires were employed not for actual screening but rather to examine the covariance of psychosis risk symptoms with other demographic and/or clinical covariates within a given sample. Studies in this last category were further considered only if authors also reported the “screening results” (that is, how many participants were “positive screens” and/or how screening results aligned with clinician diagnoses; e.g. Kobayashi et al., 2011). The included articles describe a total of 13 different measures that had been used as psychosis risk screeners. A list of measures and details regarding each of the 34 studies appears in Table 1. All instruments were administered as self-report (as opposed to clinicianor parent-focused checklists) measures unless otherwise specified. 4. Discussion 4.1. Review of screening research The systematic review identified 13 self-report measures used in 14 countries with the goal of identifying individuals at clinical high risk for psychosis. Studies comparing screening results to CHR/psychosis diagnoses based on more validated interview measures (e.g., SIPS, CAARMS) reported a wide range of psychometric data. Depending on the setting, population, measure, score threshold, and gold standard assessment, sensitivities ranged from 0.50 to 1.00 and specificities ranged from 0.10 to 1.00. No single measure has demonstrated both sensitivity and specificity exceeding 0.70 on more than two studies. Overall, given the imperfect ability of screening to predict clinician diagnoses, studies in which self-report scores serve as a proxy for face-to-face psychosis risk evaluations should be interpreted bearing the limitations of this approach. The measures themselves vary in the extent to which they focus on attenuated positive symptoms vs. assessing a range of symptoms associated with high-risk states. Although the psychosis prodrome is characterized by a number of cognitive, behavioral, and affective changes (Yung and McGorry, 1996), the widely-used SIPS and CAARMS emphasize the emergence of positive symptoms in their CHR syndrome criteria. Given this emphasis on positive symptoms for risk identification, it is not surprising that screeners with a narrow focus on attenuated positive symptoms (e.g., Prodromal Questionnaire — Brief [PQ-B], Prime Screen — Revised [PS-R], Youth Psychosis At-Risk Questionnaire — Brief [YPARQ-B], Eppendorf Schizophrenia Inventory [ESI]) appear to align more reliably with SIPSand CAARMS-based diagnoses. Measures such as the PROD-Screen (PROD) and General Health Questionnaire (GHQ) that contain many

E. Kline, J. Schiffman / Schizophrenia Research 158 (2014) 11–18

13

Table 1 Screening tools for detection of psychosis risk states. Measure

First author, year

Sample

Findings

Prodromal Questionnaire (PQ; 92 items)

Loewy et al., 2005

US adolescents and young adults ages 12–35 referred for CHR evaluation (N = 113)

Loewy et al., 2007

US university students (N = 1020)

Lindgren, 2010

Finnish adolescents ages 15–18 seeking psychiatric treatment (N = 836)

Ising et al., 2012

Dutch young adults ages 18–35 seeking mental health services (N = 420)

Loewy et al., 2012

Finnish adolescents ages 15–18 seeking psychiatric treatment (N = 408)

Rietdijk et al., 2012

Dutch young adults ages 18–35 seeking psychiatric treatment (N = 3671)

van der Gaag et al., 2012

Dutch adolescents and young adults ages 14–35 seeking psychiatric treatment (N = 5800)

Loewy et al., 2011

US adolescents and young adults ages 12–35 referred for CHR evaluation (N = 141)

Jarrett et al., 2012

United Kingdom incarcerated men age 21 and older (N = 750)

Kline et al., 2012

US adolescents and young adults ages 12–22 receiving mental health services (N = 49)

Kline et al., 2014

US adolescents and young adults ages 12–22 receiving mental health services (N = 66)

Zhang et al., 2014

Chinese outpatients ages 15–45 seeking psychiatric treatment (N = 2101)

Ising et al., 2012

Dutch young adults ages 18–35 seeking mental health services (N = 420)

Chen et al., 2014

Chinese college students ages 16–22 (N = 579)

Using a cutoff of eight or more positive symptom endorsements, the PQ achieved sensitivity of 0.90, specificity of 0.49, and PPV of 0.78 with regard to SIPS CHR or psychosis diagnoses. 43% of respondents scored above the threshold of ≥8 positive symptom responses. 2% endorsed eight or more positive symptoms that they described as distressing. No follow-up clinical interviews were conducted. Measure used as a first-step screener for possible inclusion in a study investigating neurocognition in CHR youth. Of those scoring above the cutoff of ≥18 positive symptom endorsement (15% of those screened), 49% (n = 62) met criteria for a SIPS CHR diagnosis. (Participants with psychosis were not included in analysis sample.) An optimized threshold score of ≥18 positive symptoms yielded sensitivity of 0.90, specificity of 0.90, and PPV of 0.52 with regard to CAARMS CHR or psychosis diagnoses. Authors report that a threshold of 18 positive symptom endorsements yielded optimal psychometrics within a pilot subsample of participants. After screening 408 help-seeking adolescents, investigators conducted SIPS assessment with the highest-scoring 20%, as well as 10% of the remaining sample (interviewed N = 99). A threshold of 18 positive symptom endorsements achieved sensitivity of 0.82, specificity of 0.49, and PPV of 0.51 with regard to SIPS CHR diagnoses (8 participants with psychosis were excluded from analyses). 14 high PQ participants were followed for 1 year, 3 of whom transitioned to psychotic illness. Measure administered upon first contact with the mental health system. Screening identified 420 patients (11%) who endorsed ≥18 positive symptoms. Of these, 199 (47%) met CAARMS criteria for ARMS (n = 147, 35%) or psychosis (n = 52, 12%). Screening yielded a greater number of identified cases than clinician referrals, and screen-recruited cases were more likely to progress to psychosis. At 12-month follow-up, 21 of 78 (27%) positive screens had converted to psychosis. PQ used to screen for inclusion in a high-risk intervention trial. 5800 patients were screened with the measure. Those scoring ≥18 on the positive symptom scale (864 or 15%) were assessed using the CAARMS. Of those interviewed, 302 were determined to be high-risk and 112 had a current or past psychotic disorder, yielding an overall PPV of 0.48. Authors created PQ-B by reducing the total number of items to 21 and adding a ‘distress scale’ on which respondents use a 5-point Likert scale to indicate the degree of distress associated with each endorsed symptom. An optimized cut-off score of ≥3 endorsements on the PQ-B yielded sensitivity of 0.89, specificity of 0.58, and PPV of 0.93 with regard to SIPS CHR or psychosis diagnoses. A ‘distress’ score of ≥6 yielded sensitivity of 0.88, specificity of 0.68, and PPV of 0.95. PQ-B used to screen 750 newly arrived inmates at a men's prison. All of the 329 positive-screen participants (44% of the screened population) completed CAARMS interviews; 60 negative-screen participants were also interviewed. A screening threshold of ≥4 items that cause distress yielded sensitivity of 0.90, specificity of 0.44, and PPV of 0.29 with regard to psychosis or ARMS diagnoses. The author-recommended screening threshold (≥6) yielded sensitivity of 0.95, specificity of 0.28, and PPV of 0.48. The within-sample optimized threshold was substantially higher (≥38) and yielded sensitivity of 0.70, specificity of 0.82, and PPV of 0.74 with regard to SIPS CHR or psychosis diagnoses. A distress score of ≥18 yielded sensitivity of 0.77, specificity of 0.68, and PPV of 0.61 with regard to SIPS CHR or psychosis diagnosis. Counting only items for which respondents positively endorsed distress, a threshold of ≥4 endorsements yielded sensitivity of 0.73, specificity of 0.83, and PPV of 0.73. 80% of respondents scored above the author-recommended score threshold of 6 or greater, and a subsample of 1461 (representing both positive and negative screens) was assessed via SIPS. According to a flowchart provided by the authors, the measure yielded sensitivity of 0.97 and specificity of 0.06 with regard to SIPS CHR or psychosis diagnoses. Researchers employed a step-wise logistic regression to select the most predictive items from the PQ-92. Similar to the PQ-B, the PQ-16 contains a distress scale. Using a total score cutoff of ≥6, PQ-16 demonstrated sensitivity of 0.87, specificity of 0.87, and PPV of 0.44 with regard to CAARMS CHR or psychosis diagnoses within this sample. 54 (9%) of screened participants scored over a score threshold of ≥6. 49 of the positive screen cases and 50 negative screen cases were further assessed via SIPS. Within this subsample of 99, 20 were found to meet SIPS CHR criteria, yielding sensitivity of 1.00, specificity of 0.63, and PPV of 0.41. No participants were found to have psychosis on interview.

Prodromal Questionnaire — Brief (PQ-B; 21 items)

Prodromal Questionnaire-16 (16 items)

(continued on next page)

14

E. Kline, J. Schiffman / Schizophrenia Research 158 (2014) 11–18

Table 1 (continued) Measure

First author, year

Sample

Prime Screen — Revised (PS-R; 12 items)

Miller et al., 2004

Using a score threshold of ≥2 endorsements of “somewhat” or “definitely agree” to indicate a possible positive case yielded sensitivity of 0.90 and specificity of 1.00 with regard to SIPS CHR or psychosis diagnoses. (Information was gathered from a poster indicating that data collection was ongoing.) Japanese outpatients ages 16–30 (N = 528) 28% of respondents scored above a moderately revised screening threshold. A randomly selected subsample (N = 115) was selected for further assessment using the SIPS. Within the interviewed subsample considering either risk or psychosis, authors found PS-R sensitivity to be 1.00, specificity to be 0.73, and positive predictive value (PPV) to be 0.46. Using six-month followup interviews to detect transition to psychosis, authors determined the ‘predictive’ sensitivity to be 1.00, specificity to be 0.60, and PPV to be 0.11. Japanese high school (N = 285) and university University students' mean total (14.5) was significantly higher than the (N = 496) students high school students' (12.9); however, a greater number of high school students (13%) scored above the screening threshold of 2 or more ‘agree’ endorsements relative to university students (9%). The factor structure was similar in the 2 groups. No clinical interviews were conducted. Japanese outpatients ages 16–30 (N = 750) and Within the outpatient sample, 27% of respondents scored above the high school students (N = 781) screen threshold on the PS-R; within the high school sample, 10% scored over threshold. No clinical interviews were conducted. US adolescents and young adults ages 12–22 The author-recommended screening threshold (≥2) yielded sensitivity receiving mental health services (N = 49) of 0.80, specificity of 0.48, and PPV of 0.52 with regard to SIPS CHR/psychosis diagnoses. The within-sample optimized threshold was three or more endorsements and yielded sensitivity of 0.75, specificity of 0.66, and PPV of 0.60. US adolescents receiving mental health services A caregiver version (CGPS-R) of the PS-R was created by changing the (N = 52) phrasing of PS-R items from “I/me” to “my child.” The PS-R, CGPS-R, and SIPS were administered to 52 adolescent–parent dyads referred for CHR evaluation. The PS-R alone yielded sensitivity of 0.81, specificity of 0.60, and PPV of 0.69 with regard to SIPS CHR/psychosis diagnoses. Combining data from both parent and teen reports yielded sensitivity of 0.74, specificity of 0.76, and PPV of 0.77. Kenyan youth ages 14–29 recruited from a Nairobi Culturally modified version. A threshold of three “5” (somewhat agree) neighborhood (N = 182) or one “6” (definitely agree) responses yielded sensitivity of 0.40, specificity of 0.64, and a PPV of .12 with regard to SIPS diagnoses (authors do not specify whether any individuals met criteria for a psychotic disorder following SIPS assessment). Palau high school students (N = 648) 15% of respondents scored above a threshold of ≥11 or more endorsements. 74 high scoring and 56 other participants underwent clinician interviews (the study authors used a modified version of the Kiddie Schedule for Affective Disorder and Schizophrenia (K-SADS) to assess psychosis risk symptoms). With regard to clinician diagnosis from the K-SADS, the authors found the sensitivity of 0.98, specificity of 0.81, and PPV of 0.82. US university students (N = 998) 5% of respondents scored above the author-recommended threshold of ≥11 endorsements; responses used as selection criteria for (psychometrically-defined) high-risk sample recruitment.

Kobayashi et al., 2008

Kobayashi et al., 2010

Kobayashi et al., 2011 Kline et al., 2012

Kline et al., 2013

Owoso et al., 2014

Ord et al., 2004 Youth Psychosis At-Risk Questionnaire Brief Version (YPARQ-B; 28 items)

Bedwell and Donnelly, 2005; Bedwell and Orem, 2008 Kline et al., 2012

PROD-screen (21 items)

Eppendorf Schizophrenia Inventory (ESI; 40 items)

Findings

United States (US) adolescents and young adults referred for CHR evaluation (N = 36)

US adolescents and young adults ages 12–22 receiving mental health services (N = 49)

Heinimaa et al., 2003 Epidemiologically mixed (outpatient, general population, genetic high-risk) Finnish adult sample (N = 132) Granö et al., 2011

Finnish adolescents ages 12–20 referred for CHR evaluation (N = 87)

Niessen et al., 2010

Dutch patients ages 16–35 referred to high-risk treatment program (N = 160)

Kang et al., 2012

Korean high school students (N = 1190)

Chung et al., 2013

Korean high school students ages 16–17 (N = 1002)

The author-recommended screening threshold (≥11) yielded sensitivity of 0.65, specificity of 0.76, and PPV of 0.65 with regard to SIPS CHR/psychosis diagnoses. The within-sample optimized threshold was slightly higher (≥13) and yielded sensitivity of 0.65, specificity of 0.90, and PPV of 0.81. 41% of respondents scored above the authors' threshold of ≥2 ‘specific’ symptoms. Using this threshold, the PROD achieved sensitivity of 0.80, specificity of 0.75, and PPV of 0.57 with regard to CHR status as determined by SIPS interview. Participants completed the PROD screen as a written self-report measure, then answered PROD items in face-to-face interview with clinician. The mean self-report score (3.89) was significantly lower than the mean interview score (2.30), suggesting that participants endorsed fewer items when assessed via clinician interview. A self-report score of ≥2 yielded sensitivity of 1.00, specificity of 0.50, and PPV of 0.70 with regard to interview-based PROD diagnoses of CHR/psychosis. Participants were assessed using a structured clinical interview (SIPS or BSABS). The authors report sensitivities ranging from 0.50 to 0.81 and specificities of 0.52–0.94 using various score thresholds to distinguish CHR/psychosis cases from low-risk individuals. Translated and modified for Korean. 78 students (7%) scored above the Korean ESI (K-ESI) threshold of 29 or higher. Of those who scored above the threshold, 15 (19% of high scorers, or 1% of the sample) met CAARMS high-risk criteria. Translated and modified for Korean. Students completed a general mental health screeners, the Korean Youth Self Report (K-YSR). 120 students who screened above K-YSR clinical thresholds completed an ESI and CAARMS. 13 (1% of the study sample) met CAARMS UHR criteria. The authors found that an ESI score of ≥29 yielded sensitivity of 0.77 and specificity of 0.70 with regard to CAARMS high-risk diagnosis.

E. Kline, J. Schiffman / Schizophrenia Research 158 (2014) 11–18

15

Table 1 (continued) Measure

First author, year

Sample

Early Recognition Inventory (ERIraos) Checklist (17 items)

Häfner et al., 2004; Bechdolf et al., 2005

German participants seeking mental health services ages 19–35 (N = 1212)

Quijada et al., 2010

BASC Atypicality Scale (9 items within a 176-item survey)

Thompson et al., 2013

Thompson et al., 2014

Composite Psychosis Risk Questionnaire (15 items)

Liu et al., 2013

Early Detection Primary Care Checklist (PCCL; 20 item and 6 item versions)

French et al., 2012

General Health Questionnaire Razali et al., 2011 (GHQ-12; 12 items) Community Assessment of Psychic Experiences (CAPE; 42 items)

Mossaheb et al., 2012

Findings

Checklist and scoring instructions appear in Häfner et al., 2004. Screener was administered by general practitioners, counselors, psychiatrists, and psychotherapists. 994 respondents (82%) scored over the screening threshold of ≥6. Of these, 388 (39%) met high-risk criteria as defined by the clinician-administered Early Recognition Inventory (ERIraos symptom list). Spanish patients ages 12–56 seeking mental Any person scoring ≥3 on the checklist was referred for specialized health care (screened N not reported) assessment and completed a SIPS. Of 55 cases assessed, 29 (53%) met criteria for high-risk state or emergent psychosis. US adolescents and young adults ages 12–22 The BASC-2 scoring system provides standardized t-scores for each receiving mental health services (N = 70) assessed domain, with a t-score of ≥60 indicating likely clinical problems. Using this threshold for the atypicality subscale of the measure yielded sensitivity of 0.65, specificity of 0.87, and PPV of 0.80 with regard to SIPS CHR/psychosis diagnosis. US adolescents and young adults ages 12–22 The BASC-2 self- and parent-report forms were completed by 62 carereceiving mental health services (N = 63) giver–youth dyads. A threshold t-scores of ≥60 on the atypicality subscale of the self-report form yielded sensitivity of 0.68, specificity of 0.79, and PPV of 0.79 with regard to SIPS CHR/psychosis diagnosis. When parent reports were considered alongside self-report, a mean score of ≥60 yielded sensitivity of 0.82, specificity of 0.79, and PPV of 0.82. Taiwanese participants ages 16–32 designated as Items were drawn from Mandarin translations of the Schizophrenia high-risk (N = 111), low-risk help-seeking Proneness Scale, the Schizotypal Personality Questionnaire, as well as a (N = 95), or controls (N = 129) basic symptoms scale. Fifteen items with the strongest ability to distinguish high-risk from low-risk cases within this sample were selected from a 231-item pool. A cutoff of 8 or more endorsements within these 15 items yielded sensitivity of 0.56, specificity of 0.88, and PPV of 0.70 with regard to UHR diagnoses (participants with psychosis were excluded from analysis sample). The PCCL was completed by primary care practitioners who referred UK adolescents and young adults ages 14–34 positive screens for specialized psychiatric assessment. With regard to referred for CHR assessment by a primary care CAARMS CHR/psychosis diagnoses, the PCCL was found to have excellent provider (N = 136) sensitivity (0.96) but poor specificity (0.10). An optimized 6-item version yielded a sensitivity of .88 and specificity of .47; an optimized 20-item version sensitivity of .89 and specificity of .60. Malaysian relatives of patients with schizophrenia Following CAARMS assessment, only 3 participants were determined to ages 13–30 (N = 111) meet criteria for a high-risk state, thus precluding further analysis of screener effectiveness. Austrian adolescents and young adults ages 13–24 A threshold of 3.20 on the positive symptom scale yielded sensitivity of referred for a psychosis risk evaluation (N = 165) 0.67, specificity of 0.73, and PPV of 0.72 with regard to CAARMS CHR diagnoses (15 participants found to have psychosis were excluded from analyses). A threshold of 2.80 yielded sensitivity of 0.83, specificity of 0.49, and PPV of 0.63.

items assessing non-positive symptoms (e.g., mood and sleep disturbances) may be too non-specific to distinguish between SIPS/CAARMS identified CHR respondents and those with non-psychotic affective disorders, although they may effectively capture other relevant clinical concerns. Most studies that used a screener for clinical assessment have done so within the context of help-seeking populations and psychiatric clinics. Individuals presenting for psychiatric care represent a selfselected “enriched” group in that they are, at a minimum, experiencing some type of distress and/or disturbance in functioning. Screeners have been used effectively within general psychiatric or counseling settings to triage or refer likely high-risk patients for more specialized evaluations. When used as the basis for referrals to specialized clinics, positive predictive values for screeners have ranged from 39 to 53%, indicating that these tools are clinically useful for selecting a group with heightened psychosis risk as opposed to unrelated psychiatric or behavioral concerns (Bechdolf et al., 2005; Lindgren et al., 2010; Niessen et al., 2010; Quijada et al., 2010; Ising et al., 2012; Rietdijk et al., 2012; van der Gaag et al., 2012). In particular, the Prodromal Questionnaire, PQB, and PQ-16 have collectively garnered the most “real world” evidence for their screening utility use relative to less tested measures. In contrast, studies in which investigators screened general populations such as those found in school, primary care, and community settings generally found poor specificity (French et al., 2012; Owoso et al., 2014) and/or a very low case prevalence (Razali et al., 2011; Kang et al., 2012). Due to such limitations, these studies were generally

less effective in leveraging screening tools for CHR identification. It is possible that a higher threshold or more specific items would be necessary to minimize false positives in general population screening; alternatively, the incidence of psychosis within the general population may be too low to justify the costs of screening. The specific goals and contexts of various screening efforts will likely dictate the justifiable costs. A recruitment effort in which investigators hope to identify every possible case would prioritize a highly sensitive instrument, even at the cost of reduced specificity or an overall low yield of identified cases. Alternatively, a setting with high need and limited resources for conducting lengthy clinical evaluations would demand a highly specific tool that would quickly narrow the pool of likely CHR individuals, even at the risk of missing some cases. Because findings regarding CHR prevalence and appropriate scoring thresholds have been inconsistent across heterogeneous samples, assessors should be sensitive to context when choosing a measure and/or threshold score for screening. 4.2. Estimating CHR prevalence Studies that conducted a two-step screening and interview procedure using an accepted gold-standard assessment for CHR status (i.e., the CAARMS or SIPS) enable estimation of CHR prevalence within these samples. Within unselected help-seeking adolescent and young adult populations, the prevalence of identified CHR cases was estimated to be quite consistent at 4–5% across several studies (Lindgren et al., 2010; Ising et al., 2012; Jarrett et al., 2012; Rietdijk et al., 2012; van

16

E. Kline, J. Schiffman / Schizophrenia Research 158 (2014) 11–18

der Gaag et al., 2012; Zhang et al., 2014). Researchers who conducted screening and interviews within adolescent and young adult general population samples estimated prevalence of 1% (Kang et al., 2012; Chung et al., 2013), 3% (Chen et al., 2014), and 4% (Owoso et al., 2014). Unsurprisingly, samples in which participants were referred for specialized psychosis evaluations reported higher proportions (33– 51%) of CHR cases, regardless of whether participants were formally screened prior to SIPS/CAARMS assessment (Loewy et al., 2005; Niessen et al., 2010; Granö et al., 2011; Kline et al., 2012; Mossaheb et al., 2012). Although these studies were not intended to describe CHR epidemiology and could potentially underestimate prevalence due to the assumption that negative screens are in fact “true” negatives, these figures may prove useful for planning CHR evaluation and treatment services. 4.3. Criteria for effective screening Across instances of illness, clinicians must consider similar disease, screening, and outcome criteria in order to determine whether a screening protocol is justified and/or likely to reduce morbidity in a given population. To this end, an article reviewing “ten criteria for effective screening” (Table 2), offers a valuable review of important considerations for any screening mechanism (Obuchowski et al., 2001). Although this list of criteria is certainly not exhaustive, the criteria provide a useful heuristic for examining the potential benefits and pitfalls of screening for psychosis risk symptoms. Given literature to date on the costs and long-term disability associated with schizophrenia and related disorders, it is clear that psychosis is associated with substantial morbidity and mortality (Saha et al., 2007; criterion one from Table 2). Most individuals on a trajectory toward psychosis experience a recognizable prodromal phase, which has been described and codified within the current standards for high-risk identification (Fusar-Poli et al., 2013; criterion two). The body of research reviewed highlights problems – as well as potential solutions – regarding existing screening tools and procedures (criteria three and four). Intervention trials utilizing cognitive behavioral therapies (Morrison et al., 2007; Hutton and Taylor, 2014; Okuzawa et al., in press) and pharmacologic agents (Stafford et al., 2012; van der Gaag et al., 2013) have demonstrated some evidence that detection and treatment of clinical high-risk states may have the potential to prevent or minimize the impact of psychosis (criteria five and eight); however, more research is needed to establish the safety and efficacy of intervention in CHR populations relative to standard treatment (criteria nine and ten). Finally, although concerns regarding stigma and clinician burden pose challenges regarding criteria six and seven, several of the studies featured in Table 1 suggest that some screening programs have successfully managed these possible barriers (e.g., Quijada et al., 2010; Bechdolf et al., 2012; Jarrett et al., 2012; Rietdijk et al., 2012; van der Gaag et al., 2012). These criteria help to highlight gaps in the current knowledge that must be addressed in order to implement effective screening. 4.4. Limitations of the current review The current review represents a rapidly expanding field of study, and thus is subject to many limitations that preclude any broad conclusions about the ultimate success or failure of psychosis risk screening efforts. First, it is difficult to compare the use of a single screener across settings and populations. Given these differences, as well as the variety of aims of the screening studies included in this review, screener effectiveness should be considered in the context of each investigation. Second, the measures themselves – including their content, number of items, and score thresholds – vary widely. Third, the “gold standard” against which screener psychometrics are estimated varies across studies. Finally, few of the studies included in this review contain longitudinal data. Long-term follow-up after screening will help to determine the

extent to which screeners are able to select a group with increased likelihood of developing psychosis over time. 4.5. Future directions for research The instability of optimal screening thresholds across populations and settings constitutes a major barrier to successful screening efforts. This variability may be due to local differences in culture, development, and clinical severity across a wide range of settings. Administering screeners to large general population samples is crucial for establishing culturally- and developmentally-sensitive norms. Without normative data and sensitivity to different populations, it is difficult to determine what constitutes a clinically meaningful elevation. Beyond the aim of initial detection and diagnosis, the expansion of evidence-based care will require tools for ongoing symptom monitoring. Pending further controlled trials investigating potential intervention approaches for preventing psychosis among CHR patients, current practice guidelines emphasize frequent clinical monitoring to detect symptom exacerbation or transition to psychosis (Fusar-Poli et al., 2013). Although the use of brief self-report assessments at regular intervals to monitor clinical status has become routine in many clinical practices (Lambert et al., 2003), no instrument assessing attenuated psychotic symptoms has yet been validated for this purpose. Finally, this review does not address the issue of psychosis stigma, which may profoundly impact the risk to benefit ratio of screening efforts. Whether screening creates distress for those referred for further evaluation, or negatively impacts an individual's trust in his or her mental health care providers, is likely to vary widely across settings and cultures. Assessing attenuated psychosis symptoms within a broader clinical evaluation may help to minimize the stigma that could accompany psychosis risk screening (Thompson et al., 2013). Patients' perceptions of psychosis risk screening have not yet been studied and should be weighed alongside psychometric and cost considerations. 4.6. Conclusion Effective screening would allow the low-cost identification of individuals at clinical high risk for psychosis, either in the general population or at their first contact with mental health services. This systematic review identified 34 studies reporting the results of psychosis risk screening investigations. Although screeners have successfully been used to identify high-risk cases, no screener has emerged that can reliably predict the result of face-to-face evaluations such as the SIPS and CAARMS across all contexts and populations. An improved understanding of normative and elevated scores, as well as thoughtful consideration of how to minimize the cost and stigma incurred by screening, will help to improve the usefulness of existing measures. Findings to date suggest that screening appears to be effective for identifying those who may benefit from a more specialized, clinician-administered evaluation within indicated (i.e., help-seeking) populations. Role of the funding source The funders were not involved in study design, analyses, manuscript preparation, or decision to submit for publication. Table 2 Ten criteria for effective screening (from Obuchowski et al., 2001). 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

The disease has serious consequences There is high prevalence of a detectable pre-clinical phase Screening detects little pseudodisease Screening test has high accuracy Screening detects disease before a critical point Screening causes little morbidity Screening is affordable and available Treatment exists Treatment is more effective when applied during stage detected by screening Treatment is not too risky

E. Kline, J. Schiffman / Schizophrenia Research 158 (2014) 11–18 Contributors Emily Kline conducted the systematic review and prepared the manuscript. Jason Schiffman served as advisor and editor for all components of the project.

Conflict of interest The authors have no actual or potential conflicts of interest to disclose.

Acknowledgment This work was supported in part by funding from the Maryland Department of Health and Mental Hygiene, Mental Hygiene Administration through the Center for Excellence on Early Intervention for Serious Mental Illness (OPASS# 14-13717G/M00B4400241) and the 1915(c) Home and Community-Based Waiver Program Management, Workforce Development and Evaluation (OPASS# 13-10954G/M00B3400369).

References Bechdolf, A., Ruhrmann, S., Wagner, M., Kühn, K.U., Janssen, B., Bottlender, R., et al., 2005. Interventions in the initial prodromal states of psychosis in Germany: concept and recruitment. Br. J. Psychiatry 187 (48), s45–s48. Bechdolf, A.A., Wagner, M.M., Ruhrmann, S.S., Harrigan, S., Putzfeld, V., Pukrop, R., et al., 2012. Preventing progression to first-episode psychosis in early initial prodromal states. Br. J. Psychiatry 200 (88), 22–29. Bedwell, J.S., Donnelly, R.S., 2005. Schizotypal personality disorder or prodromal symptoms of schizophrenia. Schizophr. Res. 80 (2–3), 263–269. Bedwell, J.S., Orem, D.M., 2008. The effect of red light on backward masking in individuals with psychometrically defined schizotypy. Cogn. Neuropsychiatry 13 (6), 491–504. Chen, F., Wang, L., Heeramun-Aubeeluck, A., Wang, J., Shi, J., Yuan, J., et al., 2014. Identification and characterization of college students with attenuated psychosis syndrome in China. Psychiatry Res. 216 (3), 346–350. Chung, Y.C., Kang, N.I., Im, Y.J., Kim, S.W., Cho, I.H., Lee, Y.M., et al., 2013. Validation of the Korean version of the Eppendorf Schizophrenia Inventory as a screening measure to detect adolescents at ultra‐high risk for psychosis. Early Interv. Psychiatry 7 (1), 71–79. Cicero, D.C., Martin, E.A., Becker, T.M., Docherty, A.R., Kerns, J.G., 2014. Correspondence between psychometric and clinical high risk for psychosis in an undergraduate population. Psychol. Assess. (epub ahead of print, in press). French, P., Owens, J., Parker, S., Dunn, G., 2012. Identification of young people in the early stages of psychosis: validation of a checklist for use in primary care. Psychiatry Res. 200 (2–3), 911–916. Fusar-Poli, P., Bonoldi, I., Yung, A.R., Borgwardt, S., Kempton, M.J., Valmaggia, L., et al., 2012. Predicting psychosis: meta-analysis of transition outcomes in individuals at high clinical risk. JAMA Psychiatry 69 (3), 220–229. Fusar-Poli, P., Borwardt, S., Bechdolf, A., Addington, J., Reicher-Rössler, A., Schultze-Lutter, F., et al., 2013. The psychosis high-risk state: a comprehensive state-of-the-art review. JAMA Psychiatry 70 (1), 107–120. Granö, N., Karjalainen, M., Itkonen, A., Anto, J., Edlund, V., Heinimaa, M., et al., 2011. Differential results between self-report and interview-based ratings of risk symptoms of psychosis. Early Interv. Psychiatry 5 (4), 309–314. Häfner, H., Maurer, K., Ruhrmann, S., Bechdolf, A., Klosterkötter, J., Wagner, M., et al., 2004. Early detection and secondary prevention of psychosis: facts and visions*. Eur. Arch. Psychiatry Clin. Neurosci. 254 (2), 117–128. Heinimaa, M., Salokangas, R.K.R., Ristkari, T., Plathin, M., Huttunen, J., IIonen, T., et al., 2003. PROD-screen: a screen for prodromal symptoms of psychosis. Int. J. Methods Psychiatr. Res. 12 (2), 92–104. Hutton, P., Taylor, P.J., 2014. Cognitive behavioural therapy for psychosis prevention: a systematic review and meta-analysis. Psychol. Med. 44, 449–468. Ising, H.K., Veling, W., Loewy, R.L., Rietveld, M.W., Rietdijk, J., Dragt, S., et al., 2012. The validity of the 16-item Prodromal Questionnaire (PQ-16) to screen for ultra high risk of developing psychosis in the general help-seeking population. Schizophr. Bull. 38 (6), 1288–1296. Jarrett, M., Craig, T., Parrott, J., Forrester, A., Winton-Brown, T., Maguire, H., et al., 2012. Identifying men at ultra high risk of psychosis in a prison population. Schizophr. Res. 136 (1–3), 1–6. Kang, N.I., Park, T.W., Yang, J.C., Oh, K.Y., Shim, S.H., Chung, Y.C., 2012. Prevalence and clinical features of Thought–Perception–Sensitivity Symptoms: results from a community survey of Korean high school students. Psychiatry Res. 198 (3), 501–508. Kline, E., Wilson, C., Ereshefsky, S., Denenny, D., Thompson, E., Pitts, S.C., et al., 2012. Psychosis risk screening in youth: a validation study of three self-report measures of attenuated psychosis symptoms. Schizophr. Res. 141 (1), 72–77. Kline, E., Thompson, E., Schimunek, C., Reeves, G., Bussell, K., Pitts, S.C., et al., 2013. Parent–adolescent agreement on psychosis risk symptoms. Schizophr. Res. 147 (1), 147–152. Kline, E., Thompson, E., Bussell, K., Pitts, S.C., Reeves, G., Schiffman, J., 2014. Psychosis-like experiences and distress among adolescents using mental health services. Schizophr. Res. 152 (2–3), 498–502. Kobayashi, H., Nemoto, T., Koshikawa, H., Osono, Y., Yamazawa, R., Murakami, M., et al., 2008. A self-reported instrument for prodromal symptoms of psychosis: testing the clinical validity of the PRIME Screen — Revised (PS-R) in a Japanese population. Schizophr. Res. 106 (2–3), 356–362. Kobayashi, H., Yamazawa, R., Nemoto, T., Murakami, M., Kashima, H., Mizuno, M., 2010. Correlation between attenuated psychotic experiences and depressive symptoms among Japanese students. Early Interv. Psychiatry 4 (3), 200–205.

17

Kobayashi, H., Nemoto, T., Murakami, M., Kashima, H., Mizuno, M., 2011. Lack of association between psychosis-like experiences and seeking help from professionals: a casecontrolled study. Schizophr. Res. 132 (2–3), 208–212. Kwapil, T.R., Barrantes-Vidal, N., Silvia, P.J., 2008. The dimensional structure of the Wisconsin Schizotypy Scales: factor identification and construct validity. Schizophr. Bull. 34 (3), 444–457. Lambert, M.J., Whipple, J.L., Hawkins, E.J., Vermeersch, D.A., Nielsen, S.L., Smart, D.W., 2003. Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clin. Psychol. 10 (3), 288–301. Lindgren, M., Manninen, M., Laajasalo, T., Mustonen, U., Kalska, H., Suvisaari, J., et al., 2010. The relationship between psychotic-like symptoms and neurocognitive performance in a general adolescent psychiatric sample. Schizophr. Res. 123 (1), 77–85. Liu, C.C., Tien, Y.J., Chen, C.H., Chiu, Y.L., Hsieh, M.H., Liu, C.M., et al., 2013. Development of a brief self-report questionnaire for screening putative pre-psychotic states. Schizophr. Res. 143 (1), 32–37. Loewy, R.L., Bearden, C.E., Johnson, J.K., Raine, A., Cannon, T.D., 2005. The Prodromal Questionnaire (PQ): preliminary validation of a self-report screening measure for prodromal and psychotic syndromes. Schizophr. Res. 79 (1), 117–125. Loewy, R.L., Johnson, J.K., Cannon, T.D., 2007. Self-report of attenuated psychotic experiences in a college population. Schizophr. Res. 93 (1–3), 144–151. Loewy, R.L., Pearson, R., Vinogradov, S., Bearden, C.E., Cannon, T.D., 2011. Psychosis risk screening with the Prodromal Questionnaire — Brief version (PQ-B). Schizophr. Res. 129 (1), 42–46. Loewy, R.L., Therman, S., Manninen, M., Huttunen, M.O., Cannon, T.D., 2012. Prodromal psychosis screening in adolescent psychiatry clinics. Early Interv. Psychiatry 6 (1), 69–75. Marshall, M., Lewis, S., Lockwood, A., Drake, R., Jones, P., Croudace, T., 2005. Association between duration of untreated psychosis and outcome in cohorts of first-episode patients: a systematic review. JAMA Psychiatry 62 (9), 975–983. McGlashan, T., Walsh, B., Woods, S., 2010. The Psychosis Risk Syndrome: Handbook for Diagnosis and Follow-up. Oxford University Press, New York. Miller, T.J., McGlashan, T.H., Rosen, J.L., Cadenhead, K., Ventura, J., McFarlane, W., et al., 2003. Prodromal assessment with the Structured Interview for Prodromal Syndromes and the Scale of Prodromal Symptoms: predictive validity, interrater reliability, and training to reliability. Schizophr. Bull. 29 (4), 703–715. Miller, T.J., Cicchetti, D., Markovich, P.J., McGlashan, T.H., Woods, S.W., 2004. The SIPS screen: a brief self-report screen to detect the schizophrenia prodrome. Schizophr. Res. 70 (1), 78. Morrison, A.P., French, P., Parker, S., Roberts, M., Stevens, H., Bentall, R.P., et al., 2007. Three-year follow-up of a randomized controlled trial of cognitive therapy for the prevention of psychosis in people at ultrahigh risk. Schizophr. Bull. 33 (3), 682–687. Mossaheb, N., Becker, J., Schaefer, M.R., Klier, C.M., Schloegelhofer, M., et al., 2012. The Community Assessment of Psychic Experience (CAPE) questionnaire as a screeninginstrument in the detection of individuals at ultra-high risk for psychosis. Schizophr. Res. 141 (2), 210–214. Müller, M., Riecher, A., Kammermann, J., Stieglitz, R.D., Stettbacher, G.A., Vetter, S., 2009. Prediction of caseness for mental pathology in Swiss conscripts: the Self-Screen Prodrome. Mil. Med. 174 (12), 1270–1275. Müller, M., Vetter, S., Buchli-Kammermann, J., Stieglitz, R.D., Stettbacher, A., RiecherRössler, A., 2010. The Self-screen-Prodrome as a short screening tool for prepsychotic states. Schizophr. Res. 123 (2), 217–224. Niessen, M.A., Dingemans, P.M., van de Fliert, R., Becker, H.E., Nieman, D.H., Linszen, D., 2010. Diagnostic validity of the Eppendorf Schizophrenia Inventory (ESI): a self-report screen for ultrahigh risk and acute psychosis. Psychol. Assess. 22 (4), 935–944. Obuchowski, N., Graham, R., Baker, M., Powell, K., 2001. Ten criteria for effective screening: their application to multi-slice CT screening for pulmonary and colorectal cancers. Am. J. Roentgenol. 176 (6), 1357–1362. Okuzawa, N., Kline, E., Fuertes, J., Negi, S., Reeves, G., Himelhoch, S., et al., 2014. Psychotherapy for adolescents and young adults at high risk for psychosis: a systematic review. Early Interv. Psychiatry (electronic only available, in press). Ord, L., Myles-Worsley, M., Blailes, F., Ngiralmau, H., 2004. Screening for prodromal adolescents in an isolated high-risk population. Schizophr. Res. 71 (2), 507–508. Owoso, A., Ndetei, D.M., Mbwayo, A.W., Mutiso, V.N., Khasakhala, L.I., Mamah, D., 2014. Validation of a modified version of the PRIME screen for psychosis-risk symptoms in a non-clinical Kenyan youth sample. Compr. Psychiatry 55 (2), 380–387. Quijada, Y., Tizón, J.L., Artigue, J., Parra, B., 2010. At‐risk mental state (ARMS) detection in a community service center for early attention to psychosis in Barcelona. Early Interv. Psychiatry 4 (3), 257–262. Raine, A., 1991. The SPQ: a scale for the assessment of schizotypal personality based on DSM-III-R criteria. Schizophr. Bull. 17 (4), 555. Razali, S.M., Othman, Z., Abidin, Z.Z., Yassin, M.A.M., 2011. Early intervention in psychosis: efficacy of the screening program. Int. Med. J. 18 (2), 117–120. Rietdijk, J.J., Klaassen, R.R., Ising, H.H., Draqt, S., Nieman, D.H., van de Kamp, J., et al., 2012. Detection of people at risk of developing a first psychosis: comparison of two recruitment strategies. Acta Psychiatr. Scand. 126 (1), 21–30. Saha, S., Chant, D., McGrath, J., 2007. A systematic review of mortality in schizophrenia: Is the differential mortality gap worsening over time? JAMA Psychiatry 64 (10), 1123–1131. Stafford, M.R., Jackson, H., Mayo-Wilson, E., Morrison, A.P., Kendall, T., 2012. Early interventions to prevent psychosis: systematic review and meta-analysis. BMJ 346, f185. Thompson, E., Kline, E., Reeves, G., Pitts, S., Schiffman, J., 2013. Identifying youth at-risk for psychosis using the Behavior Assessment System for Children. Schizophr. Res. 151, 238–244. Thompson, E., Kline, E., Reeves, G., Pitts, S., Bussell, K., Schiffman, J., 2014. Using parent and youth reports from the Behavior Assessment System for Children, To identify

18

E. Kline, J. Schiffman / Schizophrenia Research 158 (2014) 11–18

individuals at clinical high-risk for psychosis, Second edition. Schizophr. Res., 154 (13), 107–12. van der Gaag, M., Nieman, D.H., Rietdijk, J., Dragt, S., Ising, H.K., Klaassen, R.M.C., et al., 2012. Cognitive behavioral therapy for subjects at ultrahigh risk for developing psychosis: a randomized controlled clinical trial. Schizophr. Bull. 38 (6), 1180–1188. van der Gaag, M., Smit, F., Bechdolf, A., French, P., Linszen, D.H., Yung, A.R., et al., 2013. Preventing a first episode of psychosis: meta-analysis of randomized controlled prevention trials of 12 month and longer-term follow-ups. Schizophr. Res. 149 (1), 56–62.

Yung, A.R., McGorry, P.D., 1996. The prodromal phase of first-episode psychosis: past and current conceptualizations. Schizophr. Bull. 22 (2), 353–370. Yung, A.R., Yuen, H.P., McGorry, P.D., Phillips, L.J., Kelly, D., Dell'Olio, M., et al., 2005. Mapping the onset of psychosis: the Comprehensive Assessment of At‐Risk Mental States. Aust. N. J. Z. Psychiatry 39 (11–12), 964–971. Zhang, T., Li, H., Woodberry, K.A., Seidman, L.J., Zheng, L., Li, H., et al., 2014. Prodromal psychosis detection in a counseling center population in China: an epidemiological and clinical study. Schizophr. Res. 152 (2), 391–399.