Available online at www.sciencedirect.com
Psychiatry Research 159 (2008) 376 – 381 www.elsevier.com/locate/psychres
A short DSM-IV screening scale to detect posttraumatic stress disorder after a natural disaster in a Chinese population Aizhong Liu a,b,⁎,1 , Hongzhuan Tan a , Jia Zhou a , Shuoqi Li a , Tubao Yang a , Xuemin Tang a , Zhenqiu Sun a , Xin Yang a , Chengqiu Wu a , Shi Wu Wen a,b a b
School of Public Health, Central South University, Xiangya Road 110, Changsha, Hunan 410078, PR China Departments of Obstetrics & Gynecology, and of Epidemiology & Community Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada, K1H 8L6 Received 20 November 2006; received in revised form 21 June 2007; accepted 21 August 2007
Abstract The objective of the study was to construct a short screening scale for posttraumatic stress disorder (PTSD). We used data from our previous study on PTSD among flood victims in 1998 and 1999 in Hunan, China, which was a representative population sample of 27,267 subjects from 16 to 94 years old. Multistage sampling was used to select the subjects from the flood areas and PTSD was ascertained with the Diagnostic and Statistical Manual of Mental Disorders: 4th Edition (DSM-IV). We randomly assigned 80% (n =21,762) of study subjects to construct the screening scale (construct model) and the remaining 20% (n =5505) to test the model. Logistic regression analysis and receiver operating characteristic analysis were used to select a subset of items (symptoms) from the full scale that would effectively predict PTSD. A seven-symptom screening scale for PTSD was selected. A score of 3 or more on this scale was used to define positive cases of PTSD, with a sensitivity of 87.9%, specificity of 97.9%, positive predictive value of 81.3%, and negative predictive value of 98.7%. The short screening scale developed in this study is highly valid, reliable, and predictable. It is an efficient tool to screen PTSD in epidemiological and clinical studies. © 2007 Elsevier Ireland Ltd. All rights reserved. Keywords: Posttraumatic stress disorder; Screening scale; Flood; China
1. Introduction Posttraumatic stress disorder (PTSD) is an anxiety disorder that develops in response to severe traumatic life ⁎ Corresponding author. Department of Epidemiology & Health Statistics, School of Public Health, Central South University, 110 Xiangya Rd, Changsha, Hunan, PR China, 410078. Tel.: +86 731 4805465; fax: +86 731 4805454. E-mail address: [email protected]
(A. Liu). 1 Current address: University of Ottawa, 501 Smyth Rd, Box 241, Ottawa, Canada, K1H 8L6. Tel.: +1 613 737 8899x74238; fax: +1 613 739 6266.
stress (American Psychiatric Association, 1994). The optimal measurement for diagnosing PTSD is a multimode approach in which a variety of different types of data (e.g. social, cognitive, emotional, familial, and vocational functioning), collected from different sources (e.g. self-report, clinician, collateral) and by different methods (e.g. interviews, behavioral observation, physiological and psychological tests), are gathered and evaluated (Keane et al., 1985; Wolfe et al., 1987; Allen, 1994). Such an exhaustive approach, however, is rarely feasible under most clinical and/or research circumstances. There are no simple measurement tools for PTSD that are
0165-1781/$ - see front matter © 2007 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.psychres.2007.08.015
A. Liu et al. / Psychiatry Research 159 (2008) 376–381 Table 1 DSM-IV criterion symptoms of PTSD reported by 27,267 subjects who were exposed to floods in Hunan, China Symptoms of PTSD
B. Re-experiencing B1 intrusive recollections B2 repeating nightmares B3 acting as if flood is recurring B4 feeling worse when reminded of flood B5 reactivity to flood reminders C. Avoidance and numbing C1 efforts to avoid thoughts or feelings associated with the flood C2 efforts to avoid activities that arouse recollections of the flood C3 shows amnesia C4 diminished interest C5 detachment or estrangement C6 restricted range of affect C7 experiences sense of foreshortened future D. Arousal D1 difficulty falling or staying asleep D2 irritability or anger D3 difficulty concentrating D4 hypervigilance D5 exaggerated startle response
24.1 13.0 17.5 8.2 9.6 7.7 14.5 24.4 22.0 19.8 11.6 8.0 4.8 5.6 29.6 32.8 6.5 8.2 33.4 6.2
acknowledged and confirmed by specialists. More commonly, the PTSD diagnosis is ascertained via administration of a structured interview schedule and/or the PTSD symptom self-report (Smith et al., 1999). There are many PTSD diagnostic tools now in use (Horowitz et al., 1979; Robins et al., 1981; Keane et al., 1984, 1988; Blake et al., 1990; Spitzer et al., 1990; Watson et al., 1991; Foa et al., 1993; Weathers et al., 1993; Stuber et al., 1994), but they are difficult to use, costly and time-consuming, especially for large population studies. Since floods always strike many people suddenly, we are in need of a simple measurement tool, in order to shorten the emergency response time, reduce the cost and minimize the burden on respondents in such an urgent situation. A short screening scale is useful in rapidly selecting the suspects who are likely to have PTSD for more intensive diagnostic assessment in the subsequent stage. Although various PTSD screening scales have been developed (Breslau et al., 1999; Brody et al., 1999;
Zimmerman and Mattia, 2001; Brewin et al., 2002; Dobie et al., 2002; Franklin et al., 2002; Chou et al., 2003; Mori et al., 2003; Zelst et al., 2003; Germain et al., 2005; Lang and Stein, 2005), they were aimed at different targets. The previous screening scales focused on a general health survey in a community or on populations of primary health care patients, older adults, earthquake victims, and sexual assault survivors. Floods are one of the most common and most severe forms of natural disasters, accounting for up to one-half of all natural disasters. When floods strike, they are always quick and likely to injure large numbers of people, physically and psychologically as well. As an indicator of psychological damage, the presence of PTSD is often investigated to evaluate the severity of mental suffering of victims of natural disasters. This information is essential for rapidly assessing the intensity of a flood's impact, for anticipating medical service demands after a flood, and for predicting the amount of supplies and number of personnel needed. Unfortunately, people's psychological response to different kinds of natural disasters may vary in different cultural and ethnic groups. However, no PTSD screening scale has been developed for flood victims, especially for Chinese people. The objective of this study is to develop an effective short screening scale for the epidemiological study of PTSD among Chinese flood victims. 2. Methods 2.1. Subjects We used a multistage sampling method to select subjects. The representative samples in this study were the victims who had been directly exposed to the 1998–1999 floods in Hunan, China. In the first stage of sampling, 8 counties were randomly selected from 38 counties that suffered from the flood. In the second stage of sampling, 40 townships (50%) were randomly selected from the 8 study counties. In the third stage of sampling, 310 villages (50%) were randomly selected from the study townships. Finally, 13,450 households (50%) were
Table 2 Logistic regression analyses of three nested best models for predicting diagnosis of PTSD in 21,762 (80%) subjects who were exposed to floods in Hunan, China Model
List of DSM-IV symptoms in model
Positive predictive value (%)
Negative predictive value (%)
Six symptoms Seven symptoms Eight symptoms
B2, B4, C1, C3, C4, and D1 B2, B4, C1, C3, C4, C5, and D1 B1, B2, B4, C1, C3, C4, C5, and D1
83.6 90.4 89.5
99.5 99.5 99.4
94.8 95.2 93.9
98.3 98.3 98.9
A. Liu et al. / Psychiatry Research 159 (2008) 376–381
randomly sampled from the study villages. All family members of the selected villages who were 16 years of age or older were asked to participate in the study. 2.2. Procedure Between January and May 2000, we undertook a faceto-face interview by trained research assistants (with onsite supervision by psychologists) with a pre-constructed questionnaire, which included 17 items according to the Diagnostic and Statistical Manual of Mental Disorders: 4th Edition (DSM-IV) (American Psychiatric Association, 1994). All subjects were diagnosed for the presence or absence of PTSD using DSM-IV. We took DSM-IVas the “gold standard”, and randomly assigned 80% (n = 21,762) of study subjects to construct the screening scale (construct model) and the remaining 20% (n = 5505) to test the model. Forward stepwise logistic regression analysis was used to determine the best combinations of independent variables, with or without PTSD as the dependent variable, and 17 diagnosis items as independent variables. The entry threshold was set at P = 0.10 and the exit threshold at P = 0.05 for all variables. For each step, one variable was entered and the sensitivity, specificity, and positive and negative predictive values of different variable combinations were tested. The optimal variable combination was determined by comparing their sensitivity, specificity, and positive and negative predictive values. We chose the best scale based on the predicted probability that resulted in the maximum sum of positive and negative predictive values. The total score of this scale was based on the total of symptoms positively reported by the respondents, any one item among the symptoms positively being recorded as a score of 1 with equal weight given to each symptom. The ROC (receiver operating characteristic) curve was used to determine the threshold and estimate the sensitivity, specificity, and positive and negative predictive value. We then used the remaining 20% of samples to evaluate the validity, reliability and predictable value of this screening scale by
Fig. 1. The receiver operating characteristic (ROC) curve of the logistic probability model.
comparing the results blindly with the PTSD diagnostic standard of DSM-IV. In the meantime, we evaluated the validity, reliability and predictive value of Breslau's, Chou's and our screening scale, by comparing the results of PTSD detected in the same sample with the DSM-IV criteria for PTSD. All analyses were performed using SPSS Version 11.0 statistical software. 3. Results Of the 27,267 participants, 14,350 were men (52.6%) and 12,917 were women (47.4%); DSM-IV criteria for PTSD were satisfied by 2595 (9.5%) subjects. The highest rate was noted for symptom D4, hypervigilance (33.4%), and the lowest for C6, restricted range of affect (4.8%), among symptoms in the full diagnostic scale (Table 1).
Table 3 Sensitivity, specificity, crude agreement, Youden's index, and positive and negative predictive values of different cutoff points on the seven-symptom scale for PTSD applied to 21,762 (80%) subjects who were exposed to floods in Hunan, China Cutoff point
Crude agreement (%)
Positive predictive value (%)
Negative predictive value (%)
1 2 3 4 5 6 7
99.9 98.3 87.9 64.0 41.6 7.4 3.0
68.2 87.1 97.9 99.9 100.0 100.0 100.0
71.2 88.1 97.0 96.5 94.5 91.3 90.9
0.681 0.858 0.858 0.639 0.416 0.074 0.030
24.6 44.3 81.3 98.5 100.0 100.0 100.0
100.0 99.8 98.7 96.4 94.3 91.2 90.8
A. Liu et al. / Psychiatry Research 159 (2008) 376–381
The results of the logistic regression analysis showed that the three combinations of six-, seven-, and eightsymptom scales at steps 6, 7, and 8 had a high sensitivity, specificity, and positive and negative predictive value (Table 2). The areas under the ROC curve were 98.3%, 98.5%, and 98.9%. The seven-symptom scale had the highest positive predictive value, sensitivity, and specificity, while the other statistics (negative predictive value) were constant across the three scales. Two of these symptoms were from the re-experiencing symptom group (including B2, repeating nightmares, and B4, feeling worse when reminded of flood), four were from the avoidance and numbing symptom group (including C1, efforts to avoid thoughts, feelings associated with the flood, C3, shows amnesia, C4, diminished interest in activities, and C5, detachment or estrangement), and one was from the hyperarousal symptom group (D1, difficulty falling or staying asleep). Information on sensitivity, specificity, crude agreement, Youden's index, and positive and negative predictive values of this seven-symptom scale for screening PTSD among different cutoff points, formed by summing up the number of positive items on the symptoms, is presented in Table 3. A score of 3 appeared to be the best overall cutoff point in terms of sensitivity, specificity, and positive and negative predictive values. The ROC curve is a work curve for screening tests, which uses the sensitivity as the y-axis and (1-specificity) as the x-axis. Fig.1 reveals a series of different sensitivity and specificity values when using different cutoff points. A score of 3 or more on this scale was used to define positive cases of PTSD, with a sensitivity of 87.9%, specificity of 97.9%, positive predictive value of 81.3%, and negative predictive value 98.7% (Table 3). The remaining 20% of randomized samples were measured Table 4 Comparison of the performance of Breslau's and Chou's scales with the performance of our scale Screening scale
Our scale Positive Negative Breslau's Positive Negative Chou's Positive Negative
for PTSD using a score of 3 or more on this scale and DSM-IV (Table 4). The sensitivity and specificity of the scale were 89.9% and 97.8%, and positive and negative predictive values were 81.4% and 98.9%. The sensitivity, specificity, and predictive values of Breslau's and Chou's screening scales were lower than those of our scale (Table 4). The result of the Kappa analysis showed that our scale and DSM-IV were highly consistent on PTSD measurement (κ = 0.84, u = 67.53, P b 0.01). We performed separate analyses of the data for men and women as well as different age groups, and all showed similar results (data not shown). 4. Discussion To be effective and efficient, a screening scale should ideally be short and contain the minimum number of items necessary for accurate case identification. It should be simple and preferably not require respondents to ponder over a large number of alternative scale points. It should be written in language that is easy to understand. It should be acceptable to respondents. For easy administration, self-report questionnaires would be the most feasible solution (Brewin et al., 2002). Our screening scale appears to meet most of these criteria. A short screening scale will be useful for surveys in a disaster-affected population to select a subset of respondents who are likely to have a disorder for more intensive diagnostic assessment in the next stage (Breslau et al., 1999; Newman et al., 1990). We developed a short screening scale for PTSD. The screening scale was designed to quickly test for PTSD in individuals exposed to floods. A score of 3 or more on the seven-symptom scale has the following characteristics for screening PTSD: sensitivity = 87.9%, specificity = 97.9%, positive predictive value= 81.3% and negative predictive value= 98.7%. The predictive value of a screening test is related to its sensitivity, specificity, and prevalence rate, so we should know the effect of the prevalence rate. When the rate of the disorder is low, application of Bayesian theory predicts that the target condition will not be present. At low prevalence rates, the positive predictive value is reduced because a large number of subjects scoring positive for the disorder are actually false positives. Thus, when the PTSD prevalence rate is low, the rate of identification of true cases is also low (Allen, 1994). In our previous article (Liu et al., 2006), the rate of PTSD in flood victims was 9.5%. Because the prevalence rate of post-flood PTSD is not high, the positive predictive value is not high. A score of 3 on the seven-symptom screening scale was identified as the optimal cutoff point for separating subjects with or without PTSD. This cutoff
A. Liu et al. / Psychiatry Research 159 (2008) 376–381
point minimized the probability of missing true cases of PTSD at the expense of somewhat raising the probability of classifying subjects without the disorder as having it. This tradeoff is particularly suitable for two-phase assessments in which the first phase is designed to maximize the number of true cases of PTSD diagnosed, and the second phase is expected to reclassify those who were wrongly classified as having the disorder. Other more advantageous cutoff points may be determined for different applications of PTSD screening scales. Although this screening scale is not a substitute for a psychiatric diagnosis (Breslau et al., 1999), it may be used to identify suspected cases of PTSD among the victims of catastrophic events, after which these cases might be selected for future diagnosis (Chou et al., 2003). Breslau's, Chou's, and our screening scales are all seven-symptom scales, but the items included are not the same. In Breslau's scale, five are from the avoidance and numbing symptom group and two from the arousal symptom group. In Chou's scale, three are from the reexperiencing symptom group, three from the avoidance and numbing symptom group, and one from the arousal symptom group. The threshold of 3 in our scale was the same as that of Chou's scale for earthquakes (Chou et al., 2003) but lower than that of Breslau's scale of 4 for community surveys (Breslau et al., 1999). This indicates that psychological responses might vary in different kinds of disasters and among different populations. The study results also showed that the sensitivity, specificity, and predictive value of Breslau's and Chou's scales were all lower than those of ours in detecting PTSD among the same remaining 20% of samples. This further demonstrates the usefulness of our screening scale. Our large population-based study suggests that the seven-symptom screening scale developed in this study is highly valid, reliable, and predictable. It is an effective tool to screen quickly for PTSD in large epidemiologic studies among victims of natural disasters, such as floods. Acknowledgements This project was supported by grant CMB 98-689 from the Chinese Medicine Board (New York). The authors show great appreciation for Xiumin Zhang, the director of the Center of Disease Prevention and Control (CDC) of Anxiang county, Linbao Xiang, the director of CDC of Yiyang City, Huaxian He, the director of CDC of Yueyang City, Linlin Li, the director of CDC of Xiangxi autonomy region, and Senlin Tang, the director of CDC of Datong lake district, all located in Hunan, China, for their kind cooperation in this study.
References Allen, S.N., 1994. Psychological assessment of post-traumatic stress disorder: psychometrics, current trends and future directions. Psychiatric Clinics of North America 17, 327–349. American Psychiatric Association, 1994. Diagnostic and Statistical Manual of Mental Disorders 4th ed. APA, Washington, DC, pp. 257–294. Blake, D.D., Weathers, F.W., Nagy, L.N., Kaloupek, D.G., Gusman, F.D., Charney, D.S., 1990. A clinician rating scale for assessing current and lifetime PTSD: the CAPS-1. Behavior Therapy 18, 187–188. Breslau, N., Peterson, E.L., Kessler, R.C., Schultz, L.R., 1999. Short screening scale for DSM-IV posttraumatic stress disorder. American Journal of Psychiatry 156, 908–911. Brewin, C.R., Rose, S., Andrews, B., Green, J., Tata, P., Mcevedy, C., 2002. Brief screening instrument for post-traumatic stress disorder. British Journal of Psychiatry 181, 158–162. Brody, S.M., Churchill, E., Davidson, J.R.T., 1999. Derivation of the SPAN, a brief diagnostic screening test for post-traumatic stress disorder. Psychiatry Research 88, 63–70. Chou, F.H.C., Su, T.T.P., Yang, W.C.O., Chien, I.C., Lu, M.K., Chou, P., 2003. Establishment of a disaster-related psychological screening test. Australian and New Zealand Journal of Psychiatry 37, 97–103. Dobie, D.J., Kivlahan, D.R., Maynard, C., Bush, K.R., McFall, M., Epler, A.J., 2002. Screening for post-traumatic stress disorder in female Veteran's Affairs patients: validation of the PTSD checklist. General Hospital Psychiatry 24, 367–374. Foa, E.B., Riggs, D., Dancu, D., Rothman, B., 1993. Reliability and validity of a brief instrument for assessing post-traumatic stress disorder. Journal of Traumatic Stress 6, 459–473. Franklin, C.L., Sheeran, T., Zimmerman, M., 2002. Screening for trauma histories, posttraumatic stress disorder (PTSD), and subthreshold PTSD in psychiatric outpatients. Psychological Assessment 14, 467–471. Germain, A., Hall, M., Krakow, B., Shear, M.K., Buysse, D.J., 2005. A brief sleep scale for posttraumatic stress disorder: Pittsburgh Sleep Quality Index Addendum for PTSD. Anxiety Disorders 19, 233–244. Horowitz, M.J., Wilner, N., Alvarez, W., 1979. Impact of events scale: a measure of subjective stress. Psychosomatic Medicine 41, 209–218. Keane, T.M., Malloy, P.F., Fairbank, J.A., 1984. Empirical development of an MMPI subscale for the assessment of combat-related PTSD. Journal of Consulting and Clinical Psychology 62, 888–891. Keane, T.M., Fairbank, J.A., Caddell, J.M., Zimering, R.T., Bender, M.E., 1985. A behavioral approach to assessing and treating PTSD in Vietnam veterans. In: Figley, C.R. (Ed.), Trauma and Its Wake. Brunner/Mazel, New York, pp. 257–294. Keane, T.M., Caddell, J.M., Taylor, K.L., 1988. Mississippi Scale for combat-related post-traumatic stress disorder: three studies in reliability and validity. Journal of Consulting and Clinical Psychology 56, 1–6. Lang, A.J., Stein, M.B., 2005. An abbreviated PTSD checklist for use as a screening instrument in primary care. Behaviour Research and Therapy 43, 585–594. Liu, A.Z., Tan, H.Z., Zhou, J., Li, S.Q., Yang, T.B., Wang, J.R., Liu, J., Tang, X.M., Sun, Z.Q., Wen, S.W., 2006. An epidemiologic study of posttraumatic stress disorder in flood victims in Hunan China. Canadian Journal of Psychiatry 51, 350–354. Mori, D.L., Lambert, J.F., Niles, B.L., Orlander, J.D., Grace, M., LoCastro, J.S., 2003. The BAI-PC as a measure of anxiety, depression, and PTSD in primary care. Journal of Clinical Psychology in Medical Settings 10, 187–192.
A. Liu et al. / Psychiatry Research 159 (2008) 376–381 Newman, S.C., Shrout, P.E., Blan, R.C., 1990. The efficiency of twophase designs in prevalence surveys of mental disorders. Psychological Medicine 20, 183–193. Robins, L., Helzer, J.D., Croughan, J., Ratcliff, K.S., 1981. The NIMH diagnostic interview schedule: its history, characteristics and validity. Archives of General Psychiatry 38, 381–389. Smith, M.Y., Redd, W.H., Peyser, C., Vogl, D., 1999. Post-traumatic stress disorder in cancer: a review. Psycho-oncology 8, 521–537. Spitzer, R.L., Williams, J.B., Gibbon, M., First, M.B., 1990. Structured Clinical Interview for DSM-III-R—Non-Patient Edition (SCIDNP, version 1.0). American Psychiatric Press, Washington, DC. Stuber, M.L., Meeske, K., Gonzalez, S., 1994. Post-traumatic stress disorder after children cancer I: the role of appraisal. Psychooncology 3, 305–312. Watson, C., Juba, M., Manifold, V., Kucala, T., Anderson, P., 1991. The PTSD interview: rationale, description, reliability and
concurrent validity of DSM-III based technique. Journal of Clinical Psychology 47, 179–188. Weathers, F., Litz, B., Herman, D., Huska, J., Keane, T., 1993. The PTSD Checklist (PCL): reliability, validity and diagnostic utility. [Abstract] Presented at the 9th Annual meeting of the International Society for Traumatic Stress Studies. San Antonio, TX. Wolfe, J., Keane, T.M., Lyons, J.A., Gerardi, R.J., 1987. Current trends and issues in the issues in the assessment of combat-related posttraumatic stress disorder. Behavior Therapy 12, 27–32. Zelst, W.H., Beurs, E., Beekman, A.T.F., Deeg, D.J.H., Bramsen, I., Dyck, R., 2003. Criterion validity of the self-rating inventory for posttraumatic stress disorder (SRIP) in the community of older adults. Journal of Affective Disorders 76, 229–235. Zimmerman, M., Mattia, J.I., 2001. A self-report scale to help make psychiatric diagnoses. The Psychiatric Diagnostic Screening Questionnaire. Archives of General Psychiatry 58, 787–794.