Journal of Child & Adolescent Substance Abuse, 20:63–81, 2011 Copyright # Taylor & Francis Group, LLC ISSN: 1067-828X print=1547-0652 online DOI: 10.1080/1067828X.2011.534366
Adolescent Alcohol Use Self-Report Stability: A Decade of Panel Study Data AUDREY M. SHILLINGTON, JOHN D. CLAPP, MARK B. REED, and SUSAN I. WOODRUFF San Diego State University, San Diego, CA, USA
This study analyzed six waves of panel data from the National Longitudinal Survey of Youth (NLSY). These analyses were conducted to test the stability of self-reported lifetime use and age of onset. Intraclass correlation coefficients (ICCs) indicated that the stability of age of onset reports decreased with longer time frames between follow-ups. The percentage of youths who had discrepancies in self-reported ever use of alcohol at two-year follow-up ranged from 15% to 35%. Higher discrepancy rates were found for males and younger respondents. Differences in report stability as a function of race=ethnicity were minimal. Questions related to lifetime use and age of onset have implications for the study of lifetime trajectories of use and the timing of prevention programs. KEYWORDS adolescent, alcohol, longitudinal, reliability, report stability
INTRODUCTION In the United States, the prevalence rate of alcohol use among adolescents has decreased during the past decade but still remains high. By grade eight, nearly two-fifths of youths report using alcohol during the past year (Johnston, O’Malley, Bachman, & Schulenberg, 2008) and this prevalence rate increases to 72.2% by senior year in high school. In addition, among This study was funded by a grant from the National Institute on Alcohol Abuse and Alcoholism 5R21AA016769, A. M. Shillington, PI. Address correspondence to Audrey M. Shillington, PhD, Center for Alcohol and Other Drug Studies and Services, 6386 Alvarado Court, Suite 224, San Diego, CA 92120, USA. E-mail:
[email protected] 63
64
A. M. Shillington et al.
twelfth-graders who drink, more than half report being drunk during the past year. For past-month alcohol use, 44.4% of high school seniors and 33.4% of sophomores report use. Such adolescent alcohol use has been found to be linked with a myriad of negative consequences including fatal traffic accidents, unintentional injury=death, unprotected sexual activity, suicide, and poor academic performance. Miller, Levy, Spicer, and Taylor (2006) estimated that the cost to society per drink consumed by an underage drinker was $3.00, cumulating in a total cost of $61.9 billion, which included medical costs, property loss, work losses, and quality of life lost. Prevalence and incidence rates of alcohol use and problems typically are based on self-reports. Epidemiological surveys commonly query whether one has used a particular substance in one’s lifetime, past year or past 30 days. Logically, if a respondent reports any lifetime use, such use should be consistently reported in subsequent panel surveys. A number of studies have utilized retrospective self-reported age of onset of various substances to identify sequences and trajectories for substance use of adolescents into young adulthood (Golub, Labouvie, & Johnson, 2000; Perkonigg et al., 2008; Wittchen et al., 2008). Early age of onset for alcohol use has been linked with more severe later use and subsequent problems (Hingson, Heeren, Jamanka, & Howland, 2000). Hingson and colleagues reported that relative to those who began drinking after age 21, those who started before age 14 as well as older teens were more likely to be injured due to the influence of alcohol even when controlling for family history of alcoholism, their own history of heavy drinking, and dependence. Theories of substance use trajectories posit that individuals who begin to use alcohol and tobacco at early ages are at greater risk to progress to the use of illicit substances as well as an increased risk of progressing to problem use (Funkhouser, Goplerud, & Bass, 1992; Grant, 1998; Grant & Dawson, 1997; Hawkins, Catalano, & Miller 1992; Mills & Noyes, 1984). Individuals may be incorrectly identified as more ‘‘at risk’’ because they are younger when interviewed and more accurately report their age of onset compared to those interviewed at an older age. Older adolescents or adults who report an older age of onset may be reporting inaccurately because of recall biases such as telescoping, and thus may not be identified correctly as ‘‘at risk.’’ The implications for the unreliability of age of onset impacts the ability to (1) predict individual outcomes, (2) accurately examine alcohol use trajectories, patterns, and problems, and (3) determine the appropriate time to offer prevention programs (Golub, Labouvie, & Johnson, 2000). Psychometrically, reliability of an assessment is a measure of its repeatability, typically established by interviewing and re-interviewing respondents a week to several weeks apart. Similar to reliability is stability (Rice et al., 1992). Conceptually, stability is an extension of reliability assessed
Adolescent Alcohol Use Self-Report Stability
65
when test-retest interviews are spaced temporally much more widely apart. The two measurement properties are similar in that they consist of two or more assessments at different points in time. However, with reliability, when the assessment is days apart, there is a possibility that the respondent will recall their prior answers. With stability, a report is spaced a year or several years apart and it is unlikely the respondent will recall prior responses. If a respondent repeatedly endorses a behavior or symptom then it is assumed to be stable and that it reflects a true clinical state and not simply the ability to recall earlier endorsements of interview questions (Rice et al., 1992).
Report Stability To date there have been a few studies that have examined the stability or consistency of alcohol use interview or survey responses across longer time periods (Barnea, Rahav, & Teichman, 1987; Fendrich & Rosenbaum, 2003; O’Malley, Bachman, & Johnston, 1983). Barnea and colleagues (1987) examined the stability of use across two interviews with high school students spaced 10 to 12 months apart and observed correlation coefficients of .72 to .80 for wine and beer. As part of a DARE evaluation, Fendrich and Rosenbaum (2003) examined the stability of self-reported alcohol use across four years of annual data collection and found nearly half of all users recanted their use at a follow-up. Once someone reports ‘‘yes’’ to lifetime use of alcohol, then logically, the response should be ‘‘yes’’ at all later follow-ups.
Age of Onset A slightly larger number of publications are available that report on the stability of reported age of onset of alcohol use. When respondents report an age of onset at follow-up older than the originally reported age, the response pattern represents forward telescoping. Conversely, when respondents report age of first use as being younger than the originally reported age, the response patterns reflect backward telescoping. Golub, Labouvie, and Johnson (2000) interviewed adolescents regarding their age of onset at baseline and three years later. They found that among their sample forward telescoping was as likely to occur as backward telescoping generally but more alcohol users tended to exhibit forward telescoping. Johnson and Mott (2001) examined national data and found consistent forward telescoping among their youth sample; however, generally about half of alcohol users reported consistent ages of onset within one year of the previously stated age (Johnson & Mott, 2001; Shillington & Clapp, 2000). Golub, Labouvie, and Johnson (2000) also found strong evidence for forward telescoping and, similar to Johnson and Mott (2001), reported that the
66
A. M. Shillington et al.
change in stated age of onset for alcohol use was more than two years older from what was originally reported. Prause, Dooley, Ham-Rowbottom, and Emptage (2007) also found forward telescoping to a lesser degree for alcohol use but they also reported low agreement in age of onset from one wave of data collection to the next. The psychometric properties of self-report substance abuse inventories among treatment or pre-treatment adolescent populations have been examined (Stinchfield, 1997; Winters et al., 1991). There are fundamental differences in treatment and general population adolescents and this approach has utility for treatment professionals and researchers concerned with examining the efficacy of treatment. Some teens will minimize their alcohol or other drug use when queried prior to treatment as a function of their addiction (Aiken, 1986). On the other hand, other adolescents may exaggerate their use to gain access to treatment or avoid incarceration (Winters et al., 1991). Therefore, youths entering or in treatment have varying reasons for inconsistent self-reports. So research examining the reliability of self-reports using treatment or pre-treatment populations is less applicable to researchers interested in AOD epidemiology at the population level. Only a few studies have used national samples to examine the stability of self-reported alcohol use and age of onset at one-year follow-up (Barnea, Rahav, & Teichman, 1987; Golub, Labouvie, & Johnson, 2000; Shillington & Clapp, 2000) while others studying this topic have used smaller convenience samples (Barnea et al., 1987; Fendrich & Rosenbaum, 2003). The purpose of this article is to analyze 6 waves of national panel data spanning 10 years to understand if there are consistent errors in self-report over time. This study will also examine if there are differences in report stability of use and reported ages of onset for alcohol use for sex, ethnicity, and age groups.
METHODS Sample The data for this study are from the National Longitudinal Survey of Youth (NLSY). Detailed descriptions of the samples and procedures can be found elsewhere (Baker, Keck, Mott, & Quinland, 1993; Center for Human Resources Research, 1997). The study design uses a multistage stratified random sampling technique. The original participants have been interviewed annually since 1979 with a retention rate at 12-year follow-up of 90.5% (Baker & Mott, 1989). In 1986, the NLSY started to survey the children of the original female respondents. The children, when age 10 years and older, were asked to complete a self-report instrument titled the Child Self-Administered Supplement (CSAS). The children were interviewed every two years on a number of topics
Adolescent Alcohol Use Self-Report Stability
67
including alcohol and drug use. In 1994 a new instrument was added to the protocol. While children ages 10 to 14 still completed the CSAS, the new supplement was given to adolescents 15 years and older. The new survey was titled the Young Adult Survey (YAS). The YAS asked about substance use, sexual activity, delinquent activities, and relationships. The sample for this study comprised all youths age 10 and older beginning with those interviewed from 1988 to 1998. The sample size varied wave to wave. In 1988 the sample size of those 10 years and older was 1,157 with a mean age of 11.78 (SD 1.58) and by 1998 the sample size increased to 5,549 and the mean age was 16.06 (SD 3.88). The final sample in 1998 was approximately half male (50.9%). The ethnicity was 22.0% Hispanic, 35.2% black, and 42.8% non-Hispanic, non-black. The N size for those who reported alcohol use in 1988 and 1990 was N ¼ 94; for 1996 and 1998 the pairs matched reports increased to 780.
Measures For the purposes of this study, the first wave of data (Time 1) consisted of responses to alcohol use questions in the 1988, 1990, 1992, 1994, 1996, and 1998 surveys. Across this time period, many children aged out of the CSAS and started responding to the YAS. Although the responses to the alcohol use questions were sometimes from two different survey instruments (CSAS and YAS), the questions themselves were nearly the same. A respondent would answer a question asking if he or she ever drank alcohol (no or yes). However, there were one or two waves of data (depending on whether it was the CSAS or YAS) to which the youth was asked to respond regarding the number of times they’d drank alcohol. If a youth reported ‘‘0’’ they were coded as ‘‘no’’ for lifetime use and if he or she reported 1 or more the code was a ‘‘1’’ for ‘‘yes’’ lifetime use. The second wave (Time 2) is always the subsequent wave of data collection. For example, if Time 1 is 1990 then Time 2 is 1992. Nearly all the waves of data had alcohol use queried by a question asking the adolescent if he or she ever used alcohol. For the 1998 YAS, a drink was defined as a full glass of beer, wine, or hard liquor. Since this was the only wave and survey that defined it with greater specificity, we could not test if this enhanced report stability across time. For this study we used two variables to examine external consistency— agreement and discrepancy—for lifetime alcohol use wave-to-wave. We also examined incident cases; that is, when one would report they had never used alcohol at Time 1 but then did report use at Time 2. Because the two sources of data are from the adolescents’ self-reports, it is not possible to validate if new users from Time 1 to Time 2 are true incident cases. Thus incident cases are presented separately and not considered a type of report agreement. Those reporting ‘‘never used’’ at both Time 1 and Time 2 were excluded from our analyses for two reasons. First, prior research has shown that
68
A. M. Shillington et al.
youths who report no use for each substance category are significantly younger than ever users (Shillington & Clapp, 2000). Second, the inclusion of never users in an analysis of report stability artificially decreases discrepancy rates (Bailey, Flewelling, & Rachal, 1992).
External Consistency Users of alcohol could be coded as either ‘‘consistent’’ or ‘‘discrepant’’. Consistent reports for both Time 1 and Time 2 included adolescents reporting alcohol use in each survey. For example if a respondent reported lifetime use of alcohol at Time 1 and then reported such use again two years later they were categorized as in agreement or consistent. Respondents could also be categorized as discrepant in their reports. Discrepant reports included those who reported use of alcohol at Time 1 but two years later reported ‘‘never used’’ alcohol. For example, a discrepant case would be a youth who reported lifetime use during the 1992 survey but in 1994 responded that they never drank alcohol. A variable was created for consistent use for six interviews across five time frames (1988–1990, 1990–1992, 1992–1994, 1994–1996, and 1996–1998) with a result of five different consistency variables. We also examined the stability of age of onset from one wave to another. A variable was created in which the difference was calculated between the age of onset reported at Time 2 and the age of onset reported at Time 1. This calculation could result in a ‘‘0’’, which would indicate that the age of onset was the same at both times. However, a value of ‘‘þ3’’, for example, would indicate that a youth reported their age of onset three years older at Time 2 compared to Time 1. A negative value would be for those reporting a younger age of onset from Time 2 compared to Time 1. Longer wave-to-wave comparisons are conducted that examine the stability of age of onset for time spans that range from 4 to 10 years and the exact age reported is utilized for this. As per Johnson and Mott (2001) and Shillington and Clapp (2000) we included data for inspection that demonstrates the differences between the percentage of youths who reported the age of onset exactly the same wave-to-wave as well as those who reported their age of onset 1 year from that reported previously.
Internal Consistency To examine internal consistency of reports (consistency based on the logical skip pattern of the surveys), responses to two variables at each wave of data collection were compared. If respondents reported lifetime alcohol use then they were next asked to report how recently they drank. If they reported drinking within the past 30 days they were asked to report the frequency of past 30-day use. Internally consistent reports were defined
Adolescent Alcohol Use Self-Report Stability
69
as those respondents who reported 30-day use and then reported drinking 1þ more times during the past 30 days. Inconsistent reports were defined as those respondents who reported past 30-day use but reported that they had no 30-day frequency; thus, the survey respondents’ responses did not logically match. Inconsistent reports could also be defined as a situation where the youth reported no 30-day use but answered the 30-day frequency question. These questions were available in the 1994, 1996, and 1998 interviews.
Demographic Variables The data set consisted of all youths age 10 or older by the last wave of data collection in 1998. Ethnicity was coded as 1 ¼ Hispanic, 2 ¼ African American, and 3 ¼ non-Hispanic=non-African American. Age ranges changed based upon the year of interview. The minimum age was always 10 years but the upper age range increased with each interview as did the mean age. In 1988 the age range was 10 to 18 years with the upper age range increasing by two years at each later wave of data collection. By 1998 the mean age was 16.26 years with a range of 10 to 28 years. When age-related analyses were conducted, only the chronological age at the time of the survey was used for the analyses. For example, when examining if differences existed in stability reports by chronological age, a separate analysis for each year’s stability variable and the same year’s age were conducted.
Statistical Methods Wave-to-wave comparison analyses were conducted using SPSS version 15.0.1.1. Chi-square analyses were conducted to compare the report agreement and discrepancy in reports by gender and ethnicity. Chronological age differences for those with report agreement and discrepancies were examined using analysis of variance and Scheffe´ post hoc means tests to reduce type I error. Exact agreement for reported age of onset for alcohol use was tested using the intraclass correlation coefficient (ICC), which is an agreement-based analysis for reliability. The two-way mixed model analysis of variance where the respondents were considered the random factor and the two time points for comparisons were the fixed factor (Bartko, 1966; McGraw & Wong, 1996; Shrout & Fleiss, 1979) was used for ICC calculations. This test-retest agreement is considered excellent when the ICCs are 0.75 or higher; considered good from 0.60–0.74; fair for values of 0.40–0.59; and poor for values below 0.40 (Fendrich, Weissman, Warner, & Mufson, 1990; Johnson & Mott, 2001). Logistic regression analyses were conducted to test the association between the demographic variables and recanting alcohol use. When testing for internal consistency with logically linked questions the analyses included a phi and Cramer’s V coefficient. These analyses were
70
A. M. Shillington et al.
conducted for each wave-to-wave comparison as well as the within-year interviews for the internal consistency analyses.
RESULTS External Consistency Table 1 shows consistency of self-report. We found the percentage of consistent reporters increased across time. Those who reported use at both waves was lower at 64.9% and 69.2% from 1988–1990 and 1990–1992, respectively, but increased to more than 80% in the latter 3 wave-to-wave comparisons. The percentage of recanting ranged from about 15% to 35%. The data were stratified to identify if there were any differences on ethnicity or gender for report consistency (not reported in a table). The only comparison in which differences on ethnicity were found was 1996–1998, in which the highest rate of consistent self-reports were found among non-Hispanic, non-black teens, followed by Hispanic youths and then black youths. Two comparisons resulted in significant differences when examining report stability and gender. We found in both sets of analyses that males had significantly higher rates of recanting compared to females at about 24% compared to 15%. Report discrepancies were also examined by age; these results are presented in Table 2. Results showed for each wave-by-wave comparison the mean age of youths with report discrepancies were significantly younger compared to those who had consistent reports. The differences in the means ranged from about 1 year younger (1990–1992) to 2.65 years younger (1992–1994).
Age of Onset Age of onset was examined several ways. First we examined the percentage of youths who reported their age of onset for alcohol use as the exact same age from wave 1 to wave 2 (Table 3, column 1). The stability of reported age TABLE 1 Report Agreement, Report Discrepancy, and Incident Cases for Alcohol Users Over Ten Years Years of Interviews 1988–90 1990–92 1992–94 1994–96 1996–98
Total N Reported Use Time 1 or Time 2 178 368 713 1,038 1,193
Agreement Yes-Yes N (%) 61 128 297 564 633
(64.9%) (69.2%) (80.0%) (85.1%) (81.1%)
Use Discrepancy Yes-No N (%) 33 57 74 99 147
(35.1%) (30.8%) (20.0%) (14.9%) (18.9%)
Incident Cases N (%) 84 183 342 375 413
(47.2%) (49.7%) (48.0%) (36.1%) (34.6%)
Adolescent Alcohol Use Self-Report Stability
71
TABLE 2 Mean Chronological Age Differences between Those Who are Consistent or Inconsistent Reporters for Lifetime Alcohol Use from Wave 1 to Wave 2 2-Year Comparison 1988–1990 1990–1992 1992–1994 1994–1996 1996–1998
Report Alcohol Use Both Waves Mean Age (SD) 15.25 15.73 17.08 18.57 18.50
(1.74) (1.91) (2.08) (2.33) (1.83)
Denied Use at Wave 2 Mean Age (SD) 13.79 14.74 14.43 16.07 16.40
(0.93) (1.85) (1.64) (2.32) (2.38)
ANOVA F Value 20.06 10.98 103.78 96.72 138.29
p < .001.
of onset was rather low when restricting the analysis to the exact age with stability rates being as low as 15.6%; even at the highest it only reached 25%. With a one-year allowance the rate increased with the stability rates, increasing to around 50% with a range of 42.3% to 56.1%. The last two columns in Table 3 present the telescoping analyses. We found forward telescoping to be more common than backward telescoping. In other words, more youths reported their age of onset two or more years older at Time 2 than the age of onset reported at Time 1. Forward telescopers made up about 40% of the age of onset inconsistencies with approximately 10% of the inconsistencies being backward telescopers (all others were within the one-year allowance from their baseline age or reported the exact same age at both times). In Table 4 the intraclass correlations are presented for the comparisons of absolute consistency for multiyear comparisons. As can be seen, the ICCs tend to be excellent to good when the age of onset report consistency is tested with a two-year difference, but the ICCs fall into the good or even poor range once the time between interviews increases to four years or greater. We also ran age of onset analyses to examine differences for ethnicity, sex, and chronological age. Only one set of analyses (not presented in a table) found a difference between ethnic groups, the result being that the non-black, non-Hispanic teens were much more likely to be forward telescopers (68.9%) compared to Hispanic (54.5%) and black respondents (55.9%). Hispanic youths were more likely to report the exact same age wave to wave (25.8%) than black (18.1%) or non-black, non-Hispanic youths (19.3%). Black youths were more likely to be backward telescopers (26.0%) than Hispanic (21.7%) or non-black, non-Hispanic 11.8%) (c2 ¼ 10.01, p < .05). No significant differences were identified between males and females for the accuracy of age of onset. We also examined mean chronological age of the three age of onset groups (exact same, backward telescopers, and forward telescopers). Significant differences were found between the three groups for the last three year-by-year comparisons (Table 5). The means tests analyses indicate that
72
Years of Interviews 1988–90 1990–92 1992–94 1994–96 1996–98
Reported Age of Onset Exactly the Same Wave to Wave
15.6% 16.7% 20.1% 25.4% 22.2%
42.3% 50.0% 47.1% 56.1% 54.1%
Reported Age of Onset 1 Year Older or Younger 17.2% 8.3% 10.6% 10.5% 7.4%
Reported Age of Onset 2 or More Years YOUNGER at Wave 2
42.2% 41.7% 42.3% 33.4% 38.5%
Reported Age of Onset 2 or More Years OLDER at Wave 2
TABLE 3 Percentage Who Reported Their Age of Onset for Alcohol Use Accurately and the Variability of Inaccuracy across Waves of Data Collection
73
Adolescent Alcohol Use Self-Report Stability TABLE 4 Reliability of Age of Onset for Alcohol Use across Multiple Waves of Reports N Size 1988 N ¼ 169 64 54 82 73 16 1990 N ¼ 248 132 171 154 1992 N ¼ 463 312 229 158 1994 N ¼ 813 556 369 1996 N ¼ 1,202 646
Year-to-Year Comparison
Intraclass Correlation Coefficient (ICC)
Significance
1988–1990 1988–1992 1988–1994 1988–1996 1988–1998
.559 .313 .031 .240 .050
.001 .009 ns .002 ns
1990–1992 1990–1994 1990–1996 1990–1998
.659 .361 .289 No cases
.000 .000 .000
1992–1994 1992–1996 1992–1998
.666 .474 .225
.000 .000 .000
1994–1996 1994–1998
.714 .492
.000 .000
1996–1998
.659
.000
for all three sets of comparisons, forward telescopers were significantly younger than either the backward telescopers or those reporting the same age. Thus the younger alcohol users were reporting their age of onset as older at Wave 2 compared to that given at Wave 1.
Internal Consistency After responding to questions regarding lifetime use of alcohol and age of onset, youths were further asked recency and frequency questions. TABLE 5 Mean Differences in Chronological Age for Those Reporting Age of Onset for Alcohol Use Consistently, Backward and Forward Telescopers
Years of Interviews 1988–90 1990–92 1992–94 1994–96 1996–98
Reported Age of Onset Exactly the Same Wave to Wave Mean Age at Wave 2
Backward Telescopers Mean Age at Wave 2
Forward Telescopers Mean Age at Wave 2
ANOVA F Test
15.60 16.32 17.27 18.96 18.87
15.67 15.87 18.12 18.94 18.59
15.02 15.64 16.72 18.33 18.45
ns ns 11.62a 5.18a 6.25a
a Scheffe and Bonferroni means test significant difference between backward and forward telescopers; exact age and forward telescopers. p < .01; p < .001.
74
A. M. Shillington et al.
The results of examining the consistency within a wave of data and if a youth logically answered questions consistently are shown in Table 6. These questions asked if alcohol was used in the past 30 days or less recently and then the frequency of use during the time periods during the surveys 1994–1998. Internal consistency was high ranging, from 84% to 90%. The bulk of inconsistencies were found among youths who reported no past 30-day use but then reported a frequency of use of 1þ during the prior month. Phi and Cramer’s V were significant, ranging from .72 to .78. Internal or logical consistency was examined by chronological age. The youths with consistent reports were significantly older than the youths with inconsistencies. The age differences ranged from 0.93 to 1.8 years across the three waves (ANOVA F value at the lowest was 9.25, p < .01 and highest 32.43, p < .001).
Multiple Logistic Regression To test if chronological age continues to be associated with self-report inconsistencies for alcohol use it was entered into logistic regression models along with sex and ethnicity. For ethnicity, the referent group was non-Hispanic, non-African American, and for sex the referent group was female. Results, reported in Table 7, indicate that younger age was significantly associated with report inconsistency for each year-to-year comparison. The odds ratio for age ranged from 1.32 in 1992 to 2.20 in 1994. So for each year of increased age the odds of being a consistent reporter of alcohol use increased significantly. Ethnicity remained significant in two models. For the 1992–1994 model, it was found that African Americans were more likely than the referent group to recant their prior use, and in 1996–1998 the Hispanic youths were more likely to recant their prior use compared to the referent group. TABLE 6 Internal Consistency Between the Recency of Alcohol Use Question and the Frequency of Alcohol Use Question Recency of Last Use Substance Frequency of 30 Day 1994 Used 0 times past 30 days Used 1þ times past 30 days 1996 Used 0 times past 30 days Used 1þ times past 30 days 1998 Used 0 times past 30 days Used 1þ times past 30 days
p < .001.
No Use Past 30 Days N (%)
Used Past 30 Days N (%)
Total Inconsistencies
Phi and Cramer’s V
193 (35.2) 76 (13.9)
6 (1.1) 273 (49.8)
82 (15.0)
.723
204 (27.8) 80 (10.9)
19 (2.6) 423 (58.8)
99 (16.1)
.716
122 (19.2) 40 (6.3)
12 (1.9) 460 (72.6)
52 (10.1)
.777
75
1990 Sex—Male Hispanic African Am. Age in 1990 1992 Sex—Male Hispanic African Am. Age in 1992 1994 Sex—Male Hispanic African Am. Age in 1994 1996 Sex—Male Hispanic African Am. Age in 1996 1998 Sex—Male Hispanic African Am. Age in 1998
ns ns ns 2.12 1.40–3.22
Report Consistency 1990 OR 95% CI
ns ns ns 1.32 1.11–1.58
Report Consistency 1992 OR 95% CI
ns ns 2.97 1.20–7.29 2.20 1.80–2.70
Report Consistency 1994 OR 95% CI
TABLE 7 Logistic Regression Results for Alcohol Use Report Consistency and Demographics
ns ns ns 1.62 1.45–1.81
Report Consistency 1996 OR 95% CI
1.82 1.21–2.73 2.73 1.68–2.54 ns 1.68 1.52–1.86
Report Consistency 1998 OR 95% CI
76
A. M. Shillington et al.
Finally, after controlling for chronological age, the sex differences only remain statistically significant in the 1996–1998 model, with males recanting their prior use more than their female counterparts.
DISCUSSION Regardless of the depth and breadth of alcohol epidemiological data collected, it is very important to understand the accuracy of self-reports of lifetime use or the age of onset. The heavy reliance of epidemiological research on self-reports of past experiences in alcohol use underscores the need to understand accurately the stability of such reports. This study examined the stability of self-reported lifetime use and age of onset for alcohol use with a national sample of children and young adults spanning a decade. The findings indicate that the continued report of alcohol use was good but not excellent. Self-report stability for alcohol use was found to be highly impacted by the chronological age of the respondent. Overall, consistency rates increased from the earliest waves of data collection to the later waves. Concurrently the mean age of the sample increased nearly 5 years across the 10 years of data collection. Further investigation revealed that youths who denied=recanted=forgot their prior alcohol use were from one to more than two years younger than those who reported consistently. This was found for all wave-to-wave comparisons across the 10 years and was further supported from the logistic regression models while controlling for the other demographics under study. Ethnic differences were not found to be consistently associated with recanting. Race and ethnic differences in AOD self-report reliability have been previously identified (Fendrich & Rosenbaum, 2003; Johnson & Mott, 2001; Shillington & Clapp, 2000). However, in this study this was not a consistent finding. In fact the bulk of the results indicated that there are not meaningful differences between the three groups in report stability rates, suggesting ethic differences might not be critical to understanding report stability of alcohol use reports. In the multivariate analysis, differences were found in two models, but the results were not consistent. This study also found differences in the stability of self-reported alcohol use among males and females, with males having lower stability. This was identified in two of the wave-to-wave comparisons but sex only remained in one of the five logistic regression models. Sex differences in recanting have been examined by other studies that found similar results (Percy, McAlister, Higgins, McCrystal, & Thornton, 2005; Shillington & Clapp, 2000), although Fendrich and Rosenbaum (2003) found that females were more likely to recant their prior reports of alcohol use relative to males. One reason why alcohol use recanting may be more problematic compared to other substance use is that respondents’ definition of the word ‘‘use’’
Adolescent Alcohol Use Self-Report Stability
77
may be ambiguous. Many may be offered small quantities of alcohol—say a sip or two of alcohol at very young ages. It is not clear whether respondents will consistently report this type of low-level introductory use as ‘‘use’’ on a survey like the NLSY. Furthermore, such use may qualify as ‘‘use’’ when younger but at a later interview those sips may not longer be perceived as ‘‘use.’’ So question interpretation may be a central issue. Age of onset is a key variable for tracking risk and AOD use trajectories. Across six waves of national data, the stability of this variable was low if the operationalization required the age of onset be an exact match from one wave to the next with only 25%, at the highest, reporting the same age. Stability improves if there is a one-year allowance for a year older or a year younger than the previously stated age of onset. The stability increased to about 50%. The bulk of self-reports that did not fall into that one-year allowance were forward telescopers, which are youths who reported their second age of onset as older than the age of onset previously reported. For these analyses, the age was reported at a minimum of two years older than the age of onset reported just two years previously. These findings are similar to those of Golub and colleagues (2000), who reported forward telescoping of three years for alcohol use. As noted through the intraclass correlations, the reliability of reported age of onset does decrease meaningfully as more years elapse between the anchor year and each subsequent wave of data collection. In fact, the ICCs tend to be near the excellent range when comparing reports with two years between interviews, but they quickly fall into good by four years and the poor within six years. Another study that has examined the ICCs for age of onset report reliability has only reported up to a four-year time span (Johnson & Mott, 2001). However, for alcohol use, they reported similar findings to those presented here, although this study tested for much larger time frames. Similar to Prause and colleagues (2007) we showed that, in terms of the age of onset, chronological age impacted the stability of these self-reports. To our knowledge, this is the first study to examine the chronological age of the respondent as it relates to telescoping. In the last three wave-to-wave comparisons, we found forward telescopers were significantly younger than backward telescopers and those who reported the same age at both times. However, when examining age of onset with ethnicity, our results suggest that ethnic differences might not be critical to understanding report stability of age of onset of alcohol use in that only one of five analyses related to ethnicity was statistically significant. The final type of consistency examined was the logical consistency within waves. The results indicated that the youths were largely consistent in responding to subsequent questions regarding frequency and recency of use with 85% to 90% of self-reports being consistent. For the 10% 15% of inconsistencies, we found that chronological age was important. Youths with
78
A. M. Shillington et al.
inconsistent reports for frequency and recency were significantly younger than those who answered the questions in a logical manner. To our knowledge, this is the first study to examine chronological age and its impact on a respondent’s ability to logically answer self-administered alcohol survey questions.
Strengths and Limitations A major strength of this study is that we were able to use data from a national panel study. We had the ability to examine report stability for use, age of onset, and logical consistency for the same youths. Furthermore, this is the first study to examine such measures of stability across a decade of self-reported alcohol use. One limitation of this research is that as the NLSY surveys went to field, some changes were made to the ever use questions for one or two survey years. The different wording of questions could have impacted how a youth interpreted this question. However, the data do not reveal any significant shifts in report consistency related to such changes. Furthermore, such changes were in place for all respondents so this would not explain the differences identified here. Another limitation is that the youths in this study are the older children to young mothers. So although the sample is a national sample and the mothers are nationally generalizable, the findings from the children are not generalizable to all youths in this age range. Lifetime use questions were phrased as ‘‘ever use’’ questions. Some of the report instability may be due to the issue of threshold. Children who reported prior use and denied it later may have reported ‘‘yes’’ to drinking but had only a few sips. Later denial may be a result of the youths deciding that a few sips should not be considered drinking. Further research may examine this issue of question phrasing. So, with the lack of data trends coinciding with wording changes and the need for research that examines this important issue, the limitation is noted but thought to be acceptable. Because there is so little research that examines these specific issues, it is important to advance our understanding due to so many AOD studies collecting data with cross-sectional designs.
Future Research and Conclusions Ethnic differences are not very clear from this study. Some differences were found; however, the findings do not lend themselves to any specific conclusions. As noted previously, other research has identified strong ethnic differences in report consistency; however, none have examined this issue across so many waves of data collection. More research is needed to understand under which circumstances such differences may exist and why.
Adolescent Alcohol Use Self-Report Stability
79
This topic needs to be examined within the context of a broader theoretical perspective regarding factors affecting the reporting of potentially sensitive behavior. To move the field forward and potentially improve the process of survey research on alcohol use, future research needs to conduct qualitative research and longitudinal research with a theoretical framework to guide queries into report discrepancies identified as they occur. Clinical implications would include the stability of adolescents reporting their use of alcohol with diagnostic instruments. Such inconsistencies may lead to some adolescents being under-diagnosed and referred to treatment. Because we don’t know the reasons for the inconsistencies, it is difficult to state whether the older adolescents are better reporters or if they have motivations to disclose their alcohol use that differ from younger adolescents. Our findings indicate a strong impact of chronological age on report accuracy. In every type of alcohol report stability examined and for nearly every set of analyses, younger respondents were more likely to have report errors compared to their older peers. More research is needed to understand the reasons for this. It is not possible to delve into this issue with the current longitudinal data. Such research should include a focus on cognitive developmental issues and their impact on self-reports. Future research should be designed to specifically identify report instability and query the youths in-depth regarding such inconsistency. Because the findings from this research indicate that younger age is important to understanding self-report stability, a strong focus on cognitive development would be essential. This is particularly important in light of recent suggestions that causal models for underage drinking risk be expanded into early childhood years well before age 10 (Zucker, Donovan, Masten, Mattson, & Moss, 2008). Another line of research would include looking at whether younger teens who report alcohol use may be ‘‘riskier’’ drinkers; perhaps there is something inherently different about them and their recall or reporting.
REFERENCES Aiken, L. S. (1986). Retrospective self-reports by clients differ from original reports: Implications for the evaluation drug treatment programs. The International Journal of the Addictions, 21, 767–788. Bailey, S. L., Flewelling, R. L., & Rachal, V. J. (1992). The characterization of inconsistencies in self-reports of alcohol and marijuana use in a longitudinal study of adolescents. Journal of Studies on Alcohol, 53, 636–647. Baker, P. C., Keck, C. K., Mott, F. L., & Quinlan, S. V. (1993). NLSY child handbook– Revised edition: A guide to the 1986–1990 National Longitudinal Survey of Youth Data. Columbus, OH: Center for Human Resource Research. Baker, P. C., & Mott, F. L. (1989). NLSY child handbook 1989: A guide and resource document for the National Longitudinal Survey of Youth 1986
80
A. M. Shillington et al.
child data. Columbus, OH: Center for Human Resource Research, The Ohio State University. Barnea, Z., Rahav, G., & Teichman, M. (1987). The reliability and consistency of self-reports on substance use in a longitudinal study. British Journal of Addiction, 82, 891–898. Bartko, J. J. (1966). The intraclass correlation coefficient as a measure of reliability. Psychological Reports, 19, 3–11. Fendrich, M., & Rosenbaum, D. P. (2003). Recanting of substance use reports in a longitudinal prevention study. Drug and Alcohol Dependance, 70, 241–253. Fendrich, M., Weissman, M. M., Warner, V., & Mufson, L. (1990). Two-year recall of lifetime diagnoses in offspring at high and low risk for major depression. Archives of General Psychiatry, 47, 1121–1127. Funkhouser, J. E., Goplerud, E. N., & Bass, R. O. (1992). The current status of prevention strategies. In M. A. Jansen (Ed.), A promising future: Alcohol and other drug problem prevention services improvement (pp. 17–82). [OSAP Prevention Monograph No. 10, DHHS) Pub No. (ADM) 92-1807]. Washington, DC: Government Printing Office. Golub, A., Labouvie, E., & Johnson, B. D. (2000). Response reliability and the study of adolescent substance use progression. Journal of Drug Issues, 30, 103–118. Grant, B. F. (1998). Age at smoking onset and its association with alcohol consumption and dependence: Results from the National Longitudinal Alcohol Epidemiologic Survey. Journal of Substance Abuse, 10, 59–73. Grant, B. F., & Dawson, D. A. (1997). Age at onset to alcohol use and its association with DSM-IV alcohol abuse and dependence: Results from the National Longitudinal Alcohol Epidemiologic Survey. Journal of Substance Abuse, 9, 103–110. Hawkins, B., Catalano, R. P., & Miller, J. Y. (1992). Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: Implications for substance abuse prevention. Psychological Bulletin, 112, 64–105. Hingson, R., Heeren, T., Jamanka, A., & Howland, J. (2000, September). Age of drinking onset and unintentional injury involvement after drinking. JAMA: Journal of the American Medical Association, 284(12), 1527–1533. Johnson, T. P., & Mott, J. A. (2001). The reliability of self-reported age of onset of tobacco, alcohol and illicit drug use. Addiction, 96, 1187–1198. Johnston, L. D., O’Malley, P. M., Bachman, J. G., & Schulenberg, J. E. (2008). Monitoring the future national results on adolescent drug use: Overview of key findings, 2007 (NIH Publication No. 08-6418). Bethesda, MD: National Institute on Drug Abuse. Retrieved April 21, 2008, from http://monitoringthefuture. org/pubs.html McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46. Miller, T., Levy, D., Spicer, R., & Taylor, D. (2006, July). Societal costs of underage drinking. Journal Studies on Alcohol, 67(4), 519–528. Retrieved July 23, 2009, from PsycINFO database.
Adolescent Alcohol Use Self-Report Stability
81
Mills, C. J., & Noyes, H. L. (1984). Patterns and correlates of initial and subsequent drug use among adolescent. Journal of Consulting and Clinical Psychology, 52, 231–243. O’Malley, P. M., Bachman, J. G., & Johnston, L. D. (1983). Reliability and consistency in self-reports of drug use. International Journal of the Addictions, 18, 805–824. Percy, A., McAlister, S., Higgins, K., McCrystal, P., & Thornton, M. (2005). Response consistency in young adolescents’ drug use self-reports: A recanting rate analysis. Addiction, 100, 189–196. Perkonigg, A., Goodwin, R., Behrendt, S., Breesdo, K., Lieb, R., Wittchen, H., et al. (2008, September). Cannabis use—Do we have solutions? A reply to Macleod. Addiction, 103(9), 1575. Prause J., Dooley, D., Ham-Rowbottom, K. A., & Emptage, N. (2007). Alcohol drinking onset: A reliability study. Journal of Child and Adolescent Substance Abuse, 16(4), 79–90. Shillington, A. M., & Clapp, J. D. (2000). Self-report stability of adolescent substance use: A first look at gender, ethnicity, and behavioral problems. Drug and Alcohol Dependance, 60(1), 19–27. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428. SPSS, Inc. (2007). Statistical package for the social sciences, version 15.0.1.1. Chicago, IL: Author. Stinchfield, R. (1997). Reliability of adolescent self-reported pretreatment alcohol and other drug use. Subtance Use & Misuse, 32, 425–434. Winters, K. C., Stinchfield, R. D., Henly, G. A., & Schwartz, R. H. (1991). Validity of adolescent self-report of alcohol and other drug involvement. International Journal of the Addictions, 25(11a), 1379–1395. Wittchen, H. U., Behrendt, S., Hofler, M., Perkonigg, A., Lieb, R., Buhringer, G., et al. (2008). What are the high risk periods for incident substance use and transitions to abuse and dependence? Implications for early intervention and prevention. International Journal of Methods in Psychiatric Research, 17, S16–S29. Zucker, R. A., Donovan, J. E., Masten, A. S., Mattson, M. E., & Moss, H. B. (2008). Early development processes and the continuity of risk for underage drinking and problem drinking. Pediatrics, 121, S252–S272.
Copyright of Journal of Child & Adolescent Substance Abuse is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
