GASTROENTEROLOGY 2011;140:65–72
Analysis of Administrative Data Finds Endoscopist Quality Measures Associated With Postcolonoscopy Colorectal Cancer NANCY N. BAXTER,*,‡,§ RINKU SUTRADHAR,‡,储 SHAWN S. FORBES,‡ LAWRENCE F. PASZAT,‡,§,储 REFIK SASKIN,‡ and LINDA RABENECK‡,§,储,¶,# *Department of Surgery and Keenan Research Centre, Li Ka Shing Knowledge Institute, St Michael’s Hospital, University of Toronto; ‡Institute for Clinical Evaluative Sciences; §Department of Health Policy, Management, and Evaluation, 储Dalla Lana School of Public Health, and #Department of Medicine, University of Toronto; and ¶ Odette Cancer Centre, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
See related articles, Kahi CJ et al, on page 42; and Sung JJY et al on page 47 in CGH; see editorial on page 19.
BACKGROUND & AIMS: Most quality indicators for colonoscopy measure processes; little is known about their relationship to patient outcomes. We investigated whether characteristics of endoscopists, determined from administrative data, are associated with development of postcolonoscopy colorectal cancer (PCCRC). METHODS: We identified individuals diagnosed with colorectal cancer in Ontario from 2000 to 2005 using the Ontario Cancer Registry. We determined performance of colonoscopy using Ontario Health Insurance Plan data. Patients who had complete colonoscopies 7 to 36 months before diagnosis were defined as having a PCCRC. Patients who had complete colonoscopies within 6 months of diagnosis had detected cancers. We determined if endoscopist factors (volume, polypectomy and completion rate, specialization, and setting) were associated with PCCRC using logistic regression, controlling for potential covariates. RESULTS: In the study, 14,064 patients had a colonoscopy examination within 36 months of diagnosis; 584 (6.8%) with distal and 676 (12.4%) with proximal tumors had PCCRC. The endoscopist’s specialty (nongastroenterologist/nongeneral surgeon) and setting (non– hospital-based colonoscopy) were associated with PCCRC. Those who underwent colonoscopy by an endoscopist with a high completion rate were less likely to have a PCCRC (distal: odds ratio [OR], 0.73; 95% confidence interval [CI], 0.54 – 0.97; P ⫽ .03; proximal: OR, 0.72; 95% CI, 0.53– 0.97; P ⫽ .002). Patients with proximal cancers undergoing colonoscopy by endoscopists who performed polypectomies at high rates had a lower risk of PCCRC (OR, 0.61; 95% CI, 0.42– 0.89; P ⬍ .0001). Endoscopist volume was not associated with PCCRC. CONCLUSIONS: Endoscopist characteristics derived from administrative data are associated with
development of PCCRC and have potential use as quality indicators. Keywords: Colonoscopy Quality; Colorectal Cancer Detection; Quality Indicators.
N
umerous organizations have called for the development of performance measures to provide meaningful information on quality of care.1–3 Often, performance measures are based on administrative data because these data are readily available and in many cases comprehensive in coverage. However, quality indicators derived from administrative data frequently measure processes of care (ie, elements of medical care believed to have an impact on outcomes) and not outcomes of care. In fact, in many cases, particularly when adverse outcomes are rare or difficult to measure, measurement of processes may be the only way to practically assess quality; however, it is essential to understand the relationship between process measures and actual patient outcomes to ensure the validity of these indicators. Colonoscopy is a common procedure and is advocated as the procedure of choice for screening and prevention of colorectal cancer (CRC) by many.4 – 6 However, there is good evidence that the quality of colonoscopy varies and that this variation has an impact on effectiveness.7–9 Although a number of quality indicators for this procedure have been recommended,10 most proposed indicators cannot be assessed using administrative data. In an attempt by the American Medical Association Physician Consortium for Performance Improvement to identify quality indicators for use in the Centers for Medicare and Medicaid Services Pay for Quality Reporting Initiative, only 3 quality indicators for colonoscopy were approved; Abbreviations used in this paper: CI, confidence interval; CRC, colorectal cancer; ICD-9, International Classification of Diseases, Ninth Revision; OR, odds ratio; PCCRC, postcolonoscopy colorectal cancer. © 2011 by the AGA Institute 0016-5085/$36.00 doi:10.1053/j.gastro.2010.09.006
CLINICAL– ALIMENTARY TRACT
CLINICAL—ALIMENTARY TRACT
66
BAXTER ET AL
CLINICAL– ALIMENTARY TRACT
of these, 2 relate to intervals between colonoscopies and one relates to quality of reporting; none measure performance.11 We designed this study to determine if performancebased quality indicators of colonoscopy could be identified at the endoscopist level using administrative data and if these indicators are valid predictors of an important outcome, postcolonoscopy CRC (PCCRC), the diagnosis of CRC within 3 years following a colonoscopy in which the cancer was not detected. We prefer the term “PCCRC” to “interval cancer” because the latter refers to cancers that occur following a negative finding on a screening episode in the interval before the next screening is due.
Patients and Methods Overview of Study Design This is an observational study of all individuals diagnosed with CRC between January 1, 2000, and December 31, 2005, with a known site of primary who underwent a colonoscopy within 36 months of diagnosis.
Data Sources We used 5 data sources. 1. The Ontario Cancer Registry includes information on all incident cancers diagnosed since 1964 in Ontario and is estimated to be more than 95% complete.12 2. The Ontario Health Insurance Plan database contains information on claims billed by physicians for services, permitting identification of virtually all medical procedures occurring in Ontario (other than in limited geographical areas where physicians use an alternative payment plan). 3. The Canadian Institute for Health Information hospital discharge abstract database contains information on every patient discharged from a hospital or same day surgery unit in Ontario and includes patient demographic information, major diagnoses, procedures, and discharge status. 4. The Registered Persons Database is a roster of all Ontario Health Insurance Plan beneficiaries (virtually all residents of Ontario) and includes demographic information and length of eligibility. 5. The Ontario Physicians Human Resources Data Centre has a registry of all licensed physicians practicing in Ontario and is the definitive source of information regarding the physician workforce in the province.
Patients We identified patients from the Ontario Cancer Registry diagnosed with CRC from January 1, 2000, to December 31, 2005. We included only those with location of cancer (assessed by registrars of the Ontario Cancer Registry from pathology reports and hospital discharge summaries) recorded. We defined patients with cancers
GASTROENTEROLOGY Vol. 140, No. 1
proximal to the splenic flexure as having proximal cancers (International Classification of Diseases, Ninth Revision [ICD-9] codes 153.4, 153.6, 153.0, 153.1). Those with cancers at or distal to the splenic flexure were considered to have distal cancers (ICD-9 codes 154.1, 154.0, 153.3, 153.2, 153.7; Table 1). Of the 34,312 patients diagnosed with a CRC in Ontario with a defined primary site, we excluded individuals aged younger than 20 years (n ⫽ 12), those with a previous diagnosis of CRC (n ⫽ 93), those residing in the geographic areas of Ontario where physicians do not bill directly for services (n ⫽ 1650), and those with a history of Crohn’s disease or ulcerative colitis (n ⫽ 305). We included only individuals who had a complete colonoscopy (code Z555: colonoscopy to the descending colon plus E747 [to cecum] or E705 [to terminal ileum]) within 36 months of diagnosis (n ⫽ 14,342). Of these, we excluded 275 individuals with a previous colon resection and 3 individuals with missing demographic information.
Identification of Missed and Detected Cancers Similar to previous research,13,14 we defined individuals who developed a PCCRC after colonoscopy as those who had a complete colonoscopy 7 to 36 months before diagnosis of cancer. Those undergoing complete colonoscopy within 6 months of diagnosis with no preceding colonoscopy were considered to have a detected cancer. This definition assumes that virtually all patients with CRC identified at colonoscopy would have a definitive diagnosis recorded in the Ontario Cancer Registry within 6 months of initial colonoscopic detection. For patients who had more than one colonoscopy within 36 months, assignment as a PCCRC or detected cancer was based on the date of the first complete colonoscopy performed; this became the colonoscopy of interest for our analysis.
Determining Covariates We determined patient demographic information including age, sex, and geographic location by health care region. Comorbid conditions were evaluated based on ICD-9 codes for those with hospital discharges within 60 months of the index date. The Charlson score15 was computed for each patient and categorized as 0, 1, 2, or ⱖ3. Those with no hospital admission were assigned a score of 0.
Endoscopist Characteristics We hypothesized that a variety of endoscopist characteristics would be associated with the likelihood of missing a cancer or precursor lesion, including training (endoscopist specialty16), setting of colonoscopy (in hospital vs out of hospital17), and potential indicators of process quality at the endoscopist level (procedure vol-
January 2011
QUALITY INDICATORS FOR COLONOSCOPY
67
Diagnostic codes Right-sided cancer Left-sided cancer Diagnosis of inflammatory bowel disease Procedure codes Any attempted colonoscopy Complete colonoscopy
Polypectomy
Previous colorectal resections
Cecum to transverse colon: ICD-9 codes 153.4, 153.6, 153.0, 153.1 Splenic flexure to rectum: ICD-9 codes 154.1, 154.0, 153.3, 153.2, 153.7 ICD-9 codes 556.0 –556.9, 555.0 –555.9 ICD-10 codes K50.0 –50.9, K51.0 –51.9 OHIP billing code Z555: colonoscopy to descending colon OHIP billing code Z555 with accompanying codes E747: colonoscopy to cecum or E705: colonoscopy to terminal ileum OHIP billing codes Z571: excision polyp, through colonoscope (first polyp) E720: excision of polyp through colonoscope (up to 2 additional) OHIP billing codes S162, S166, S167, S169, S172, S171, S168, S170, S173, S174, S188, S177, S213, S214, S215, S216, S217 CCP codes 57.5x, 57.6x, 60.4x, 60.5x CCI codes 1.NM.87x, 1.NM.89x, 1.NM.91x, 1.NQ.87x, 1.NQ.89x, 1.NQ.90x
ume,18 polypectomy rate,8,19 and completion rate20,21). For each patient in our study, we determined the endoscopist performing the colonoscopy using billing codes. We ascertained the self-designated specialty for the billing endoscopist from the Ontario Physicians Human Resources Data Centre. For each endoscopist, procedure volume, polypectomy rates (adenoma detection rates could not be obtained from administrative data), and completion rates were calculated based on the billings of the endoscopist in the 2 years before each colonoscopy associated with a detected or PCCRC; this enabled endoscopist volume and rates to change over time. Specifically, the endoscopist procedure volume for each patient was calculated by taking the yearly average of all colonoscopies (irrespective of completion) performed by the relevant endoscopist for the 2 years before the date of the patient’s colonoscopy. The polypectomy rate for that endoscopist was calculated as the proportion of colonoscopies (irrespective of completion) that were associated with a code indicating removal of a polyp ⱖ3 mm over the same 2-year period. Polyps ⬍3 mm removed by biopsy or fulguration were not included. (A single code is used for polyps 3–30 mm in size, so restricting to polypectomies greater than 5 or 10 mm was not possible.) The completion rate was calculated as the proportion of colonoscopies performed by the relevant endoscopist over a 2-year period that were complete to the cecum. We determined the overall completion rate and completion rate stratified by patient sex. Where there was a Canadian Institute for Health Information admission record that corresponded to the Ontario Health Insurance Plan billing record, the colonoscopy was considered hospital based. Hospital setting was further subdivided into academic or community hospital based on the hospital designation. Patients who had only physician billing records for colonoscopy were considered to have had a non– hospital-based colonoscopy.
Analysis We calculated descriptive statistics for study variables stratified by primary site. The outcome of interest was the development of a PCCRC. Using generalized estimating equations22 to account for clustering at the level of the endoscopist, we performed logistic regression stratified by primary site to calculate odds ratio (OR) estimates and 95% confidence intervals (CIs) for endoscopist characteristics including specialty of endoscopist (gastroenterologist, general surgeon, nongastroenterologist/nongeneral surgeon [including internists, general practitioners, and family physicians]), adjusting for patient covariates including age (treated as continuous), sex, Charlson comorbidity score (0, 1, 2, 3, or more), and setting of colonoscopy (academic hospital, community hospital, nonhospital setting). Potential quality indicators at the endoscopist level included percent complete overall and by sex (⬍80%, 80%– 84%, 85%– 89%, 90%–94%, ⬎94%), percentage of colonoscopies associated with a polypectomy code (⬍10%, 10%–14%, 15%–19%, 20%–24%, 25%–29%, ⬎29%), and procedure volume. We modeled procedure volume in a variety of ways: continuous, log transformed, and categorical. The cut points used to categorize procedure volume were optimally selected based on graphical diagnostic plots.23 We tested for interactions between covariates. We analyzed data using SAS version 9.1 (SAS Institute, Inc, Cary, NC). All statistical tests were 2 sided, and P values less than .05 were considered statistically significant. The study was approved by the Research Ethics Board of Sunnybrook Health Sciences Centre (Toronto, Ontario, Canada). All data analysis was conducted at the Institute for Clinical Evaluative Sciences, a Section 45 (1) prescribed entity in Ontario’s Personal Health Information Protection Act.
CLINICAL– ALIMENTARY TRACT
Table 1. Diagnostic and Procedure Codes Used in the Study
386.5 (1.5–1819.0) 387.5 (0.5–1965.5) 382.0 (1.5–1645.5) 384.5 (1.5–1819.0) 391.0 (0.5–1965.5)
reports these values per patient and a single endoscopist may contribute multiple values aTable
0.89 (0.05–1.00) 0.90 (0.00–1.00)
394.5 (0.5–1826.5)
0.90 (0.16–1.00) 0.90 (0.04–1.00) 0.89 (0.05–1.00)
207 (16.5) 919 (72.9) 134 (10.6) 0.17 (0.00–0.78) 2116 (16.5) 9848 (76.9) 840 (6.6) 0.19 (0.00–1.00)
0.90 (0.00–1.00)
1278 (16.0) 6176 (77.1) 554 (6.9) 0.19 (0.00–1.00)
228 (39.0) 275 (47.1) 81 (13.9) 2790 (34.8) 4459 (55.7) 759 (9.5) 241 (35.6) 339 (50.2) 96 (14.2) 469 (37.2) 614 (48.7) 177 (14.1) 4741 (37.0) 6841 (53.4) 1222 (9.5)
1951 (40.7) 2382 (49.7) 463 (9.6)
446 (76.4) 74 (12.7) 38 (6.5) 26 (4.4) 7085 (88.5) 567 (7.1) 228 (2.8) 128 (1.6) 538 (79.6) 72 (10.7) 34 (5.0) 32 (4.7) 4101 (85.5) 417 (8.7) 167 (3.5) 111 (2.3) 984 (78.1) 146 (11.6) 72 (5.7) 58 (4.6) 11,186 (87.4) 984 (7.7) 395 (3.0) 239 (1.9)
69 (25–97) 248 (42.5) 66 (21–96) 3027 (37.8) 72 (27–93) 348 (51.5) 71 (21–98) 2523 (52.6) 71 (25–97) 596 (47.3) 68 (21–98) 5550 (43.4)
Median (range) age (y) Female (%) Charlson score, n (%) 0 1 2 ⱖ3 Specialty of endoscopist, n (%) Gastroenterologist Surgeon Other Setting of endoscopist, n (%) Academic hospital Community hospital Nonhospital a2-Year median polypectomy rate of endoscopist performing procedure (range) a2-Year median completion rate of endoscopist performing procedure (range) a2-Year median procedure volume of endoscopist performing procedure (range)
PCCRC (n ⫽ 584) Detected (n ⫽ 8008) PCCRC (n ⫽ 676) Detected (n ⫽ 4796) PCCRC (n ⫽ 1260) Detected cancer (n ⫽ 12,804)
Proximal cancers (n ⫽ 5472)
Table 2. Demographic Information for Patients With Detected Cancer and Patients With PCCRC
CLINICAL– ALIMENTARY TRACT
Of the 34,312 patients in Ontario diagnosed with CRC in our study period, 14,064 had a complete colonoscopy within 36 months of diagnosis and met all study criteria. Of these, 1260 (9.0%) were considered to have a new or missed cancer (PCCRC). Overall, 61.1% of patients with CRC were diagnosed with a distal CRC. Patients with proximal CRC were more likely to have a PCCRC than patients with distal CRC (676 patients with proximal cancer had a PCCRC [12.4%] vs 584 patients [6.8%] with distal cancer; P ⬍ .0001) (Table 2). A total of 724 endoscopists performed at least one colonoscopy that detected or missed a cancer in the study. A majority of colonoscopies were performed either by gastroenterologists (5210 [37.0%]) or surgeons (7455 [53.0%]). Most colonoscopies were performed in a hospital setting (13,090 [93.1%]). For the endoscopists, median colonoscopy volume over a 2-year period was 258 (range, 1–1678), median polypectomy rate was 17.7% (range, 0.0%–72.5%), and median completion rate was 87.6% (range, 0.0%–100.0%). We initially attempted to develop a single multivariate model; however, there were multiple significant interactions between site of cancer (proximal vs distal) and endoscopist measures. We therefore developed separate models for patients with proximal and distal cancers (Table 3). The odds ratio for age was reported for every 10-year increase. Patient factors, including older age, female sex, and a higher comorbidity score, were statistically significantly associated with diagnosis of a PCCRC for patients with distal CRC while only a higher comorbidity score was associated with proximal PCCRC. Endoscopy factors were associated with both proximal and distal PCCRC; endoscopist specialty (nongastroenterologist, nonsurgeon) was associated with the diagnosis of a PCCRC, as was setting of the colonoscopy (office vs hospital based). After controlling for patient and endoscopy factors, patients undergoing colonoscopy performed by an endoscopist with a completion rate of ⱖ95% were less likely to have a PCCRC than if performed by an endoscopist with a ⬍80% completion rate for proximal (OR, 0.72; 95% CI, 0.53– 0.97) and distal (OR, 0.73; 95% CI, 0.54 – 0.97) cancers. For patients with proximal cancers, polypectomy rate was also associated with the diagnosis of a PCCRC; patients undergoing colonoscopy performed by an endoscopist with a polypectomy rate ⱖ30% were less likely to develop a PCCRC than if colonoscopy were performed by an endoscopist with a ⬍10% polypectomy rate for proximal PCCRC (OR, 0.61; 95% CI, 0.42– 0.89). Polypectomy rate was not associated with diagnosis of a distal PCCRC. Procedure volume was not found to be significant in any model. There appeared to be more variation in endoscopists’ completion rate for female patients than for male patients or for patients overall. We repeated our logistic multivariate models including all covariates, changing
Distal cancers (n ⫽ 8592)
Results
101 (17.3) 414 (70.9) 69 (11.8) 0.18 (0.00–0.78)
GASTROENTEROLOGY Vol. 140, No. 1
106 (15.7) 505 (74.7) 65 (9.6) 0.16 (0.00–0.71)
BAXTER ET AL
838 (17.5) 3672 (76.5) 286 (6.0) 0.19 (0.00–0.76)
68
January 2011
QUALITY INDICATORS FOR COLONOSCOPY
69
Table 3. Multivariate Model Generalized Estimating Equation Logistic Predicting PCCRC
OR (95% CI) Age (for every 10-year increase) Sex Female Male Charlson score 0 1 2 3⫹ Log (endoscopist volume) % Completeness for endoscopist ⬍80% 80%–84% 85%–89% 90%–94% 95%⫹ % Polypectomy for endoscopist ⬍10% 10%–14% 15%–19% 20%–24% 25%–29% 30%⫹ Specialty of endoscopist Gastroenterologist Surgeon Other Setting of colonoscopy Academic hospital Community hospital Nonhospital
Distal cancers P value
OR (95% CI)
P value
1.05 (0.98–1.13)
.17
1.18 (1.08–1.28)
.0001
1.00 (referent) 1.02 (0.86–1.20)
.83
1.00 (referent) 0.79 (0.66–0.93)
.005
1.00 (referent) 1.27 (0.98–1.65) 1.60 (1.09–2.34) 2.02 (1.34–3.03) 1.00 (0.89–1.13)
.004
1.00 (referent) 1.88 (1.44–2.46) 2.29 (1.58–3.31) 2.78 (1.78–4.35) 0.94 (0.84–1.05)
⬍.0001
1.00
.28
1.00 (referent) 1.16 (0.86–1.56) 0.69 (0.51–0.93) 0.66 (0.50–0.87) 0.72 (0.53–0.97)
.002
1.00 (referent) 0.90 (0.65–1.25) 0.65 (0.47–0.89) 0.71 (0.54–0.93) 0.73 (0.54–0.97)
.03
1.00 (referent) 1.11 (0.81–1.53) 0.75 (0.54–1.04) 0.75 (0.52–1.07) 0.52 (0.35–0.79) 0.61 (0.42–0.89)
.0001
1.00 (referent) 0.99 (0.73–1.35) 0.78 (0.57–1.06) 0.82 (0.58–1.16) 0.87 (0.61–1.24) 0.79 (0.54–1.14)
.39
1.00 (referent) 1.23 (0.96–1.57) 1.87 (1.34–2.60)
.006
1.00 (referent) 0.96 (0.73–1.25) 1.67 (1.13–2.46)
.001
1.00 (referent) 1.11 (0.83–1.50) 1.88 (1.2–2.92)
.05
1.00 (referent) 0.96 (0.73–1.25) 1.67 (1.13–2.46)
.05
only the method of assessing completion rate (overall completion rate, completion rate in women only, completion rate in men only) to determine if completion rate for women or men was more strongly associated with PCCRC than overall completion rate (Table 4). Completeness in women was more strongly associated with PCCRC than completeness in men or completeness overall for both proximal and distal cancers.
Discussion Colonoscopy is a common procedure and a key component of CRC control strategies at the population level. Previous studies have documented variation in performance of colonoscopy; adenoma detection rate, for example, is known to vary between endoscopists19,24 and is associated with factors such as scope withdrawal time,8 a proxy for careful colonoscopy technique. Variation in the quality of colonoscopy likely has an important impact on effectiveness and patient outcomes, and this has led to a call for the routine assessment of quality, particularly for screening examinations.25–30 However, the literature evaluating the impact of quality variation on meaningful outcomes is limited and there is an inade-
quate research base evaluating candidate quality metrics, particularly population-based measures using administrative data. Completion rates have been proposed as a quality metric by a number of societies, researchers, and clinicians because it is reasonable to assume that the effectiveness of colonoscopy is limited if the entire colon is not routinely examined.5,28,29 Similarly, because removal of adenomatous tissue underlies the preventive potential of colonoscopy, adenoma detection rate has also largely been accepted as a quality metric.10,29 Despite the acceptance of these measures, little research has evaluated the relationship between endoscopist performance on these metrics and patient outcomes. A recent study31 has shown an association between endoscopists’ adenoma detection rate (but not cecal intubation rates) and development of “interval cancer” in a large study of screening colonoscopy; however, only 42 interval cancers were included in the analysis. Our study, based on administrative data including far greater numbers of PCCRC and endoscopists, adds significantly to the understanding of the relationship between measurable quality indicators and clinically relevant outcomes.
CLINICAL– ALIMENTARY TRACT
Proximal cancers
70
BAXTER ET AL
GASTROENTEROLOGY Vol. 140, No. 1
Table 4. Association of Sex-Stratified Completion Rate and PCCRC in the Multivariate Generalized Estimating Equation Logistic Model % Completeness Measure of endoscopist completion rate CLINICAL– ALIMENTARY TRACT
Association between completion rate and proximal PCCRC Completion rate in all patients Completion rate in female patients Completion rate in male patients Association between completion rate and distal PCCRC Completion rate in all patients Completion rate in female patients Completion rate in male patients
⬍80%
80%–84%
85%–89%
90%–94%
95%⫹
P value
1.00 (referent) 1.16 (0.86–1.56) 0.69 (0.51–0.93) 1.00 (referent) 0.81 (0.61–1.07) 0.61 (0.46–0.81) 1.00 (referent) 0.91 (0.65–1.26) 0.99 (0.74–1.34)
0.66 (0.50–0.87) 0.55 (0.42–0.74) 0.71 (0.53–0.94)
0.72 (0.53–0.97) 0.64 (0.48–0.85) 0.69 (0.51–0.94)
.002 .0003 .007
1.00 (referent) 0.90 (0.65–1.25) 0.65 (0.47–0.89) 1.00 (referent) 0.66 (0.47–0.91) 0.64 (0.46–0.88) 1.00 (referent) 0.90 (0.62–1.29) 0.80 (0.57–1.13)
0.65 (0.47–0.89) 0.68 (0.52–0.88) 0.75 (0.56–1.01)
0.79 (0.54–1.14) 0.70 (0.53–0.93) 0.80 (0.60–1.06)
.03 .003 .02
NOTE. Model includes patient age, sex, Charlson score, endoscopist volume, polypectomy rate, and specialty and setting of colonoscopy. Only the measure of endoscopist completion rate is varied between these models.
Importantly, our study focused on measures obtained from administrative data, a readily available source of information for quality measurement. After controlling for patient factors and endoscopy factors (including setting of the colonoscopy and training of the endoscopist), past performance of the endoscopist in the preceding 2 years with respect to completion of colonoscopy and the percentage of colonoscopies associated with a polypectomy were statistically significantly associated with patients developing a PCCRC within 3 years of the procedure. The association between PCCRC and completion rate was strongest when the endoscopist’s performance for only female patients over the past 2 years was considered the quality indicator (as compared with completion rate overall or in men alone). Interestingly, we did not find that procedure volume in the preceding 2 years was associated with PCCRC. We found the proportion of patients diagnosed with CRC who underwent colonoscopy 6 to 36 months before diagnosis (PCCRC) to be 9%, a higher proportion than previously reported. A retrospective study from 20 Indiana hospitals reported that 47 (5%) of 941 patients with CRC who had a colonoscopy within 3 years before their diagnosis had reportedly normal findings on colonoscopy.32 In addition, an Ontario population-based cohort study of 12,487 persons with CRC diagnosed from 1997 to 2002 reported that 2% to 6% had a PCCRC within 3 years of colonoscopy, with right-sided CRC associated with a greater risk.33 Of interest is a recently completed US study of 69,591 persons aged 69 years or older with a new diagnosis of CRC. That study, which used linked SEER-Medicare data, reported a PCCRC rate of 7.5%.17 Differences in study population, time period, and methods may explain the difference. In future work, it will be important to validate the definition of PCCRC. Nonetheless, taken together, these studies show that PCCRC occurs in a small but clinically meaningful number of individuals.
This study is population based, which is a distinct strength because the results do not reflect only the practices of expert high-volume endoscopists in specialized centers; our results reflect not only the general population of patients undergoing colonoscopy but also the general population of endoscopists providing colonoscopy in Ontario. However, because of the data available for analysis, there are distinct limitations of our study. We have included any patients developing CRC within 3 years of colonoscopy and are unable to differentiate screening from diagnostic or confirmatory examinations. However, it is unlikely the predictors we found to be associated with PCCRC would differ depending on reason for colonoscopy. Also, endoscopist-level variables used in the analysis were not based on the colonoscopy associated with the PCCRC or detected cancer; rather, the performance of the endoscopist in the previous 2 years (including diagnostic and screening examinations) was considered. The use of administrative data has limitations; coding errors may have resulted in misclassification, although previous studies of our data sources have generally shown that procedures and diagnoses are coded accurately.34 Additionally, if miscoding did occur in procedure billing data, almost certainly the miscoding would be in the direction of better than true performance. Such miscoding would bias the study toward the null. Our data enabled us to determine polypectomy rate versus adenoma detection rate, and although this has limitations, 2 recent studies have shown that polypectomy rate is a valid proxy for adenoma detection rate.35,36 Additionally, we could not assess a number of other aspects of care that are likely to have an impact on colonoscopy performance and effectiveness, including factors such as quality of preparation and sedation. Although our definition of PCCRC has proven useful in prior research,14,16,33 further information regarding the validity of this definition is needed and this research is ongoing.
Although the use of administrative data has limitations, our findings indicate that meaningful quality indicators can be derived from these data. These findings could be applied with relative ease by payers to assess “quality” of colonoscopy at the endoscopist level potentially as part of an audit and feedback program or even for pay for performance. It is less clear, however, that the measurement and use of these indicators in performance management would lead to an improvement in the quality of colonoscopy or a reduction in the number of PCCRCs in the population. In fact, the introduction of quality indicators derived from administrative data that are surrogate measures (eg, the measurement of polypectomy rate vs adenoma detection rate) might lead to “gaming” the system, where performance on the indicator improves without any true benefit. This reflects the limited evidence base for such quality improvement strategies overall. In conclusion, our large population-based study including 14,064 patients with CRC and 724 endoscopists has shown that polypectomy rate and completion rate (particularly completion rate in female patients) as measured by administrative data are associated with development of PCCRC, an inarguably important outcome, supporting the use of these measures as quality indicators. Future research should be directed at determining the best way to use quality indicators for colonoscopy in a manner that results in improved patient care and outcomes. References 1. US Department of Health and Human Services. Value driven healthcare quality standards. Available at: http://archive.hhs. gov/valuedriven/fourcornerstones/quality/index.html. Accessed June 4, 2010. 2. National Quality Forum Measuring Performance. Available at: http://www.qualityforum.org/Measuring_Performance/Measuring_ Performance.aspx. Accessed June 4, 2010. 3. American Medical Association Physician Consortium for Performance Improvement (PCPI). Available at: http://www.ama-assn. org/ama/pub/physician-resources/clinical-practice-improvement/ clinical-quality/physician-consortium-performance-improvement. shtml. Accessed June 4, 2010. 4. Rex DK, Johnson DA, Anderson JC, et al. American College of Gastroenterology guidelines for colorectal cancer screening 2009. Am J Gastroenterol 2009;104:739 –750. 5. Levin B, Lieberman DA, McFarland B, et al. American Cancer Society Colorectal Cancer Advisory Group; US Multi-Society Task Force; American College of Radiology Colon Cancer Committee. Screening and surveillance for the early detection of colorectal cancer and adenomatous polyps, 2008: a joint guideline from the American Cancer Society, the US Multi-Society Task Force on Colorectal Cancer, and the American College of Radiology. Gastroenterology 2008;134:1570 –1595. 6. Klabunde CN, Lanier D, Nadel MR, et al. Colorectal cancer screening by primary care physicians: recommendations and practices, 2006 –2007. Am J Prev Med 2009;37:8 –16. 7. Chen SC, Rex DK. Endoscopist can be more powerful than age and male gender in predicting adenoma detection at colonoscopy. Am J Gastroenterol 2007;102:856 – 861.
QUALITY INDICATORS FOR COLONOSCOPY
71
8. Barclay RL, Vicari JJ, Doughty AS, et al. Colonoscopic withdrawal times and adenoma detection during screening colonoscopy. N Engl J Med 2006;355:2533–2541. 9. Cotton PB, Connor P, McGee D, et al. Colonoscopy: practice variation among 69 hospital-based endoscopists. Gastrointest Endosc 2003;57:352–357. 10. Rex DK, Petrini JL, Baron TH, et al. ASGE/ACG Taskforce on Quality in Endoscopy. Quality indicators for colonoscopy. Am J Gastroenterol 2006;101:873– 885. 11. American Gastroenterological Society. Finally . . . endoscopy quality measures become a reality. Available at: https:// secureweb.gastro.org/wmspage.cfm?parm1⫽6301. Accessed June 7, 2010. 12. Robles SC, Marrett LD, Clarke EA, et al. An application of capture recapture methods to the estimation of completeness of cancer registration. J Clin Epidemiol 1988;41:495–501. 13. Baxter NN, Goldwasser MA, Paszat LF, et al. Association of colonoscopy and death from colorectal cancer. Ann Intern Med 2009;150:1– 8. 14. Bressler B, Paszat LF, Chen Z, et al. Rates of new or missed colorectal cancers after colonoscopy and their risk factors: a population-based analysis. Gastroenterology 2007;132:96 – 102. 15. Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chron Dis 1987;40:373–383. 16. Rabeneck L, Paszat LF, Saskin R. Endoscopist specialty is associated with incident colorectal cancer after a negative colonoscopy. Clin Gastroenterol Hepatol 2010;8:275–279. 17. Cooper GS, Xu F, Koroukian S, et al. Prevalence and predictors of new or missed colorectal cancer: a population-based US study. Gastroenterology 2010;138(Suppl 1):S24. 18. Harewood GC. Relationship of colonoscopy completion rates and endoscopist features. Dig Dis Sci 2005;50:47–51. 19. Imperiale TF, Glowinski EA, Juliar BE, et al. Variation in polyp detection rates at screening colonoscopy. Gastrointest Endosc 2009;69:1288 –1295. 20. Dafnis G, Granath F, Påhlman L, et al. The impact of endoscopists’ experience and learning curves and interendoscopist variation on colonoscopy completion rates. Endoscopy 2001;33: 511–517. 21. Wexner SD, Garbus JE, Singh JJ. SAGES Colonoscopy Study Outcomes Group. A prospective analysis of 13,580 colonoscopies. Reevaluation of credentialing guidelines. Surg Endosc 2001;15:251–261. 22. Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika 1986;73:13–22. 23. Williams BA, Mandrekar JN, Mandrekar SJ, et al. Finding optimal cutpoints for continuous covariates with binary and time-to-event outcomes. Department of Health Sciences Research, Mayo Clinic, Minnesota. Technical Report Series, 2006. 24. Chen SC, Rex DK. Endoscopist can be more powerful than age and male gender in predicting adenoma detection at colonoscopy. Am J Gastroenterol 2007;102:856 – 861. 25. Lieberman D. Colon cancer screening and surveillance controversies. Curr Opin Gastroenterol 2009;25:422– 427. 26. Macken E, Moreels T, Pelckmans P, et al. Flemish Society of Gastroenterology VVGE. Quality assurance and recommendations for quality assessment of screening colonoscopy in Belgium. Acta Gastroenterol Belg 2009;72:17–25. 27. Bourke MJ. Making every colonoscopy count: ensuring quality in endoscopy. J Gastroenterol Hepatol 2009;24(Suppl 3):S43– S50. 28. Rabeneck L, Rumble RB, Axler J, et al. Cancer Care Ontario’s Colonoscopy Standards Expert Panel. Cancer Care Ontario Colonoscopy Standards: standards and evidentiary base. Can J Gastroenterol 2007;21(Suppl D):5D–24D.
CLINICAL– ALIMENTARY TRACT
January 2011
72
BAXTER ET AL
CLINICAL– ALIMENTARY TRACT
29. NHS Bowel Cancer Screening Programme. Quality assurance guidelines for colonoscopy. http://www.cancerscreening.nhs. uk/bowel/publications/nhsbcsp06.pdf. Accessed June 10, 2010. 30. Lieberman D, Nadel M, Smith RA, et al. Standardized colonoscopy reporting and data system: report of the Quality Assurance Task Group of the National Colorectal Cancer Roundtable. Gastrointest Endosc 2007;65:757–766. 31. Kaminski MF, Regula J, Kraszewska E, et al. Quality indicators for colonoscopy and the risk of interval cancer. N Engl J Med 2010; 362:1795–1803. 32. Rex DK, Rahmani EY, Haseman JH, et al. Relative sensitivity of colonoscopy and barium enema for detection of colorectal cancer in clinical practice. Gastroenterology 1997;112:17–23. 33. Bressler B, Paszat LF, Chen Z, et al. Rates of new or missed colorectal cancers after colonoscopy and their risk factors: a population-based analysis. Gastroenterology 2007;132:96 –102. 34. Williams JI, Young W. Appendix: a summary of studies on the quality of health care administrative databases in Canada. In: Goel V, Williams JI, Anderson GM, et al, eds. Patterns of health care in Ontario. The ICES practice atlas. 2nd ed. Ottawa: Canadian Medical Association, 1996:339 –345. 35. Hilsden RJ, Rostom A, Dube C, et al. Is polyp detection rate a valid proxy for adenoma detection rate for measuring the technical quality of colonoscopy? Gastroenterology 2010;138:S57. 36. Williams JE, Le TD, Faigel DO. Polypectomy rate as a surrogate marker for adenoma detection in quality measurement for colonoscopy. Gastrointest Endosc 2010;71:AB146.
GASTROENTEROLOGY Vol. 140, No. 1
Received June 26, 2010. Accepted September 9, 2010. Reprint requests Address requests for reprints to: Nancy N. Baxter, MD, PhD, Division of General Surgery, St Michael’s Hospital, University of Toronto, 30 Bond Street 16CC-40, Toronto, Ontario, M5B 1W8, Canada. e-mail:
[email protected]; fax: 416-360-0637. Conflicts of interest The authors disclose no conflicts. Funding of interest Supported by an American College of Gastroenterology Cancer Prevention Action Plan Grant, the Ontario Institute for Cancer Research, Cancer Care Ontario and by the Institute for Clinical Evaluative Sciences, which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care. N.N.B. holds the Cancer Care Ontario Health Services Research Chair and an Early Researchers Award from the Ontario Ministry of Research and Innovation. The funding sources played no role in design, conduct, or reporting of this study. The opinions, results, and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by the Institute for Clinical Evaluative Sciences or the Ontario Ministry of Health and Long-Term Care is intended or should be inferred.
