ORIGINAL ARTICLE
Systematic Review of Comorbidity Indices for Administrative Data Mansour T.A. Sharabiani, MD, MRes, PhD, Paul Aylin, FFPHM, and Alex Bottle, PhD
Background: Adjustment for comorbidities is common in performance benchmarking and risk prediction. Despite the exponential upsurge in the number of articles citing or comparing Charlson, Elixhauser, and their variants, no systematic review has been conducted on studies comparing comorbidity measures in use with administrative data. We present a systematic review of these multiple comparison studies and introduce a new meta-analytical approach to identify the best performing comorbidity measures/ indices for short-term (inpatient+r30 d) and long-term (outpatient+ > 30 d) mortality. Methods: Articles up to March 18, 2011 were searched based on our predefined terms. The bibliography of the chosen articles and the relevant reviews were also searched and reviewed. Multiple comparisons between comorbidity measures/indices were split into all possible pairs. We used the hypergeometric test and confidence intervals for proportions to identify the comparators with significantly superior/inferior performance for short-term and longterm mortality. In addition, useful information such as the influence of lookback periods was extracted and reported. Results: Out of 1312 retrieved articles, 54 articles were eligible. The Deyo variant of Charlson was the most commonly referred comparator followed by the Elixhauser measure. Compared with baseline variables such as age and sex, comorbidity adjustment From the Dr Foster Unit, Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, UK. Funded by the National Institute for Health Research Health Services Research Programme (project number 09/2001/32). The Dr Foster Unit at Imperial is principally funded by a research grant by Dr Foster Intelligence, an independent health care information company and joint venture with the Information Centre of the NHS. The Dr Foster Unit at Imperial is affiliated with the Imperial Centre for Patient Safety and Service Quality at Imperial College Healthcare NHS Trust, which is funded by the National Institute of Health Research. The Department of Primary Care & Public Health is grateful for support from the National Institute for Health Research Biomedical Research Centre Funding Scheme. The views and opinions expressed here are those of the authors and do not necessarily reflect those of the HSR Programme, NIHR, NHS or the Department of Health. The authors declare no conflict of interest. Reprints: Mansour T.A. Sharabiani, MD, MRes, PhD, Dr Foster Unit, Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, UK EC1A 9LA. E-mail: mansour.
[email protected]. Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Website, www.lww-medical care.com. Copyright r 2012 by Lippincott Williams & Wilkins ISSN: 0025-7079/12/000-000
Medical Care
Volume 00, Number 00, ’’ 2012
methods seem to better predict long-term than short-term mortality and Elixhauser seems to be the best predictor for this outcome. For short-term mortality, however, recalibration giving empirical weights seems more important than the choice of comorbidity measure. Conclusions: The performance of a given comorbidity measure depends on the patient group and outcome. In general, the Elixhauser index seems the best so far, particularly for mortality beyond 30 days, although several newer, more inclusive measures are promising. Key Words: administrative data, comorbidity, casemix, systematic review, mortality, meta-analysis (Med Care 2012;00: 000–000)
I
n risk prediction and risk adjustment modeling, comorbidity is frequently important. The increasing use and sophistication of large administrative databases has led to the construction of comorbidity indices. In predicting the risk of hospital death for 3 common operations, we have found that similar model discrimination can be attained whether routinely collected administrative data or clinical databases are used.1 However, a common and key limitation of the administrative databases is the lack of present-on-admission information, which meant that some index developers aimed to exclude many acute conditions that could be complications of care. Charlson2 and Elixhauser3 are the best known comorbidity indices, with the latter designed for databases lacking present-on-admission flags. There is only partial overlap in the set of comorbidities that these 2 cover, and many diseases are not covered by either. This has led to attempts to be more inclusive. They were designed or suggested for general purpose use, whereas many researchers want to adjust for sets of diseases that are of importance to a specific patient group. Further, the indices were originally used to predict 1-year mortality (Charlson) or length of stay, hospital charges, and in-hospital death (Elixhauser), whereas other outcomes may be of interest, perhaps needing a different formulation. The comorbidities included in Charlson index had been defined based on clinical data; however, others translated them into International Classification of Disease (ICD) codes that are used by administrative data4–7 and Read/OXMIS codes that are used by the General Practice Research Database.8 The few reviews of comorbidity indices in the literature9–13 mainly provide descriptive summaries based on www.lww-medicalcare.com |
1
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Sharabiani et al
limited numbers of indices, studies, and/or pertain only to specific patient groups. The exponential upsurge in the number of articles citing Charlson, Elixhauser, and their variants14 highlights the increasing interest in the use of comorbidity adjustment measures. Thus, there is a need for a comprehensive review of these measures and indices and a comparative analysis of their performances in administrative data. We conducted a systematic review of multiple comparison studies on comorbidity measures/indices in use with administrative data. We also present a new meta-analytical approach to identify the best-performing comorbidity measures/indices that can be used for adjusting short-term and long-term mortality risk after hospital admission. We summarize the studies by year of publication, country, database, patient group, outcome measures, other variables included in the models, and any special features such as the “lookback” period. We do not describe the absolute performance of any given index for any given patient group. We analyze the comparisons between indices (or different versions of the same index), discuss the various approaches that have been taken to develop an index and the difficulties in comparing them, and end with some suggestions for further work.
MATERIALS AND METHODS Search Strategy The literature search was conducted first using 3 electronic databases, Medline, EMBASE, and PubMed, up to March 18, 2011. Search terms included Charlson, Elixhauser, comorbidity, casemix, case-mix, mortality, and morbidity. In addition, the bibliography of the chosen articles and the relevant reviews were searched.9–13 Two authors independently reviewed the titles and abstracts and the methodology section of the articles and selected the relevant potential articles. All disagreements were resolved by consensus.
Inclusion and Exclusion Criteria Articles were included if administrative data were used and comparisons between predictive performances of at least 2 indices (perhaps also including common covariates) were made. Articles were excluded if clinical data without ICD codes were used or if indices were used for the purpose of adjustment only, without any comparison of indices. For details see Supplementary Methods (http://links.lww.com/ MLR/A314).
Data Abstraction Information relating to study design, methodology, results, and conclusion was extracted in a standardized way using a template designed for this study.
Comparative Analysis We aimed to identify the comorbidity measures/indices with the best predictive performance for short-term and longterm mortality from the above studies. We developed a metaanalytical approach to summarize these comparisons. As the first step, we considered the basic unit or building block of our approach to be a comparison between a pair of
2 | www.lww-medicalcare.com
Medical Care
Volume 00, Number 00, ’’ 2012
comorbidity measures/indices. Studies comparing 3 indices therefore contributed 3 pairs. To compare the predictive performances of the measures/indices, the c-statistics and the confidence intervals, G2, or Akaike Information Criterion, or other appropriate statistics were looked at if they were available. If c-statistics alone without confidence intervals (or P-values) were provided, an arbitrary Z0.02 cutoff point was chosen to define the difference between 2 measures/indices as significant, unless the authors of the study explicitly stated otherwise (only 1 article). To ensure whether the arbitrary cutoff point for c-statistics or authors’ judgment influenced our results, we performed sensitivity analyses based first on Z0.01 and second on Z0.03 cutoff points, also excluding the 1 article that used the authors’ judgment. Details of the sensitivity analysis and the comparative analysis of the most common comorbidity measures are described and presented in the Supplementary Methods and Results and Supplementary Table 1 (http://links.lww.com/MLR/A316) and Table 3 (http://links.lww.com/MLR/A318).
Paired Comparison of Comparators (PCC) We used the term “PCC” to refer to comparison of the predictive performances of any pair of comorbidity measures/indices. For any PCC, we take into account the arbitrary order or the direction of the comparison. For instance, if index X is compared with index Y, we refer to X as the “first comparator” and Y (the benchmark index) as the “second comparator.” Considering the order or direction of comparison enables us to classify PCCs into 3 categories: superior PCC (SPCC), inferior PCC (IPCC), and neutral PCC (NPCC); in addition, nonsuperior PCC (NSPCC) refers to both NPCC and IPCC categories together. Suppose that a study shows that Charlson predicts better than Elixhauser. In this case, if we consider Charlson as the ‘first comparator’ and the Elixhauser the ‘second comparator,’ then the Charlson versus Elixhauser PCC will be SPCC. Consequently, Elixhauser versus Charlson PCC will be IPCC. If a study shows that Charlson and Elixhauser perform equally, then both the Charlson versus Elixhauser and Elixhauser versus Charlson comparisons will be NPCC.
Statistical Analysis All mortality outcomes were divided into 2 major groups: (i) “short term”—all inpatient mortality and any mortality within 30 days of admission and (ii) “long term”— outpatient mortality or that later than 30 days of admission. Each set of outcomes contained a pool of paired comparisons (PCCs) between different comorbidity measures/indices including Charlson and its Ghali, Deyo, Romano, Dartmouth, D’Hoore, Quan, and Holman adaptations as well as Elixhauser and the van Walraven version of it. PCCs also included comparisons made for the transition from ICD-9 to ICD-10. Both sets of outcomes contained paired comparisons between “Baseline,” “Counts of diagnoses,” “Empirical weights,” and a separate cluster of comorbidity measures called “Other measures.” This last group included the indices that have been introduced and used only once to date or indices that partially met our criteria such as the Chronic r
2012 Lippincott Williams & Wilkins
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Medical Care
Volume 00, Number 00, ’’ 2012
Disease Score that uses pharmacy information; this is partially met, because only some administrative databases maintain the type of information required for these measures/ indices. Further examples of “Other measures” are APRDRGs, AHRQ’s Clinical Classification Software, and DxCG’s Diagnosis Cost Groups. For full details of this group, see Supplementary Results (http://links.lww.com/ MLR/A315) and Supplementary Table 2 (http://links. lww.com/MLR/A317). The term “empirical weights” indicates that the study authors did not change any comorbidities but derived new weights calibrated to their own dataset instead of using the original published weights. For each outcome, PCCs were clustered by their “first comparators.” To crudely estimate the relative superiority of any given comorbidity measure/index, we allocated a score for each cluster by subtracting the number of IPCCs from SPCCs and dividing the result by the total number of PCCs in the cluster. Subsequently, we scaled this score to between 0 and 1, where 0 represents worst and 1 represents best predictive performances. We call the score the Scaled Ranking Score (SRS), because we used it to rank the measures’/indices’ performance. For more detailed information on its calculation and also a hypothetical example of clustering PCCs, see the Supplementary Appendix Methods. We calculated the percentages of SPCCs, NPCCs, and IPCCs for each cluster. SRS crudely estimates the relative superiority of any given comorbidity measure. To examine which measures/indices perform significantly better or worse than the majority of the others, we used 2 statistical tests: the hypergeometric test (HPGT) and confidence intervals for proportions (CIP). In the results, we report only those measures/indices with at least an arbitrary threshold of 10 comparisons with other measures/indices.
HPGT We used HPGT to measure the probability of an observed number of SPCCs in a given cluster, taking into account the total number of PCCs in the cluster and the total number of SPCCs and PCCs in all clusters. Thus, a significant overrepresentation of SPCCs in a given measure/ index (cluster) would be interpreted as an overall better performance of that particular measure compared with the majority of the measures/indices.
CIP We calculated 95% CIP based on the binomial distribution for the SPCCs in each cluster and compared it with the overall proportion of SPCCs in all clusters to identify those clusters whose SPCC proportions are significantly different from the overall SPCC proportion.
RESULTS The Literature Selection As shown in Figure 1, after removing duplicates from different databases, we retrieved 1312 studies to review. Of them, 1195 studies were obtained from the electronic databases and 117 from the other sources described in the r
2012 Lippincott Williams & Wilkins
Systematic Review of Comorbidity Indices
Materials and methods section. After the review, 179 publications were screened on the bases of title and abstract. The screened articles underwent a full-text assessment of eligibility according to our predefined criteria. As result, 54 articles were found to be eligible for data abstraction. Criteria for excluding the articles during full-text assessment included: (a) the index could not be calculated based on administrative data, which was clear in the abstract (75%) and (b) the index was used only for adjustment purposes and no comparison between different indices was made (25%). Detailed characteristics of the selected studies are described in the Supplementary Results and Supplementary Table 2.
General Characteristics of the Studies Table 1 summarizes the general characteristics of 54 eligible articles included in the review. As expected, the number of comparative studies increased overall each year from 1995 till March 2011. The majority of the comparative studies were carried out in the United States (54%) and in Canada (35%). Studies covered all age groups from 15–16 years of age and above15,16 to 65–70 years of age and above.14,17–24 Thirty-one percent of studies were related to cardiovascular diseases including ischemic heart disease. Acute myocardial infarction alone accounted for 19% of studies. Cancer, renal and respiratory diseases, as well as orthopedic, conditions accounted for 15%, 13%, 11%, and 9% of studies, respectively. Twenty percent of studies included all hospitalizations. Medicare and Medicaid (Centers for Medicare and Medicaid Services) data were most frequently used. English (hospital episode statistics) and Canadian hospital discharge data were also used, as were registries such as for cancer, organ replacement, atrial fibrillation, and trauma and databases for the US Veteran Affairs, governmental17,22,25 and national insurance and social programmes. Seventy-four percent of the studies used the ICD-9, 9% used the ICD-10, and 15% both. A few studies carried out chart review, mainly for the purpose of validation. Comorbidity was most commonly adjusted for age, sex, race, socioeconomic status including the measures of income and education as well as insurance-related variables (Table 1) and less commonly for admission-related and hospital characteristics (rural, urban, teaching, or nonteaching). Some of the variables were related to the time of admission and a defined time period before the admission (baseline and/or “lookback” period), such as the number of physician visits for any reason during the baseline year or the number of hospitalizations for any reason and any length during the baseline year.17 Variables relating to patient group and/or disease or conditions have also been adjusted for.
Outcome Measures Inpatient mortality and 1-year mortality were the most common outcomes considered, by 41% and 35% of the studies, respectively (Table 1). Mortality over other lengths of time was also common including 30 days, 90 days,26 6 months,14,26–28 2 years,22,23,29–32 and between 2 and up to 10 years were also considered.22,33–36 Noncancer mortality www.lww-medicalcare.com |
3
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Medical Care
Sharabiani et al
Volume 00, Number 00, ’’ 2012
Literature search Databases: Medline: (690 records) EMBASE: (876 records) PubMed: (954 records) Limits: From 1st of January 1987 to 18th March 2011 Search terms: comorbidity or casemix or case-mix AND Identification
mortality or morbidity AND Charlson or Elixhauser
Records identified through database searching: 2520
Records identified through other sources: 117
Eligibility
Screening
Records after duplicates removed: 1312
Articles screened on basis of title and abstract: 1312
Records excluded: 1133
Full-text articles assessed for eligibility: 179
Full-text articles excluded: 125 (Out of scope)
Included
Included in qualitative synthesis: 54 Includes research articles comparing at least 2 comorbidity measures including at least versions of the same measure or one measure with different data feed.
FIGURE 1. Flow diagram of literature search results.
over 2 years was another form of outcome23,30,31 as was the mortality by person-year.37 Length of stay was considered as an outcome by 11% of the studies.38–43 Two studies had readmission as one of the outcomes27,39 and similar outcomes included hospitalizations during the follow-up year17 and hospitalization probability.44 For more details see Supplementary Results.
Comparators Most studies focused on a few comparators, that is, different versions of the Charlson index and the Elixhauser measure. This was expected, partially due to our search strategy and because the relevant studies are normally ex-
4 | www.lww-medicalcare.com
pected to test at least one of the well-known measures and pioneering works. The Deyo adaptation4 was the most commonly compared version of Charlson index (Table 1). Elixhauser’s set of comorbidities and its score15,42 made up another set of commonly compared comorbidity measures. Baseline characteristics and count of comorbidities/diagnoses were considered as independent comparators by 39% and 28% of studies, respectively. The insensitivity of the latter measure to potential miscoding of diagnoses in administrative/claims data was viewed as a potential advantage.38 Empirical weights were also commonly compared in the studies.17,20,23,25,26 For more details see Supplementary Results. r
2012 Lippincott Williams & Wilkins
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Medical Care
Volume 00, Number 00, ’’ 2012
Systematic Review of Comorbidity Indices
TABLE 1. Summary of General Characteristics of the Studies Study Features Year of publication
Country
Scale
Patient group
ICD version Database
Common covariates
Outcome
Comparator
Most Common Characteristics
Article (Counts)
References
Up to 1999 2000–2004 2005–2009 2010–March 2011 United States Canada Australia Other countries One/few hospitals National Regional Multinational All hospitalizations All CVD* AMI Stroke Cancer Renal diseasesz Respiratory Orthopedic Other conditions ICD-9 ICD-10 ICD-9 and 10 Medicare and Medicaid Canadian discharge HES/other dischargew Registries US Veteran Affair Other governmental Age Sex Race SESy Admission type Admission source8 Inpatient mortality One-year mortality 30-day mortality Length of stay Readmission Deyo Romano D’Hoore Dartmouth-Manitoba Ghali Elixhauser van Walraven Comorbidities count Baseline as comparator
6 15 24 9 28 18 5 7 7 22 21 4 11 21 10 5 8 7 6 5 11 40 5 8 12 6 6 5 4 7 48 42 18 13 6 3 22 19 8 6 4 25 11 5 4 3 15 4 15 21
D’Hoore and colleagues6,26,29,32,38,53,54 Schneeweiss and colleagues17–21,30,36,46,55–61 Moore and colleagues16,22–24,27,31,33,35,37,39–42,62–71 Gagne and colleagues14,15,25,28,34,43,44,72,73 Gagne and colleagues14,18–21,23,24,26–32,34,35,37,38,46,53–58,60,66,69 D’Hoore and colleagues6,15–17,21,22,33,41,42,44,57,61,62,64,67,68,70,71 Quan and colleagues25,36,39,65,67 Quan and colleagues25,40,43,63,67,72,73 van Walraven and colleagues15,18,32,42,54,60,71 Reid and colleagues 19,23,24,27–29,31,33–35,37,38,40,41,43,44,58,59,61,66,72,73 D’Hoore and colleagues6,14,16,17,20,22,26,30,36,39,46,53,55,56,62–65,68–70 Schneeweiss and colleagues21,25,57,67 Gagne and colleagues14,15,18,20,21,25,28,55,58,62,65; all surgical Atherly et al60 D’Hoore and colleagues6,17,21,22,26–29,36,37,39,43,46,53,54,57,59,63,64,68,69,72,73 D’Hoore and colleagues6,22,28,29,36,39,46,57,59,64,72 D’Hoore and colleagues6,26–28,70,73 Reid and colleagues19,23,30,31,39,40,56 Cleves and colleagues26,33–35,39,43,68 Cleves and colleagues26,39,46,63,72,73 Radley and colleagues24,26,38,43,66 D’Hoore and colleagues6,16,26,28,32,39,41,44,55,68,71 D’Hoore and colleagues6,14–24,26–32,34–39,41,46,53–61,66,69,71,72 Quan and colleagues25,40,43,67,73 van Walraven and colleagues42,44,62–65,68,70 Gagne and colleagues14,20,21,24,26,28,29,35,37,38,56,69 D’Hoore and colleagues6,44,61,62,67,68,70 Nuttall and colleagues40,43,46,54,64,67 Moore and colleagues16,33,37,56 Berlowitz and colleagues27,32,58,66 Schneeweiss and colleagues17,20–22,25,56,72 D’Hoore and colleagues6,14–27,29–41,43,44,46,53–60,63–65,68–73 D’Hoore and colleagues6,14–26,29–31,33–41,43,44,46,53–55,57,59,60,63,64,68–73 Reid and colleagues19,24,29,31,33–35,37–39,41,46,54–56,59,69,72 Reid and colleagues19,22,24,30,31,38,39,41,44,55,56,59,69 van Walraven and colleagues15,17,41,43,46,54 Moore and colleagues16,41,58 D’Hoore and colleagues6,15,16,25,40–43,46,53,54,58,61–65,67,68,70–73 Gagne and colleagues14,17,18,20–22,24,25,39,40,43,44,56,57,59,64,66,68,72 Gagne and colleagues14,25,26,38,57,60,64 Melfi and colleagues38–43 Schneeweiss and colleagues17,27,39,44 Desai and colleagues18,19,21,26,27,29,34,37–41,46,53–57,59,61,62,65,66,72 Gagne and colleagues14,16,17,20,21,24,26,37,60,69,72 D’Hoore and colleagues6,17,21,22,36 Schneeweiss and colleagues17,28,40,57 Schneeweiss and colleagues17,21,53 Li and colleagues28,35,41,42,44,46,58,61,62,66,68–70,72,73 Gagne and colleagues14,15,20,42 Moore and colleagues16–19,21,22,32,33,38,43,56,58,65,71 Moore and colleagues16,17,20,22,26,27,30,31,33,37,38,40,43,44,46,56,58,59,63,70,73
*All cardiovascular conditions including AMI, coronary artery bypass graft, aortic valve replacement, atrial fibrillation, and stroke. w Hospital episodes. z Noncancer renal diseases. y SES also including measures of income and education; insurance related: inpatient and physician claims, and the original reason for Medicare entitlement. 8 Direct admission or transfer. AMI indicates acute myocardial infarction; AVR, aortic valve replacement; CABG, coronary artery bypass graft; CVD, cardiovascular diseases; HES, Hospital Episode Statistics; ICD, International Classification of Disease; SES, socioeconomic status.
Comparative Analysis For the rest of this article, we refer to inpatient mortality or mortality within 30 days of admission as short-term mortality and to mortality later than 30 days as long-term mortality. The comparisons for short-term and long-term mortality have been summarized in Supplementary Table 1, using a particular visualization method that we have inr
2012 Lippincott Williams & Wilkins
troduced and described in Supplementary Method (Comparison matrix). Tables 2 and 3 summarize the results of the comparative analysis for mortality. For short-term mortality, there were 13 clusters of comorbidity measures and 366 PCCs including 100 SPCCs. For long-term mortality, there were 15 clusters of comorbidity measures and 432 PCCs www.lww-medicalcare.com |
5
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Medical Care
Sharabiani et al
Volume 00, Number 00, ’’ 2012
TABLE 2. Hypergeometric Analysis Results of PCCs for Short-term Mortality Counts of PCCs Comparators Baseline Deyo Quanw Elixhauser Romano Counts of Diag. ICD-9 to ICD-10 Empirical weights Other measures
% of PCCs
Overall Proportion: 0.273
Rank*
SPCC
NPCC
IPCC
Total PCCs
SPCC
NPCC
IPCC
Hypergeometric Test (P)
0.20 0.40 0.46 0.49 0.50 0.55 0.59 0.67
0 13 2 13 7 2 2 9
22 26 17 28 24 7 9 10
33 26 4 14 7 1 0 2
55 65 23 55 38 10 11 21
0 0.20 0.09 0.24 0.18 0.20 0.18 0.43
0.40 0.40 0.74 0.51 0.63 0.70 0.82 0.48
0.60 0.40 0.17 0.25 0.18 0.10 0.00 0.10
< 0.001 0.043 0.021 0.108 0.069 0.264 0.233 0.052
0 0.200 0.087 0.236 0.184 0.200 0.182 0.429
0.78
43
19
5
67
0.64
0.28
0.07
< 0.001
0.642 (0.522, 0.746)
Comparators’ 95% CIP (0, 0.065) (0.121, 0.313) (0.024, 0.268) (0.144, 0.364) (0.092, 0.334) (0.057, 0.510) (0.051, 0.477) (0.245, 0.635)
Inpatient mortality ± mortality within 30 days (short-term mortality). Only clusters with a minimum of 10 comparisons are shown. *Rank of relative performance of comparators based SRS described in the Materials and methods section and in Supplementary Method. w ICD-10 version of Charlson index. CIP indicates confidence interval for proportions; ICD, International Classification of Disease; IPCC, inferior PCC; NPCC, neutral PCC; PCC, paired comparison of comparators; SPCC, superior PCC; SRS, Scaled Ranking Score.
including 160 SPCCs. The performance of the measures improves (although not necessarily significantly) in ascending order in both Tables 2 and 3 according to our SRS described in the Materials and methods section and Supplementary Methods. For both outcomes, only including the baseline variables performed significantly less well than including any of the comorbidity measures. Romano adaptation of Charlson index had better predictive performance than the majority of the indices for longterm mortality: SPCC proportion 0.51 (0.38, 0.63) versus 0.37 overall (59 comparisons), P = 0.008 for HPGT. Deyo adaptation of Charlson index shows a lower capability than the majority of the measures/indices in predicting short-term mortality that is significant by HPGT (P = 0.043), but for long-term mortality it performs similarly to the majority. This is similar to the predictive performance of Quan index (ICD-10 adaptation of Charlson index) in short-term and long-term mortality. The predictive performance difference
between short-term and long-term mortality is more noticeable for the Elixhauser method (Tables 2, 3).
DISCUSSION In this article, we aimed to summarize studies comparing two or more comorbidity measures used in administrative data. Fifty-four articles that met our criteria and were included in our analysis provided 366 and 432 comparisons of the performances of comorbidity measures in predicting short-term and long-term mortality, respectively. Many studies analyzed all admissions combined, although various patient groups were also of interest. It must be emphasized that, based on our analysis, we cannot report on absolute prediction for short-term and longterm mortality. However, if we assume that the predictive performance of baseline variables such as age and sex remains similar for both short-term and long-term mortality,
TABLE 3. Hypergeometric Analysis Results of PCCs for Long-term Mortality Counts of PCCs Comparators Baseline Charlson (original) Ghali D’Hoore Quanw Deyo Count of diagnoses Empirical weights Other measures Romano Elixhauser
% of PCCs
Overall Proportion: 0.37
Rank*
SPCC
NPCC
IPCC
Total PCCs
SPCC
NPCC
IPCC
Hypergeometric Test (P)
0 0.15 0.44 0.47 0.50 0.53 0.54 0.56 0.68 0.69 0.77
0 2 6 5 3 28 11 4 36 30 25
0 1 3 4 9 24 6 11 12 21 15
69 14 8 6 3 24 9 2 14 8 2
69 17 17 15 15 76 26 17 62 59 42
0 0.12 0.35 0.33 0.20 0.37 0.42 0.24 0.58 0.51 0.60
0.00 0.06 0.18 0.27 0.60 0.32 0.23 0.65 0.19 0.36 0.36
1.00 0.82 0.47 0.40 0.20 0.32 0.35 0.12 0.23 0.14 0.05
< 0.001 0.017 0.201 0.208 0.088 0.104 0.138 0.108 < 0.001 0.008 0.001
Comparators’ 95% CIPs 0 0.118 0.353 0.333 0.200 0.368 0.423 0.235 0.581 0.509 0.595
(0, 0.053) (0.033, 0.343) (0.173, 0.587) (0.152, 0.583) (0.071, 0.452) (0.269, 0.481) (0.255, 0.611) (0.096, 0.473) (0.457, 0.695) (0.384, 0.632) (0.445, 0.730)
Outpatient mortality after >30 days (long-term mortality). Only clusters with a minimum of 10 comparisons are shown. *Rank of relative performance of comparators based SRS described in the Materials and methods section and in Supplementary Method. w ICD-10 version of Charlson index. CIP indicates confidence interval for proportions; ICD, International Classification of Disease; IPCC, inferior PCC; NPCC, neutral PCC; PCC, paired comparison of comparators; SPCC, superior PCC; SRS, Scaled Ranking Score.
6 | www.lww-medicalcare.com
r
2012 Lippincott Williams & Wilkins
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Medical Care
Volume 00, Number 00, ’’ 2012
then it would seem that the majority of those comorbidity measures/indices are better able to predict long-term mortality than short-term, regardless of patient group, at least compared with the baseline variables. This pattern can be inferred by comparing short-term and long-term predictive performances of baseline variables, Romano, Deyo, Quan, and Elixhauser methods. A potential explanation is coding bias, because comorbidities are usually underreported in patients with serious acute conditions.3 This is because coding guidelines typically require the recording of only those comorbidities considered relevant to the admission. In England, however, a list of 61 conditions that should always be recorded was introduced in April 2010, which covers many of the Charlson items. For short-term mortality, the use of empirical weights (of whichever index) or various other comorbidity measures performed best. For long-term mortality, the “Other measures,” the Romano version of Charlson and the Elixhauser measure performed significantly better. Assuming that patients who die in the short term as opposed to the long term are more likely to be seriously ill at the time of admission, coding bias may be greater for these patients because, as mentioned earlier, their comorbidities that are less relevant to the main diagnosis are less likely to be recorded. As Elixhauser requires more comorbidity information than Charlson, the gap in recording the comorbidity data of patients who die in the short term (seriously ill patients) and those who die in the long term might be larger. In addition, probably in the short term, mortality is likely to be affected more by the severity of the disease and its complications, whereas in the long term, the patients’ overall health (reflected by their comorbidities) may play an increasing role.
Comorbidity Measure or Index Various sets of baseline variables (eg, age and sex) had significantly less ability to predict both short-term and longterm mortality than comorbidity, regardless of the patient groups and/or type of comorbidity measure/index.
“Other Measures” and Other Measures/Indices Overall, this cluster significantly outperformed other comparators in prediction of mortality both in the short and in long term. However, this cluster does not represent a particular comorbidity adjustment measure or index—for full details, including the information on paired comparisons of this group, see Supplementary Table 2. The highly significant superiority of this cluster in predicting short-term and long-term mortality could be partly due to publication bias—there would be little point in publishing a new index that could not outperform an existing index (this cluster includes newly introduced measures that have not been used in any other study). Similarly, publication bias may also affect the “baseline cluster” in the opposite direction, because it will not be desirable to publish a new index that cannot significantly outperform baseline variables. A second potential reason for the superior performance of “Other measures” is that some members of this group, such as Diagnosis-related Groups, include information on r
2012 Lippincott Williams & Wilkins
Systematic Review of Comorbidity Indices
complications of care. The presence of such complications is often associated with poorer outcomes and higher costs. In contrast, a stated goal of Elixhauser and colleagues was to exclude these. If the goal of the modeling is risk-adjusting hospital outcomes for benchmarking, then one should not adjust for complications, as they are at least partly within the control of the hospital. Third, some members include information such as disease severity or from pharmacies that is not always available from administrative databases, and this extra information may improve model performance. “Charlson-original” is a specific cluster in the sense that it normally includes all the studies that have used Charlson index as one of the comparators without specifying any particular adaptation or variant of it. This cluster includes studies in which the application of the Charlson index was compatible with administrative data (eg, Medicare). Charlson-original predicted long-term mortality significantly less well than did the majority of measures/indices. The Ghali and D’Hoore adaptations were examined for long-term mortality only, and they performed slightly better than Charlson-original. Simple counts of comorbidities and empirical weights performed more or less the same in predicting short-term and long-term mortality, although both ranked higher than most of the comparators based on our SRS scores. It must be noted that many studies that have used Romano have actually used empirical weights as recommended by Romano’s paper. Therefore, the higher performance of Romano in predicting long-term mortality may partially reflect the impact of using empirical weights in general.
Information Beyond the Analysis Apart from the inherent capability of comorbidity measures/indices, a few other factors such as adding/prolonging the “lookback period” and linking data sources may also improve prediction (see Supplementary Results for more detail). This information, which is not included in the analysis nor is necessarily limited to the chosen articles, also includes observations related to outcomes other than mortality and comparisons of surgical with nonsurgical admissions. This information will be useful when using or developing a new comorbidity measure.
Lookback Period Zhang et al45 showed improved 1-year mortality prediction for Charlson by using a 1-year inpatient lookback period, 1-year auxiliary claims lookback, and the second year inpatient lookback periods into the regression model. Similar improvements for both Charlson and Elixhauser were shown by adding inpatient and outpatient claims in the 12 months before the index hospitalization28 or by separately adding prior hospitalization(s).46 Preen et al47 showed that the explanatory power of the models was influenced by both the length of lookback periods and the chosen outcome. A 1-year lookback period fit best for 1-year mortality, and a longer lookback period fit best for readmission.47 For more details see Supplementary Results. www.lww-medicalcare.com |
7
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Sharabiani et al
Approaches to Selecting Comorbidities Normand et al29 initially grouped secondary diagnoses based on a minimum prevalence of 0.01% and seeking expert opinion (cardiologists) to divide comorbidities into 2 sets: one set reflecting the clinical status on admission and the other set that could be used to assess interventions that were likely to be related to the interventions. Desai et al18 initially selected high-risk medical conditions by reviewing the medical literature, seeking expert opinion, and, according to the preliminary analysis of their data, considering those conditions that had a prevalence of Z2% and had either an unadjusted rate ratio Z1.5 or an attributable risk Z5%. Fleming et al48 developed a comorbidity model according to the prevalence of disease and comparison with Charlson index as well as “clinical insight and the desire to include chronic rather than acute conditions” over 6 rounds of refinements. To predict the future health care costs, Ash and colleagues grouped 15,000 ICD-9-CM codes into 543 categories, calling them DxGROUPs. Subsequently, they clustered DxGROUPs into 118 diagnosis-based conditional categories that consisted of medically related problems with the same level of expected costs after consulting a panel of experts.49 Holman et al39 chose a given comorbidity if it either was one of the top 100 most frequent comorbid conditions (excluding explicit codes for complications) or appeared in the comorbidity lists of Elixhauser and/or Deyo adaptation of the Charlson index. Gagne et al and Thombs et al simply combined all the conditions listed in Elixhauser and the Deyo adaptation of the Charlson index14,50 except those conditions thought to be related to the main diagnosis.50
Limitations The amount of information that we could extract from the literature was not sufficient to enable us to determine the most appropriate method (index) for every outcome and every patient group. In addition, publication bias to some extent might have influenced our results, as the number of the studies that purely compared various comorbidity adjustment methods makes up a small fraction of the studies in the review and analysis. However, it must be said that most of the studies compared their methods with >1 preexisting method. Therefore, publication bias could be of less concern for the comparisons between the preexisting methods. In the analysis, we considered the clusters to be independent. If cluster 1 is superior to cluster 2, it means that the ratio of superior PCCs to nonsuperior PCCs in cluster 1 is bigger than this ratio in cluster 2. This does not mean that, for example, each IPCC that corresponds to an SPCC in cluster 1 necessarily appears in cluster 2. Indeed, we have assumed that corresponding IPCCs, SPCCs, and NPCCs from a given cluster are randomly scattered across the other clusters, although in practice this may not always be the case. However, it is also worth mentioning that “nonsignificant” measures/indices in our results should not be interpreted as “no value” methods. A large group of nonsignificant measures means that the measures in the group perform more or less at the same level and that all, for instance, perform better than baseline variables.
8 | www.lww-medicalcare.com
Medical Care
Volume 00, Number 00, ’’ 2012
For studies that did not test the statistical significance of differences in c-statistics, we chose an arbitrary cutoff of 0.02 as representing an important difference. The value of an improvement of 0.02 or more will depend on the purpose of the analysis. However, sensitivity analysis showed that moving the cutoff point to either Z0.01 or Z0.03 did not influence the results. Finally, despite an arbitrary minimum threshold of 10 PCCs per each cluster to aid robust inference, it is still possible that our general results, to some extent, have been driven by unbalanced numbers of patients, patient groups, and studies for different measures/indices. We did not take sample size into account as would be the case with a conventional meta-analysis.
Further Work and Conclusions Comorbidity adjustment, regardless of the type of measure/index, certainly provides added prognostic value to a model above baseline variables such as age and sex. At least partly for potential reasons such as coding bias and guidelines, overall comorbidity adjustment methods showed better prediction for long-term than for short-term mortality. Elixhauser seems to be the superior method of predicting long-term mortality followed by the Romano adaptation of Charlson index. However, for short-term mortality, using empirical weights of whatever index seems to perform better. In the future, it would be intriguing to see whether the prevalence of comorbidities related and unrelated to the main diagnosis differs in strata of patients with short-term and long-term mortality. This would assist the quantification of coding biases mentioned above that are incurred due to uncertainty over whether the comorbidity is related to the patient’s main problem. Application of more sophisticated methods and models such as multilevel modeling is gaining popularity in comorbidity adjustment modeling. Because of the conceivable existence of multiple interactions and nonlinear relationships between diagnoses, comorbidities, treatment methods, patient characteristics, and factors such as coding bias, more advanced methods might be better. For instance, compared with logistic regression, Bayesian Additive Regression Trees have been shown to significantly improve prediction based on the same set of predictors in a complex network of cytokines.51 However, the use of computationally intensive Bayesian methods with administrative data that normally contain millions of records may not be practical. In contrast, nonparametric learning methods including support vector machines in comorbidity adjustment seem to be promising.52 REFERENCES 1. Aylin P, Bottle A, Majeed A. Use of administrative data or clinical databases as predictors of risk of death in hospital: comparison of models. BMJ. 2007;334:1044. 2. Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–383. 3. Elixhauser A, Steiner C, Harris DR, et al. Comorbidity measures for use with administrative data. Med Care. 1998;36:8–27. 4. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45:613–619. r
2012 Lippincott Williams & Wilkins
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Medical Care
Volume 00, Number 00, ’’ 2012
5. D’Hoore W, Bouckaert A, Tilquin C. Practical considerations on the use of the Charlson comorbidity index with administrative data bases. J Clin Epidemiol. 1996;49:1429–1433. 6. D’Hoore W, Sicotte C, Tilquin C. Risk adjustment in outcome assessment: the Charlson comorbidity index. Methods Inf Med. 1993;32: 382–387. 7. Romano PS, Roos LL, Jollis JG. Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives. J Clin Epidemiol. 1993;46:1075–1079; discussion 1081–1090.. 8. Khan NF, Perera R, Harper S, et al. Adaptation and validation of the Charlson Index for Read/OXMIS coded databases. BMC Fam Pract. 2010;11:1. 9. de Groot V, Beckerman H, Lankhorst GJ, et al. How to measure comorbidity: a critical review of available methods. J Clin Epidemiol. 2003;56:221–229. 10. Extermann M. Measuring comorbidity in older cancer patients. Eur J Cancer. 2000;36:453–471. 11. Hall WH, Jani AB, Ryu JK, et al. The impact of age and comorbidity on survival outcomes and treatment patterns in prostate cancer. Prostate Cancer Prostatic Dis. 2005;8:22–30. 12. Leal JR, Laupland KB. Validity of ascertainment of co-morbid illness using administrative databases: a systematic review. Clin Microbiol Infect. 2010;16:715–721. 13. Needham DM, Scales DC, Laupacis A, et al. A systematic review of the Charlson comorbidity index using Canadian administrative databases: a perspective on risk adjustment in critical care research. J Crit Care. 2005;20:12–19. 14. Gagne JJ, Glynn RJ, Avorn J, et al. A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol. 2011;64:749–759. 15. van Walraven C, Escobar GJ, Greene JD, et al. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2010;63:798–803. 16. Moore L, Lavoie A, Sage NL, et al. Using information on preexisting conditions to predict mortality from traumatic injury. Ann Emerg Med. 2008;52:356e2–364e2. 17. Schneeweiss S, Seeger JD, Maclure M, et al. Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data. Am J Epidemiol. 2001;154:854–864. 18. Desai MM, Bogardus ST Jr, Williams CS, et al. Development and validation of a risk-adjustment index for older patients: the high-risk diagnoses for the elderly scale. J Am Geriatr Soc. 2002;50:474–481. 19. Reid BC, Alberg AJ, Klassen AC, et al. A comparison of three comorbidity indexes in a head and neck cancer population. Oral Oncol. 2002;38:187–194. 20. Schneeweiss S, Wang PS, Avorn J, et al. Improved comorbidity adjustment for predicting mortality in Medicare populations. Health Serv Res. 2003;38:1103–1120. 21. Schneeweiss S, Wang PS, Avorn J, et al. Consistency of performance ranking of comorbidity adjustment scores in Canadian and U.S. utilization data. J Gen Intern Med. 2004;19(pt 1):444–450. 22. Grunau GL, Sheps S, Goldner EM, et al. Specific comorbidity risk adjustment was a better predictor of 5-year acute myocardial infarction mortality than general methods. J Clin Epidemiol. 2006;59:274–280. 23. Klabunde CN, Legler JM, Warren JL, et al. A refined comorbidity measurement algorithm for claims-based studies of breast, prostate, colorectal, and lung cancer patients. Ann Epidemiol. 2007;17:584–590. 24. Radley DC, Gottlieb DJ, Fisher ES, et al. Comorbidity risk-adjustment strategies are comparable among persons with hip fracture. J Clin Epidemiol. 2008;61:580–587. 25. Quan H, Li B, Couris CM, et al. Updating and validating the charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol. 2011;173: 676–682. 26. Cleves MA, Sanchez N, Draheim M. Evaluation of two competing methods for calculating Charlson’s comorbidity index when analyzing short-term mortality using administrative data. J Clin Epidemiol. 1997;50:903–908. 27. Berlowitz DR, Hoenig H, Cowper DC, et al. Impact of comorbidities on stroke rehabilitation outcomes: does the method matter? Arch Phys Med Rehabil. 2008;89:1903–1906.
r
2012 Lippincott Williams & Wilkins
Systematic Review of Comorbidity Indices
28. Li P, Kim MM, Doshi JA. Comparison of the performance of the CMS Hierarchical Condition Category (CMS-HCC) risk adjuster with the Charlson and Elixhauser comorbidity measures in predicting mortality. BMC Health Serv Res. 2010;10:245. 29. Normand SL, Morris CN, Fung KS, et al. Development and validation of a claims based index for adjusting for risk of mortality: the case of acute myocardial infarction. J Clin Epidemiol. 1995;48:229–243. 30. Klabunde CN, Potosky AL, Legler JM, et al. Development of a comorbidity index using physician claims data. J Clin Epidemiol. 2000;53:1258–1267. 31. Baldwin L-M, Klabunde CN, Green P, et al. In search of the perfect comorbidity measure for use with administrative claims data: does it exist? Med Care. 2006;44:745–753. 32. Rochon PA, Katz JN, Morrow LA, et al. Comorbid illness is associated with survival and length of hospital stay in patients with chronic disability. A prospective comparison of three comorbidity indices. Med Care. 1996;34:1093–1101. 33. Jassal SV, Schaubel DE, Fenton SSA. Baseline comorbidity in kidney transplant recipients: a comparison of comorbidity indices. Am J Kidney Dis. 2005;46:136–142. 34. Liu J, Huang Z, Gilbertson DT, et al. An improved comorbidity index for outcome analyses among dialysis patients. Kidney Int. 2010;77: 141–151. 35. Machnicki G, Pinsky B, Takemoto S, et al. Predictive ability of pretransplant comorbidities to predict long-term graft loss and death. Am J Transplant. 2009;9:494–505. 36. O’Connell RL, Lim LL. Utility of the Charlson comorbidity index computed from routinely collected hospital discharge diagnosis codes. Methods Inf Med. 2000;39:7–11. 37. Yan Y, Birman-Deych E, Radford MJ, et al. Comorbidity indices to predict mortality from Medicare data: results from the national registry of atrial fibrillation. Med Care. 2005;43:1073–1077. 38. Melfi C, Holleman E, Arthur D, et al. Selecting a patient characteristics index for the prediction of medical outcomes using administrative claims data. J Clin Epidemiol. 1995;48:917–926. 39. Holman CDAJ, Preen DB, Baynham NJ, et al. A multipurpose comorbidity scoring system performed better than the Charlson index. J Clin Epidemiol. 2005;58:1006–1014. 40. Nuttall M, van der Meulen J, Emberton M. Charlson scores based on ICD-10 administrative data were valid in assessing comorbidity in patients undergoing urological cancer surgery. J Clin Epidemiol. 2006;59:265–273. 41. Myers RP, Quan H, Hubbard JN, et al. Predicting in-hospital mortality in patients with cirrhosis: results differ across risk adjustment methods. Hepatology. 2009;49:568–577. 42. van Walraven C, Austin PC, Jennings A, et al. A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Med Care. 2009;47:626–633. 43. Armitage JN, van der Meulen JH, Royal G, College of Surgeons Comorbidity Consensus. Identifying co-morbidity in surgical patients using administrative data with the Royal College of Surgeons Charlson Score. Br J Surg. 2010;97:772–781. 44. Lix LM, Quail J, Teare G, et al. Performance of comorbidity measures for predicting outcomes in population-based osteoporosis cohorts. Osteoporosis Int. 2011;22:2633–2643. 45. Zhang JX, Iwashyna TJ, Christakis NA. The performance of different lookback periods and sources of information for Charlson comorbidity adjustment in Medicare claims. Med Care. 1999;37:1128–1139. 46. Stukenborg GJ, Wagner DP, Connors AF Jr. Comparison of the performance of two comorbidity measures, with and without information from prior hospitalizations. Med Care. 2001;39:727–739. 47. Preen DB, Holman CD, Spilsbury K, et al. Length of comorbidity lookback period affected regression model performance of administrative health data. J Clin Epidemiol. 2006;59:940–946. 48. Fleming ST, Pearce KA, McDavid K, et al. The development and validation of a comorbidity index for prostate cancer among Black men. J Clin Epidemiol. 2003;56:1064–1075. 49. Ash AS, Ellis RP, Pope GC, et al. Using diagnoses to describe populations and predict costs. Health Care Financ Rev. 2000;21:7–28. 50. Thombs BD, Singh VA, Halonen J, et al. The effects of preexisting medical comorbidities on mortality and length of hospital stay in acute
www.lww-medicalcare.com |
9
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Sharabiani et al
51. 52. 53. 54. 55. 56. 57. 58. 59. 60. 61.
burn injury: evidence from a national sample of 31,338 adult patients. Ann Surg. 2007;245:629–634. Sharabiani MTA, Vermeulen R, Scoccianti C, et al. Immunologic profile of excessive body weight. Biomarkers. 2011;16:243–251. Syed Z, Rubinfeld I, Patton JH, et al. Using procedural codes to supplement risk adjustment: a nonparametric learning approach. J Am Coll Surg. 2011;212:1086–U198. Ghali WA, Hall RE, Rosen AK, et al. Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data. J Clin Epidemiol. 1996;49:273–278. Polanczyk CA, Rohde LE, Philbin EA, et al. A new casemix adjustment index for hospital mortality among patients with congestive heart failure. Med Care. 1998;36:1489–1499. Ward MM. Development and testing of a systemic lupus-specific risk adjustment index for in-hospital mortality. J Rheumatol. 2000;27: 1408–1413. Wang PS, Walker A, Tsuang M, et al. Strategies for improving comorbidity measures based on Medicare and Medicaid claims data. J Clin Epidemiol. 2000;53:571–578. Tu JV, Austin PC, Walld R, et al. Development and validation of the Ontario acute myocardial infarction mortality prediction rules. J Am Coll Cardiol. 2001;37:992–997. Johnston JA, Wagner DP, Timmons S, et al. Impact of different measures of comorbid disease on predicted mortality of intensive care unit patients. Med Care. 2002;40:929–940. Ash AS, Posner MA, Speckman J, et al. Using claims data to examine mortality trends following hospitalization for heart attack in Medicare. Health Serv Res. 2003;38:1253–1262. Atherly A, Fink AS, Campbell DC, et al. Evaluating alternative riskadjustment strategies for surgery. Am J Surg. 2004;188:566–570. Southern DA, Quan H, Ghali WA. Comparison of the Elixhauser and Charlson/Deyo methods of comorbidity measurement in administrative data. Med Care. 2004;42:355–360.
10 | www.lww-medicalcare.com
Medical Care
Volume 00, Number 00, ’’ 2012
62. Quan H, Fink AS, Campbell DC, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43:1130–1139. 63. Martins M, Blais R. Evaluation of comorbidity indices for inpatient mortality prediction models. J Clin Epidemiol. 2006;59:665–669. 64. So L, Evans D, Quan H. ICD-10 coding algorithms for defining comorbidities of acute myocardial infarction. BMC Health Serv Res. 2006;6:161. 65. Ho KM, Finn J, Knuiman M, et al. Combining multiple comorbidities with Acute Physiology Score to predict hospital mortality of critically ill patients: a linked data cohort study. Anaesthesia. 2007;62:1095–1100. 66. Kurichi JE, Stineman MG, Kwong PL, et al. Assessing and using comorbidity measures in elderly veterans with lower extremity amputations. Gerontology. 2007;53:255–259. 67. Sundararajan V, Quan H, Halfon P, et al. Cross-national comparative performance of three versions of the ICD-10 Charlson index. Med Care. 2007;45:1210–1215. 68. Li B, Evans D, Faris P, et al. Risk adjustment performance of Charlson and Elixhauser comorbidities in ICD-9 and ICD-10 administrative databases. BMC Health Serv Res. 2008;8:12. 69. Tang J, Wan JY, Bailey JE. Performance of comorbidity measures to predict stroke and death in a community-dwelling, hypertensive Medicaid population. Stroke. 2008;39:1938–1944. 70. Zhu H, Hill MD. Stroke: the Elixhauser Index for comorbidity adjustment of in-hospital case fatality. Neurology. 2008;71:283–287. 71. Furlan JC, Kattail D, Fehlings MG. The impact of co-morbidities on age-related differences in mortality after acute traumatic spinal cord injury. J Neurotrauma. 2009;26:1361–1367. 72. Chu Y-T, Ng Y-Y, Wu S-C. Comparison of different comorbidity measures for use with administrative data in predicting short- and longterm mortality. BMC Health Serv Res. 2010;10:140. 73. Martins M. Use of comorbidity measures to predict the risk of death in Brazilian in-patients. Rev Saude Publica. 2010;44:448–456.
r
2012 Lippincott Williams & Wilkins
Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited. View publication stats
