INT J TUBERC LUNG DIS e-publication ahead of print 6 October 2011 © 2011 The Union http://dx.doi.org/10.5588/ijtld.10.0065
REVIEW ARTICLE
Microcolony culture techniques for tuberculosis diagnosis: a systematic review E. Leung,*† J. Minion,*†‡ A. Benedetti,*† M. Pai,*† D. Menzies*† * Respiratory Epidemiology and Clinical Research Unit, Montreal Chest Institute, Montreal, Quebec, † Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montreal, Quebec, ‡ Department of Medical Microbiology and Immunology, University of Alberta, Edmonton, Alberta, Canada SUMMARY BACKGROUND:
There is considerable demand for quicker and more affordable yet accurate diagnostic tools for tuberculosis (TB). The microscopic observation drug susceptibility (MODS) assay and the thin-layer agar (TLA) assay are inexpensive, rapid microcolonybased culture methods. M E T H O D S : A systematic review and meta-analysis was performed to assess the accuracy and other test characteristics of MODS and TLA compared to a reference standard of traditional solid or liquid culture. Pooled estimates of sensitivity and specificity and their 95% confidence intervals were estimated with an exact binomial likelihood random effects meta-analysis. R E S U LT S : A total of 21 eligible studies were identified, 12 that evaluated MODS, seven that evaluated TLA and two that evaluated both. The overall pooled sensitivity and specificity of MODS were respectively 92% (95%CI 87–97) and 96% (90–100), and for TLA they were re-
spectively 87% (95%CI 79–94) and 98% (95%CI 94– 100), although there was considerable heterogeneity of results. When the studies were restricted to those assessing accuracy of MODS in sputum samples only, the sensitivity was 96% (95%CI 94–98) and the specificity 96% (95%CI 89–100). The mean intervals from reception of specimens to results were 9.2 days with MODS and 11.5 days with TLA; contamination rates averaged 6.6% with MODS and 12.3% with TLA; materials and supplies costs averaged US$1.48 for MODS and US$2.42 for TLA. C O N C L U S I O N S : MODS and TLA appear to be accurate and rapid yet inexpensive diagnostic tools for active TB. However, this review did not find sufficient evidence on the feasibility and costs of implementation of these tests, nor on the impact of these tests on patient outcomes. K E Y W O R D S : tuberculosis diagnosis; MODS; thinlayer agar; tuberculosis culture; new diagnostics
TUBERCULOSIS (TB) is a major cause of illness and death worldwide, and the number of cases and the incidence have increased substantially over the last two decades.1,2 The World Health Organization has estimated that, despite 15 years of efforts in DOTS implementation and expansion, only 65% of all new smear-positive cases were diagnosed in 2007.1 The low rate of case detection has stimulated the search for new diagnostic tools that are rapid, accurate and inexpensive. These could increase TB case detection, thereby reducing morbidity, mortality and transmission of infection. Of the currently utilised methods for detection of active TB, none are rapid, accurate and inexpensive. Detection of acid-fast bacilli (AFB) using smear microscopy is the primary method of diagnosis of TB in DOTS-based programmes in low- and middle-income countries,1 providing results within hours. However, the sensitivity of this test ranges from 20% to 60%.3 Mycobacterial cultures performed using solid media such as Löwenstein-Jensen (LJ) are more sensitive than AFB smears, but are more expensive and pro-
vide results only after 4–8 weeks. Newer liquid media cultures using automated systems are even more sensitive and provide results within 2–4 weeks, but are much more expensive.1 New non-commercial culture techniques have been introduced that are less costly than the automated liquid culture systems, yet may have similar accuracy and time to results. Two of these are the microscopic observation drug susceptibility (MODS) assay and the thin-layer agar (TLA) method. We planned a systematic review, and meta-analysis if appropriate, of all published studies that reported diagnostic accuracy, time to result, contamination rates and costs of MODS or TLA assays for the detection of active TB.
METHODS We followed a standard protocol for systematic reviews and meta-analyses,4 and used methods recommended by the Cochrane Diagnostic Test Accuracy Working Group.5
Correspondence to: Dick Menzies, Montreal Chest Institute, Room K1.24, 3650 St Urbain St, Montreal, Quebec H2X 2P4, Canada. Tel: (+1) 514 934 1934; extn 32128. Fax: (+1) 514 843 2083. e-mail:
[email protected] Article submitted 27 January 2011. Final version accepted 20 May 2011.
2
The International Journal of Tuberculosis and Lung Disease
Search strategy We systematically searched the literature published from January 1990 to February 2009 in English, French or Spanish, using three databases: PubMed, EMBASE and BIOSIS. The search was updated in March 2011. All electronic searches were performed by two persons (EL and an experienced librarian). Reference lists from included studies were also hand-searched. The key words used for the electronic search were TB or tuberculosis, AND diagnosis or detection or screening or diagnostic tests or case-finding or case detection, AND microscopic-observation drug sensitivity or MODS or thin-layer agar, AND sensitivity or specificity or positive or negative predictive values or yield or utility or feasibility or feasibility or accessibility or implementation or training or operational effectiveness or costs or cost-effectiveness. Eligibility criteria All studies that reported an evaluation of MODS or TLA for diagnosis of active TB disease were included. For the primary analysis, the predetermined eligibility criteria were evaluation of MODS or TLA for detection of TB and use of an accepted reference standard. Accepted reference standards included direct or indirect culture using solid media such as LJ or Middlebrook agar, or liquid cultures such as BACTEC® or BACTEC MGIT® (BD, Towson, MD, USA). All specimens from all sites of disease were considered. Studies reporting insufficient data for the estimation of sensitivity were excluded. Studies were included if sensitivity could be calculated, even if they incorporated the index tests as part of the reference standard (meaning that the diagnosis of active TB was considered confirmed if any culture, including MODS or TLA, was positive). In these studies, specificity could not be estimated. All study designs were considered. Editorials, letters to the editor and conference abstracts were excluded. MODS and TLA can also be used for rapid detection of drug resistance.6,7 A previous meta-analysis has summarised the results of MODS or TLA for drug susceptibility testing (DST).8 Study selection All selection steps were performed by two independent reviewers (EL and JM). Titles and abstracts were screened to select studies, and any citations identified as eligible by either reviewer were selected for full text review. Articles retrieved for full text review along with reasons for exclusion are available from the authors. Disagreements were resolved through a third reviewer (DM). Data extraction A data extraction form was created and piloted with a subset of eligible studies. Data from the studies selected
for inclusion were extracted independently by two reviewers (EL and JM) using a standardised data extraction form (see Appendix*); any disagreements were resolved by consensus. Data were extracted on the following variables: type of index test, reference standard, blinding, type of specimen (pulmonary vs. extra-pulmonary, smear-positive vs. smear-negative), age and human immunodeficiency virus (HIV) status of patients, study design, selection of the specimens/ patients, costs, interval from reception until result and proportion contaminated with bacterial or fungal overgrowth. Assessment of study quality The Quality Assessment of Diagnostic Accuracy Studies (QUADAS) criteria were used to assess the quality of the studies used. Meta-analysis methods Data were analysed using a random effects metaanalysis to estimate the overall pooled estimates and 95% confidence intervals (CIs) of sensitivity and specificity using the Proc Nlmixed programme in SAS (SAS Institute, Cary, NC, USA).9 An exact binomial likelihood approach which uses a binomial distribution to approximate the distribution of the outcome of interest was used.9 This approach accounts for study size, has been demonstrated to produce less biased estimates of the pooled effect and includes a random effect to account for inter-study heterogeneity.9 Forest plots visually displaying sensitivity and specificity estimates and their 95%CIs from each study were constructed using MetaDiSc software (Microsoft, Redmond, WA, USA).10 Some of the studies included in the meta-analysis contributed both sensitivity (true-positive rate) and specificity (one minus false-positive rate). As these measures tend to be correlated and vary with the thresholds used across individual studies, a summary receiver operating characteristic curve analysis was performed to explore the effect of thresholds on results.4,11 Subgroup analysis The heterogeneity of outcomes of interest was assessed by estimating the I2 statistic and associated 95%CIs.12 Sub-group analyses were performed to minimise heterogeneity within strata defined by type of reference standard, AFB smear result, type of clinical specimen and whether the technicians were blinded. Outcome measures Results from each study were classified as true-positive (TP), false-positive (FP), false-negative (FN) and
* The Appendix is available in the online version of this article.
Accuracy of MODS or TLA for diagnosis of TB
3
true-negative (TN) values. From these data, we calculated the sensitivity and specificity for each study. Other outcomes extracted included the time from specimen receipt to availability of culture results within the laboratory, contamination rate and costs. A contaminated culture was defined as a culture with fungal or bacterial overgrowth on the first inoculation.13 Costs, expressed in 2007 $US, included all health system costs for reagents, supplies, equipment, labour, overheads and other related costs. Where costs were expressed in a foreign currency, they were converted to $US using purchasing power parity rates for the year that costs were reported in the study.14,15 Costs were then adjusted to 2007 using the consumer price index.15,16
RESULTS Characteristics of included studies As seen in Figure 1, 2067 citations were identified from the initial electronic searches and an additional 10 citations were identified from the update. After excluding duplicate citations, 1410 unique citations were left. From these, 58 potentially relevant articles were retrieved for full text review, of which 21 were considered eligible for this review. These are summarised in Table 1. Of the 21 eligible studies, 12 evaluated MODS only, seven evaluated TLA only and two evaluated both. All evaluated detection of active TB and time to detection. Fourteen of the studies used both AFB smear-positive and smear-negative clinical samples, one study included only smear-positive samples, two
Table 1
Figure 1 Study selection. MODS = microscopic observation drug susceptibility; TLA = thin-layer agar; DST = drug susceptibility testing; TB = tuberculosis.
studies included only smear-negative samples, and four studies did not report whether the samples were smear-positive or -negative. Five studies used both solid and liquid culture as the reference test, and 10 studies used only solid culture. Six studies incorporated the results of MODS or TLA into the reference standard, meaning that any positive culture was considered a true-positive; specificity could not be estimated in these studies. On average, the study size was 683 specimens per study (range 22–4213).
Characteristics of included studies*
Author, year, reference Reddy et al., 201017 Arias et al., 200718 Caws et al., 200719 Giacomazzi et al., 201020 Ha et al., 200921 Michael et al., 201022 Moore et al., 200423 Moore et al., 200624 Oberhelman et al., 200625 Oberhelman et al., 201026 Shiferaw et al., 200727 Tovar et al., 200828 Idigoras et al., 199529 Martin et al., 200930 Martin et al., 200931 Mejia et al., 199932 Mejia et al., 200433 Robledo et al., 200634 Welch et al., 199335 Caviedes et al., 200036 Irfan et al., 200637
Country
Total sample size n
Index test
India USA Viet Nam Ecuador Viet Nam India Peru Peru Peru Peru Ethiopia Peru Spain Belgium Kenya Colombia Colombia Colombia USA Peru Pakistan
889 1639 150 507 217 38 406 4213 38 22 262 111 1997 210 298 84 1809 1118 103 97 200
MODS MODS MODS MODS MODS MODS MODS MODS MODS MODS MODS MODS TLA TLA TLA TLA TLA TLA TLA Both Both
Reference test for diagnosis
Smearpositive samples %
Incorporation bias
Technicians blinded
LJ LJ L J and MGIT LJ LJ L J and BACTEC LJ L J and MBBacT LJ LJ LJ LJ LJ MGIT LJ LJ LJ LJ LJ L J and MGIT L J and BACTEC
5 36 22 10 21 24 Not reported 6 2 22 100 Not reported 2 95 0 10 Not reported 13 62 44 30
No No No No No No Yes No No No No No Yes Yes Yes No No No Yes Yes No
Yes Yes Yes Not reported Yes Yes Not reported Yes Yes Not reported Not reported Not reported Not reported Not reported Not reported Not reported Not reported Not reported Not reported Yes Not reported
* Interval from receipt of sample until result available—reported in all studies. MODS = microscopic-observation drug susceptibility; L J = Löwenstein-Jensen; MGIT = Mycobacterial Growth Indicator Tube; TLA = thin-layer agar.
4
The International Journal of Tuberculosis and Lung Disease
Figure 2
Quality of studies using QUADAS criteria. QUADAS = Quality Assessment of Diagnostic Accuracy Studies.
Quality of included studies By design, all MODS or TLA results in all studies were verified using a reference standard of diagnosis. Twelve studies used a cross-sectional study design, four were case series of only positive specimens, four used a case-control design and one used a cohort design. Fifteen studies used consecutive or random sampling, one used a convenience sample and the remainder did not describe their selection process. Nineteen studies were prospective, one was retrospective, and one did not describe directionality. In eight studies, the technicians performing the MODS or TLA were blinded; the remainder did not report on blinding. As seen in Figure 2, of the 14 QUADAS quality indicators, the number clearly met by the included studies varied between 5 and 14.
test into the reference standard (incorporation bias) were excluded, the pooled sensitivity and specificity for MODS were respectively 92% (95%CI 87–97) and 97% (95%CI 95–100), and for TLA, sensitivity was 86% (95%CI 73–99) and specificity was 97% (95%CI 95–100). When MODS results were stratified by reference standard (also Table 2), the sensitivity was 89% (95%CI 80–99) when MODS was judged against the combination of both liquid and solid cultures, compared to 94% (95%CI 88–99) if the reference was solid culture only. Specificity was 99% (95%CI 97–100) if the reference was solid plus liquid cultures compared to 90% (95%CI 75–100) with solid cultures only. There was considerable heterogeneity in accuracy estimates for MODS (Figure 3A) and TLA (Figure 3B).
Accuracy estimates As shown in Table 2, when results from all 21 studies were used, the pooled sensitivity of MODS was 92% (95%CI 88–96) and for TLA it was 87% (95%CI 79–94). When the studies that incorporated the index
Hierarchical summary receiver operating curves The sensitivity (true-positive rate) and 1-specificity (false-positive rate) were derived in a hierarchical summary receiver operating curve (HSROC) for MODS (n = 12 studies) and TLA (n = 4 studies).
Table 2
Pooled accuracy estimates of all included studies: overall, by test and by reference standard
Outcome Overall: all reference standards combined MODS only TLA only MODS only; by reference standard Liquid and solid cultures Solid cultures only Incorporation (any positive culture)*
Studies for sensitivity n
Positive test/ all with disease n/N
Sensitivity % (95%CI)
Studies for Negative test/ specificity all without disease n n/N
14 9
3516/3731 1167/1394
92 (88–96) 87 (79–94)
12 4
7041/7226 3293/3359
96 (90–100) 98 (94–100)
5 7
590/639 1038/1087
89 (80–99) 94 (88–99)
5 7
3823/3861 3218/3365
99 (96–100) 90 (74–100)
2
1888/2005
93 (84–100)
—
—
—
Specificity % (95%CI)
* Defined as studies that incorporated the results of MODS or TLA in the reference standard (all positive cultures considered true-positive). Hence specificity could not be estimated. One of these studies used both solid and liquid culture and the other used solid culture only. CI = confidence interval; MODS = microscopic-observation drug susceptibility; TLA = thin-layer agar.
Accuracy of MODS or TLA for diagnosis of TB
5
As seen in Table 3, in the one study in which all samples were smear-positive, MODS was positive in all samples that were positive with LJ culture (sensitivity 100%), but MODS was also positive in four of the 11 smear-positive specimens that were negative on LJ (‘specificity’ 64%). In the 10 studies that included both smear-positive and -negative samples, the overall sensitivity was lower and the specificity was higher. Three studies used extra-pulmonary samples alone (samples included cerebrospinal fluid, gastric aspirate, nasopharyngeal aspirate and stool specimens); all had lower sensitivities, ranging from 51% to 85%. In the studies where the technicians were blinded, the specificity was higher than in the studies that did not report blinding. Head-to-head comparisons between MODS and TLA There were two direct comparisons of MODS vs. TLA using the same specimens to detect active TB. In one, the sensitivity of MODS was 76% (95%CI 66– 84) compared to 93% for TLA (95%CI 85–97),36 but specificity was not reported. In the second, MODS sensitivity was 93% (95%CI 84–98) and TLA sensitivity was 92% (95%CI 82–97), while specificity was 87% (95%CI 80–92) for MODS and 90% (95%CI 83–94) for TLA.37
Figure 3 A) Forest plots of sensitivity and specificity of MODS for detection of active TB: studies using cultures as reference standards only. Includes 12 studies using liquid or solid culture media as the reference standard for MODS. Two studies were excluded as they incorporated all positive results as a reference standard. B) Forest plot of sensitivity and specificity of TLA for detection of active TB: studies using cultures as reference standards only. Includes four studies using liquid or solid culture media as the reference standard for TLA. Five studies were excluded as they incorporated all positive results as a reference standard. CI = confidence interval; df = degree of freedom.
The area under the curve was 98% for MODS, indicating near-perfect discriminatory ability and 96% for TLA, also indicating near-perfect discriminatory ability (see Appendix Figures). Stratified analyses Stratified analyses were restricted to studies without incorporation bias that evaluated the MODS assay.
Other test characteristics The intervals from receipt of specimens to results for MODS and TLA were respectively 9.2 days and 11.5 days (Table 4), a significant difference. These intervals for both microcolony techniques were significantly less than with traditional solid culture. In 10 studies, MODS had a pooled contamination rate of 6.6%, compared to 11.4% with solid culture and 3.9% with liquid culture, performed on the same specimens by the same laboratories (Table 5). In seven studies, TLA had a contamination rate of 12.3%, compared to 5.4% with solid culture and 10.8% with liquid culture. As seen in Table 6, in nine studies that reported costs for supplies and reagents, the average cost was US$1.48 for MODS compared to US$2.42 for TLA. (One study that reported patient costs was excluded from this analysis.17) In the only study to estimate costs for both tests at the same centre, the costs for MODS supplies and reagents were higher than for TLA. No studies considered the initial costs for equipment, construction or training, nor recurrent costs for labour, supervision or overheads associated with MODS or TLA. No study described training, supervision or quality control procedures, nor did any study assess the impact of either test on patient outcomes.
DISCUSSION Principal findings Our review identified a total of 21 studies evaluating MODS and/or TLA. MODS had a pooled sensitivity
6
The International Journal of Tuberculosis and Lung Disease
Table 3
Pooled accuracy estimates of included studies (MODS only) stratified by major covariates
Outcome By results of direct AFB smears* All positive (100%) Some positive (2–95%)† None positive Not reported By blinding of technicians performing tests* Blinding reported Unclear about blinding By type of samples* All sputum (100%) Some sputum (34–84%)† None sputum Not reported
Studies for sensitivity n
Positive test/ all with disease n/N
Sensitivity % (95%CI)
Studies for Negative test/ specificity all without disease n n/N
1 10 0 1
247/247 1343/1438 — 38/41
100 89 (83–95) — 93 (80–100)
1 10 0 1
7/11 6893/7046 — 141/169
65 (0–100) 97 (93–100) — 84 (30–100)
7 5
1236/1312 392/414
90 (81–99) 94 (88–100)
7 5
6323/6443 718/774
99 (97–100) 78 (52–100)
6 3 3 0
1413/1460 129/164 86/102 —
96 (94–99) 78 (62–95) 86 (73–99) —
6 3 3 0
5348/5472 1463/1487 230/267 —
96 (89–100) 99 (96–100) 80 (41–100) —
Specificity % (95%CI)
* The tables have excluded two studies that incorporated the results of MODS or TLA in the reference standard. † The results in the studies were not stratified by smear status, treatment status or percentage of pulmonary samples. MODS = microscopic-observation drug susceptibility; CI = confidence interval; AFB = acid-fast bacilli; TLA = thin-layer agar.
Table 4 Average interval from receipt of specimen to positive result for MODS or TLA
MODS Studies, n Samples, n Mean time to result, days (SD) Difference from solid reference in days (MODS/ TLA faster by n days) Difference from liquid reference in days (MODS/ TLA faster by n days)
Diagnosis of active TB with MODS
Diagnosis of active TB with TLA
10 3707 9.2 (2.2)*
5 668 11.5 (1.9)*
16.1 (n = 8)†
11.8 (n = 5)†
2.6 faster (n = 6)‡
1.5 slower (n = 1)‡
* The difference between MODS and TLA was significant (P < 0.05). † The differences between MODS and solid cultures, and between TLA and solid cultures were significant (P < 0.005). ‡ The differences between MODS and liquid cultures, and between TLA and liquid cultures were not significant. MODS = microscopic-observation drug susceptibility; TLA = thin-layer agar; TB = tuberculosis; SD = standard deviation.
of 92% and specificity of 96%, while TLA had a pooled sensitivity of 87% and specificity of 98%. However, there was considerable heterogeneity in these estimates, even in stratified analyses. Costs of reagents and materials for both tests were low, but other costs were not described. The average interval
Table 6 Average cost per sample for MODS or TLA for reagents and supplies only* Costs for supplies and reagents, US$/sample
Author, year, reference MODS Caviedes et al., 200036 Caws et al., 200719 Michael et al., 201022 Moore et al., 200624 Oberhelman et al., 200625 Tovar et al., 200828 Average cost per sample for MODS TLA Caviedes et al., 200036 Martin et al., 200930 Mejia et al., 199932 Robledo et al., 200634 Average cost per sample for TLA
0.96 0.53 2.06 2.06 1.30 1.94 1.48 0.36 2.53 3.69 3.08 2.42
* All costs reported in or converted to 2007 $US. MODS = microscopic observation drug susceptibility; TLA = thin layer agar.
from receipt to results within the laboratory was 9.2 days for MODS and 11.5 days for TLA—faster than with conventional solid and liquid cultures in the same studies.38 The proportion of contaminated specimens with both assays was low, and comparable to conventional cultures.
Table 5 Proportion of contaminated specimens using MODS or TLA compared to the reference method* Proportion contaminated using MODS
contaminated†
Proportion Solid culture Liquid culture
Proportion contaminated using TLA
n
% (range)
P value
n
% (range)
P value
10 9 5
6.6 (0.4–11.2) 11.4 (1–22) 3.9 (0.9–6.3)
