Special Issue Paper Received 14 March 2012,
Accepted 15 March 2012
Published online in Wiley Online Library
(wileyonlinelibrary.com) DOI: 10.1002/sim.5396
Responses to commentaries Michael S Lauer* †
I appreciate the opportunity to respond to the thoughtful comments of Drs. Ellenberg, Basu, Hernan, Kunz, Yeh, and Normand. I largely agree with Dr. Ellenberg. In particular, I agree that we must distinguish the roles of explanatory and pragmatic trials, that evidence should be considered in totality, that the randomized trial ‘is to remain the cornerstone of interventional research in CER’, and that we as a community should seek to improve and understand the limits of the inferential capabilities of observational research. The National Heart, Lung, and Blood Institute supported a study that showed that observational studies that focus on incident—as opposed to prevalent—use of statins were more likely to yield credible effect sizes [1]. Dr. Ellenberg argues that we may need to rely on observational studies when efficacy data are not available. The absence of efficacy data sometimes represents an unfortunate, and potentially preventable, state of affairs. We almost never require efficacy, let alone effectiveness, data for diagnostic tests, meaning that many, if not most, diagnostic tests are implemented without knowledge about whether they improve clinical outcomes [2]. For some interventions, like intracranial stenting, we implement and adopt, only later finding out that they harm, even kill [3]. Dr. Ellenberg also argues that we may need to rely on observational data when there is loss of equipoise in the medical community. This is another unfortunate state of affairs, one which again is potentially preventable if the medical community had the discipline to insist on high-quality evidence before allowing a practice to become part of the established culture [4]. In an ideal world, we might follow the example of lung-cancer screening with helical computed tomography, where a large-scale pragmatic trial was done before widespread adoption [5]. Dr. Basu argues that the clinical trial enterprise needs ‘reinvention rather than reinvigoration’. This is because under current practice ‘the target parameters for the randomized clinical trials are often misplaced.’ By focusing on average results, typical clinical trials fail to help stakeholders make optimal decisions; after all, wise economic decisions are based on marginal, rather than average, findings. I agree with Dr. Basu that a reinvention is in order, a reinvention in which the clinical enterprise incorporates large-scale, low-cost, high-impact practical trials that are specifically designed to answer the marginal questions of greatest interest to stakeholders. For example, a large simple trial could identify which patients gain meaningful clinical benefit from esomeprazole as opposed to omeprazole; with large numbers, robust estimates could determine where heterogeneity of treatment effects leads to meaningful benefits on the margin, where ultimately all decisions must be made. Dr. Hernan agrees that randomization is the most powerful tool for dealing with confounding but points out two settings ‘in which the superiority of randomized trials is uncertain’. Randomized trials are not superior to observational studies for identifying unintended effects of treatment as well as for assessing the long-term effects of time-varying treatments. He points out, for example, that use of the intention-to-treat principle to assess long-term side effects of hormone therapy may lead to
Division of Cardiovascular Sciences, National Heart, Lung, and Blood Institute, Bethesda, MD *Correspondence to: Michael S Lauer, Division of Cardiovascular Sciences, National Heart, Lung, and Blood Institute, 6701 Rockledge Drive, Room 8128, Bethesda, MD 20892, USA. † E-mail:
[email protected]
Copyright © 2012 John Wiley & Sons, Ltd.
Statist. Med. 2012
M. S. LAUER
inappropriately reassuring conclusions. I agree with Dr. Hernan that randomized trials serve best when they are properly designed to assess hypothesized effects and when investigators carefully consider the effects of long-term adherence. Drs. Kunz, Yeh, and Normand use the example of carotid atherosclerosis to illustrate limitations of randomized trials as they are currently practiced. Despite a plethora of randomized efficacy trials, we do not know which treatment is best for elderly patients with asymptomatic carotid atherosclerosis, which treatment is most effective when administered by typical physicians in the community, and which treatments may be best for certain subgroups, such as the extreme elderly, women, and high-risk surgical candidates. Under certain circumstances, Kunz, Yeh, and Normand argue that observational studies may be able to answer important questions of comparative effectiveness and that ‘insistence on randomization as the only vehicle to comparative effectiveness research is dangerous.’ Whereas I agree that we should ‘embrace multiple strategies to create evidence’, I think one point many are missing is that our current deficits of evidence stem from our widespread failure to include most patients in randomized trials. There was a time, for example, when more than 80% of children with cancer were enrolled in randomized trials, making it possible for the field to make rapid advances of relevance to most, if not nearly all, patients. Within the community of pediatric oncology, there was a culture in which participation in randomized trials was considered the norm, not the exception. Some authorities have argued that there is a civic obligation to participate in clinical research, given that clinical research is a public good from which all benefit [6]. Imagine if over the last 10 to 20 years, we had enrolled more than 80% of patients with carotid atherosclerosis into large-scale practical trials. I suspect that by now we would be able to provide confident answers to Kunz’, Yeh’s, and Normand’s questions.
References 1. Danaei G, Tavakkoli M, Hernan MA. Bias in observational studies of prevalent users: lessons for comparative effectiveness research from a meta-analysis of statins. American Journal of Epidemiology 2012; 175:250–262. 2. Lauer MS. Elements of danger—the case of medical imaging. New England Journal of Medicine 2009; 361:841–843. 3. Chimowitz MI, Lynn MJ, Derdeyn CP, Turan TN, Fiorella D, Lane BF, Janis LS, Lutsep HL, Barnwell SL, Waters MF, Hoh BL, Hourihane JM, Levy EI, Alexandrov AV, Harrigan MR, Chiu D, Klucznik RP, Clark JM, McDougall CG, Johnson MD, Pride GL Jr., Torbey MT, Zaidat OO, Rumboldt Z, Cloft HJ. Stenting versus aggressive medical therapy for intracranial arterial stenosis. New England Journal of Medicine 2011; 365:993–1003. 4. Turka LA, Caplan A. What is the evidence for our standards of care? Journal of Clinical Investigation 2011; 121:2530. 5. Aberle DR, Adams AM, Berg CD, Black WC, Clapp JD, Fagerstrom RM, Gareen IF, Gatsonis C, Marcus PM, Sicks JD. Reduced lung-cancer mortality with low-dose computed tomographic screening. New England Journal of Medicine 2011; 365:395–409. 6. Schaefer GO, Emanuel EJ, Wertheimer A. The obligation to participate in biomedical research. Journal of the American Medical Association 2009; 302:67–72.
Copyright © 2012 John Wiley & Sons, Ltd.
Statist. Med. 2012
