Jerry J. Zimmerman, MD, PhD, FCCM,
Most critical care providers recognize the importance of investigation to improve outcomes of critically ill patients,1–3 but actual evidence identifying best practices remains sparse.4 The intensive care unit (ICU) represents an ideal environment for the conduct of clinical research. In addition to the electronic medical record, detailed physiologic monitoring represents usual practice in the ICU. Similarly, biomarker samples and imaging studies, frequently collected in the ICU for clinical decision-making, are also available for research.
Despite this surfeit of resources, clinical research may be less likely in the ICU setting because the high intensity of clinical work may leave little time to pursue investigation. Research in the ICU is uniquely challenging because the environment, the patients’ conditions, and trial design are typically and simultaneously complex.
Effects of this aggregate complexity were demonstrated in an investigation that quantified challenges of enrolling adult patients into research studies.5 Nearly 60% of occasions for recruiting eligible critically ill patients into various investigations were not realized, primarily because of missed opportunities (lack of research staff, expired enrollment window, decisions not to approach) and inability to consent (difficulty in contacting decision-makers, primary physician refusal, prohibition of co-enrollment). Interestingly, as with pediatric clinical research, 90% of adult consent encounters involve surrogate decision-makers. On the other hand, because critical care is expensive and associated with high morbidity and mortality, clinical research involving critically ill patients should be a healthcare priority.6 In addition, intensive care providers’ personalities may be more inclined to action than to investigation. With the ongoing threats of death and disability, there is a need for urgent decision-making, and critical care practitioners may be reluctant to contemplate a (complex) research protocol, preferring clinical decisionmaking based on clinical experience and pathophysiologic plausibility.7
In contrast to research activities in the ICU, in the hematology/oncology or hematopoietic stem cell transplantation units, virtually every patient is enrolled in one or more research protocols. Probably the most convincing example of iterative randomized controlled trials (RCTs) translating into gradual clinical improvements relates to the amazing success story of long-term outcomes for children with acute lymphocytic leukemia (ALL).8 In 1950, ALL was generally fatal within three months. During the following decades, the first chemotherapy trials were initiated.9,10 As a result of gradual improvements in both chemotherapy and supportive care, complete remission rates in children with ALL now approach 99%.11 In fairness, decreased mortality of the cancer patient population has been attributable in part to interventions received in the ICU.12
Discussion of research as an aspect of standard of practice in critical care would be incomplete without mentioning the potential harm to patients and families of proceeding with empiric practice not founded on rigorous evidence.13 Use of a new technology or device in the realm of standard practice is associated with unknown safety and efficacy, and such practice is not subject to any review. But within a research protocol, patients have the opportunity to make informed choices. When physicians use innovative therapies outside of research, institutional safeguards are typically lacking. A number of therapies that were innovatively introduced subsequently evolved into standards of practice without adequate testing of safety and efficacy and later were found to cause harm.14 Research subjects are typically protected by multiple layers of oversight, unlike everyday ICU patients, who may be subjected to anecdotal experimentation. Patients receiving innovative or so-called compassionate-use therapy have less protection than patients enrolled in an organized research protocol. Accordingly, innovative use of a drug, device, or biologic agent may be riskier to patients than the same use in the context of an appropriately designed and conducted clinical trial.15 It has been argued that the ethics and regulatory requirements for clinical practice, quality improvement, and clinical research should be identical.13 If the relative safety and efficacy of complex therapies in critically ill patients are unknown (due to lack of research), practitioners have an ethical imperative to investigate and identify such knowledge.
Critical care practitioners face two major problems in terms of including their patients in clinical trials, namely, identification of practical, clinically meaningful primary outcome measures and maintenance of relative equipoise regarding the research question. An outcome measure must be accurately determined, easy to record, and responsive to change. Clearly, outcome measures for RCTs must discriminate between the experimental and control arms of the trial and must be carefully considered.16 An outcome should demonstrate a causal relationship to the disease process under study (biological plausibility) and must be clinically relevant to patients, families, and providers. Surrogate outcomes must have direct, valid relationships with clinically meaningful measures.17
Relatively few RCTs conducted in the ICU setting that have used mortality as a primary outcome measure have shown a beneficial impact of the experimental intervention. Accordingly, there has been an evolving consensus to identify primary end points other than crude differences in all-cause mortality.18–20 In this regard, various measures of long-term health-related quality of life appear promising.21–23 However, numerous methodologic challenges remain for studies examining long-term outcomes; these include patient heterogeneity, premorbid conditions, subsequent uncharacterized insults, variable home environments, lack of follow-up, competing mortality, and the individuality of coping mechanisms.24
Although it has been argued that the critical aspect of equipoise relates to lack of agreement within a relevant clinical or scientific community,25 sometimes when an unstable patient is spiraling toward death or disability, individual physician uncertainty is more likely to be influenced by bias based on anecdote and personal experience.26 For trials involving a placebo arm, attainment of community clinical equipoise as well as individual physician equipoise is essential but often difficult to achieve.27 Only when both community and individual equipoise exist concurrently can an investigation be legitimately conducted with true equipoise.28
The National Institutes of Health’s Roadmap for Medical Research emphasized the need to translate basic research more expeditiously into human studies and to identify appropriate tests and treatments.29 Clinical trials are generally classified into one of two categories: 1) those that test variations of current practices, where the primary hypothesis is based on clinical experience; and 2) those that test fundamentally new concepts, where the primary hypothesis is based on pathophysiology and preliminary investigations.30
Before conducting a formal RCT, pre-RCT investigations are almost always beneficial for refining various aspects of the research design. This approach typically involves descriptive, epidemiologic, and observational studies. Systematic reviews and meta-analyses may also inform design of large RCTs,31,32 and may be considered the ethical and scientific foundation for all clinical trials.32
Single-center trials predominate because they are logistically easier to conduct, less expensive, do not typically require prolonged negotiation for study design or funding, utilize simplified data collection tools, enroll a less heterogeneous population, permit better planning for definitive trials, and are useful for hypothesis generation.33 Problems with single-center trials frequently include limited external validity, implausible hypothesized effect size, unequal allocation of resources, possible lack of blinding, loss of equipoise for definitive trials, and unwarranted evolution into “standard of care.”33,34 However, launching large, multicenter RCTs without performing pilot, single-center trials would be foolish and wasteful.
Traditionally, the double-blind, placebo-controlled RCT has represented the gold standard of clinical trial design.33 Some have advocated that evidence-based practice should continue to rely on RCTs.30 Given the history of critical care evidence-based medicine, it has been argued that at least two beneficial RCTs are necessary, with at least one of them being a confirmatory trial.35 If results of the clinical trial are destined for dissemination into clinical practice, both the entire intervention as well as the methodologic and clinical contexts of the intervention need to be understood.
There are numerous difficulties in conducting RCTs in critically ill patients.16 If some clinicians view assignment of their patients to a placebo arm as a potential threat, individual physician equipoise will need to be ascertained to ensure a trial’s success; this may require a potential culture shift toward intellectual honesty.
Critically ill patients, even those who have specific syndromes such as severe sepsis or acute respiratory distress syndrome, frequently represent heterogeneous populations. For this reason, a signal from a specific intervention may be more difficult to detect. Critically ill patients commonly receive a variety of concurrent interventions. Typically, these are accounted for by randomization, assuming the sample size is large enough. If not, a post hoc regression analysis may be required to cull out influences of various noncontrolled concurrent interventions. Criticisms of the RCT in this context include homogeneous patient populations, measuring study outcomes that are not important to patients, using protocols that are overly complex and conducted in specialized centers, and using study treatments that are inconsistent with usual care. Although RCTs emphasizing strict protocol adherence are essential for demonstrating the efficacy of a particular approach, they might not address effectiveness in more generalized practice settings.
Comparative effectiveness research involves “generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care.”36 Such observational studies in general are recognized as being inherently limited by indication bias and effects of unmeasured confounding variables.37,38 “Observational research methods cannot escape a key limitation: characteristics of the patient that drive real-life clinical decisions may also influence clinical outcomes leading to uncertainty about whether they or the intervention itself causes the outcomes.”36
The key role of individual clinical research performance sites has been relatively underappreciated. Competing agendas including conflict of commitment, financial pressure, regulatory burdens, risk aversion, and multiple research priorities stress the local research mission and infrastructure.39 Suggestions to address these real impediments to clinical research include instituting a process of clinical research improvement methodology, responding to the actual needs of site-based research, identifying clinical research as a key mission of the institution, establishing a clear process for reviewing and assigning research priorities, and improving local public understanding of the role of clinical research. Meaningful research collaborations between physician-scientist and clinician-educator faculty represents an important mechanism to facilitate clinical/ translational research.40
Increasingly the pharmaceutical and device industries fund the large, expensive, definitive assessment, phase 3 efficacy trials. However, because of commercial motives— actual or potential—bias represents a significant risk to the integrity of such studies.41 Site investigators can maximize the benefit of industry collaboration in clinical research by insisting on the key principles summarized in Table 1. Table 1. Key Principles of Industry Collaboration.42
Improving clinically meaningful patient-centered outcomes should be a goal of all ICU clinical research. Ultimately research must be focused on interventions that both lower costs and improve quality43 in order to increase the value of care provided in the ICU. Critical care practitioners should support the goals of evidence-based medicine and maintain equipoise on important research quests. Unfortunately only a trivial number of critically ill patients are enrolled in clinical trials, while new technologies, drugs, interventions, and treatment plans are constantly implemented.44 To identify and promote optimal care for critically ill patients, researchers need to increase both the number of clinical trials and the number of patients enrolled in such trials.44 In this regard everyone in the ICU can promote the concept of a learning healthcare environment, where clinical care, traditional clinical research, and quality improvement are so integrated and intercalated that they are basically inseparable. Each activity informs and benefits the others, ultimately generating an evidence basis for critical care practice.
- Cook D, Brower R, Cooper J, Brochard L, Vincent JL. Multicenter clinical research in adult critical care. Crit Care Med. 2002 Jul;30(7):1636-1643.
- Juni P, Altman DG, Egger M. Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ. 2001 Jul 7;323(7303):42-46.
- Cook DJ, Sibbald WJ, Vincent JL, Cerra FB. Evidence based critical care medicine; what is it and what can it do for us? Evidence Based Medicine in Critical Care Group. Crit Care Med. 1996 Feb;24(2):334-337.
- Zijlstra JG, Ligtenberg JJ, Girbes AR. Randomized controlled trials in critical care medicine. JAMA. 2008 Jul 2;300(1):43; author reply 43-44.
- Burns KE, Zubrinich C, Tan W, et al; Canadian Critical Care Trials Group. Research recruitment practices and critically ill patients. Am J Respir Crit Care Med. 2013 Jun 1;187(11):1212-1218.
- Coopersmith CM, Wunsch H, Fink MP, et al. A comparison of critical care research funding and the financial burden of critical illness in the United States. Crit Care Med. 2012 Apr;40(4):1072-1079.
- Tonelli MR, Curtis JR, Guntupalli KK, et al; ACCP/ATS/SCCM Working Group. An official multi-society statement: the role of clinical research results in the practice of critical care medicine. Am J Respir Crit Care Med. 2012 May 15;185(10):1117-1124.
- Simone JV. History of the treatment of childhood ALL: a paradigm for cancer cure. Best Pract Res Clin Haematol. 2006;19(2):353-359.
- Djerassi I, Farber S, Abir E, Neikirk W. Continuous infusion of methotrexate in children with acute leukemia. Cancer. 1967 Feb;20(2):233-242.
- Mukherjee S. The Emperor of All Maladies. New York: Scibner; 2010.
- Pui CH. Acute lymphoblastic leukemia in children. Curr Opin Oncol. 2000 Jan;12(1):3-12.
- Fiser RT, West NK, Bush AJ, Sillos EM, Schmidt JE, Tamburro RF. Outcome of severe sepsis in pediatric oncology patients. Pediatr Crit Care Med. 2005 Sep;6(5):531-536.
- Wootton SH, Evans PW, Tyson JE. Unproven therapies in clinical research and practice: the necessity to change the regulatory paradigm. Pediatrics. 2013 Oct;132(4):599-601.
- Silverman WA. Overtreatment of neonates? A personal retrospective. Pediatrics. 1992 Dec;90(6):971-976.
- Nelson RM. Challenges in the conduct of emergency research in children: a workshop report. Am J Bioeth. 2006 Nov-Dec;6(6):W1-W9.
- Hebert PC, Cook DJ, Wells G, Marshall J. The design of randomized clinical trials in critically ill patients. Chest. 2002 Apr;121(4):1290-1300.
- Fleming TR, DeMets DL. Surrogate end points in clinical trials: are we being misled? Ann Intern Med. 1996 Oct 1;125(7):605-613.
- Curley MA, Zimmerman JJ. Alternative outcome measures for pediatric clinical sepsis trials. Pediatr Crit Care Med. 2005 May;6(3 Suppl):S150-S156.
- Marshall JC, Vincent JL, Guyatt G, et al. Outcome measures for clinical research in sepsis: a report of the 2nd Cambridge Colloquium of the International Sepsis Forum. Crit Care Med. 2005 Aug;33(8):1708-1716.
- Ospina-Tascón GA, Büchele GL, Vincent JL. Multicenter, randomized, controlled trials evaluating mortality in intensive care: doomed to fail? Crit Care Med. 2008 Apr;36(4):1311-1322.
- Hackbarth RM, Rzeszutko KM, Sturm G, Donders J, Kuldanek AS, Sanfilippo DJ. Survival and functional outcome in pediatric traumatic brain injury: a retrospective review and analysis of predictive factors. Crit Care Med. 2002 Jul;30(7):1630-1635.
- Gemke RJ, Bonsel GJ. Reliability and validity of a comprehensive health status measure in a heterogeneous population of children admitted to intensive care. J Clin Epidemiol. 1996 Mar;49(3):327-333.
- Varni JW, Burwinkle TM, Seid M, Skarr D. The PedsQL 4.0 as a pediatric population health measure: feasibility, reliability, and validity. Ambul Pediatr. 2003 Nov-Dec;3(6):329-341.
- Rubenfeld GD. Improving clinical trials of long-term outcomes. Crit Care Med. 2009 Jan;37(1 Suppl):S112-S116.
- Freedman B. Equipoise and the ethics of clinical research. N Engl J Med. 1987 Jul 16;317(3):141-145.
- Fried C. Medical Experimentation: Personal Integrity and Social Policy. Amsterdam: North-Holland Publishing Company; 1974.
- Holubkov R, Dean JM, Berger J, et al; Eunice Kennedy Shriver National Institute of Child Health and Human Development Collaborative Pediatric Critical Care Network. Is “rescue” therapy ethical in randomized controlled trials? Pediatr Crit Care Med. 2009 Jul;10(4):431-438.
- Schwab AP. Determining research through underdetermined treatment. Virual Mentor. 2004 Nov 1;6(11).
- Zerhouni E. Medicine. The NIH roadmap. Science. 2003 Oct 3;302(5642):63-72.
- Reade MC, Angus DC. The clinical research enterprise in critical care: what’s right, what’s wrong, and what’s ahead? Crit Care Med. 2009 Jan;37(1 Suppl):S1-S9.
- Cook DJ, Sackett DL, Spitzer WO. Methodologic guidelines for systematic reviews of randomized control trials in health care from the Potsdam Consultation on Meta-Analysis. J Clin Epidemiol. 1995 Jan;48(1):167-171.
- Chalmers I. Academia’s failure to support systematic reviews. Lancet. 2005 Feb 5-11;365(9458):469.
- Bellomo R, Warrillow SJ, Reade MC. Why we should be wary of single-center trials. Crit Care Med. 2009 Dec;37(12):3114-3119.
- Arnold DM, Burns KE, Adhikari NK, Kho ME, Meade MO, Cook DJ; McMaster Critical Care Interest Group. The design and interpretation of pilot trials in clinical research in critical care. Crit Care Med. 2009 Jan;37(1 Suppl):S69-S74.
- Sweeney DA, Danner RL, Eichacker PQ, Natanson C. Once is not enough: clinical trials in sepsis. Intensive Care Med. 2008 Nov;34(11):1955-1960.
- Sox HC, Greenfield S. Comparative effectiveness research: a report from the Institute of Medicine. Ann Intern Med. 2009 Aug 4;151(3):203-205.
- Deeks JJ, Dinnes J, D’Amico R, et al; International Stroke Trial Collaborative Group; European Carotid Surgery Trial Collaborative Group. Evaluating non-randomised intervention studies. Health Technol Assess. 2003;7(27):iii-x, 1-173. Review.
- Bosco JL, Silliman RA, Thwin SS, et al. A most stubborn bias: no adjustment method fully resolves confounding by indication in observational studies. J Clin Epidemiol. 2010 Jan;63(1):64-74.
- Califf RM. Clinical research sites—the underappreciated component of the clinical research system. JAMA. 2009 Nov 11;302(18):2025-2027.
- Marks AR. How to support the basic sciences. J Clin Invest. 2005 Jan;115(1):2.
- Molenberghs G, Imrey P, Drake C. Conflicts of interest and independent data analysis in industry-funded studies. JAMA. 2005 Nov 23;294(20):2575-2576; author reply 2576-2577.
- Abraham E. Better infrastructure: industry-academia partnerships—a marriage of convenience? Crit Care Med. 2009 Jan;37(1 Suppl):S159-S164.
- Emanuel EJ. The future of biomedical research. JAMA. 2013 Apr 17;309(15):1589-1590.
- Randolph AG. The unique challenges of enrolling patients into multiple clinical trials. Crit Care Med. 2009 Jan;37(1 Suppl):S107-S111.