H. Alternative and Emerging Study Designs Relevant to HTA
Primary data collection methods are evolving in ways that affect the body of evidence used in HTA. Of great significance is the recognition that clinical trials conducted for biomedical research or to gain market approval or clearance by regulatory agencies do not necessarily address the needs of decision makers or policymakers.
Comparative effectiveness research (CER) reflects the demand for real-world evidence to support practical decisions. It emphasizes evidence from direct (“head-to-head”) comparisons, effectiveness in real-world health care settings, health care outcomes (as opposed to surrogate or other intermediate endpoints), and ability to identify different treatment effects in patient subgroups. As traditional RCTs typically do not address this set of attributes, CER can draw on a variety of complementary study designs and analytical methods. Other important trends in support of CER are the gradual increase in use of electronic health records and more powerful computing and related health information technology, which enable more rapid and sophisticated analyses, especially of observational data. The demand for evidence on potentially different treatment effects in patient subgroups calls for study designs, whether in clinical trials or observational studies, that can efficiently discern such differences. Another powerful factor influencing primary data collection is the steeply increasing costs of conducting clinical trials, particularly of RCTs for new drugs, biologics, and medical devices; this focuses attention on study designs that require fewer patients, streamline data collection, and are of shorter duration.
Investigators continue to make progress in combining some of the desirable attributes of RCTs and observational studies. Some of the newer or still evolving clinical trial designs include: large simple trials, pragmatic clinical trials, cluster trials, adaptive trials, Bayesian trials, enrichment trials, and clinical registry trials (Lauer 2012), as described below.
Large simple trials (LSTs) retain the methodological strengths of prospective, randomized design, but use large numbers of patients, more flexible patient entry criteria and multiple study sites to generate effectiveness data and improve external validity. Fewer types of data may be collected for each patient in an LST, easing participation by patients and clinicians (Buring 1994; Ellenberg 1992; Peto 1995; Yusuf 1990). Prominent examples of LSTs include the GISSI trials of thrombolytic treatment of acute myocardial infarction (AMI) (Maggioni 1990), the ISIS trials of alternative therapies for suspected AMI (Fourth International Study of Infarct Survival 1991), and the CATIE trial of therapies for schizophrenia (Stroup 2003).
Pragmatic (or practical) clinical trials (PCTs) are a related group of trial designs whose main attributes include: comparison of clinically relevant alternative interventions, a diverse population of study participants, participants recruited from heterogeneous practice settings, and data collection on a broad range of health outcomes. PCTs require that clinical and health policy decision makers become more involved in priority setting, research design, funding, and other aspects of clinical research (Tunis 2003). Some LSTs are also PCTs.
Cluster randomized trials involve randomized assignment of interventions at the level of natural groups or organizations rather than at the level of patients or other individuals. That is, sets of clinics, hospitals, nursing homes, schools, communities, or geographic regions are randomized to receive interventions or comparators. Such designs are used when it is not feasible to randomize individuals or when an intervention is designed to be delivered at a group or social level, such as a workplace-based smoking cessation campaign or a health care financing mechanism. These are also known as “group,” “place,” or “community” randomized trials (Eldridge 2008).
Adaptive clinical trials use accumulating data to determine how to modify the design of ongoing trials according to a pre-specified plan. Intended to increase the quality, speed, and efficiency of trials, adaptive trials typically involve interim analyses, changes to sample size, changes in randomization to treatment arms and control groups, and changes in dosage or regimen of a drug or other technology (FDA Adaptive Design 2010; van der Graaf 2012).
A current example of an adaptive clinical trial is the I-SPY 2 (Investigation of Serial Studies to Predict Your Therapeutic Response with Imaging and Molecular Analysis 2), which is investigating multiple drug combinations and accompanying biomarkers for treating locally advanced breast cancer. In this adaptive trial, investigators calculate the probability that each newly enrolled patient will respond to a particular investigational drug combination based on how previous patients in the trial with similar genetic “signatures” (i.e., set of genetic markers) in their tumors have responded. Each new patient is then assigned to the indicated treatment regimen, accordingly, with an 80% chance of receiving standard chemotherapy plus an investigational drug and a 20% chance of receiving standard chemotherapy alone (Barker 2009; Printz 2013).
Bayesian clinical trials are a form of adaptive trials that rely on principles of Bayesian statistics. Rather than waiting until full enrollment and completion of follow-up for all enrolled patients, a Bayesian trial allows for assessment of results during the course of the trial and modifying its design to arrive at results more efficiently. Such midcourse modifications may include, e.g., changing the ratio of randomization to treatment arms (e.g., two patients randomized to the investigational group for every one patient randomized to the control group) to favor what appear to be more effective therapies, adding or eliminating treatment arms, changing enrollee characteristics to focus on patient subgroups that appear to be better responders, changing hypotheses from non-inferiority to superiority or vice-versa, and slowing or stopping patient accrual as certainty increases about treatment effects. These trial modifications also can accumulate and make use of information about relationships between biomarkers and patient outcomes (e.g., for enrichment, as described below). These designs enable more efficient allocation of patients to treatment arms, with the potential for smaller trials and for patients to receive better treatment (Berry 2006). Recent advances in computational algorithms and high-speed computing enable the calculations required for the complex design and simulations involved in planning and conducting Bayesian trials (FDA Guidance for the Use of Bayesian 2010; Lee 2012).
Enrichment refers to techniques of identifying patients for enrollment in clinical trials based on prospective use of patient attributes that are intended to increase the likelihood of detecting a treatment effect (if one truly exists) compared to an unselected population. Such techniques can decrease the number of patients needed to enroll in a trial; further, they can decrease patient heterogeneity of response, select for patients more likely to experience a disease-related trial endpoint, or select for patients (based on a known predictive biomarker) more likely to respond to a treatment (intended to result in a larger effect size). In adaptive enrichment of clinical trials, investigators seek to discern predictive markers during the course of a trial and apply these to enrich subsequent patient enrollment in the trial (FDA 2012). While these techniques improve the likelihood of discerning treatment effects in highly-selected patient groups, the findings of such trials may lack external validity to more heterogeneous patients. In one form of enrichment, the randomized-withdrawal trial, patients who respond favorably to an investigational intervention are then randomized to continue receiving that intervention or placebo. The study endpoints are return of symptoms or the ability to continue participation in the trial. The patients receiving the investigational intervention continue to do so only if they respond favorably, while those receiving placebo continue to do only until their symptoms return. This trial design is intended to minimize the time that patients receive placebo (IOM Committee on Strategies for Small-Number-Participant Clinical Research Trials 2001; Temple 1996).
Clinical registry trials are a type of multicenter trial design using existing online registries as an efficient platform to conduct patient assignment to treatment and control groups, maintain case records, and conduct follow-up. Trials of this type that also randomize patient assignment to treatment and control groups are randomized clinical registry trials (Ferguson 2003; Fröbert 2010).
N-of-1 trials are clinical trials in which a single patient is the total population for the trial and in which a sequence of experimental and control interventions are allocated to the patient (i.e., a multiple crossover study conducted in a single patient). A trial in which random allocation is used to determine the sequence of interventions is given to a patient is an N-of-1 RCT. N-of-1 trials are used to determine treatment effects in individuals, and sets of these trials can be used to estimate heterogeneity of treatment effects across a population (Gabler 2011).
Patient preference trials are intended to account for patient preferences in the design of RCTs, including their ability to discern the impact of patient preference on health outcomes. Among the challenges to patient enrollment and participation in traditional RCTs are that some patients who have a strong preference for a particular treatment may decline to proceed with the trial or drop out early if they are not assigned to their preferred treatment. Also, these patients may experience or report worse or better outcomes due to their expectations or perceptions of the effects of assignment to their non-preferred or preferred treatment groups. Any of these actions may bias the results of the trial. Patient preference trials enable patients to express their preferred treatment prior to enrolling in an RCT. In some of these trials, the patients with a strong preference, e.g., for a new treatment or usual care, are assigned to a parallel group receiving their preferred intervention. The patients who are indifferent to receiving the new treatment or usual care are randomized into one group or another. Outcomes for the parallel, non-randomized groups (new intervention and usual care) are analyzed apart from the outcomes for the randomized groups.
In addition to enabling patients with strong preferences to receive their preferred treatment and providing for comparison of randomized groups of patients who expressed no strong preference, these trials may provide some insights about the relative impact on outcomes of receiving one’s preferred treatment. However, this design is subject to selection bias, as there may be systematic differences in prognostic factors and other attributes between patients with a strong preference for the new treatment and patients with strong preferences for usual care. Selection bias can also affect the indifferent patients who are randomized, as there may be systematic differences in prognostic factors and other attributes between indifferent patients and the general population, thereby diminishing the external validity of the findings. To the extent that patients with preferences are not randomized, the time and cost required to enroll a sufficient number of patients for the RCT to achieve statistical power will be greater. Patient preference trials have alternative designs, e.g., partially randomized preference trials and fully randomized preference trials. In the fully randomized preference design, patient preferences are recorded prior to the RCT, but all patients then randomized regardless of their preference. In that design, subgroup analyses enable determining whether receiving one’s preferred treatment has any impact on treatment adherence, drop-outs, and outcomes (Howard 2006; Mills 2011; Preference Collaborative Review Group 2008; Silverman 1996; Torgerson 1998).