Scientific validity
Scientific validity of the research is one of the essential ethical requirements [18] as the overarching goal of a clinical trial is to provide evidence that can support clinical decision-making [19]. Here, we outline some of the major challenges referring to scientific validity in basket and umbrella trials.
The first issue that can be a threat to scientific validity is the design of a treatment that matches only a single mutation, while tumors may harbor multiple mutations at a time. Cancer’s heterogeneity can be distinguished not only within the primary tumor (intratumoral heterogeneity), but also between the primary tumor and its metastases (intertumoral heterogeneity) and between patients (interpatient heterogeneity) [4, 20, 21], which indicates that every tumor is unique. Thus, focusing only on molecular therapy targeting single mutation without considering the complexity of tumor biology, may introduce bias. It is unclear how many patients in NCI-MATCH and Lung-MAP trials are diagnosed to harbor more than one mutation. There may be more patients that only partially match the intervention (harbor multiple genetic changes) than the ones that totally match (harbor only one genetic change). Moreover, in a trial with randomization, like in the Lung-MAP trial, patients in both arms (experimental and standard of care) should harbor similar genetic changes within a tumor to be comparable and represent the same patient population. If the complexity and heterogeneity of tumors are neglected, the result may be a treatment failure and the impossibility to produce scientifically reliable findings.
The flexible structure of basket and umbrella trials allows for testing multiple interventions simultaneously, closing ineffective ones and opening new ones without writing a new protocol, which saves time and financial resources. Nevertheless, after closing a treatment, the results of the trial sub-study should be published as soon as possible as a full journal publication because the profound results of completed clinical trials are crucial for decision making in evidence-based medicine and inform future research. Unfortunately, summary results of NCI-MATCH and Lung-MAP sub-studies are incomplete. For example, it is hard to say in which arm of the five sub-studies in the Lung-MAP study subjects benefitted more (intervention or standard of care), because response rates in the standard of care arms have not yet been published (Table 2). Since we do not know whether responses are higher or lower in the standard of care arm, we claim that the risk of publication bias is the second challenge of the scientific validity requirement.
The flexibility enables also for changes in the protocol. Giving the exact example is the initial design of the Lung-MAP trial as a study including randomization to a control arm, which was further modified during the trial into single-arm study [17]. There is no explanation why it was changed. This can mean that the design was flawed from the beginning and no one had foreseen that. The freedom for unexplained modifications in the protocol is a third serious threat to scientific validity.
The main advantage of basket trials is that patients with rare cancers have the opportunity to be enrolled into the study. For example, in the NCI-MATCH trial, about 61% of patients have less common tumors [22]. However, a fourth serious issue referred to scientific validity is insufficient patient accrual to treatment arms, which may affect statistical methods and power and oppugn the reliability of the findings. For instance, in the NCI-MATCH trial only 8 of the 30 sub-studies reached the minimum patient accrual goal of 35 [23]. A research that cannot enroll sufficient subjects cannot generate valid scientific knowledge and is unethical [3]. Furthermore, waiting for more patients that harbor a specific mutation prolongs the study and delays the publication of trial results. There are also doubts whether drugs tested on an insufficient number of patients could be approved without confirmed efficacy based on surrogate endpoints which are considered low-grade evidence [24].
Benefits and risks
Another important ethical requirement of conducting clinical trials is a favorable risk/benefit ratio, which is met when: 1) the risk for participants is minimized; 2) the expected benefits are maximized; and 3) the possible benefits to participants and society outweigh or are proportional to the risks associated with participation in the study. The risk-benefit proportionality criterion considers the fundamental ethical principles of non-maleficence and beneficence. It also serves as a protection for participants against their exploitation [3].
There are three types of possible benefit in clinical trials: aspirational, direct and collateral [25]. Aspirational benefit is the benefit to society and to future patients, which arises from the results of the study [25]. Novel clinical trials serve as exploratory trials of both tumor and pathways and they can gain important knowledge, which can be used in future trials to develop effective therapies. Thus, it is extremely important that the results of a trial are published after each sub-trial completes. Moreover, the aspects mentioned before in the scientific validity section: the heterogeneity of tumors harboring multiple mutations, problems with sufficient patient accrual, risk of publication bias may generate unreliable and unpowered findings, which negatively influences the aspirational benefit, wastes resources and negatively affects decision making in medicine [18].
Direct benefit is the benefit to research subjects arising from receiving the intervention being studied [25]. In cancer trials an optimal direct benefit could be the one achievable in patient-centered outcomes, such as overall survival (OS) and/or quality of life (QoL) [26]. However, in basket and umbrella trials the main measured outcomes are surrogate endpoints such as: progression-free survival (PFS), time to progression (TTP), tumor shrinkage, the percentage of patients responding to a drug or biomarkers that can predict clinical outcomes like survival. Surrogate endpoints substitute clinically meaningful endpoints and they are used to indicate whether treatment works. The use of surrogate endpoints in certain phases of research is justified [2, 24, 26] and their advantage is that they can yield information about the effect of a drug more rapidly than long-term clinical outcomes. Unfortunately, there is recently mounting evidence illustrating that surrogate endpoints do not necessarily translate to patient-centered outcomes [24, 27,28,29]. This also means that the direct benefit achieved by participants of umbrella and basket trials is arguable.
Other data suggest that the minority of patients who have been treated with genome-driven therapy benefited to date [30]. However, other findings show that overall response rate in all published basket trials in cancer medicine until March 2018 was 25%, which seems very promising [31]. But published results of NCI-MATCH and Lung-MAP trials are not equally promising. In NCI-MATCH, all three arms (Table 1) failed to meet their primary endpoint (25% or more of the patients whose tumors have a complete or partial response to treatment). Nevertheless, there were 17% or more patients with prolonged stable disease. Similarly, in the Lung-MAP study (Table 2),v all of the five subprotocols failed to meet their primary endpoint (25% or more of the patients whose tumors have a complete or partial response to treatment). Surprisingly, participants in the non-match sub-study benefitted more than the ones who matched, which may indicate that the treatment adjusted to a genetic change is less effective than the treatment that does not match.
The third type of benefit – “collateral” benefit, refers to benefit arising from being a research subject, like free medical care or the personal gratification of altruism [25]. This kind of benefit is easily discernible in basket and umbrella trials as thousands of patients may be screened and find out more about their disease and take part in the research process. However, not every patient can be screened as some tumors are too small to be collected for research purposes [32]. Even if they are eligible to be screened, the genetic profiling may fail to detect any actionable mutation. If there is an alteration, sometimes there is a lack of standard treatment or ongoing trial to offer. For example, in the Lung-MAP trial every patient can be treated, because a “non-match” study exists, unlike in the NCI-MATCH study, in which patients without actionable mutations are out of the study. In the NCI-MATCH trial 5963 tumors were screened for 30 treatment arms from opening in 2015 thru 16th July 2017 and in 5546 (93%) the assay was successful. Among them, 998 patients were assigned to matched interventions, but only 689 (69%) enrolled in all treatment arms, which gives 12% of all screened patients who finally started the therapy [22].
Despite the fact that basket and umbrella trials allow for recruitment of patients with rare malignancies, which can be considered as their main advantage, it is unclear why only some provide a “default arm” and some do not. It may be worth to explore various rare tumors and enable patients with no standard treatment options to be part of the research and contribute to gaining knowledge that can be used to develop new therapies.
There are many different types of risk or harm to research subjects, such as: physical, psychological, economic, legal or dignitary [33]. The first two types - physical and psychological, may appear at different stages during the research process of basket and umbrella trials. In these studies, some invasive procedures e.g., biopsy or surgical resection are required to collect an adequate tumor sample for evaluation via genetic profiling. Patients only take part in this research process if it is feasible and supposedly safe to obtain tumor material for molecular and genomic studies [32]. A tumor biopsy may be considered safe, but it can be stressful and uncomfortable for volunteers [34]. There is always a possibility of complications, especially when dealing with patients who progressed after chemotherapy and their organisms are weak. In turn, Overman et al. [35] raise issues with underreporting of results derived from research biopsies and provide recommendations to improve such reporting. Biopsy findings are particularly important in basket and umbrella trial designs as they are crucial to include or exclude a large number of potential participants from further steps of the study and they are supposed to generate knowledge and justify the risk.
While large volumes of tumor tissue are required for assessment and optimal diagnosis, sometimes depending on the localization of the lesion, it is challenging or even impossible to obtain a sufficient amount of tissue. Moreover, a biopsy material may be of low quality and/or not accurately capture the complete genomic landscape of the patient’s cancer, so only a limited geographical region of tumor is analyzed [21]. These both may result in mismatching and assigning the patient to inappropriate study arm and therefore, pose harm. Allocation to a substandard treatment either after a biomarker test result or after randomization is the most serious problem in basket and umbrella trials. They are designed to test one targeted treatment against one specific abnormality found in the patient’s tumor but there are instances where more than one genetic change is diagnosed. Such precarious single-target treatment may be insufficient and not match the entire heterogeneous tumor, resulting in disease progression. In this case, the intervention that targets the whole tumor may be more beneficial to the participant.
Consider the case of two patients enrolled to e.g., S1400B sub-study (Table 2): one with a single genetic defect, and the other with multiple genetic changes clearly identified, who are randomized to receive either chemotherapy or experimental intervention. If the first one is allocated to the chemotherapy arm and the second to the experimental arm it may not only be disadvantageous for them to obtain direct therapeutic benefit but also for acquiring reliable and reproducible results.
The process of screening and genome sequencing must be efficient and quick. From the patient’s perspective, the screening delay means being untreated for 2 weeks while awaiting results. For some oncology patients, 14 or 16 days is not many, for others every single day may be crucial and full of hope, stress and uncertainty, whereas some may even not survive to be enrolled.
Other risks in basket and umbrella trials are the same as in other areas of medicine. For example, there is a problem how to cope with any incidental findings found during genetic sequencing, which contain important health or reproductive information about participants [32]. Moreover, the recruitment of thousands of participants generates a huge amount of data that must not only be rapidly processed, but also reliably and safely stored, so that undesirable people have no access to it.
The ethics of research with human beings demands that patient-subject burdens are redeemed by gains in generalizable knowledge. Taking into consideration both risks and benefits, we claim that there is a low chance of direct therapeutic benefit to participants due to the major flaw in the design of these trials, which include patients with multiple genetic aberrations and test treatments against a single aberration, which can harm patients and may fail in gaining reliable findings. If this flaw is reduced by including patients with only one actionable mutation, then the risks can be justifiable.
Informed consent
Informed consent is the cornerstone ethical principle of biomedical research [19] and it can be challenged in oncology clinical trials, including basket and umbrella trials. The perception of patients with life-threatening diseases is often affected by a desperate hope for the therapeutic benefit. They may understand the nature of the research process, but sometimes a clinical trial is their last hope and the last chance for any therapeutic benefit. This is where therapeutic misunderstanding can appear [36].
One such misunderstanding, the main ethical concern in cancer research, is a therapeutic misconception defined as “the belief that the purpose of a clinical trial is to benefit the individual patient rather than to gather data for the purpose of contributing to scientific knowledge” [37]. This tendency is common in all types of trials and can affect both patients willing to participate in a trial, and investigators/physicians who feel a therapeutic duty to deliver the best medical care to patients.
In basket and umbrella trial designs patients are divided into two groups – those who “match” and “not match” the experimental treatment. In the NCI-MATCH study, only the group that “matches” remains in the trial and receives intervention and the rest are excluded from the trial, while in the Lung-MAP study both those who match and do not match are treated. We think that the group that “matches” may have overestimated expectations and personal conviction that intervention is directly adjusted to each individual, although all subjects are treated according to the protocol. The same goes for another form of misunderstanding called “therapeutic misestimation”, which is described as “misunderstanding the probability of direct benefit or harms that may result from participating in research” [38] - subjects that “match” may overestimate potential chances to receive benefit and underestimate the risks, because they are “the chosen ones”. The problem of such misunderstandings can be intensified in trials with “non-match” sub-study without blinding of participants where everyone knows to which sub-trial they were assigned. In these trials, subjects in arms that “matched” may be 100% confident that they receive the treatment that best matches their disease and they will surely benefit, while patients in “non-match” may believe that the therapy they receive does not work at all, which can pose problems with reporting adverse events and assessing therapy efficacy.
What can even deepen therapeutic misunderstandings among those who want to participate in novel precision medicine trials is the excessive use of phrases like “personalized” or “individualized medicine”, and “tailored” therapy. Such terminology can be misleading, and falsely indicate that the trial’s goal is to provide personalized care with regard to the patient’s best interest standard and direct therapeutic benefit. We propose to avoid such phrases in master protocols and use more generalizable ones, such as “therapy based on genomics”.
We suggest that the problem of misunderstandings in precision medicine trials should be evaluated and reduced in view of giving valid informed consent. Without fully understanding the purpose of the trial as well as the real consequences of participation, a subject’s informed decision to take part in research is ethically challenged. Additionally, patients should be provided with information about the prospect of benefitting from the screening process while considering to enter the study. They may not be aware that after the screening process they may not be enrolled into the study, but they may have the opportunity to undergo another already approved treatment. The exclusion from being a trial participant does mean that they cannot be treated after all. Still, it is unclear how many patients in basket and umbrella trials do not enter the study because there are better treatment options for them and how many of them remain untreated due to the lack of any existing therapy.
