General considerations about adaptive trials

Conduct

Most considerations discussed in this section should also be considered by researchers at the planning and design stage of an adaptive clinical trial as they influence planning and some may need to be stipulated in the protocol.

Measures to minimise operational bias

Operational bias may arise when investigators use emerging data to modify their conduct of the trial or decision making in a way that was not previously approved in the trial design process. Such ad hoc changes may call into question the integrity and reliability of the trial’s conclusions. It is common for researchers to evaluate interventions that they have some links to – be it financial, emotional or otherwise – and whilst this does not itself invalidate the study any consumer of research findings will inevitably want assurance that the study was conducted as objectively as possible. Researchers should ensure consistency in the conduct of the trial throughout (before and after trial adaptation decisions).

Bias can arise in many types of study far beyond adaptive designs. Adaptive designs are particularly prone to the problem and careful pre-emptive measures need to be taken in order to reduce both the actual impact and the perceived impact of operational bias.

Adaptive designs can raise additional concerns brought by reviewing interim outcome data to inform trial adaptations. Access to interim data can reveal the treatments participants have been allocated to and/or the emerging treatment effect and so members of the trial team should be blinded to such information 7. This will help to avoid introducing bias and inconsistencies in the conduct of the trial. For example, knowledge of which patients are receiving an active treatment or a placebo may influence clinicians or trialists to consciously manage patients or assess patient outcomes inconsistently across treatment arms (see 1 for more discussion with examples). As a result, the reliability of the results can be jeopardised. To address this problem, adaptive designs require additional safeguards to ensure that the interim analyses and decision-making processes do not introduce operational biases that could diminish the credibility of trial results. This includes clarity on:

  • who sees unblinded data;
  •  who performs interim analyses;
  •  who reviews interim results and make trial adaptation recommendations;
  •  how interim data are transferred to relevant parties;
  •  how interim results and trial adaptation recommendations are communicated and to whom;
  •  the roles of the sponsor/funder and;
  •  how different parties involved interact with each other.
Importantly, these procedures and processes should be documented to enable readers of trial reports to judge sources of potential biases that could influence the interpretation and credibility of results.

Although there is no universal model that is applicable across all adaptive clinical trials, it is advisable to have an independent body for reviewing interim results and making recommendations to adapt the trial. An adaptation committee (ideally consisting of experts in adaptive designs and relevant statistical methods) or an independent data monitoring committee can assume this responsibility (see 2, 3, 4). This will ensure that those involved with the day-to-day running of the trial will not have access to unblinded data and results. An additional firewall should be considered to ensure that those tasked with performing interim analyses are independent of the trial conduct. Interim decision and communication models that can be adopted by researchers have been suggested in the literature (e.g., 2, 3, 5).

References

1. Fleming et al. Maintaining confidentiality of interim data to enhance trial integrity and credibility. Clin Trials. 2008;5(2):157-167.
2. Sanchez-Kam et al. A practical guide to data monitoring committees in adaptive trials. Ther Innov Regul Sci. 2014;48(3):316-326.
3. Chow et al. On the independence of data monitoring committee in adaptive design clinical trials. J Biopharm Stat. 2012;22(4):853-867.
4. Bhattacharyya et al. The changing landscape of data monitoring committees—Perspectives from regulators, members, and sponsors. Biometrical J. 2019;61:1232–41.
5. Gallo P. Confidentiality and trial integrity issues for adaptive designs. Drug Inf J. 2006;40(4):445-449.
6. Iflaifel et al. Blinding of study statisticians in clinical trials: a qualitative study in UK clinical trials units. Trials. 2022;23(1):1–11.

Data management considerations for adaptive trials

Adaptive designs need trial data before the trial ends to decide how it should proceed. Whilst this can provide a more efficient trial, it is imperative that the required interim data are accurate for reliable interim decisions and are available in a timely manner, which in turn needs appropriate resourcing. In addition, data or information should be accessed or shared in a controlled manner to avoid introducing operational bias in the conduct of the trial. Some key data management considerations are summarised here (see 1, 2).

1. Awareness of the trial adaptations and implications on database features 

It is essential to make the data management team aware of the potential trial adaptations, their implications on the conduct of the trial, and what is expected from the data management team. This includes how the potential outcomes of the interim decisions will affect the trial’s data management processes, case report forms, and any specific areas or features of the trial database. For instance, the decision to cease recruitment to a treatment arm for futility requires unblinded outcome data; likewise, adding or removing arms requires amending both data capture forms and the database. For some adaptive designs (such as adaptive population enrichment), case report forms must be updated or certain data fields blocked in the database if an interim decision is made to stop further recruitment of patients with certain characteristics, in response to a pre-planned change in eligibility criteria.

2. Key data for interim analyses and how they are captured

The trial manager and data management team should be aware of the data that are required for each interim analysis, where those data are obtained, and how they are recorded in the database. This will enable them to chase up missing data (e.g., from sites) or seek relevant approvals prior to the interim analysis. If some of the information is captured outside the database, there needs to be a plan for how it will be incorporated into the database or linked to other patient records, whether data will be timely obtained and whether access to these data will be timely granted by relevant external parties for adaptations to be smoothly implemented.

3. Real-time data capture and cleaning

Once we know the data needed for interim analyses (which is decided at the design stage) and how they are captured in the database, priority should be given to ensure that these data are of suitable quality (i.e., no missing values, or reasons given as to why data are missing), and cleaned in a timely manner (to ensure they are accurately recorded) ready for analysis. This will ensure reliable interim decisions based on good quality data. The objective is to avoid a situation where decisions are made based on poor quality data but these decisions become inconsistent with the data once errors are corrected after the adaptations (e.g., a treatment arm could be incorrectly dropped as futile due to poor interim data quality).

Sometimes additional data (e.g., adverse events) may be required to aid interim decisions even though the adaptation method may not formally require it (e.g., using serious adverse events data to accompany recommendations for whether a treatment arm should be dropped or not). The inclusion of additional data in these analyses should ideally be stated in advance and documented (e.g., in the protocol and/or statistical analysis plan) and discussed upfront with involved parties, including the trial management team and the independent data monitoring committee or adaptation committee.

4. Actions required during interim analyses and the decision-making process

Whether recruitment is paused or continued as normal during interim analyses until interim decisions are made should be decided upfront as this affects some features of the database and randomisation system during this period. For instance, some recruitment-related features of the database or randomisation system should be temporarily revoked during this period if recruitment is paused. This will involve contacting recruitment sites to pause recruitment until they are notified to resume and letting them know which features of the database or randomisation system have been temporarily blocked out.

5. Confidentiality – data access and communication

In general, access to trial data with interim decisions should only be allowed to authorised parties. That is, those blinded to treatment allocation and/or interim results should remain so throughout the trial. A clear plan should be written detailing what interim data should be transferred or shared by who and to whom; when this should be done by; and how this process will be audited. This also includes trial-related information that should be communicated to and by different parties during and after the interim analyses in line with their roles and responsibilities. For example, if an interim decision is made to stop further recruitment of patients with certain characteristics who were previously eligible, then those involved in patient recruitment at sites should be notified.

6. Mapping tasks and milestones

For each task that contributes to the interim analyses, interim decision-making process, and communication of appropriate decisions, there should be clear timelines and milestones for the data management team to follow. Other stakeholders may need to be engaged during specific tasks (e.g., trial statisticians). In summary, the following should be clear:

  • what is needed by each stakeholder to perform their duties and by when;
  • who should perform the tasks to provide the information needed by each stakeholder and what they need to achieve these tasks;
  • what should be communicated to each party, by when, and who is the contact person.

7. Documentation of interim analyses data

The data management team and the trial statistician should document the data used for each interim analysis for the purpose of reproducing the analysis and for audit trail.

References

1. Cytel. Adaptive designs: A data management perspective. 2019. https://www.cytel.com/blog/adaptive-designs-data-management-perspective. Date accessed, December 9, 2019.
2. He et al . Practical considerations for adaptive trial design and implementation. Springer. 2014. 

Considerations when writing an adaptive design protocol

There are general resources, such as the SPIRIT guidance, to help researchers write a high-quality trial protocol. Specific to adaptive designs, SPIRIT item 21b covers aspects of interim analyses, stopping rules, access to interim results, and interim decision-making. However, there are additional aspects that should be considered when writing a trial protocol for an adaptive design and researchers may use other guidelines on adaptive designs to assist them. These include FDA regulatory guidance 1, the Adaptive designs CONSORT Extension (ACE) 2, 3, adaptive protocol guidance in early phase trials 4, and other recommendations 5. Here, we summarise the most important aspects from these documents, as well as these additional aspects that should be considered when developing an adaptive trial protocol, and these points apply to the entire protocol and not just the trial design and statistical sections.

  1. Describe the adaptive design clearly, including full details of trial adaptations which may be made. A diagram may be helpful especially for complex adaptive designs;
  2. Describe how different adaptation decisions at interim analyses, such as dropping treatment arms or changing eligibility criteria, affect research governance and trial management, and how patients are clinically managed after interim decisions. For example, what will be the end of the study for patients recruited into dropped arms? Will patient information sheets be updated following trial adaptations?
  3. Describe all outcomes used to inform the trial adaptations (adaptation outcomes) including what they are, how they will be assessed, and when they will be assessed. These adaptation outcomes can be different from the primary outcome or in some cases, a combination of both can be used;
  4. Describe the planned decision rules that will lead to changes being made and when interim analyses will be performed to inform whether these changes should be made (timing and frequency of interim analyses). Researchers should think carefully about the level of information they should disclose in the protocol while the trial is ongoing. If certain information is deemed confidential to minimise potential operational bias, then this should remain confidential during the trial and not be disclosed until results are disseminated at the end. Such information may include the exact calculation of certain statistical quantities to inform trial adaptations;
  5. Describe how the sample sizes were determined and the operating characteristics or statistical properties of the adaptive design considered. It is sufficient to reference published statistical methods and software package(s) or program(s) used. If these were evaluated via statistical simulation 6, a summary of the results including simulation parameters used and decision-making probabilities should be provided 5. Ideally, expanded details should be included in a standalone simulation protocol and report;
  6. Address the feasibility/logistic aspects of implementing the adaptive design and implications on other aspects such as intervention delivery and data management;
  7. Address any ethical implications as a result of the trial adaptations which may be made (see point ‎2 above); 
  8. Describe measures put in place to maintain confidentiality and to minimise the potential introduction of operational bias focusing on (see general considerations):
    • safeguards to ensure that those that are blinded at the beginning of the trial will remain so,
    • who will perform interim analyses and how will data be shared?
    • who will have access to interim results?
    • who will review interim results and recommend trial adaptations and how will recommendations be communicated and to who?
    • who will make decisions on the recommended trial adaptations?

  9. Detail any potential risks posed by the adaptive design and how these will be managed to preserve the credibility and validity of results; 
  10. Summarise key statistical methods that will be used at interim and final analyses. This should also include how treatment effects will be estimated at the end of the trial (see Analysis). Detailed statistical methods are expected to be in the interim or final statistical analysis plan;
  11. Summarise key methods that will be used for evaluating other trial objectives that can be impacted by trial adaptations such as health economics analyses (see 7);
  12. Consider specific standard operating procedures to deal with specific operational aspects of running an adaptive trial.  For example, detailed processes and procedures to be followed by different parties involved in interim analyses and adaptation decisions to maintain confidentiality of interim data and results.
PANDA users may wish to read recommendations or considerations on master protocols and sub-protocols 8 that can be adopted for some complex adaptive trials such as in adaptive platform trials  5,9. These adaptive trials allow the addition of new study treatment arms or subpopulations to an ongoing trial. The literature discusses specific practical considerations when running adaptive platform trials 10, 11.

References

1. FDA. Adaptive designs for clinical trials of drugs and biologics guidance for industry. 2019.
2. Dimairo et al. The Adaptive designs CONSORT Extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design. BMJ. 2020;369:m115.
3. Dimairo et al. The adaptive designs CONSORT extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design. Trials. 2020;21(1):528.
4. Lorch et al. Three steps to writing adaptive study protocols in the early phase clinical development of new medicines. BMC Med Res Methodol. 2014;14(84):1-9.
5. The Adaptive Platform Trials Coalition. Adaptive platform trials: definition, design, conduct and reporting considerations. Nat Rev Drug Discov. 2019;18:797–807.
6. Mayer et al. Simulation practices for adaptive trial designs in drug and device development. Stat Biopharm Res. 2019;11(4):325-335.
7. Flight. The use of health economics in the design and analysis of adaptive clinical trials. Thesis. 2021.
8. Meyer et al. The evolution of master protocol clinical trial designs: A systematic literature review. Clin Ther. 2020;42(7):1330-1360. doi:10.1016/j.clinthera.2020.05.010
9. FDA. Master protocols: Efficient clinical trial design strategies to expedite development of oncology drugs and biologics. 2018.
10. Schiavone et al. This is a platform alteration: A trial management perspective on the operational aspects of adaptive and platform and umbrella protocols. Trials. 2019;20(1):264.
11. Hague et al. Changing platforms without stopping the train: experiences of data management and data management systems when adapting platform protocols by adding and closing comparisons. Trials. 2019;20(1):294.