1) What are the costs of stopping early? 

2) Interpreting studies which stop early 

• seasonality: is the trial period done in a time frame which is favourable or unfavourable to the effect of the treatment? Would it work as well had it covered extended periods (e.g., both cold and warm seasons)? 
• learning effect: did the trial end for futility before the trial team were fully versed in its delivery? 
• follow-up duration: are the outcomes measured over a period long enough to convince readers of the value of the findings? 
• baseline imbalances: are baseline prognostic factors imbalanced at interim analyses, especially when interim sample sizes are small and what are the potential impact on the results? 

3) Will the interim data be ready in time? 

4) Are interim data sufficient to inform early stopping? 

In context

References

• the criteria researchers wish to stop the trial early for (e.g., futility only, efficacy only, either futility or efficacy, non-inferiority, etc);
• the appropriate stopping rules for making early trial stopping decisions;
• measures of treatment effect (and when this is observed is important to inform early stopping); 
• how many times interim data will be analysed (frequency of interim analyses);
• when will interim data be analysed (timing of interim analyses).

1. MAPAC trial ⁴ 

2. REBOA trial ^8,9

References

Key message

References

References

References

References

RATPAC ³ retrospective trial 

1) Design

• futility – where the point of care shows minimal or no difference on interim analysis, 
• efficacy – overwhelming effect of point of care. 

References

Code
# fixed sample size design without continuity correction is approx. 3130 (1565 per arm)

R code: 
nfx<-nBinomial(p1=0.50, p2=0.55, alpha = 0.025, beta = 0.2, delta0 = 0, scale = "Difference") 
nfx

Stata code: 
sampsi 0.50 0.55, power(0.8) nocont alpha(0.05)
  

R code using gsDesign()

#install and load package 
install.packages(“gsDesign”) 
library(gsDesign)

#fixed sample size  
nfx<-nBinomial(p1=0.5, p2=0.55, alpha = 0.025, beta = 0.2, delta0 = 0, scale = "Difference")

# set a vector indicating  when interim analysis will happen relative to maximum sample size (information fraction) 
analysis_time<-c(0.5, 0.70, 1)

# derive the group sequential design 
design1<-gsDesign(k=3, test.type=4, alpha=0.025, beta=0.2, astar=0,
         timing=analysis_time, sfu=sfHSD, sfupar=c(-7), sfl=sfHSD, sflpar=c(-2), 
         tol=0.000001, r=80, endpoint="binomial", n.fix = nfx, delta1=-0.05, delta0=0, 
         delta = 0, overrun=0 
         ) 

# a summary note of the design and details 
summary(design1) 
design1

# a summary of efficacy and futility boundaries at each analysis on z, p-value and difference in proportion scales 
# note parameterisation of treatment effect in the results is soc-poc rather than poc-soc  
bsm<-gsBoundSummary(design1, digits = 4) 
bsm

# a function to calculate repeated confidence intervals 
gsBinRCI<-function(d, x1, x2, n1, n2){ 
        y<-NULL   
        rname<-NULL
        nanal<-length(x1)
        for (i in 1:nanal) { 
               y<-c(y, ciBinomial(x1 = x1[i], x2 = x2[i], n1=n1[i], n2 = n2[i], alpha = 2*pnorm(-d$upper$bound[i])))     
               rname<-c(rname, paste("Interim analysis", i))   
              }   
        ci<-matrix(y, nrow = nanal, ncol = 2, byrow = T) 
        rownames(ci)<-rname  
        colnames(ci)<-c("LCI", "UCI")
        ci 
       } 

# observed data and events at the first interim analysis (n=1659) 
n1<-c(833) 
n2<-c(826) 
events1 <-c(254) 
events2 <-c(119)
  
# compute repeated intervals using the above function 
rci<-gsBinRCI(design1, events1, events2, n1, n2) 
rci
 
# observed events and MLE of the difference in rates 
p1<-round(events1/n1, 3) 
p2<-round(events2/n2, 3) 
propdif<- round(events1/n1 - events2/n2, 3) 
p1 
p2 
propdif
  

R code using rpact()

# installing and loading package 
install.packages(“rpact”) 
library(rpact) 

# set up the group sequential design 
ratdesign<-getDesignGroupSequential( 
    kMax = 3, alpha = 0.025, 
    beta = 0.2, sided = 1, informationRates = c(0.5, 0.7, 1), 
    typeOfDesign = "asHSD", gammaA = -7, typeBetaSpending = "bsHSD", gammaB = -2, 
    bindingFutility = FALSE, twoSidedPower = FALSE 
  ) 
ratdesign 

# get the sample sizes of the above design we set up (at each interim and maximum) 
ratpac1<-getSampleSizeRates( 
    design=ratdesign, groups = 2, thetaH0 = 0.00, pi1=0.55, pi2=0.5, 
    normalApproximation = TRUE 
  ) 
summary(ratpac1) 

# first interim analysis when 1659 patients were recruited 
# input data and observed events in each arm 

ratpacdata<-getDataset( 
  n1 = c(833), n2 = c(826), 
  events1 = c(254), events2 = c(119) 
  ) 

interim1res<-getAnalysisResults(ratdesign, ratpacdata) 
interim1res 

# action to take given the results and corresponding stopping boundaries 
getTestActions(ratdesign, stageresults) 

# produce the final results 
getFinalConfidenceInterval(ratdesign, ratpacdata) 

# get stage results first, and final overall p-value 
stageresults<-getStageResults(ratdesign, ratpacdata, stage=1) 
stageresults 
getFinalPValue(ratdesign, stageresults) 

2) Stopping boundaries 

 3) Performance of the design

• are the chances of stopping for futility reasonable or meet our expectations when we know that the study treatment is futile? 
• is the evidence convincing enough to justify early stopping? 
• are the chances of stopping for efficacy when we know the effect is overwhelmingly good enough for the design to be worthwhile? 
• do the benefits of the design justify its use against additional logistical and practical challenges? 

4) Impact and potential benefits 

5) First interim analysis results 

References

References

References