Skip to contents

Fixed effect estimates and prior distributions

Johannes Cepicka

Source: vignettes/Fixed_and_prior_distributions.Rmd
Fixed_and_prior_distributions.Rmd

Suppose we are planning a drug development program testing the superiority of an experimental treatment over a control treatment. Our drug development program consists of an exploratory phase II trial which is, in case of promising results, followed by a confirmatory phase III trial.

The drugdevelopR package enables us to optimally plan such programs using a utility-maximizing approach. Up until now, we presented a very basic example on how the package works in Introduction to planning phase II and phase III trials with drugdevelopR. In this article, we want to expand the basic setting and want to introduce you to the modelling of the assumed treatment effect on a prior distribution.

The example setting

We are in the same setting as in the introduction, i.e. we suppose we are developing a new tumor treatment, exper. The patient variable that we want to investigate is the difference in tumor width between the one-year visit and baseline. This is a normally distributed outcome variable.

The parameters we insert into the function optimum_normal are the same parameters we also inserted in the basic setting. However this time, we set fixed = "FALSE", hence the assumed true treatment effect is not fixed but follows a prior distribution. Again, we start by loading the drugdevelopR package.

library(drugdevelopR)
#> Loading required package: doParallel
#> Loading required package: foreach
#> Loading required package: iterators
#> Loading required package: parallel

Parameters of the prior distribution

Additionally to the parameters in the baseline scenario with fixed treatment effects, we now need further input parameters:

  • Differing from the baseline scenario we now have not only one treatment effect, but two: Delta1 and Delta2. The input parameter Delta1 is the one we got from some randomized controlled pilot trial that our team conducted earlier. Its value is given as the standardized difference in means (Δ=μcontro−μexperσ\Delta=\frac{\mu_{contro} - \mu_{exper}}{\sigma}) and its value was determined to be 0.625. However, we are not so sure about this result, as another research group conducted a similar study and got an treatment effect of 0.9, which will now be our value for Delta2. Of course, the choice of Δ1\Delta_1 and Δ2\Delta_2 need not be built on two clinical studies, but can also be derived from different sources for forming a prior belief, e.g. clinical experience.
  • We now have to specify how sure we are about the two prior beliefs Δ1\Delta_1 and Δ2\Delta_2. This is done by the two additional parameters in1 and in2. We call these parameters the “amount of information”. They refer to the sample sizes of the studies on which we base our prior beliefs. If our pilot study was conducted with 300 participants, the value for in1 is set to be 300. Let’s assume that the study of the other research group was conducted with 600 participants, so the parameter value for in2 is 600. The higher amount of information, the lower the variance we attribute to that prior belief.
  • Now, a weight parameter w has to be defined, that allows us to weigh the two treatment effects. If we want to trust our results more, than we can set a higher parameter value for w. (Note that w has to be between 0 and 1, a parameter value of 1 would put all the weight on our results and none on the results of the second study). If we think the results of the other group are more reliable we reduce the value for w, thus putting more weight on Delta2. Note that by exchanging the values of Delta1 and Delta2 (and the corresponding values for in1 and in2) and setting wnew=1−woldw_{new} = 1 - w_{old} our final results will not change. In our case, we want to put more trust on our results and thus set the parameter w to be 0.6. Setting the weight to 11 would effectively mean ignoring the second treatment effect, which is also possible.
  • The prior distribution used in the package is the sum of two truncated normal distributions. Hence, we need truncation values a as the lower boundary for the truncation and b as the upper boundary. In our case we set a = 0.25 and b = 0.75.

The prior distribution for the standardized true difference in means is then given by Δ∼w·𝒩[0.25,0.75]t(0.625,4/300)+(1−w)·𝒩[0.25,0.75]t(0.9,4/600)\Delta ∼ w · \mathcal{N}^t_{[0.25,0.75]} (0.625, 4/300) + (1 − w) · \mathcal{N}^t_{[0.25,0.75]} (0.9, 4/600) where N[a,b]t(μ,σ2)N^t_{[a,b]} (\mu, \sigma^2) denotes the truncated normal distribution with mean μ\mu, variance σ2\sigma^2, truncated below at a and above at b. To see how different input values change the prior distribution we refer to the Shiny app.

 res <- optimal_normal(Delta1 = 0.625, Delta2 = 0.8, fixed = FALSE, # treatment effect
                       n2min = 20, n2max = 400, # sample size region
                       stepn2 = 4, # sample size step size
                       kappamin = 0.02, kappamax = 0.2, # threshold region
                       stepkappa = 0.02, # threshold step size
                       c2 = 0.675, c3 = 0.72, # maximal total trial costs
                       c02 = 15, c03 = 20, # maximal per-patient costs
                       b1 = 3000, b2 = 8000, b3 = 10000, # gains for patients
                       alpha = 0.025, # significance level
                       beta = 0.1, # 1 - power
                       w = 0.6, in1 = 300, in2 = 600, #weight and amount of information
                       a = 0.25, b = 0.75) #truncation values

Interpreting the output

After setting all these input parameters and running the function, let’s take a look at the output of the program.

res
#> Optimization result:
#>  Utility: 3147.32
#>  Sample size:
#>    phase II: 80, phase III: 188, total: 268
#>  Probability to go to phase III: 0.99
#>  Total cost:
#>    phase II: 69, phase III: 155, cost constraint: Inf
#>  Fixed cost:
#>    phase II: 15, phase III: 20
#>  Variable cost per patient:
#>    phase II: 0.675, phase III: 0.72
#>  Effect size categories (expected gains):
#>   small: 0 (3000), medium: 0.5 (8000), large: 0.8 (10000)
#>  Success probability: 0.85
#>  Joint probability of success and observed effect of size ... in phase III:
#>    small: 0.68, medium: 0.16, large: 0.01
#>  Significance level: 0.025
#>  Targeted power: 0.9
#>  Decision rule threshold: 0.06 [Kappa] 
#>  Parameters of the prior distribution: 
#>    Delta1: 0.625, Delta2: 0.9, in1: 300, in2: 600,
#>    a: 0.25, b: 0.75, w: 0.6
#>  Treatment effect offset between phase II and III: 0 [gamma]

The program returns a total of thirteen values and the input values. Once again, we will only focus at the most important ones:

  • res$n2 is the optimal sample size for phase II and res$n3 the resulting sample size for phase III. We see that the optimal scenario requires 80 participants in phase II and 188 participants in phase III. The number of participants in both phases was reduced compared to the setting with a fixed treatment effect.
  • res$Kappa is the optimal threshold value for the go/no-go decision rule. We see that we need a treatment effect of more than 0.06 in phase II in order to proceed to phase III, which is the same as in the baseline scenario.
  • res$u is the expected utility of the program for the optimal sample size and threshold value. In our case it amounts to 3147.32, i.e. we have an expected utility of 314 732 000$. This is a 6.83% increase over the scenario without the prior distribution. The increase in the expected utility can be attributed to the additional weight which was put on the second treatment effect Delta2, which was more promising than our treatment effect.

Of course, the differences in the output values compared to the fixed setting heavily depend on the choice of the prior.

Prior distributions for time-to-event outcomes

Note that in the setting of time-to-event outcomes, the following input parameters have to be specified, which differ from the setting with normally distributed outcomes:

  • Instead of the parameters in1 and in2 for the “amount of information”, we have to use the parameters id1 and id2 which represent the “number of events”. They refer to the number of events which were observed in the study to determine the treatment effect. If in our study 210 events could be observed, then the value for id1 is set to be 210. If we assume assume that in the study of the other research group 420 events could be observed, the parameter value for id2 is 420.
  • Moreover, we do not need truncation values, i.e. values for the parameters a and b.

Where to go from here

This tutorial explains how to use the parameters needed for the prior distribution when setting the parameter fixed to be "FALSE".

For more information on how to use the package, see: