More parameters to adapt to specific settings
Johannes Cepicka
Source:vignettes/More_Parameters.Rmd
More_Parameters.Rmd
Suppose we are planning a drug development program testing the superiority of an experimental treatment over a control treatment. Our drug development program consists of an exploratory phase II trial which is, in case of promising results, followed by a confirmatory phase III trial.
The drugdevelopR package enables us to optimally plan such programs using a utility-maximizing approach. To get a brief introduction, we presented a very basic example on how the package works in Introduction to planning phase II and phase III trials with drugdevelopR. However, there are many more parameters, which you can provide to adapt the program to your specific needs. In this article, we therefore want to present the options you have.
The example setting and the input parameters
We are in the same setting as in the introduction, i.e. we suppose we are developing a new tumor treatment, exper. The patient variable that we want to investigate is the difference in tumor width between the one-year visit and baseline. This is a normally distributed outcome variable.
The parameters we insert into the function
optimum_normal
are the same parameters we also inserted in
the basic setting. Thus, for more information on the input values we
refer to the introduction.
We start by loading the drugdevelopR package.
library(drugdevelopR)
#> Loading required package: doParallel
#> Loading required package: foreach
#> Loading required package: iterators
#> Loading required package: parallel
Back then, we got the following output:
res
#> Optimization result:
#> Utility: 2946.07
#> Sample size:
#> phase II: 92, phase III: 192, total: 284
#> Probability to go to phase III: 1
#> Total cost:
#> phase II: 77, phase III: 158, cost constraint: Inf
#> Fixed cost:
#> phase II: 15, phase III: 20
#> Variable cost per patient:
#> phase II: 0.675, phase III: 0.72
#> Effect size categories (expected gains):
#> small: 0 (3000), medium: 0.5 (8000), large: 0.8 (10000)
#> Success probability: 0.85
#> Success probability by effect size:
#> small: 0.72, medium: 0.12, large: 0
#> Significance level: 0.025
#> Targeted power: 0.9
#> Decision rule threshold: 0.06 [Kappa]
#> Assumed true effect: 0.625 [Delta]
#> Treatment effect offset between phase II and III: 0 [gamma]
We now want to see how different modifications, i.e. using more parameters change the outcome of the program.
Fixed treatment effect estimates or prior distributions
Setting the parameter fixed
to be FALSE
models the assumed true treatment effect using a prior distribution.
This is explained in detail in the article Fixed
effect estimates and prior distributions.
Constrained optimization
The are three options how to include constrained optimization, by setting maximum cost limits, by setting a maximum number of participants or by demanding a minimum probability of a successful program. These options can be applied separately or combined, as it is best suited to your specific needs.
Cost constraint – maximum total costs
The total costs are given as sum of the cost in phase II and phase
III, i.e. as \(K2\) + \(K3\). In our example we get total costs of
\(72+139=211\) (in 10^5 $). Let’s
assume an investor in your drug or a pharmaceutical company is only
willing to pay 20,000,000 $. You can now set a maximum cost constraint
by providing an parameter value of K = 200
into the
optimum_normal
function. The default value for
K
is Inf
.
resK <- optimal_normal(Delta1 = 0.625, fixed = TRUE, # treatment effect
n2min = 20, n2max = 400, # sample size region
stepn2 = 4, # sample size step size
kappamin = 0.02, kappamax = 0.2, # threshold region
stepkappa = 0.02, # threshold step size
c2 = 0.675, c3 = 0.72, # maximal total trial costs
c02 = 15, c03 = 20, # maximal per-patient costs
b1 = 3000, b2 = 8000, b3 = 10000, # gains for patients
alpha = 0.025, # one-sided significance level
beta = 0.1, # 1 - power
Delta2 = NULL, w = NULL, in1 = NULL, in2 = NULL,
a = NULL,b = NULL, # setting all unneeded parameters to NULL
K = 200 # cost constraint
)
We get the following results:
resK
#> Optimization result:
#> Utility: 2846.69
#> Sample size:
#> phase II: 76, phase III: 158, total: 234
#> Probability to go to phase III: 0.97
#> Total cost:
#> phase II: 66, phase III: 133, cost constraint: 200
#> Fixed cost:
#> phase II: 15, phase III: 20
#> Variable cost per patient:
#> phase II: 0.675, phase III: 0.72
#> Effect size categories (expected gains):
#> small: 0 (3000), medium: 0.5 (8000), large: 0.8 (10000)
#> Success probability: 0.82
#> Success probability by effect size:
#> small: 0.7, medium: 0.11, large: 0
#> Significance level: 0.025
#> Targeted power: 0.9
#> Decision rule threshold: 0.18 [Kappa]
#> Assumed true effect: 0.625 [Delta]
#> Treatment effect offset between phase II and III: 0 [gamma]
We see that due to the cost constraint the cost in phase II decreases from 77 to 66 and the cost in phase III decreases from 158 to 133, so overall the constraint is met. Moreover, the expected utility at the optimum decreases from 294 607 000 to 284 669 000 dollars.
Sample size constraint – maximum number of participants
Very similarly to the cost constraint, we can set a constraint on the
maximum number of participants in our trial. Without constraints, we
need a total number of 250 participants in the optimum. However, one can
certainly imagine a situation, where the only 200 persons willing to
participate in a trial, for example because the incidence of the illness
is not very high within the target population. To model this, we can
easily set a sample size constraint, in this case of
N = 200
. This allows for an optimal allocation of the
participants between phase II and III. The default value for
N
is Inf
.
resN <- optimal_normal(Delta1 = 0.625, fixed = TRUE, # treatment effect
n2min = 20, n2max = 400, # sample size region
stepn2 = 4, # sample size step size
kappamin = 0.02, kappamax = 0.2, # threshold region
stepkappa = 0.02, # threshold step size
c2 = 0.675, c3 = 0.72, # maximal total trial costs
c02 = 15, c03 = 20, # maximal per-patient costs
b1 = 3000, b2 = 8000, b3 = 10000, # gains for patients
alpha = 0.025, # significance level
beta = 0.1, # 1 - power
Delta2 = NULL, w = NULL, in1 = NULL, in2 = NULL,
a = NULL,b = NULL, # setting all unneeded parameters to NULL
N = 200 # sample size constraint
)
We get the following results:
resN
#> Optimization result:
#> Utility: 2658.9
#> Sample size:
#> phase II: 48, phase III: 150, total: 198
#> Probability to go to phase III: 0.93
#> Total cost:
#> phase II: 47, phase III: 127, cost constraint: Inf
#> Fixed cost:
#> phase II: 15, phase III: 20
#> Variable cost per patient:
#> phase II: 0.675, phase III: 0.72
#> Effect size categories (expected gains):
#> small: 0 (3000), medium: 0.5 (8000), large: 0.8 (10000)
#> Success probability: 0.76
#> Success probability by effect size:
#> small: 0.65, medium: 0.11, large: 0
#> Significance level: 0.025
#> Targeted power: 0.9
#> Decision rule threshold: 0.2 [Kappa]
#> Assumed true effect: 0.625 [Delta]
#> Treatment effect offset between phase II and III: 0 [gamma]
Due to our sample size constraint to optimal number of participants in phase II is 48 and 150 in phase III (down from 92 and 192 respectively) within this framework. The expected utility in the optimum decreases to 265 890 000$.
Success probability constraint - minimum probability of a successful program
The last option provided for constraint optimization is the minimum
probability of a successful program. In our basic setting without
constraints, we had a probability that the program was successful of
0.85, i.e. we had a probability of 0.85 that the program would proceed
from phase II to phase III and that a positive treatment effect could be
detected in phase III. Let’s say, you are willing to have higher costs
and in return want this probability to be 0.87. You can include this
within the framework, by including the parameter S
and by
setting it to S = 0.87
. The default value for
S
is -Inf
.
resS <- optimal_normal(Delta1 = 0.625, fixed = TRUE, # treatment effect
n2min = 20, n2max = 400, # sample size region
stepn2 = 4, # sample size step size
kappamin = 0.02, kappamax = 0.2, # threshold region
stepkappa = 0.02, # threshold step size
c2 = 0.675, c3 = 0.72, # maximal total trial costs
c02 = 15, c03 = 20, # maximal per-patient costs
b1 = 3000, b2 = 8000, b3 = 10000, # gains for patients
alpha = 0.025, # significance level
beta = 0.1, # 1 - power
Delta2 = NULL, w = NULL, in1 = NULL, in2 = NULL,
a = NULL,b = NULL, # setting all unneeded parameters to NULL
S = 0.87 #minimum success probability
)
We get the following results:
resS
#> Optimization result:
#> Utility: 2906.94
#> Sample size:
#> phase II: 172, phase III: 138, total: 310
#> Probability to go to phase III: 1
#> Total cost:
#> phase II: 131, phase III: 119, cost constraint: Inf
#> Fixed cost:
#> phase II: 15, phase III: 20
#> Variable cost per patient:
#> phase II: 0.675, phase III: 0.72
#> Effect size categories (expected gains):
#> small: 0 (3000), medium: 0.5 (8000), large: 0.8 (10000)
#> Success probability: 0.87
#> Success probability by effect size:
#> small: 0.76, medium: 0.11, large: 0
#> Significance level: 0.025
#> Targeted power: 0.9
#> Decision rule threshold: 0.06 [Kappa]
#> Assumed true effect: 0.625 [Delta]
#> Treatment effect offset between phase II and III: 0 [gamma]
The probability of a successful program now increased from 0.85 to 0.87, mainly by increasing the number of participants in phase II. The expected utility decreased to 290 694 000 $.
Customized effect size categories
Up until now, the effect size categories for a small, medium or large
treatment effect were predefined and set to \(0-0.5\) for a small treatment effect, \(0.5-0.8\) for a medium treatment effect and
\(\geq 0.8\) for a large treatment
effect (each in standardized differences in mean). These parameter
values were taken from Cohen (1988). However, one can easily customize
the effect size categories by inserting threshold values for each effect
size. Let’s say you only consider a program successful if the treatment
effect is above 0.1. Furthermore, you want to assign a medium benefit if
the treatment effect is between 0.6 and 1 and a large benefit if the
treatment effect is above 1. Note that there is a one-to-one relation
between effect size categories and benefit categories, i.e. the large
treatment effect corresponds to the large benefit and so on. If you want
to implement this scenario you can do this by defining the parameters
steps1
, stepm1
and stepl1
. Here,
we set them to steps1 = 0.1
, stepm1 = 0.6
and
stepl1 = 1
.
res <- optimal_normal(Delta1 = 0.625, fixed = TRUE, # treatment effect
n2min = 20, n2max = 400, # sample size region
stepn2 = 4, # sample size step size
kappamin = 0.02, kappamax = 0.2, # threshold region
stepkappa = 0.02, # threshold step size
c2 = 0.675, c3 = 0.72, # maximal total trial costs
c02 = 15, c03 = 20, # maximal per-patient costs
b1 = 3000, b2 = 8000, b3 = 10000, # gains for patients
alpha = 0.025, # significance level
beta = 0.1, # 1 - power
Delta2 = NULL, w = NULL, in1 = NULL, in2 = NULL,
a = NULL,b = NULL, # setting all unneeded parameters to NULL
steps1 = 0.1, stepm1 = 0.6, stepl1 = 1 # step sizes for effect size categories
)
Differences in the treatment effect
Due to different population structures in phase II and phase III, the
assumed true treatment effect Delta1
may not be the same in
both phases. Phase III trials are often conducted in populations that
are more heterogeneous than those of the preceding phase II trials
(Kirby et al., 2012), so the effect in phase III may differ from that in
phase II. In order to model these differences, one can use the parameter
gamma
. The treatment effect in phase III is then calculated
as follows: \(\Delta_3 = \Delta_2 +
\gamma\). Values of gamma below zero indicate a more pessimistic
view, values above zero indicate a more optimistic view about the
treatment effect in the confirmatory phase. The default value of
gamma
is zero.
Skipping phase II
If enough information is available, one may decide to skip the phase
II trial and move directly to a confirmatory trial. This can be
integrated in the framework by using the parameter skipII
.
If we set the parameter TRUE
, the program calculates the
expected utility for the case when phase II is skipped and compares it
to the situation when phase II is not skipped. When skipping phase II,
the assumed treatment effect used for planning the phase III trial is
calculated as the median of the prior distribution. The results are then
returned as a two-row data frame, res[1, ]
being the
results when including phase II and res[2, ]
when skipping
phase II. The default value is skipII = FALSE
.
resII <- optimal_normal(Delta1 = 0.625, fixed = TRUE, # treatment effect
n2min = 20, n2max = 400, # sample size region
stepn2 = 4, # sample size step size
kappamin = 0.02, kappamax = 0.2, # threshold region
stepkappa = 0.02, # threshold step size
c2 = 0.675, c3 = 0.72, # maximal total trial costs
c02 = 15, c03 = 20, # maximal per-patient costs
b1 = 3000, b2 = 8000, b3 = 10000, # gains for patients
alpha = 0.025, # significance level
beta = 0.1, # 1 - power
Delta2 = NULL, w = NULL, in1 = NULL, in2 = NULL,
a = NULL,b = NULL, # setting all unneeded parameters to NULL
skipII = TRUE #skipping phase II
)
We get the following results:
resII
#> Optimization result including phase II:
#> Utility: 2946.07
#> Sample size:
#> phase II: 92, phase III: 192, total: 284
#> Probability to go to phase III: 1
#> Total cost:
#> phase II: 77, phase III: 158, cost constraint: Inf
#> Fixed cost:
#> phase II: 15, phase III: 20
#> Variable cost per patient:
#> phase II: 0.675, phase III: 0.72
#> Effect size categories (expected gains):
#> small: 0 (3000), medium: 0.5 (8000), large: 0.8 (10000)
#> Success probability: 0.85
#> Success probability by effect size:
#> small: 0.72, medium: 0.12, large: 0
#> Significance level: 0.025
#> Targeted power: 0.9
#> Decision rule threshold: 0.06 [Kappa]
#> Assumed true effect: 0.625 [Delta]
#> Treatment effect offset between phase II and III: 0 [gamma]
#>
#> Result when skipping phase II:
#> Utility: 3080.46
#> Sample size:
#> phase II: 0, phase III: 108, total: 108
#> Probability to go to phase III: 1
#> Total cost:
#> phase II: 0, phase III: 98, cost constraint: Inf
#> Fixed cost:
#> phase II: 0, phase III: 20
#> Variable cost per patient:
#> phase II: 0, phase III: 0.72
#> Effect size categories (expected gains):
#> small: 0 (3000), medium: 0.5 (8000), large: 0.8 (10000)
#> Success probability: 0.9
#> Success probability by effect size:
#> small: 0.81, medium: 0.09, large: 0
#> Significance level: 0.025
#> Targeted power: 0.9
#> Decision rule threshold: -Inf [Kappa]
#> Assumed true effect: 0.62 [Delta]
#> Treatment effect offset between phase II and III: 0 [gamma]
#>
#> Skipping phase II is the optimal option with respect to the maximal expected utility.
It can be seen that in this case skipping phase II yields a higher
utility than including phase II. Skipping phase II automatically sets
n2 = O
which leads to costs K2 = 0
. Hence,
overall costs are smaller in our example. Furthermore, note that the
optimal threshold value Kappa
is also automatically set to
-Inf
and the probability to go to phase III
pgo = 1
, as we go to phase III every time, when skipping
phase II.
Parallel computing
As some of the programs may have very long run-times, especial more
complex ones, faster computing is enabled by parallel programming. The
parameter num_cl
allows you to set the number of cores for
parallel computing. It is advisable to check the number of cores
available by using the function detectCores()
. The default
value is num_cl = 1
. Note that using this parameters does
not change the results, but can decrease run-time.
Where to go from here
This tutorial presents more parameters, so you can adapt the program to your specific needs. For more information on how to use the package, see:
- Introduction to drugdevelopR: See how the package works in a basic example.
- Different outcomes: Apply it to binary endpoints and time-to-event endpoints.
- Interpreting the rest of the output: Obtain further details on your drug development program.
- Fixed or prior: Model the assumed treatment effect on a prior distribution.
- Complex drug development programs: Adapt to situations with biased effect estimators, multiple phase III trials, multi-arm trials, or multiple endpoints.