NONMEM Users Network Archive

Hosted by Cognigen

Re: VPC appropriateness in complex PK

From: Nick Holford <n.holford>
Date: Mon, 21 Sep 2009 17:53:42 +1200


Like Leonid, I am having trouble understanding how trials originally
conducted with adaptive designs can be used for predictive checks if the
simulation dose regimen is not based on the randomly assigned individual
PK parameters. If the original adapted doses ("obtained from the CRF")
are used then the simulated concentrations will not approach the
adaptive design target as they would have done in the original data.
Thus the distribution of simulated concentrations will be wider than the
distribution of observed concentrations (see Bergstrand et al 2009
Example 3 left hand plot).

Traditional visual predictive checks using the original doses will
clearly show that the distributions of observations and simulated
concentrations are different and would wrongly reject an adequate PK model.

I would expect methods based on statistical predictive checks (PDE
(including SVPC), NPDE) would detect that the distribution of prediction
discrepancies is not as expected (uniform for PDE; normal for NPDE) and
also wrongly reject an adequate PK model.

PRED-corrected VPCs will not detect a difference between the
PRED-corrected simulated concentrations and the PRED-corrected
observations. This is because the PRED correction process is equivalent
to normalizing all subjects to the same dose at each time point. For a
linear PK model the variability in concs will have all the dose
information removed and thus the adaptive changes in dose become
irrelevant. Note that the PRED-corrected 'observations' will be quite
different from the original observations and the trend of the
PRED-corrected 'observations' variability will be quite unlike that seen
in the data (see Bergstrand et al 2009 Example 3 right hand plot). This
could be confusing but it should not lead to wrongly rejecting an
adequate model.

If the simulations are done using an adaptive dosing algorithm that is
similar to that used in the original study then the statistical
predictive checks and visual predictive checks (without or with
PRED-correction) should not reject an adequate PK model.

A non-PRED-corrected visual predictive check (NPC-VPC) should also
correctly represent the actual observations and the simulated
distributions if it used an adaptive dosing model. I think this is a key
difference between the empirical PRED-corrected and mechanism based
adaptive dose model approaches to a VPC. The mechanism based approach
gives more visual reassurance that the combined models i.e. the PK model
and the adaptive dosing model, can describe the data. This will give
visual support for using the model combination for future trial
simulations. The empirical PRED-corrected VPC does not give this kind of
support for future use of the PK model under an adaptive design scenario.


Bergstrand M, Hooker AC, Wallin JE, Karlsson MO. Prediction corrected
visual predictive checks ACOP.

Leonid Gibiansky wrote:
> Diane, Martin,
> I think you are correct that SVPC (percentiles plotted versus time)
> and PC-VPC should work in the adaptive-study example that I provided.
> Still, conceptually, the simulation process should mimic the actual
> study. If adaptation was used in the actual study, it is better to do
> VPC-type simulations using adaptation rules, the same as in the
> original study.
> For adaptive trials, individual dose depends on individual parameters
> while in the non-adaptive simulations, simulated random effect
> distributions are dose-independent. It is not obvious why the
> resulting simulated distributions should always be similar to the
> observed distributions even if the model is correct.
> Thanks
> Leonid
> --------------------------------------
> Leonid Gibiansky, Ph.D.
> President, QuantPharm LLC
> web:
> e-mail: LGibiansky at
> tel: (301) 767 5566
> Wang, Diane wrote:
>> Leonid,
>> I agree with you that VPC can not be used for concentration, effect
>> controlled trials or trials with adaptive design. However SVPC does
>> work in all these situations. The dosing record for each patient
>> obtained from CRF is the adjusted dose based on the individual
>> parameters. Therefore, 95 and 5 percentiles of the simulated
>> concentrations based on this individual's information should only
>> reflect random effect as all fixed effects are the same.
>> Compared to PC-VPC, SVPC doesn't have the disadvantage Martin indicated
>> in his Acop abstract "Prediction Corrected Visual Predictive Checks"
>> that "PC-VPC only accounts for differences in typical subject
>> predictions; there may also be differences in expected variability
>> around this prediction". Rather than correcting for typical subject's
>> predictions, SVPC uses each individual's exact design template without
>> approximation and is not affected by the variability/uncertainty of the
>> predictions.
>> Diane
>> -----Original Message-----
>> From: Leonid Gibiansky [mailto:LGibiansky
>> Friday, September 18, 2009 5:28 PM
>> To: Wang, Diane
>> Cc: Dider Heine; nmusers
>> Subject: Re: [NMusers] VPC appropriateness in complex PK
>> Diane,
>> I probably worded it incorrectly. I was going to say that for
>> concentration or effect controlled trails you cannot use straightforward
>> VPC simulation based on the actual dosing history; you have to be
>> more careful. Let me show the example that illustrates how VPC/SVPC
>> behaves in the concentration or effect controlled trials.
>> Assume that we conduct the two-dose study. The first dose (same for
>> all subjects) is given to learn the kinetics. The second dose is
>> adjusted (based on the previous data) in order to get the same Cmax
>> for all subjects. For simplicity, assume that the world is nearly
>> perfect: no or
>> small residual variability, no or small inter-occasion variability. Then
>> the dose adjustment can be perfect, and the second-dose Cmax for all
>> subjects would hit the target. No or small second-dose Cmax
>> variability would be observed.
>> Now, let's do VPC. If you simulate based on the actual dosing history
>> (even from the from the true model), your first-dose Cmax will be
>> distributed similar to the observed data. However, your second-dose Cmax
>> will vary significantly (even more than the first-dose Cmax) because you
>> use the actual dose, rather than adjust the dose based on the individual
>> parameters. Thus, standard VPC/SVPC/NPDE/PC-VPC/etc. will be misleading.
>> One needs to simulate using the same dose adjustment algorithm as in the
>> actual study. Only for these simulations predictive check plots can
>> be used.
>> Thanks
>> Leonid
>> --------------------------------------
>> Leonid Gibiansky, Ph.D.
>> President, QuantPharm LLC
>> web:
>> e-mail: LGibiansky at
>> tel: (301) 767 5566
>> Wang, Diane wrote:
>>> Leonid,
>>> Thank you for the explanation. I was writing the response but found
>>> your email stated it even better than what I could do myself. :)
>>> Basically, VPC is less sensitive, when your data set is not
>> homogeneous,
>>> for evaluating random effects, because the 95% percentile interval of
>>> predicted concentrations based on the full model reflects not only the
>>> random effects (inter- and intra-subject variability) but also fixed
>>> effects (difference in study design and covariate effects). SVPC
>> solved
>>> this problem as you described.
>>> Regarding stratifying the plots by dose and influential covariates
>> when
>>> using SVPC, our solution is to group subjects by the covariate of
>>> interest (e.g. dose, and influential covariates) using different
>> colors
>>> and see if the colors are uniformly distributed in the SVPC plot.
>> This
>>> can also be used to identify potential covariates. I have a couple
>>> examples in the presentation slides.
>>> I am not sure I agree with you that SVPC can not be used for
>>> concentration or efficacy controlled trials. In concentration or
>>> efficacy controlled trials, patient's dose is adjusted based on
>>> concentration or efficacy observed. As long as we have the dosing
>>> record, we should be able to get the percentile for each observation
>> of
>>> each patient based on predicted PK/PD endpoints using this patient's
>>> dosing record, and then pool all observation percentiles together
>>> regardless of each patient's dose and dosing schedule.
>>> Thanks,
>>> Diane
>>> Diane D. Wang, Ph.D.
>>> Director
>>> Clinical Pharmcology Oncology Business Unit
>>> Pfizer La Jolla
>>> 10555 Science Center Dr. (CB10/2408)
>>> San Diego, CA 92121
>>> Office Phone: (858) 622-8021
>>> Cell Phone: (858) 761-3667
>>> email:
>>> -----Original Message-----
>>> From: owner-nmusers
>> [mailto:owner-nmusers
>>> On Behalf Of Leonid Gibiansky
>>> Sent: Friday, September 18, 2009 12:44 PM
>>> To: Dider Heine
>>> Cc: nmusers
>>> Subject: Re: [NMusers] VPC appropriateness in complex PK
>>> Hi Dider,
>>> VPC is very good when your data set is homogeneous: same or similar
>>> dosing, same or similar sampling, same or similar influential
>> covariates
>>> that results in similar PK or PD predictions. In cases of diverse data
>>> sets, traditional VPC is more difficult to implement, and it may not
>> be
>>> useful.
>>> To see the problem, consider VPC (without stratification) for the data
>>> with two dose groups, 1 and 100 units (with the rest being similar).
>>> Obviously, all data that exceed 95% CI would come from the high
>>> dose, and all data below 5th percentile would come from the low
>>> dose, and overall, VPC plots and stats will not be useful. With two
>>> doses, it is
>>> easy to fix: just stratify by dose. If you have more diverse groups,
>> you
>>> have to either do VPC by group, or find the way to plot all values
>>> in one scale. In cases of dose differences and linear kinetics, one can
>> do
>>> VPC with all values normalized by dose. In nonlinear cases, it is more
>>> difficult.
>>> SVPC offers the way out of this problem. In this procedure, each
>>> observation is compared with the distribution of observations at the
>>> same time point, with the same dosing, and with the same covariate set
>>> as in the original data. Position of the observation in the
>> distribution
>>> of simulated values is characterized by the percent of simulated
>> values
>>> that is above (or below) the observed value. If the model is
>>> correct, then percentiles should be uniformly distributed in the
>>> range of 0 to 100. This should hold for any PRED value, and dose,
>>> any time post-dose
>>> etc.
>>> It is important not to combine all observed points together (to
>>> study overall distribution of the SVPC percentiles): in this case
>>> the test
>> in
>>> not sensitive. SVPC is useful when these percentile values are plotted
>>> versus time, time post dose, or PRED (but not IPRED or DV !!)
>>> values. Then, they can be use to see the problems with the model,
>>> similar to
>> how
>>> WRES vs TIME and WRES vs PRED plots are used. The disadvantage is that
>>> you loose visual part: your percentile versus time profiles should
>> look
>>> like a square filled with the points rather than like
>> concentration-time
>>> profiles. Even in this procedure, it make sense to stratify your plots
>>> by dose, influential covariates, etc. to see whether the plots are
>>> uniformly good. Dose, covariate, time or PRED dependencies of the SVPC
>>> plots may indicate some deficiency of the model.
>>> Note that none of these procedures can be used to evaluate the
>>> concentration or effect controlled trials, or trials with non-random
>>> drop out. In order to use VPC-based procedures for these cases, you
>> need
>>> to simulate accordingly: with dosing that depend on simulated values
>>> (for concentration or effect controlled trials) or with the drop-out
>>> models.
>>> Thanks
>>> Leonid
>>> --------------------------------------
>>> Leonid Gibiansky, Ph.D.
>>> President, QuantPharm LLC
>>> web:
>>> e-mail: LGibiansky at
>>> tel: (301) 767 5566
>>> Dider Heine wrote:
>>>> Dear NMusers:
>>>> The Visual predictive check (VPC,
>>>> , and
>>> JPKPD,
>>>> Volume 35, Number 2 / April, 2008) has been touted as a useful tool
>>> for
>>>> assessing the perfomance of population pharmacokinetic models.
>>> However
>>>> I recently came across this abstract from the 2009 PAGE meeting:
>>> ictive%20Check%20in%20Model%20Evaluation%20-%20PAGE2009%20submit.pdf
>>>> .
>>>> This abstract states that situations when VPC is not feasible but a
>>>> "Standardized Visual Predictive Check (SVPC) can be used are as
>>> follows:
>>>> - Patients received individualized dose or there are a small number
>> of
>>>> patients per dose group and PK or PD is nonlinear, thus observations
>>> can
>>>> not be normalized for dose
>>>> - There are multiple categorical covariate effects on PK or PD
>>> parameters
>>>> - Covariate is a continuous variable which made stratification
>>> impossible
>>>> - Study design and execution varies among individuals, such as
>>> adaptive
>>>> design, difference in dosing schedule, dose changes and dosing time
>>>> varies during study, protocol violations
>>>> - Different concomitant medicines and food intake among individuals
>>> when
>>>> there are drug-drug interactions and food effect on PK
>>>> However, the original VPC articles seem to suggest that these are the
>>>> exact situations when the VPC alone is an ideal tool for model
>>>> validation. Is there any justification for one approach over the
>>>> other? Has anyone ever seen an SVPC utilized elsewhere, I have found
>>>> nothing. Are these truly weaknesses of a VPC?
>>>> Cheers!
>>>> Dider

Nick Holford, Professor Clinical Pharmacology
Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
mobile: +64 21 46 23 53
Received on Mon Sep 21 2009 - 01:53:42 EDT

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to:

Once subscribed, you may contribute to the discussion by emailing: