NONMEM Users Network Archive

Hosted by Cognigen

Re: VPC appropriateness in complex PK

From: Leonid Gibiansky <LGibiansky>
Date: Sun, 20 Sep 2009 17:23:03 -0400

Diane, Martin,

I think you are correct that SVPC (percentiles plotted versus time) and
PC-VPC should work in the adaptive-study example that I provided. Still,
conceptually, the simulation process should mimic the actual study. If
adaptation was used in the actual study, it is better to do VPC-type
simulations using adaptation rules, the same as in the original study.

For adaptive trials, individual dose depends on individual parameters
while in the non-adaptive simulations, simulated random effect
distributions are dose-independent. It is not obvious why the resulting
simulated distributions should always be similar to the observed
distributions even if the model is correct.


Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
e-mail: LGibiansky at
tel: (301) 767 5566

Wang, Diane wrote:
> Leonid,
> I agree with you that VPC can not be used for concentration, effect
> controlled trials or trials with adaptive design. However SVPC does
> work in all these situations. The dosing record for each patient
> obtained from CRF is the adjusted dose based on the individual
> parameters. Therefore, 95 and 5 percentiles of the simulated
> concentrations based on this individual's information should only
> reflect random effect as all fixed effects are the same.
> Compared to PC-VPC, SVPC doesn't have the disadvantage Martin indicated
> in his Acop abstract "Prediction Corrected Visual Predictive Checks"
> that "PC-VPC only accounts for differences in typical subject
> predictions; there may also be differences in expected variability
> around this prediction". Rather than correcting for typical subject's
> predictions, SVPC uses each individual's exact design template without
> approximation and is not affected by the variability/uncertainty of the
> predictions.
> Diane
> -----Original Message-----
> From: Leonid Gibiansky [mailto:LGibiansky
> Sent: Friday, September 18, 2009 5:28 PM
> To: Wang, Diane
> Cc: Dider Heine; nmusers
> Subject: Re: [NMusers] VPC appropriateness in complex PK
> Diane,
> I probably worded it incorrectly. I was going to say that for
> concentration or effect controlled trails you cannot use straightforward
> VPC simulation based on the actual dosing history; you have to be more
> careful. Let me show the example that illustrates how VPC/SVPC behaves
> in the concentration or effect controlled trials.
> Assume that we conduct the two-dose study. The first dose (same for all
> subjects) is given to learn the kinetics. The second dose is adjusted
> (based on the previous data) in order to get the same Cmax for all
> subjects. For simplicity, assume that the world is nearly perfect: no or
> small residual variability, no or small inter-occasion variability. Then
> the dose adjustment can be perfect, and the second-dose Cmax for all
> subjects would hit the target. No or small second-dose Cmax variability
> would be observed.
> Now, let's do VPC. If you simulate based on the actual dosing history
> (even from the from the true model), your first-dose Cmax will be
> distributed similar to the observed data. However, your second-dose Cmax
> will vary significantly (even more than the first-dose Cmax) because you
> use the actual dose, rather than adjust the dose based on the individual
> parameters. Thus, standard VPC/SVPC/NPDE/PC-VPC/etc. will be misleading.
> One needs to simulate using the same dose adjustment algorithm as in the
> actual study. Only for these simulations predictive check plots can be
> used.
> Thanks
> Leonid
> --------------------------------------
> Leonid Gibiansky, Ph.D.
> President, QuantPharm LLC
> web:
> e-mail: LGibiansky at
> tel: (301) 767 5566
> Wang, Diane wrote:
>> Leonid,
>> Thank you for the explanation. I was writing the response but found
>> your email stated it even better than what I could do myself. :)
>> Basically, VPC is less sensitive, when your data set is not
> homogeneous,
>> for evaluating random effects, because the 95% percentile interval of
>> predicted concentrations based on the full model reflects not only the
>> random effects (inter- and intra-subject variability) but also fixed
>> effects (difference in study design and covariate effects). SVPC
> solved
>> this problem as you described.
>> Regarding stratifying the plots by dose and influential covariates
> when
>> using SVPC, our solution is to group subjects by the covariate of
>> interest (e.g. dose, and influential covariates) using different
> colors
>> and see if the colors are uniformly distributed in the SVPC plot.
> This
>> can also be used to identify potential covariates. I have a couple
>> examples in the presentation slides.
>> I am not sure I agree with you that SVPC can not be used for
>> concentration or efficacy controlled trials. In concentration or
>> efficacy controlled trials, patient's dose is adjusted based on
>> concentration or efficacy observed. As long as we have the dosing
>> record, we should be able to get the percentile for each observation
> of
>> each patient based on predicted PK/PD endpoints using this patient's
>> dosing record, and then pool all observation percentiles together
>> regardless of each patient's dose and dosing schedule.
>> Thanks,
>> Diane
>> Diane D. Wang, Ph.D.
>> Director
>> Clinical Pharmcology
>> Oncology Business Unit
>> Pfizer La Jolla
>> 10555 Science Center Dr. (CB10/2408)
>> San Diego, CA 92121
>> Office Phone: (858) 622-8021
>> Cell Phone: (858) 761-3667
>> email:
>> -----Original Message-----
>> From: owner-nmusers
> [mailto:owner-nmusers
>> On Behalf Of Leonid Gibiansky
>> Sent: Friday, September 18, 2009 12:44 PM
>> To: Dider Heine
>> Cc: nmusers
>> Subject: Re: [NMusers] VPC appropriateness in complex PK
>> Hi Dider,
>> VPC is very good when your data set is homogeneous: same or similar
>> dosing, same or similar sampling, same or similar influential
> covariates
>> that results in similar PK or PD predictions. In cases of diverse data
>> sets, traditional VPC is more difficult to implement, and it may not
> be
>> useful.
>> To see the problem, consider VPC (without stratification) for the data
>> with two dose groups, 1 and 100 units (with the rest being similar).
>> Obviously, all data that exceed 95% CI would come from the high dose,
>> and all data below 5th percentile would come from the low dose, and
>> overall, VPC plots and stats will not be useful. With two doses, it is
>> easy to fix: just stratify by dose. If you have more diverse groups,
> you
>> have to either do VPC by group, or find the way to plot all values in
>> one scale. In cases of dose differences and linear kinetics, one can
> do
>> VPC with all values normalized by dose. In nonlinear cases, it is more
>> difficult.
>> SVPC offers the way out of this problem. In this procedure, each
>> observation is compared with the distribution of observations at the
>> same time point, with the same dosing, and with the same covariate set
>> as in the original data. Position of the observation in the
> distribution
>> of simulated values is characterized by the percent of simulated
> values
>> that is above (or below) the observed value. If the model is correct,
>> then percentiles should be uniformly distributed in the range of 0 to
>> 100. This should hold for any PRED value, and dose, any time post-dose
>> etc.
>> It is important not to combine all observed points together (to study
>> overall distribution of the SVPC percentiles): in this case the test
> in
>> not sensitive. SVPC is useful when these percentile values are plotted
>> versus time, time post dose, or PRED (but not IPRED or DV !!) values.
>> Then, they can be use to see the problems with the model, similar to
> how
>> WRES vs TIME and WRES vs PRED plots are used. The disadvantage is that
>> you loose visual part: your percentile versus time profiles should
> look
>> like a square filled with the points rather than like
> concentration-time
>> profiles. Even in this procedure, it make sense to stratify your plots
>> by dose, influential covariates, etc. to see whether the plots are
>> uniformly good. Dose, covariate, time or PRED dependencies of the SVPC
>> plots may indicate some deficiency of the model.
>> Note that none of these procedures can be used to evaluate the
>> concentration or effect controlled trials, or trials with non-random
>> drop out. In order to use VPC-based procedures for these cases, you
> need
>> to simulate accordingly: with dosing that depend on simulated values
>> (for concentration or effect controlled trials) or with the drop-out
>> models.
>> Thanks
>> Leonid
>> --------------------------------------
>> Leonid Gibiansky, Ph.D.
>> President, QuantPharm LLC
>> web:
>> e-mail: LGibiansky at
>> tel: (301) 767 5566
>> Dider Heine wrote:
>>> Dear NMusers:
>>> The Visual predictive check (VPC,
>>> , and
>>> Volume 35, Number 2 / April, 2008) has been touted as a useful tool
>> for
>>> assessing the perfomance of population pharmacokinetic models.
>> However
>>> I recently came across this abstract from the 2009 PAGE meeting:
>> ictive%20Check%20in%20Model%20Evaluation%20-%20PAGE2009%20submit.pdf
>>> .
>>> This abstract states that situations when VPC is not feasible but a
>>> "Standardized Visual Predictive Check (SVPC) can be used are as
>> follows:
>>> - Patients received individualized dose or there are a small number
> of
>>> patients per dose group and PK or PD is nonlinear, thus observations
>> can
>>> not be normalized for dose
>>> - There are multiple categorical covariate effects on PK or PD
>> parameters
>>> - Covariate is a continuous variable which made stratification
>> impossible
>>> - Study design and execution varies among individuals, such as
>> adaptive
>>> design, difference in dosing schedule, dose changes and dosing time
>>> varies during study, protocol violations
>>> - Different concomitant medicines and food intake among individuals
>> when
>>> there are drug-drug interactions and food effect on PK
>>> However, the original VPC articles seem to suggest that these are the
>>> exact situations when the VPC alone is an ideal tool for model
>>> validation. Is there any justification for one approach over the
>>> other? Has anyone ever seen an SVPC utilized elsewhere, I have found
>>> nothing. Are these truly weaknesses of a VPC?
>>> Cheers!
>>> Dider
Received on Sun Sep 20 2009 - 17:23:03 EDT

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to:

Once subscribed, you may contribute to the discussion by emailing: