NONMEM Users Network Archive

Hosted by Cognigen

RE: FW: VPC appropriateness in complex PK

From: Martin Bergstrand <martin.bergstrand>
Date: Tue, 22 Sep 2009 15:24:21 +0200

Dear Nick and NMusers,

Thank you for your very insightful comments on these matters. I in =
agree with you all the way.

As I have said before I think that it is very useful if you can simulate
dose adaptations and use these for VPCs. However even if you can do so I
think that the PC-VPC is a useful complement. Especially since it will =
unaffected by the performance of the dos adaptation model and better
diagnose the random effect components of the model. Furthermore the =
is much simpler to implement since you don't not have to develop and =
a dos adaptation model before simulating.

After evaluating the NPDEs coming from NONMEM7 for my adaptive dose =
example I can see that these are unaffected (mean=0, variance=1). I =
don't understand exactly why this is so but I have obviously been doing
something wrong before. My humble apology to the community for spreading
this false information.

Best regards,

-----Original Message-----
From: owner-nmusers
Behalf Of Nick Holford
Sent: den 22 september 2009 12:13
To: nmusers
Subject: Re: FW: [NMusers] VPC appropriateness in complex PK


I understand it is a problem to simulate adaptive dosing when the rules
used by the clinicians are unknown (or not followed). However, I see no
reason not to use a plausible set of rules to try to simulate the know
adaptive dosing. Ignoring this will lead to differences between observed =

and predicted distributions as shown by a VPC even if the structural and =

random effects model derived from the original data is fine.

Adding a dosing regimen model to the simulation structure is not really
any different from changing other components of the original model. It
may involve a few "informed guess" parameters but if you can get a good
agreement between observations and simulated predictions then this can
be rewarding in two ways:

The first is that it may produce a VPC that helps to confirm the
structural and random effects model assumptions and parameter estimates. =

An example of this is shown in Karlsson & Holford 2008 Slide 27/28 shown =

at PAGE last year. Dropout simulation based on the simulated response
(informative missingness) led to good agreement between the observed and =

simulated distributions shown in a VPC. Dropout simulation is just an
example of adaptive design and in principle is no different from
adaptive dosing changes to the design.

The second is that the adaptive dosing model that is found to help
describe the observations can now be used with some confidence to
simulate future trials when adaptive dosing is not strictly controlled
but is likely to follow the pattern in the original study. This is not
an unreasonable assumption as we frequently make it for other parts of
the model when doing clinical trial simulations.

This brings me to your question to me. A PC-VPC may help to confirm a
model for describing the data but if it does not simulate using adaptive =

dosing, for a trial that used adaptive dosing, then it cannot help
understand what kind of model should be used to simulate adaptive dosing =

in a future design. This illustrates an important difference between
empirical (PC-VPC) and mechanism based (adaptive dose simulation).
Results from empirical methods ("confirming") speak to the past while
mechanism based methods ("learning") can help predict the future.

You mention that in your experience that SPCs are not useful for
adaptive dosing studies because of correlation between ETAs and design.
I can understand why NPCs would fail (they have the same problem as VPC
when comparisons are made directly between the distributions of
observations and predictions) but not NPDE. I have struggled with the
properties of NPDE in adaptive design but have no direct experience. I
have recently responded on nmusers to comments from Yaning Wang which
make me think that NPDE should be fine to evaluate adaptive designs
provided the original dosing is used for the simulation. Can you tell us =

more about your experiences? Do you have examples that show that NPDE
comes to the wrong conclusion about a model when the original design is
based on adaptive dosing?

Best wishes,


Karlsson MO, Holford NHG. A Tutorial on Visual Predictive Checks. PAGE
17 (2008) Abstr 1434 [wwwpage-meetingorg/?abstract=1434]. 2008.

Martin Bergstrand wrote:
> Dear NMusers,
> For some reason my last message to NMusers got lost in www-space.
> Since both Leonid and Nick have responded to my initial message I
> repost this message so that you can follow the discussion (see email
> below).
> In addition to this message I would also like to comment on the
> messages by Diane, Leonid and Nick.
> _Nick and Leonid:_ I agree that it would be useful if one could also
> simulate that adaptive design (e.g. dose adaptations) and show the
> observations on the non transformed scale. However this will in many
> cases be very hard since dos adaptations are often done not according
> to a strict algorithm and/or all information supporting the dose
> alterations is not available. It is to my experience quite commonly
> written I study protocols that dose adjustments can be done “by the
> discretion of the investigator”.
> _Diane and Leonid:_ If I understood the SVPC procedure correctly from
> Diane’s presentation it utilizes a principle similar to that behind
> Numerical Predictive Check (NPC). Most of all SVPC seem to have a
> striking similarity to the first version of the prediction
> discrepancies as described by Metré et al (1). The prediction
> discrepancies have been further developed into the normalised
> prediction distribution errors (NPDE) (2). From my experience both NPC =

> and NPDE are useful diagnostic tools but not applicable to data from
> studies with adaptive dos adjustments (correlation between ETAs and
> design). What is the unique feature with SVPC that sets it apart from
> the prediction discrepancies and makes it applicable to studies with
> adaptive dos adjustments?
> _Nick:_ Regarding this sentence “The empirical PRED-corrected VPC =
> not give this kind of support for future use of the PK model under an
> adaptive design scenario”. Why is this? If the PC-VPC can verify =
> you have an acceptable structure model and unbiased parameter
> estimates you can then simulate any type of adaptive design scenario.
> Best regards,
> Martin
> 1. Prediction discrepancies for the evaluation of … Mentré F, =
> S. JPKPD. 2006
> 2. Computing normalised prediction distribution errors ... Comets E,
> Brendel K, Mentré F. CMPB. 2008

Nick Holford, Professor Clinical Pharmacology
Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New =
mobile: +64 21 46 23 53
Received on Tue Sep 22 2009 - 09:24:21 EDT

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to:

Once subscribed, you may contribute to the discussion by emailing: