# Re: FW: PPC

From: Nick Holford <n.holford>
Date: Wed, 23 Jul 2008 22:31:52 +1200

Paul,

The procedure you describe is a way of producing a posterior predictive
check but I don't know of any good examples of its use. A simpler way of =

doing a PPC samples the population parameter estimates from a
distribution centered on the final estimates with a variance-covariance =

based on the estimated standard errors and their correlation. VPCs are
not posterior predictive checks because they do not take account of the
posterior distribution of the parameter estimates (i.e. the final
estimates with their uncertainty). A VPC typically ignores the parameter =

uncertainty and uses what has been called the degenerate posterior
distribution (See Yano Y, Beal SL, Sheiner LB. Evaluating
pharmacokinetic/pharmacodynamic models using the posterior predictive
check. J Pharmacokinet Pharmacodyn. 2001;28(2):171-92 for terminology,
methods and examples).

When I spoke of uncertainty I did not mean random variability (OMEGA and =

SIGMA). A VPC will simulate observations using the final THETA, OMEGA
and SIGMA estimates.

You can calculate distribution statistics for your observations (such as =

median and 90% intervals) by combining the observations (one per
individual) at each time point to create an empirical distribution. The
statistics are then determined from this empirical distribution. In
order to get sufficient numbers of points (at least 10 is desirable) you =

may need to bin observations into time intervals e.g. 0-30 mins, 30-60
mins etc.

Nick

Paul Matthew Westwood wrote:
> ________________________________________
> From: Paul Matthew Westwood
> Sent: 22 July 2008 13:20
> To: Nick Holford
> Subject: RE: [NMusers] PPC
>
> Nick,
>
> Thanks for your reply and apologies once again for another confusing em=
ail. I think I am using VPC, which as I understand it is simulating n dat=
asets using the final parameter estimates gained from the final model, an=
d then taking the median and 90% confidence interval (for example) for ea=
ch simulated concentration and comparing these to the real concentrations=
. Whereas, PPC is where you then run the final model through the simula=
ted datasets and compare selected statistics of these new runs with the o=
riginal. Is this correct? You mentioned including uncertainty on the para=
meter estimates in the simulated datasets. Would one usually not include =
uncertainty (fixing the error terms to zero) in the simulated datasets? D=
oing this with mine obviously produced much better concentrations with no=
negative values and no 'significant' outliers. Another thing you mention=
ed is comparing the median of the simulated concentrations with the media=
n of the original dataset concentrations, but as there is only one sample=
for any particular time point would this indicate the unsuitability of V=
PC (and furthermore PPC) for this model?
>
> Thanks again,
> Paul.
> ________________________________________
> From: owner-nmusers
half Of Nick Holford [n.holford
> Sent: 22 July 2008 10:30
> To: nmusers
> Subject: Re: [NMusers] PPC
>
> Paul,
>
> Its not clear to me if you did a VPC (visual predictive check) using
> just the final estimates of the parameters) or tried to do a posterior
> predictive check (PPC) including uncertainty on the parameter estimates=

> in the simulation.
>
> I dont have any experience with PPC but I dont think its helpful for
> model evaluation. Its more of a tool for understanding uncertainties of=

> predictions for future studies.
>
> I assume you dont have complications like informative dropout processes=

> to complicate the simulation so if you did a VPC and the median of the
> predictions doesnt match the median of the observations then your model=

> needs more work.
>
> Some negative concs are OK but 'impossibly high values' point to
>
> So I think you can safely say the VPC has worked very well -- it has
> told you that you need to think more about your model. You might find
> some ideas in these references:
>
> 1. Tod M, Jullien V, Pons G. Facilitation of drug evaluation in
> children by population methods and modelling. Clin Pharmacokinet.
> 2008;47(4):231-43.
> 2. Anderson BJ, Holford NH. Mechanism-Based Concepts of Size and
> Maturity in Pharmacokinetics. Annu Rev Pharmacol Toxicol. 2008;48:303-3=
2.
>
> Nick
>
> Paul Matthew Westwood wrote:
>
>> Hello all,
>>
>> I wonder if someone can give me some tips on PPC.
>> I am working on a midazolam dataset with a pediatric population, and h=
ave decided to use PPC as a model validation technique. The dataset I am =
modelling has up to 43 patients, at different ages, different weights, di=
fferent times of dosing and sampling, and different doses. I simulated 10=
0 datasets using NONMEM VI, fixing all parameters to the final estimates =
from the model. The simulated datasets produced had a large proportion of=
negative concentrations, and also a few impossibly large concentration v=
alues. Also the median, 5th and 95th percentiles were not very promising,=
and the resulting graphs not very clean.
>> Firstly, can I use PPC with any degree of confidence with a dataset su=
ch as this, and if so, do I omit the negative concentration values from t=
he analysis?
>>
>> Thanks in advance for any help given.
>>
>> Paul Westwood,
>> PhD Student,
>> QUB,
>> Belfast.
>>
>>
>>
>>
>
> --
> Nick Holford, Dept Pharmacology & Clinical Pharmacology
> University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Ze=
aland
> n.holford
> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
>
>
>

--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zeal=
and
n.holford
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Wed Jul 23 2008 - 06:31:52 EDT

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to: nmusers-request@iconplc.com.

Once subscribed, you may contribute to the discussion by emailing: nmusers@globomaxnm.com.