NONMEM Users Network Archive

Hosted by Cognigen

Re: VPC with Mixture Model

From: Nick Holford <n.holford>
Date: Wed, 15 Apr 2009 08:32:47 +0200

Hi,

I realized after I sent the comment below about mixture model
**simulations** that Mats was no doubt referring to the problem of
separating the two sub-populations in the **observed** data using
MIXEST. This will of course be problematic for generating prediction
percentiles for the 2 populations as Mats pointed out when there is
shrinkage of the EBEs which lead to the 'decision' about MIXEST.

This is an example of why one must be cautious interpreting a VPC when
the observed data is not 'correct' for comparison with the simulation
predictions. Another example is when there is data missing in the
observations which is not missing completely at random (e.g. dropouts
due to adverse effects). Adding a dropout model to the VPC simulation
can help sometimes in 'correcting' the simulation so that it matches the
observed data better.

Mats, can you suggest a way to try to correct for the Bayesian shrinkage
problem for obtaining MIXEST?

Nick

Nick Holford wrote:
> Mats,
>
> A VPC relies on simulation alone - there is no estimation step.
> Presumably if the population estimate of the % of poor metabolizers is
> 5% then NONMEM will simulate 5% of the population as a poor
> metaboliser. There is no Bayesian 'posthoc' step hidden in the way
> that NONMEM does simulations is there? I had assumed (but have never
> checked) that if one tabulates the value of MIXEST obtained when using
> $SIM that the proportion of MIXEST values would not be biased compared
> to the population value.
>
> Nck
>
> Mats Karlsson wrote:
>> Hi Leonid,
>>
>> I would not do it for just the reason you mention. I would not want to
>> condition my VPC on the model results. Especially as we know that the
>> subpopulation assignment suffer from the same problems as other
>> empirical
>> Bayes estimates. "Shrinkage" when it comes to subpopulation
>> assignment will
>> have the consequence that the larger of (two) subpopulations having a
>> higher
>> fraction of POSTHOC assignments than the Pmix estimate for that
>> subpopulation. This is expected and I have often seen it. So you may
>> well
>> have a situation when the population estimate of poor metabolizers is
>> 5%,
>> but only 2% are allocated to this subpopulation by the EBE step.
>>
>> Best regards,
>> Mats
>>
>>
>> Mats Karlsson, PhD
>> Professor of Pharmacometrics
>> Dept of Pharmaceutical Biosciences
>> Uppsala University
>> Box 591
>> 751 24 Uppsala Sweden
>> phone: +46 18 4714105
>> fax: +46 18 471 4003
>>
>>
>> -----Original Message-----
>> From: Leonid Gibiansky [mailto:LGibiansky
>> Tuesday, April 14, 2009 9:06 PM
>> To: Mats Karlsson
>> Cc: 'Satyendra Suryawanshi'; nmusers
>> Subject: Re: [NMusers] VPC with Mixture Model
>>
>> Hi Mats,
>>
>> Could you elaborate why you would not stratify based on the
>> subpopulations? It seems perfectly reasonable for me to simulate from
>> the model (including random assignment of subpopulations), and then
>> compare "apples to apples": observed subpopulation versus simulated
>> subpopulations. In your example of 5% poor metabolizers, I would plot
>> observed poor metabolizers (as assigned by the model) versus
>> simulated poor metabolizers (as simulated from the model). Indeed,
>> poor metabolizers assignment would be defined by the model, so this
>> VPC will be conditioned on the model posthoc EST prediction, but the
>> remaining parts of the model would be tested by this procedure. If
>> the model is good, VPC should provide good results. It is unclear to
>> me how sensitive this procedure is to model misspecification (in
>> general, I think VPC is less sensitive to model misspecification than
>> other model diagnostics)
>>
>> Thanks
>> Leonid
>>
>> --------------------------------------
>> Leonid Gibiansky, Ph.D.
>> President, QuantPharm LLC
>> web: www.quantpharm.com
>> e-mail: LGibiansky at quantpharm.com
>> tel: (301) 767 5566
>>
>>
>>
>>
>> Mats Karlsson wrote:
>>
>>> Dear Satyendra,
>>>
>>>
>>>
>>> Interesting question. I don't think there is much written about
>>> this, but I may be wrong. What I would not do is to try to stratify
>>> based on estimated subpopulation allocation ("EST"). Rather I would
>>> use the same VPCs as if there had been no mixture model. Possibly
>>> what you could do is be more careful in your choice of prediction
>>> intervals to display. For example, if you have a subpopulation of
>>> poor metabolizers of about 5%, displaying only median and
>>> interquartile range PIs may not be a good idea.
>>>
>>>
>>>
>>> Best regards,
>>>
>>> Mats
>>>
>>>
>>>
>>> Mats Karlsson, PhD
>>>
>>> Professor of Pharmacometrics
>>>
>>> Dept of Pharmaceutical Biosciences
>>>
>>> Uppsala University
>>>
>>> Box 591
>>>
>>> 751 24 Uppsala Sweden
>>>
>>> phone: +46 18 4714105
>>>
>>> fax: +46 18 471 4003
>>>
>>>
>>>
>>> *From:* owner-nmusers
>>> [mailto:owner-nmusers
>>> Suryawanshi
>>> *Sent:* Tuesday, April 14, 2009 6:50 PM
>>> *To:* nmusers
>>> *Subject:* [NMusers] VPC with Mixture Model
>>>
>>>
>>>
>>> Dear all,
>>>
>>> I have a Mixture Model with 2 subpopulation. Now I want to check its
>>> prediction. One way to see this is a Visual Predictive Check. My
>>> question is, How to perform visual predictive check with mixture model?
>>> I will be thankful for your suggestion and references.
>>>
>>>
>>>
>>> Best regards
>>>
>>>
>>>
>>> Satyendra Suryawanshi, PhD
>>>
>>> University of Tennessee Health Science Center
>>>
>>>
>>>
>>>
>>
>>
>

--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
n.holford
mobile: +33 64 271-6369 (Apr 6-Jul 17 2009)
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Wed Apr 15 2009 - 02:32:47 EDT

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to: nmusers-request@iconplc.com.

Once subscribed, you may contribute to the discussion by emailing: nmusers@globomaxnm.com.