NONMEM Users Network Archive

Hosted by Cognigen

RE: VPC with Mixture Model

From: Mats Karlsson <mats.karlsson>
Date: Wed, 15 Apr 2009 12:47:14 +0200

Hi,

Nick wrote:
"Mats, can you suggest a way to try to correct for the Bayesian =
shrinkage
problem for obtaining MIXEST?"

As the problem is created by using model estimates for stratification, =
the
simple answer is not to do it. I can't think of a situation where it =
would
be useful to stratify based on some model estimates.

Not related to VPCs, but relevant for using MIXEST: when using mixture
models both mixture assignment and ETA estimates produced by NONMEM are
simplifications and more informative output can be generated: the
probability for a subject to be in subpopulations x, y, z (rather than =
just
the best guess) and the overall best estimate of ETAs, not just that of =
the
best guess subpopulation. How to get that type of output and use it is
described in this recent publication (thanks Nock for the cue):
"Modeling subpopulations with the $MIXTURE subroutine in NONMEM: finding =
the
individual probability of belonging to a subpopulation for the use in =
model
analysis and improved decision making."
Carlsson KC, SaviŠ RM, Hooker AC, Karlsson MO.
AAPS J. 2009 Mar;11(1):148-54. Epub 2009 Mar 10.

Mats

Mats Karlsson, PhD
Professor of Pharmacometrics
Dept of Pharmaceutical Biosciences
Uppsala University
Box 591
751 24 Uppsala Sweden
phone: +46 18 4714105
fax: +46 18 471 4003


-----Original Message-----
From: owner-nmusers
On
Behalf Of Nick Holford
Sent: Wednesday, April 15, 2009 8:33 AM
To: nmusers
Subject: Re: [NMusers] VPC with Mixture Model

Hi,

I realized after I sent the comment below about mixture model
**simulations** that Mats was no doubt referring to the problem of
separating the two sub-populations in the **observed** data using
MIXEST. This will of course be problematic for generating prediction
percentiles for the 2 populations as Mats pointed out when there is
shrinkage of the EBEs which lead to the 'decision' about MIXEST.

This is an example of why one must be cautious interpreting a VPC when
the observed data is not 'correct' for comparison with the simulation
predictions. Another example is when there is data missing in the
observations which is not missing completely at random (e.g. dropouts
due to adverse effects). Adding a dropout model to the VPC simulation
can help sometimes in 'correcting' the simulation so that it matches the =

observed data better.

Mats, can you suggest a way to try to correct for the Bayesian shrinkage =

problem for obtaining MIXEST?

Nick

Nick Holford wrote:
> Mats,
>
> A VPC relies on simulation alone - there is no estimation step.
> Presumably if the population estimate of the % of poor metabolizers is =

> 5% then NONMEM will simulate 5% of the population as a poor
> metaboliser. There is no Bayesian 'posthoc' step hidden in the way
> that NONMEM does simulations is there? I had assumed (but have never
> checked) that if one tabulates the value of MIXEST obtained when using =

> $SIM that the proportion of MIXEST values would not be biased compared =

> to the population value.
>
> Nck
>
> Mats Karlsson wrote:
>> Hi Leonid,
>>
>> I would not do it for just the reason you mention. I would not want =
to
>> condition my VPC on the model results. Especially as we know that the
>> subpopulation assignment suffer from the same problems as other
>> empirical
>> Bayes estimates. "Shrinkage" when it comes to subpopulation
>> assignment will
>> have the consequence that the larger of (two) subpopulations having a =

>> higher
>> fraction of POSTHOC assignments than the Pmix estimate for that
>> subpopulation. This is expected and I have often seen it. So you may
>> well
>> have a situation when the population estimate of poor metabolizers is =

>> 5%,
>> but only 2% are allocated to this subpopulation by the EBE step.
>>
>> Best regards,
>> Mats
>>
>>
>> Mats Karlsson, PhD
>> Professor of Pharmacometrics
>> Dept of Pharmaceutical Biosciences
>> Uppsala University
>> Box 591
>> 751 24 Uppsala Sweden
>> phone: +46 18 4714105
>> fax: +46 18 471 4003
>>
>>
>> -----Original Message-----
>> From: Leonid Gibiansky [mailto:LGibiansky
>> Tuesday, April 14, 2009 9:06 PM
>> To: Mats Karlsson
>> Cc: 'Satyendra Suryawanshi'; nmusers
>> Subject: Re: [NMusers] VPC with Mixture Model
>>
>> Hi Mats,
>>
>> Could you elaborate why you would not stratify based on the
>> subpopulations? It seems perfectly reasonable for me to simulate from =

>> the model (including random assignment of subpopulations), and then
>> compare "apples to apples": observed subpopulation versus simulated
>> subpopulations. In your example of 5% poor metabolizers, I would plot =

>> observed poor metabolizers (as assigned by the model) versus
>> simulated poor metabolizers (as simulated from the model). Indeed,
>> poor metabolizers assignment would be defined by the model, so this
>> VPC will be conditioned on the model posthoc EST prediction, but the
>> remaining parts of the model would be tested by this procedure. If
>> the model is good, VPC should provide good results. It is unclear to
>> me how sensitive this procedure is to model misspecification (in
>> general, I think VPC is less sensitive to model misspecification than =

>> other model diagnostics)
>>
>> Thanks
>> Leonid
>>
>> --------------------------------------
>> Leonid Gibiansky, Ph.D.
>> President, QuantPharm LLC
>> web: www.quantpharm.com
>> e-mail: LGibiansky at quantpharm.com
>> tel: (301) 767 5566
>>
>>
>>
>>
>> Mats Karlsson wrote:
>>
>>> Dear Satyendra,
>>>
>>>
>>>
>>> Interesting question. I don't think there is much written about
>>> this, but I may be wrong. What I would not do is to try to stratify
>>> based on estimated subpopulation allocation ("EST"). Rather I would
>>> use the same VPCs as if there had been no mixture model. Possibly
>>> what you could do is be more careful in your choice of prediction
>>> intervals to display. For example, if you have a subpopulation of
>>> poor metabolizers of about 5%, displaying only median and
>>> interquartile range PIs may not be a good idea.
>>>
>>>
>>>
>>> Best regards,
>>>
>>> Mats
>>>
>>>
>>>
>>> Mats Karlsson, PhD
>>>
>>> Professor of Pharmacometrics
>>>
>>> Dept of Pharmaceutical Biosciences
>>>
>>> Uppsala University
>>>
>>> Box 591
>>>
>>> 751 24 Uppsala Sweden
>>>
>>> phone: +46 18 4714105
>>>
>>> fax: +46 18 471 4003
>>>
>>>
>>>
>>> *From:* owner-nmusers
>>> [mailto:owner-nmusers
>>> Suryawanshi
>>> *Sent:* Tuesday, April 14, 2009 6:50 PM
>>> *To:* nmusers
>>> *Subject:* [NMusers] VPC with Mixture Model
>>>
>>>
>>>
>>> Dear all,
>>>
>>> I have a Mixture Model with 2 subpopulation. Now I want to check its =

>>> prediction. One way to see this is a Visual Predictive Check. My
>>> question is, How to perform visual predictive check with mixture =
model?
>>> I will be thankful for your suggestion and references.
>>>
>>>
>>>
>>> Best regards
>>>
>>>
>>>
>>> Satyendra Suryawanshi, PhD
>>>
>>> University of Tennessee Health Science Center
>>>
>>>
>>>
>>>
>>
>>
>

--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New =
Zealand
n.holford
mobile: +33 64 271-6369 (Apr 6-Jul 17 2009)
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Wed Apr 15 2009 - 06:47:14 EDT

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to: nmusers-request@iconplc.com.

Once subscribed, you may contribute to the discussion by emailing: nmusers@globomaxnm.com.