NONMEM Users Network Archive

Hosted by Cognigen

RE: Model building

From: Jurgen Bulitta <jbulitta>
Date: Wed, 9 Dec 2009 17:07:44 -0500

Dear Luis,

I guess your biggest concern regarding the residuals is whether residuals a=
re independent and identically distributed (i.i.d.). I think residual analy=
sis becomes more important, if you have only a few samples
per subject, since it is much easier to judge the goodness of fit on a DV v=
s. TIME plot on linear and semi-log scale in the case of many samples per s=
ubject.

Could you please give us some examples where you made a real-life decision =
based on one of the types of residual plots which you would not have been a=
ble to make from a thorough analysis of individual fit vs. time plots or fr=
om basic and advanced (visual) predictive checks?

I would be especially interested in the type of residuals you found most he=
lpful, types of observed variables, the study design, and software for calc=
ulation of the residuals when you found those residuals were most helpful.

Best wishes
Juergen


Jurgen Bulitta, Ph.D., Senior Scientist
Ordway Research Institute, Albany, NY



From: owner-nmusers
 Behalf Of Luis.Pereira
Sent: Wednesday, December 09, 2009 12:29 PM
To: nmusers
Subject: RE: [NMusers] Model building

Dear All,
For the sake of correctness, Residuals Analysis is one of the most relevant=
 topics in the field of Regression. The very notion of objective function i=
s nothing more than a single characteristic of a type of residuals. So, raw=
 residuals, standardized residuals, partial residuals, studentized residual=
s and so on, are essential to assess serial correlation, collinearity, leve=
rage, hat-matrix, scedasticity, transformations, and on, and on, particular=
ly in multivariable and nonlinear regression. The fact that Nonmem only pro=
vides some residuals by default does not mean we should look at them and ev=
en calculate others. Using just the value of the objective function and a p=
redictive check measure is like stirring a boat in the fog with the eyes ju=
st glued on the bow. The picture is much broader. Please check the extensiv=
e literature on Regression and model identification.
Cheers

Luis

-----------------------
Luis Pereira, PhD
Associate Professor
Childrens' Hospital Boston
Harvard Medical School
Boston MA02115

________________________________
From: owner-nmusers
Sent: Fri 12/4/2009 10:14 PM
To: nmusers
Subject: Re: [NMusers] Model building
Leonid,

I rely on the objective function for model development. Note the word objec=
tive.

I have never looked at a RES or WRES plot except to laugh at the subjective=
 foolishness one can imagine there. Note the word subjective.

Of course one can run into problems by looking only at the objective functi=
on but that is when the VPC is most helpful. I like to use the VPC to decid=
e which model(s) are fit for purpose.

Thank you for recognizing my extreme views. I prefer to be an outlier than =
lost among the pseudo-random residuals :-)

Nick

Leonid Gibiansky wrote:
Hi Nick,
As usual, you are very extreme. VPC could be more sensitive in some cases b=
ut the first step is to get the model with good fits, RES, WRES plots. The =
original question was whether to choose the model with numerical problems b=
ut good WRES plots versus converged problem with bad WRES plots. Your answe=
r effectively means: do VPC fists, then decide.

Let me disagree and recommend the model with better WRES plots even if this=
 model does not converge.

Thanks
Leonid


--------------------------------------
Leonid Gibiansky, Ph.D.
President, QuantPharm LLC
web: www.quantpharm.com<http://www.quantpharm.com/>
e-mail: LGibiansky at quantpharm.com
tel: (301) 767 5566




Nick Holford wrote:


Indranil Bhattacharya wrote:

"So my question is whether the fits, RES, WRES plots and the ofv values hav=
e meaning even when the minimization terminates"

I do not agree with Joachim that RES, WRES are useful. IMHO these have almo=
st no diagnostic merit except for the most extreme cases of a bad model. Si=
mulation based diagnostics (VPC, SPC) have better diagnostic properties and=
 are being actively evaluated by many groups interested in modelling method=
ology.

See Karlsson MO, Savic RM. Diagnosing model diagnostics. Clin Pharmacol The=
r. 2007 Jul;82(1):17-20. for a demonstration of the problems.

Nick

Thanks Joachim, that is what I thought. I wanted to be sure that I invest t=
ime building the right model and not just some model which works (converges=
) but is biased.

Neil
On Fri, Dec 4, 2009 at 3:31 AM, Grevel, Joachim <Joachim.Grevel
.com<mailto:Joachim.Grevel
eca.com><mailto:Joachim.Grevel

    Hi Neil,


    You ask:


    "So my question is whether the fits, RES, WRES plots and the ofv
    values have meaning even when the minimization terminates"


    The answer: you bet they matter! Residual plots are the most
    informative output NONMEM gives you. They should guide you when
    you determine the basic structure of your model that is supported
    by the data. Successful termination, covariance step, standard
    errors, Eigen values, messages, warnings... are just icing on the
    cake vis-a-vis the residual plots.


    These are my two pennies worth of advice,


    Joachim



    *Joachim Grevel *

    Senior Pharmacometrician

    _____________________________________________________________________

    *AstraZeneca R&D Charnwood*

    Clinical Pharmacology & DMPK, Charnwood

    Bakewell Road, Loughborough, Leics., LE11 5RH, England

    Tel +44 (0) 1509 644035 Fax +44 (0) 1509 645576 Mobile +44 (0)
    7920 285905

    _joachim.grevel
    <mailto:joachim.grevel
ca.com>_

    _ _

     P Please consider the environment before printing this e-mail




    -----------------------------------------------------------------------=
-

    AstraZeneca UK Limited is a company incorporated in England and
    Wales with registered number: 03674842 and a registered office at
    15 Stanhope Gate, London W1K 1LN.

    *Confidentiality Notice: *This message is private and may contain
    confidential, proprietary and legally privileged information. If
    you have received this message in error, please notify us and
    remove it from your system and note that you must not copy,
    distribute or take any action in reliance on it. Any unauthorised
    use or disclosure of the contents of this message is not permitted
    and may be unlawful.

    *Disclaimer:* Email messages may be subject to delays,
    interception, non-delivery and unauthorised alterations.
    Therefore, information expressed in this message is not given or
    endorsed by AstraZeneca UK Limited unless otherwise notified by an
    authorised representative independent of this message. No
    contractual relationship is created by this message by any person
    unless specifically indicated by agreement in writing other than
    email.

    *Monitoring: *AstraZeneca UK Limited may monitor email traffic
    data and content for the purposes of the prevention and detection
    of crime, ensuring the security of our computer systems and
    checking compliance with our Code of Conduct and policies.

    *From:* owner-nmusers
m>
    <mailto:owner-nmusers
om>
    [mailto:owner-nmusers
    <mailto:owner-nmusers
om>] *On Behalf Of *Indranil
    Bhattacharya
    *Sent:* 03 December 2009 15:41

    *To:* nmusers
sers
    *Subject:* [NMusers] Model building


    Hi, I am in the process of developing a PK/PD model and have a
    naive question regarding model building.

    I currently do not have all the PD data and more data would be
    available in the future. For the current data set, I have tried
    models say A to D.

    Now model A (cell kill) and B (cell kill +transduction) converges
    using FOCE (no CV% but I am willing to live with that) but from
    the RES and WRES plots we can clearly see that there is some bias.
    The fit is OK but not great. The ofv values are around 400.

    Now models B (cell cycle specific kill), C (cell + precursor cell
    kill +transduction) and D (cell cycle specific + indirect response
    model) do not converge using FO or FOCE methods but when I look at
    the fits from the terminated runs, the fits are much better than
    those obtained with Model A, and there seems very little bias.
    Also the ofv values are between 160-250.


    So my question is whether the fits, RES, WRES plots and the ofv
    values have meaning even when the minimization terminates.


    Regards


    Neil

    -- Indranil Bhattacharya




--
Indranil Bhattacharya

--
Nick Holford, Professor Clinical Pharmacology
Dept Pharmacology & Clinical Pharmacology
University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand
tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53
email: n.holford
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford


--

Nick Holford, Professor Clinical Pharmacology

Dept Pharmacology & Clinical Pharmacology

University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand

tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53

email: n.holford

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Wed Dec 09 2009 - 17:07:44 EST

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to: nmusers-request@iconplc.com.

Once subscribed, you may contribute to the discussion by emailing: nmusers@globomaxnm.com.