NONMEM Users Network Archive

Hosted by Cognigen

Re: Models that abort before convergence

From: Nick Holford <n.holford>
Date: Wed, 19 Nov 2008 08:25:47 +1300

Dennis,

The hypothesis that NONMEM termination messages do not indicate whether
a model is fit for purpose has now been tested numerous times on
simulated and real data sets. No evidence has been found to reject this
hypothesis e.g.

Look here for my initial explorations of this problem:
http://www.cognigencorp.com/nonmem/nm/99jul152003.html
then you search on nmusers for "minimization terminated" using this URL:
http://www.mail-archive.com/nmusers
you will find several threads including:
http://www.mail-archive.com/nmusers

In addition to the discussion and references on nmusers you can also
look in these publications which report that there was no difference in
conclusions drawn by using or ignoring runs which NONMEM did not report
as being successful:
Ahn JE, Karlsson MO, Dunne A, Ludden TM. Likelihood based approaches to
handling data below the quantification limit using NONMEM VI. J
Pharmacokinet Pharmacodyn. 2008;35(4):401-21.
Byon W, Fletcher CV, Brundage RC. Impact of censoring data below an
arbitrary quantification limit on structural model misspecification. J
Pharmacokinet Pharmacodyn. 2008;35(1):101-16.

Therefore I recommend ignoring NONMEM's conclusion about whether a run
is successful or not and use more informative criteria based on common
sense evaluation of parameters and other priors plus credible
diagnostics such as VPC and NPDE:

Karlsson MO, Holford NHG. A Tutorial on Visual Predictive Checks. PAGE
17 (2008) Abstr 1434 [wwwpage-meetingorg/?abstract=1434]. 2008.
Comets E, Brendel K, Mentré F. Computing normalised prediction
distribution errors to evaluate nonlinear mixed-effect models: The npde
add-on package for R. Comput Methods Programs Biomed. 2008;90(2):154-66.

Finally, this paper reports a model that terminated with an even more
severe error message ('INFINITE OBJECTIVE FUNCTION AT NEXT ITERATION')
but the model itself was clearly OK when based on other more informative
criteria. It was also acceptable to peer reviewers.

Matthews I, Kirkpatrick C, Holford NHG. Quantitative justification for
target concentration intervention - Parameter variability and predictive
performance using population pharmacokinetic models for aminoglycosides.
British Journal of Clinical Pharmacology. 2004;58(1):8-19.

Nick


Dennis Fisher wrote:
> Colleagues,
>
> I am curious as to your thoughts about a particular NONMEM issue. I
> often find myself in a situation where a complex model does not
> converge to 3 digits ("no of digits: unreportable") yet the objective
> function is markedly better than a previous model and graphics suggest
> that the model is quite good (and better than the previous one). Nick
> Holford has advocated (and I agree) that NONMEM's SE's have minimal
> utility and the inability to calculate them is not important.
> However, I have not seen similar discussion about whether one can /
> should accept a model that did not converge.
>
> The particular situation that I dealing with at the moment is that a
> dataset that I am analyzing yielded a series of results that did not
> converge as I added parameters (despite an improving fit and a marked
> decrease in the objective function), then yet a more complicated model
> yielded 3.0 significant digits. In this case, there is no problem (I
> can use this final model for bootstrap, VPC, etc.) but what if none of
> these models had converged.
>
> Dennis
>
> Dennis Fisher MD
> P < (The "P Less Than" Company)
> Phone: 1-866-PLessThan (1-866-753-7784)
> Fax: 1-415-564-2220
> www.PLessThan.com
>

--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
n.holford
http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
Received on Tue Nov 18 2008 - 14:25:47 EST

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to: nmusers-request@iconplc.com.

Once subscribed, you may contribute to the discussion by emailing: nmusers@globomaxnm.com.