NONMEM Users Network Archive

Hosted by Cognigen

RE: Models that abort before convergence

From: Mark Sale - Next Level Solutions <mark>
Date: Wed, 19 Nov 2008 06:48:36 -0700
Leonid et al,

   I'm a little confused by t= his discussion.  To make an analogy, assume that drug company A has a = wonderful theory that drug B will treat a disease.  Theory makes sense= by your favorite epistemology criteria etc.  But of course, being goo= d scientists, we know that theories must be verified, so we do an experimen= t, and the data suggest that the theory is wrong.  Most of us would cr= iticize as unscientific someone who who discarded the data (didn't point ou= t flaws in the data, didn't provide opposing data, simply discounted it) in= favor of continuing to believe the theory.
   Why do we not a= pply the same standards here?  Theory says that models that do not con= verge (or fail covariance) are "bad".  Data (that so far as I know no = one has found to be flawed, nor provided opposing data) suggests that, by a= t least one criteria (same parameter estimates, same SD of parameter estima= tes) there are no important differences.  I don't disagree that failin= g a covariance step, or failing to converge provide information about a mod= el.  But it doesn't seem to be informative about what we probably real= ly care about -does the line go through the points, how confident are we WR= T the precision of the parameters and is the model predictive.
  I'= m not sure if the small number of published examples (of bootstrap with ~50= 0 samples) are a small number of anecdotes or a small number of trials with= N ~ 500, but I've run 5 or so myself and found the same to be consistently= the case. That is, a successful covariance step is not informative WRT the= parameter values or their precision.  I suspect others have similar e= xperience.  If there are other "studies"/anecdotes with different conc= lusions, someone should publish them.  Otherwise, it seems like we are= obligated to abandon this theory in favor of the data.

 

Mark Sale MD
Next Level Solutions, LLC
www.NextLevelSolns.com
919-846-9185

-------- Original Message --------
Subject: RE: [NMusers] Models that abort before convergence
From: LGibiansky Date: Tue, November 18, 2008 11:13 pm
To: fisher n.holford
Dennis,
I do not support extreme views (from places where people walk upside down :) ) that Nonmem error messages should be ignored: they serve the useful purpose to alert when Nonmem is having some difficulties, and should always=
be part of the picture. If the data looks good, model is simple, then we need to look for the reason for the poor convergence. Sometimes it helps to=
use SIGDIG= 5 or 6 to get 3 significant digits precision. But if you are<= br> working on the limit of the algorithms (as implemented) abilities:
nonlinear model + stiff differential equations + large range of doses and concentrations, etc., then you face the situation when you cannot force
convergence even if you try hard. On my recent project, none of the
intermediate model converged even though bootstrap provided pretty narrow CI (so it does not look like over-parametrized model), all diagnostic plots=
were good, and the visual predictive check was reasonable. Then you just blame the algorithm and move on. You loose the ability to justify your
covariate selection based on the objective function drop (which is not a good idea any way), and may need to provide a little bit more detailed
investigation to convince reviewers (regulatory and/or journal) that the model is adequate for the intended purpose.
Thanks
Leonid


Original Message:
-----------------
From: Dennis Fisher fisher Date: Tue, 18 Nov 2008 11:21:23 -0800
To: nmusers Subject: [NMusers] Models that abort before convergence


Colleagues,

I am curious as to your thoughts about a particular NONMEM issue. I
often find myself in a situation where a complex model does not
converge to 3 digits ("no of digits: unreportable") yet the objective
function is markedly better than a previous model and graphics suggest that the model is quite good (and better than the previous one). Nick Holford has advocated (and I agree) that NONMEM's SE's have minimal
utility and the inability to calculate them is not important.
However, I have not seen similar discussion about whether one can /
should accept a model that did not converge.

The particular situation that I dealing with at the moment is that a
dataset that I am analyzing yielded a series of results that did not
converge as I added parameters (despite an improving fit and a marked
decrease in the objective function), then yet a more complicated model yielded 3.0 significant digits. In this case, there is no problem (I
can use this final model for bootstrap, VPC, etc.) but what if none of these models had converged.

Dennis

Dennis Fisher MD
P < (The "P Less Than" Company)
Phone: 1-866-PLessThan (1-866-753-7784)
Fax: 1-415-564-2220
www.PLessThan.com


--------------------------------------------------------------------
mail2web.com - Microsoft® Exchange solutions from a leading provider -=
http://link.mail2web.= com/Business/Exchange


left.letterhead
(image/png attachment: left.letterhead)

Received on Wed Nov 19 2008 - 08:48:36 EST

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to: nmusers-request@iconplc.com.

Once subscribed, you may contribute to the discussion by emailing: nmusers@globomaxnm.com.