Leonid et al,
I'm a little confused by t=
his discussion. To make an analogy, assume that drug company A has a =
wonderful theory that drug B will treat a disease. Theory makes sense=
by your favorite epistemology criteria etc. But of course, being goo=
d scientists, we know that theories must be verified, so we do an experimen=
t, and the data suggest that the theory is wrong. Most of us would cr=
iticize as unscientific someone who who discarded the data (didn't point ou=
t flaws in the data, didn't provide opposing data, simply discounted it) in=
favor of continuing to believe the theory. Why do we not a=
pply the same standards here? Theory says that models that do not con=
verge (or fail covariance) are "bad". Data (that so far as I know no =
one has found to be flawed, nor provided opposing data) suggests that, by a=
t least one criteria (same parameter estimates, same SD of parameter estima=
tes) there are no important differences. I don't disagree that failin=
g a covariance step, or failing to converge provide information about a mod=
el. But it doesn't seem to be informative about what we probably real=
ly care about does the line go through the points, how confident are we WR=
T the precision of the parameters and is the model predictive. I'=
m not sure if the small number of published examples (of bootstrap with ~50=
0 samples) are a small number of anecdotes or a small number of trials with=
N ~ 500, but I've run 5 or so myself and found the same to be consistently=
the case. That is, a successful covariance step is not informative WRT the=
parameter values or their precision. I suspect others have similar e=
xperience. If there are other "studies"/anecdotes with different conc=
lusions, someone should publish them. Otherwise, it seems like we are=
obligated to abandon this theory in favor of the data.
Mark Sale MD
Next Level Solutions, LLC
www.NextLevelSolns.com
9198469185
 Original Message 
Subject: RE: [NMusers] Models that abort before convergence
From: LGibiansky Date: Tue, November 18, 2008 11:13 pm
To: fisher n.holford
Dennis,
I do not support extreme views (from places where people walk upside down
:) ) that Nonmem error messages should be ignored: they serve the useful
purpose to alert when Nonmem is having some difficulties, and should always=
be part of the picture. If the data looks good, model is simple, then we
need to look for the reason for the poor convergence. Sometimes it helps to=
use SIGDIG= 5 or 6 to get 3 significant digits precision. But if you are<=
br>
working on the limit of the algorithms (as implemented) abilities:
nonlinear model + stiff differential equations + large range of doses and
concentrations, etc., then you face the situation when you cannot force
convergence even if you try hard. On my recent project, none of the
intermediate model converged even though bootstrap provided pretty narrow
CI (so it does not look like overparametrized model), all diagnostic plots=
were good, and the visual predictive check was reasonable. Then you just
blame the algorithm and move on. You loose the ability to justify your
covariate selection based on the objective function drop (which is not a
good idea any way), and may need to provide a little bit more detailed
investigation to convince reviewers (regulatory and/or journal) that the
model is adequate for the intended purpose.
Thanks
Leonid
Original Message:

From: Dennis Fisher fisher Date: Tue, 18 Nov 2008 11:21:23 0800
To: nmusers Subject: [NMusers] Models that abort before convergence
Colleagues,
I am curious as to your thoughts about a particular NONMEM issue. I
often find myself in a situation where a complex model does not
converge to 3 digits ("no of digits: unreportable") yet the objective
function is markedly better than a previous model and graphics suggest
that the model is quite good (and better than the previous one). Nick
Holford has advocated (and I agree) that NONMEM's SE's have minimal
utility and the inability to calculate them is not important.
However, I have not seen similar discussion about whether one can /
should accept a model that did not converge.
The particular situation that I dealing with at the moment is that a
dataset that I am analyzing yielded a series of results that did not
converge as I added parameters (despite an improving fit and a marked
decrease in the objective function), then yet a more complicated model
yielded 3.0 significant digits. In this case, there is no problem (I
can use this final model for bootstrap, VPC, etc.) but what if none of
these models had converged.
Dennis
Dennis Fisher MD
P < (The "P Less Than" Company)
Phone: 1866PLessThan (18667537784)
Fax: 14155642220
www.PLessThan.com

mail2web.com  MicrosoftÂ® Exchange solutions from a leading provider =
http://link.mail2web.=
com/Business/Exchange
