NONMEM Users Network Archive

Hosted by Cognigen

Re: unscientific poll

From: Joachim.Grevel
Date: Thu, 4 Dec 2008 08:08:07 +0100

Thanks Leonid for this service to our community! I am not surpri=
sed by the results but rather by the number of replies. There are s=
upposedly 1000 NONMEM users on this beautiful planet, and 35 particip=
ated in the survey. My guess is that these are about 50% of the act=
ive participants of the usernet. Therefore we may have, let's say, 100 =
regular readers of the usernet. That is 10% of the users. That nu=
mber troubles me as it tells me that 90% of the NONMEM users have est=
ablished their routine and see little need (have little time) to read=
 the postings of the usernet. Yet, the discussions on our usernet a=
re extremely relevant to our daily modeling work. Cheer=
s to our usernet! Joachim _________________________________=
_________ Joachim GREVEL, Ph.D. MERCK SERONO International S=
.A. Exploratory Medicine 1202 Geneva Tel: +41.22.414=
.4751 Fax: +41.22.414.3059 Email: joachim.grevel
erono.net Leonid Gibiansky <LGibiansky=
  
/2008 07:00 PM To nmusers <nmusers
c Subject [NMusers] unscientific poll =

for the questions that I sent out recently. I received a total of 3=
5 replies. This e-mail consists of 4 parts, as follows: =
Part 1: Each original questions is followed by the summary of replies =
Part 2: All comments that I received are copy-pasted after the summar=
y of replies. Part 3: CSV file with the original data is=
 copy-pasted after the comments Part 4: R code that I used to su=
mmarize the results is provided Thanks to all who participated=
. Leonid -------------------------------------- Leon=
id Gibiansky, Ph.D. President, QuantPharm LLC web: www.q=
uantpharm.com e-mail: LGibiansky at quantpharm.com tel: (3=
01) 767 5566 ############ RESULTS ########################### =

syntax) > error/warning messages (134, 137, number of significant dig=
its, etc.) > and "MINIMIZATION SUCCESSFUL" messages (YES/NO): =
      question YES No Missing Q1 2 (5=
.7%) 33 (94.3%) 0 (0%) > 2. Do you remember at least o=
ne example when the run-time error message helped you to find an erro=
r in your code (YES/NO): question YES No Missing=
Q2 31 (88.6%) 3 (8.6%) 1 (2.9%) > 3=
. In your experience, run-time error messages allow you to detect m=
odel errors or problems quicker than it would be done without error m=
essages: (agree/disagree) question AGREE No Missin=
g Q3 27 (77.1%) 3 (8.6%) 5 (14.3%) > 4=
. Have you ever used in your report/publication ANY model that did =
not have $COV step completed (YES/NO): question YES =
  No Missing Q4 25 (71.4%) 9 (25.7%) 1 (2.9%)=
> 5. Have you ever used in your report/publication ANY model=
 that did not converge (YES/NO): question YES =
  No Missing Q5 13 (37.1%) 21 (60%) 1 (2.9%) =

that did > not have $COV step completed (YES/NO): question =
       YES No Missing Q6 16 (45.7%) 18 (51.=
4%) 1 (2.9%) > 7. Have you ever used in your report/public=
ation FINAL model that did > not converge (YES/NO): questi=
on YES No Missing Q7 3 (8.6%) 31 (8=
8.6%) 1 (2.9%) > 8. Define yourself as novice/intermedia=
te/experienced Nonmem user: Missing Novice Intermediate Exper=
ienced 1 3 15 16 #######=
##### Comments ########################### Honestly, nobody is p=
roposing to remove the minimization successful statement or any non sy=
ntax error message. I agree with the comments that stress it's impor=
tant to take them with a pinch of salt as these "errors" not always po=
int you in the right directions, and not always being picky about $COV=
 step or number of significant digits help in selecting the best model=
------------------------- Question 1 has at least 3 parts =
and cannot be answered YES or NO in any meaningful way. Please note=
 the messages we have discussed are not ERROR or WARNING messages. =
They are a message about the minimization status. I would be h=
appy if NONMEM stuck to the facts. It can tell me if it achieved th=
e requested sigdigs (CONVERGED) or ran out of function evals (PREMATU=
RE TERMINATION). But it should keep its subjective judgements to it=
self. If you made a list of run-time error messages and anothe=
r of run-time warning messages then perhaps your survey could be more=
 helpful in deciding which are meaningful? ---------------=
------- My 2 cents on the discussion, for what it?s worth? The =
error message is usually related to an error in the dataset / dataset=
 programming issue / initial estimates / model parameterization. G=
enerally, all need to be addressed. Just as important, I wouldn?t =
trust a MINIMIZATION SUCCESSFUL message either as you will often see =
flip-flop or 3-CMT identifiability issues that NONMEM doesn?t flag.=
  NONMEM has its flaws, but I?ve seen many of NONMEM?s error messages=
 disappear after correcting a programming error in the dataset, refin=
ing the initial estimates, or re-parameterizing the model. I think=
 the practical pharmacometrician isn?t going to accept NONMEM outputs=
 strictly at face value, but would challenge the findings to confirm =
the results. That?s just good science. ---------------------- =

nverge or failed the COV step. My guess is that individuals who fr=
equently do probably tend to be more mechanistic in their model build=
ing than I am and often push the complexity of their models beyond wh=
at can be supported by the data in hand. For those that do report =
out models that don't converge, I wonder if they have tried re-runnin=
g their models with different starting values (15-20% different) and =
see if NONMEM fails to converge at the same set of parameter estimate=
s. My guess is in many cases it won't although both sets of estima=
tes may appear "reasonable" and give similar fits and VPC. =

models, my thinking is that rather than using approximate maximum =

  model is unstable or not fully supported by the data, I think they w=
ould be better served by using a Bayesian approach. That way they =
can be explicit about the strength of their priors and they don't hav=
e to worry about convergence and COV step failures. JMHO. =
------------------ ############## Original Data #########=
######### ID,Q1,Q2,Q3,Q4,Q5,Q6,Q7,Q8 1,0,0,-1,1,1,1,0,1 =

,0,0,-1,1,0,1,0,2 6,0,1,-1,1,0,1,0,3 7,0,1,1,0,0,0,0,3 8,1,1=
,0,0,0,0,0,3 9,0,1,1,0,0,0,0,2 10,0,1,1,1,0,0,0,-1 11,0,1,1,=
1,0,1,0,3 12,0,0,1,0,0,0,0,2 13,0,1,1,1,1,1,0,2 14,0,1,1,0,0=
,0,0,3 15,0,1,1,1,1,1,0,3 16,1,-1,0,1,1,1,0,2 17,0,1,1,-1,-1=
,-1,-1,1 18,0,1,1,0,0,0,0,3 19,0,1,1,1,0,0,0,3 20,0,1,1,1,0,=
0,0,3 21,0,1,1,1,1,0,0,1 22,0,1,1,0,0,0,0,2 23,0,1,0,1,1,1,0=
,3 24,0,1,1,1,1,0,0,2 25,0,1,1,1,1,0,0,3 26,0,1,1,0,0,0,0,2=
27,0,1,1,1,0,0,0,2 28,0,1,1,1,1,1,0,2 29,0,1,1,1,0,1,0,3 =

33,0,1,1,1,1,1,0,2 34,0,1,1,1,0,0,0,2 35,0,1,1,0,0,0,0,2 =

.data <- read.table("C:/poll.csv",sep=",",header=T) res <- N=
ULL for(Qname in paste("Q",1:7,sep="") ){ x <- raw.data[=
,Qname] n.yes <- sum(x == 1) n.no <- sum(x ==
= 0) n.na <- sum(x == -1) n <- length(x) =
    temp <- data.frame(n=n,question=Qname, YE=
S=paste(n.yes," (",round(100*n.yes/n,1),"%)",sep=""), =
           No=paste(n.no," (",round(100*n.no/n,1),"%)",sep=""), =

1),"%)",sep="")) res <- rbind(res,temp) } res tab=
le(raw.data$Q8) ##############################################=
############ ---------------------=
-------------------- This message and any attachment are confidential,=
 may be privileged or otherwise protected from disclosure and are intende=
d only for use by the addressee(s) named herein. If you are not the int=
ended recipient, you must not copy this message or attachment or disclose=

sion in error, please notify the sender immediately and delete the messag=
e and any attachment from your system. Merck Serono does not accept lia=
bility for any omissions or errors in this message which may arise as a r=
esult of E-Mail-transmission or for damages resulting from any unauthoriz=
ed changes of the content of this message and any attachment thereto. I=
f verification is required, please request a hard-copy version. Merck S=
erono does not guarantee that this message is free of viruses and does no=
t accept liability for any damages caused by any virus transmitted ther=
ewith.

--=_alternative 00273275C1257515_=--
Received on Thu Dec 04 2008 - 02:08:07 EST

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to: nmusers-request@iconplc.com.

Once subscribed, you may contribute to the discussion by emailing: nmusers@globomaxnm.com.