NONMEM Users Network Archive

Hosted by Cognigen

RE: OMEGA selection

From: Mark Sale - Next Level Solutions <mark>
Date: Thu, 23 Apr 2009 08:50:39 -0700
  For details, I'd refer you to the abstract= s, I've never published this.  But, whenever I do a bootstrap I look a= t whether the samples that had a successful covariance step are different (= in mean or variability), just for my own interest.  They never have be= en different, I'd guess I've looked at 6 or so.  I have no records of = what fraction of samples had a successful covariance step.
I'd also refe= r to any number of good reference on how to decide if a model is "good" (pl= ots, biological plauability, reasonable parameters, various metrics of "goo= dness". etc.  I'd suggest that if your parameters are poorly defined b= y the data (e.g., all concentrations near EMAX, unable to define EC50) you'= ll invariably find that other metrics suggest lack of model goodness. = Whether and how successful covariance or minimization fits into this will = have to wait until we have a universally accepted metric of model "goodness= ".
I would list CI (based on bootstrap, not $COV) among my metrics of mo= del goodness, I'd even list a successful covariance step among metrics of m= odel goodness - but pretty far down the list. (everything else being equal,= I'd prefer a model that has a successful covariance step - of course every= thing else is never equal).

Mark Sale MD
Next Level Solutions, LLC

-------- Original Message --------
Subject: RE: [NMusers] OMEGA selection
From: Hang, Yaming <yaming_hang Date: Thu, April 23, 2009 10:35 am
To: "Mark Sale - Next Level Solutions" <mark Cc: <nmusers
= Hi Mark,
Very inter= esting point. In general, your logic about why the covariance step doesn't = matter in the bootstrapping case makes sense to me. However, I have some qu= estions about why such a conclusion was reached. My questions are: 1. = how many data sets are bootstrapped, 2. among them, what's the frequen= cy of failed vs. successful covariance step, 3. are parameter estimates the= mselves similar across different bootstraps, 4. are there any major di= fference among the data sets leading to successful and failed covariance st= ep?
I am imagining an example: with an Emax model, I generate two data sets,= one with good distribution with regard to the X variable (say concent= ration) and the other with ill distribution. So that the first da= ta set gives me a successful run including $COV step with re= asonable estimates for Emax and EC50, the second data set will lead to a&nb= sp;total failure in estimation, even estimates for Emax and EC50 canno= t be obtained. I guess I cannot use this as a basis to conclude that even t= he $ESTIMATE step is not reliable, since both data sets are coming from the= same population, right?
I'd love to hear your thoughts on this one.

Nick et al.
    At th= is risk of starting an discussion that probably has little mileage left in = it.  First I agree with Nick on covariance - it probably doesn't matte= r.  But, I'd like to point out what may be an error in our logic. = ;
We content that we have demonstrated that covariance doesn't matter.&= nbsp; Our evidence is that, when bootstrapping, the parameters for the samp= le that have successful covariance are not different from those that failed= .  So, we conclude that the results are the same regardless of covaria= nce outcome across sampled data sets - the independent variable in this tes= t is the data set, the model is fixed.
In model selection/building, we h= ave a fixed data set and the independent variable is the model structure.&n= bsp;  Whether covariance success is a useful predictor across differen= t models with a fixed data set is a different question than whether covaria= nce is a useful predictor across data sets with a fixed model.
But, in t= he end, I do agree that biological plausibility, diagnostic plots, reasonab= le parameters and some suggestion of numerical stability/identifiably (such= as bootstrap CIs) are more important than a successful covariance step.

Mark Sale MD
Next Level Solutions, LLC

-------- Original Message --------
Subject: Re: [NMusers] OMEGA select= ion
From: Nick Holford <n.holford ril 15, 2009 12:17 pm
To: nmusers not pay any attention to whether or not the $COV step runs or even if
= the run is 'SUCCESSFUL' to conclude anything about your model. Your
opi= nion is not supported experimentally e.g. see

NONMEM has no idea if the parameters make= sense or not and will happily
converge with models that are overparame= terised. You cannot rely on a
failed $COV step or a MINIMIZATION TERMIN= ATED message to conclude the
model is not a good one. You need to use y= our brains (NONMEM does not
have a brain) and your common sense to deci= de if your model makes sense
or is perhaps overparameterised.

Ni= ck

Ethan Wu wrote:
> Dear all,
> I am fi= tting a PD response, and the equation goes like this:
> total= response = baseline+f(placebo response) +f(drug response)
>= ; first, I tried full omega block, and model was able to converge, but
= > $COV stop failed.
> To me, this indicates that too many = parameters in the model. The
> structure model is rather simple one,= so I think probably too many Etas.
> I wonder is there a goo= d principle of Eta reduction that I could
> implement here. Any good= reference?

Nick Holford, Dept Pharmacology &= amp; Clinical Pharmacology
University of Auckland, 85 Park Rd, Private B= ag 92019, Auckland, New Zealand
n.holford 730 fax:+64(9)373-7090
mobile: +33 64 271-6369 (Apr 6-Jul 17 2009)

<= /div>
Notice:  This e-mail message, toge=
ther with any attachments, contains
information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station,
New Jersey, USA 08889), and/or its affiliates (which may be known
outside the United States as Merck Frosst, Merck Sharp & Dohme or
MSD and in Japan, as Banyu - direct contact information for affiliates is
available at that may be
confidential, proprietary copyrighted and/or legally privileged. It is
intended solely for the use of the individual or entity named on this
message. If you are not the intended recipient, and have received this
message in error, please notify us immediately by reply e-mail and
then delete it from your system.

(image/png attachment: left.letterhead)

(image/png attachment: 02-left.letterhead)

Received on Thu Apr 23 2009 - 11:50:39 EDT

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to:

Once subscribed, you may contribute to the discussion by emailing: