NONMEM Users Network Archive

Hosted by Cognigen

RE: unbalanced design

From: Bob Leary <bleary>
Date: Wed, 3 Sep 2008 09:24:54 -0400

Hi -
Here "robust" seems to be being used to have the meaning "unbiased"
(or perhaps asymptotically unbiased or even consistent).
The more usual statistical meaning
of robust with respect to an estimation method is that the method
is relatively resilient to small departures from model assumptions,
such as some degree of non-normality of residuals or random effects. =
For example,
the mean is not a robust measure of central tendency of a distribution,
whereas the median is robust. Most classical maximum likelihood-based
estimation methods based on normality assumptions are not robust,
and in this sense none of the usual NONMEM parametric methods is robust,
regardless of the experimental design.
Non-parametric methods (e.g. the median is a nonparametric estimator)
tend to be more robust.

In the sense of being asymptotially unbiased or the stronger condition
of being consistent, NONMEM FOCE and Laplacian methods
are (weakly) consistent in the sense that they will converge to
the true parameter values as (loosely speaking, since
there are degenerate cases where this is not true)
both the number of subjects and the amount of data per subject
increase without bound. The are not strongly consistent in the sense
that biased estimates will still be produced if the amount
of data increases without bound but either the number
of subjects or amount of data per subject remains bounded.

The FO method is biased regardless of the amount of data.
In fact, FO results often become worse as the amount of data per subject
increases . Alan Schumitzky has a nice example of this
in which he obtains a lower bound on the FO bias for a particular
model where this bound in fact increases with the amount of data per =
The problem is that the joint likelihood function for each individual
becomes more and more peaked around its mode (the empirical Bayes =
but the FO method is based on an implicit quadratic
extrapolation to estimate the mode position, and the quality of this
extrapolation becomes poorer as the joint likelihood becomes more =

Robert H. Leary, PhD
Principal Software Engineer
Pharsight Corp.
5520 Dillard Dr., Suite 210
Cary, NC 27511

Phone/Voice Mail: (919) 852-4625, Fax: (919) 859-6871

> This email message (including any attachments) is for the sole use of =
the intended recipient and may contain confidential and proprietary =
information. Any disclosure or distribution to third parties that is =
not specifically authorized by the sender is prohibited. If you are not =
the intended recipient, please contact the sender by reply email and =
destroy all copies of the original message.

-----Original Message-----
From: owner-nmusers
Sent: Tuesday, September 02, 2008 22:53 PM
To: 'Nick Holford'; 'Wang, Yaning'
Cc: 'Mark Sale - Next Level Solutions'; nmusers
Subject: RE: [NMusers] unbalanced design


In Nick's example, the bias in disease progression parameters may indeed =
higher in the unbalanced design compared to the full, more extensive, =
in all subjects. However, that would in my mind come from data =
Bias would be expected to be even larger when all subjects have the =
design if for example the FOCE method is used. Whenever data per subject
becomes sparser, the FOCE method becomes more like the FO method and
therefore in general more biased in the parameter estimates.
Thus, robustness would decrease in the order "rich balanced design",
"rich+sparse unbalanced design", "sparse unbalanced design". Apart from =
effect I know of no reason to expect unbalanced designs not to be robust =
the model is correctly specified.

Best regards,

Mats Karlsson, PhD
Professor of Pharmacometrics
Dept of Pharmaceutical Biosciences
Uppsala University
Box 591
751 24 Uppsala Sweden
phone: +46 18 4714105
fax: +46 18 471 4003

-----Original Message-----
From: owner-nmusers
Behalf Of Nick Holford
Sent: Tuesday, September 02, 2008 10:03 PM
To: Wang, Yaning
Cc: Mark Sale - Next Level Solutions; nmusers
Subject: Re: [NMusers] unbalanced design


Its not clear to me what Mark had in mind when he asked if " mixed
effect modeling (NONMEM in particular) is robust".

But Susan proposes its just obviously OK <grin> and Yaning suggests
reading a book for the simple case of linear models. But what about the
real world i.e. non-linear mixed models?

And surely there must be some degree of imbalance that would lead to a
non-robust description when using a mixed model? e.g. if one is trying
to described a disease progress curve and some people are followed long
enough to identify an exponential shape while others are followed for a
shorter time and appear to have a linear shape then wouldn't there be
some bias in the resulting estimates describing the curve depending on
the mix of short or long follow up times?


Willavize, Susan wrote:

Hi Mark,


This should be true just based on the nature of mixed effects modeling.
 If you are not convinced, you may want to try some examples where you
simulate balanced and unbalanced designs and then estimate. J


Best Regard

Wang, Yaning wrote:
> Linear Mixed Models for Longitudinal Data by Geert Verbeke
> Geert Molenberghs
> Yaning Wang, Ph.D.
> Team Leader, Pharmacometrics
> Office of Clinical Pharmacology
> Office of Translational Science
> Center for Drug Evaluation and Research
> U.S. Food and Drug Administration
> Phone: 301-796-1624
> Email:
> "The contents of this message are mine personally and do not
> necessarily reflect any position of the Government or the Food and
> Drug Administration."
> =
> *From:* owner-nmusers
> [mailto:owner-nmusers
> Level Solutions
> *Sent:* Tuesday, September 02, 2008 1:28 PM
> *To:* nmusers
> *Subject:* [NMusers] unbalanced design
> Does anyone have a reference to a publication assessing whetheor
> unbalances studies? I see if in a number of courses (including the
> original beginners course for NONMEM), but can't find a publication.
> thanks
> Mark Sale MD
> Next Level Solutions, LLC
> <>
> 919-846-9185

Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New =
Received on Wed Sep 03 2008 - 09:24:54 EDT

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to:

Once subscribed, you may contribute to the discussion by emailing: