From: Bob Leary <*bleary*>

Date: Wed, 3 Sep 2008 09:24:54 -0400

Hi -

Here "robust" seems to be being used to have the meaning "unbiased"

(or perhaps asymptotically unbiased or even consistent).

The more usual statistical meaning

of robust with respect to an estimation method is that the method

is relatively resilient to small departures from model assumptions,

such as some degree of non-normality of residuals or random effects. =

For example,

the mean is not a robust measure of central tendency of a distribution,

whereas the median is robust. Most classical maximum likelihood-based

estimation methods based on normality assumptions are not robust,

and in this sense none of the usual NONMEM parametric methods is robust,

regardless of the experimental design.

Non-parametric methods (e.g. the median is a nonparametric estimator)

tend to be more robust.

In the sense of being asymptotially unbiased or the stronger condition

of being consistent, NONMEM FOCE and Laplacian methods

are (weakly) consistent in the sense that they will converge to

the true parameter values as (loosely speaking, since

there are degenerate cases where this is not true)

both the number of subjects and the amount of data per subject

increase without bound. The are not strongly consistent in the sense

that biased estimates will still be produced if the amount

of data increases without bound but either the number

of subjects or amount of data per subject remains bounded.

The FO method is biased regardless of the amount of data.

In fact, FO results often become worse as the amount of data per subject

increases . Alan Schumitzky has a nice example of this

in which he obtains a lower bound on the FO bias for a particular

model where this bound in fact increases with the amount of data per =

subject.

The problem is that the joint likelihood function for each individual

becomes more and more peaked around its mode (the empirical Bayes =

estimate),

but the FO method is based on an implicit quadratic

extrapolation to estimate the mode position, and the quality of this

extrapolation becomes poorer as the joint likelihood becomes more =

peaked.

Robert H. Leary, PhD

Principal Software Engineer

Pharsight Corp.

5520 Dillard Dr., Suite 210

Cary, NC 27511

Phone/Voice Mail: (919) 852-4625, Fax: (919) 859-6871

*> This email message (including any attachments) is for the sole use of =
*

the intended recipient and may contain confidential and proprietary =

information. Any disclosure or distribution to third parties that is =

not specifically authorized by the sender is prohibited. If you are not =

the intended recipient, please contact the sender by reply email and =

destroy all copies of the original message.

-----Original Message-----

From: owner-nmusers

[mailto:owner-nmusers

Sent: Tuesday, September 02, 2008 22:53 PM

To: 'Nick Holford'; 'Wang, Yaning'

Cc: 'Mark Sale - Next Level Solutions'; nmusers

Subject: RE: [NMusers] unbalanced design

Hi,

In Nick's example, the bias in disease progression parameters may indeed =

be

higher in the unbalanced design compared to the full, more extensive, =

design

in all subjects. However, that would in my mind come from data =

sparseness.

Bias would be expected to be even larger when all subjects have the =

sparser

design if for example the FOCE method is used. Whenever data per subject

becomes sparser, the FOCE method becomes more like the FO method and

therefore in general more biased in the parameter estimates.

Thus, robustness would decrease in the order "rich balanced design",

"rich+sparse unbalanced design", "sparse unbalanced design". Apart from =

this

effect I know of no reason to expect unbalanced designs not to be robust =

if

the model is correctly specified.

Best regards,

Mats

Mats Karlsson, PhD

Professor of Pharmacometrics

Dept of Pharmaceutical Biosciences

Uppsala University

Box 591

751 24 Uppsala Sweden

phone: +46 18 4714105

fax: +46 18 471 4003

-----Original Message-----

From: owner-nmusers

On

Behalf Of Nick Holford

Sent: Tuesday, September 02, 2008 10:03 PM

To: Wang, Yaning

Cc: Mark Sale - Next Level Solutions; nmusers

Subject: Re: [NMusers] unbalanced design

Hi,

Its not clear to me what Mark had in mind when he asked if " mixed

effect modeling (NONMEM in particular) is robust".

But Susan proposes its just obviously OK <grin> and Yaning suggests

reading a book for the simple case of linear models. But what about the

real world i.e. non-linear mixed models?

And surely there must be some degree of imbalance that would lead to a

non-robust description when using a mixed model? e.g. if one is trying

to described a disease progress curve and some people are followed long

enough to identify an exponential shape while others are followed for a

shorter time and appear to have a linear shape then wouldn't there be

some bias in the resulting estimates describing the curve depending on

the mix of short or long follow up times?

Nick

Willavize, Susan wrote:

Hi Mark,

This should be true just based on the nature of mixed effects modeling.

If you are not convinced, you may want to try some examples where you

simulate balanced and unbalanced designs and then estimate. J

Best Regard

Wang, Yaning wrote:

*>
*

*>
*

*> Linear Mixed Models for Longitudinal Data by Geert Verbeke
*

*>
*

<http://www.amazon.com/exec/obidos/search-handle-url/102-2006236-4753744?=

%5F

encoding=UTF8&search-type=ss&index=books&field-author=Geert%20Ver=

beke>,

*> Geert Molenberghs
*

*>
*

<http://www.amazon.com/exec/obidos/search-handle-url/102-2006236-4753744?=

%5F

encoding=UTF8&search-type=ss&index=books&field-author=Geert%20Mol=

enberghs>

*>
*

*>
*

*>
*

*> Yaning Wang, Ph.D.
*

*> Team Leader, Pharmacometrics
*

*> Office of Clinical Pharmacology
*

*> Office of Translational Science
*

*> Center for Drug Evaluation and Research
*

*> U.S. Food and Drug Administration
*

*> Phone: 301-796-1624
*

*> Email: yaning.wang *

*>
*

*> "The contents of this message are mine personally and do not
*

*> necessarily reflect any position of the Government or the Food and
*

*> Drug Administration."
*

*>
*

*>
*

*>
*

*> =
*

------------------------------------------------------------------------

*> *From:* owner-nmusers *

*> [mailto:owner-nmusers *

*> Level Solutions
*

*> *Sent:* Tuesday, September 02, 2008 1:28 PM
*

*> *To:* nmusers *

*> *Subject:* [NMusers] unbalanced design
*

*>
*

*>
*

*> Does anyone have a reference to a publication assessing whetheor
*

*> unbalances studies? I see if in a number of courses (including the
*

*> original beginners course for NONMEM), but can't find a publication.
*

*> thanks
*

*>
*

*>
*

*> Mark Sale MD
*

*> Next Level Solutions, LLC
*

*> www.NextLevelSolns.com <http://www.NextLevelSolns.com>
*

*> 919-846-9185
*

*>
*

--

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New =

Zealand

n.holford

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Wed Sep 03 2008 - 09:24:54 EDT

Date: Wed, 3 Sep 2008 09:24:54 -0400

Hi -

Here "robust" seems to be being used to have the meaning "unbiased"

(or perhaps asymptotically unbiased or even consistent).

The more usual statistical meaning

of robust with respect to an estimation method is that the method

is relatively resilient to small departures from model assumptions,

such as some degree of non-normality of residuals or random effects. =

For example,

the mean is not a robust measure of central tendency of a distribution,

whereas the median is robust. Most classical maximum likelihood-based

estimation methods based on normality assumptions are not robust,

and in this sense none of the usual NONMEM parametric methods is robust,

regardless of the experimental design.

Non-parametric methods (e.g. the median is a nonparametric estimator)

tend to be more robust.

In the sense of being asymptotially unbiased or the stronger condition

of being consistent, NONMEM FOCE and Laplacian methods

are (weakly) consistent in the sense that they will converge to

the true parameter values as (loosely speaking, since

there are degenerate cases where this is not true)

both the number of subjects and the amount of data per subject

increase without bound. The are not strongly consistent in the sense

that biased estimates will still be produced if the amount

of data increases without bound but either the number

of subjects or amount of data per subject remains bounded.

The FO method is biased regardless of the amount of data.

In fact, FO results often become worse as the amount of data per subject

increases . Alan Schumitzky has a nice example of this

in which he obtains a lower bound on the FO bias for a particular

model where this bound in fact increases with the amount of data per =

subject.

The problem is that the joint likelihood function for each individual

becomes more and more peaked around its mode (the empirical Bayes =

estimate),

but the FO method is based on an implicit quadratic

extrapolation to estimate the mode position, and the quality of this

extrapolation becomes poorer as the joint likelihood becomes more =

peaked.

Robert H. Leary, PhD

Principal Software Engineer

Pharsight Corp.

5520 Dillard Dr., Suite 210

Cary, NC 27511

Phone/Voice Mail: (919) 852-4625, Fax: (919) 859-6871

the intended recipient and may contain confidential and proprietary =

information. Any disclosure or distribution to third parties that is =

not specifically authorized by the sender is prohibited. If you are not =

the intended recipient, please contact the sender by reply email and =

destroy all copies of the original message.

-----Original Message-----

From: owner-nmusers

[mailto:owner-nmusers

Sent: Tuesday, September 02, 2008 22:53 PM

To: 'Nick Holford'; 'Wang, Yaning'

Cc: 'Mark Sale - Next Level Solutions'; nmusers

Subject: RE: [NMusers] unbalanced design

Hi,

In Nick's example, the bias in disease progression parameters may indeed =

be

higher in the unbalanced design compared to the full, more extensive, =

design

in all subjects. However, that would in my mind come from data =

sparseness.

Bias would be expected to be even larger when all subjects have the =

sparser

design if for example the FOCE method is used. Whenever data per subject

becomes sparser, the FOCE method becomes more like the FO method and

therefore in general more biased in the parameter estimates.

Thus, robustness would decrease in the order "rich balanced design",

"rich+sparse unbalanced design", "sparse unbalanced design". Apart from =

this

effect I know of no reason to expect unbalanced designs not to be robust =

if

the model is correctly specified.

Best regards,

Mats

Mats Karlsson, PhD

Professor of Pharmacometrics

Dept of Pharmaceutical Biosciences

Uppsala University

Box 591

751 24 Uppsala Sweden

phone: +46 18 4714105

fax: +46 18 471 4003

-----Original Message-----

From: owner-nmusers

On

Behalf Of Nick Holford

Sent: Tuesday, September 02, 2008 10:03 PM

To: Wang, Yaning

Cc: Mark Sale - Next Level Solutions; nmusers

Subject: Re: [NMusers] unbalanced design

Hi,

Its not clear to me what Mark had in mind when he asked if " mixed

effect modeling (NONMEM in particular) is robust".

But Susan proposes its just obviously OK <grin> and Yaning suggests

reading a book for the simple case of linear models. But what about the

real world i.e. non-linear mixed models?

And surely there must be some degree of imbalance that would lead to a

non-robust description when using a mixed model? e.g. if one is trying

to described a disease progress curve and some people are followed long

enough to identify an exponential shape while others are followed for a

shorter time and appear to have a linear shape then wouldn't there be

some bias in the resulting estimates describing the curve depending on

the mix of short or long follow up times?

Nick

Willavize, Susan wrote:

Hi Mark,

This should be true just based on the nature of mixed effects modeling.

If you are not convinced, you may want to try some examples where you

simulate balanced and unbalanced designs and then estimate. J

Best Regard

Wang, Yaning wrote:

<http://www.amazon.com/exec/obidos/search-handle-url/102-2006236-4753744?=

%5F

encoding=UTF8&search-type=ss&index=books&field-author=Geert%20Ver=

beke>,

<http://www.amazon.com/exec/obidos/search-handle-url/102-2006236-4753744?=

%5F

encoding=UTF8&search-type=ss&index=books&field-author=Geert%20Mol=

enberghs>

------------------------------------------------------------------------

--

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New =

Zealand

n.holford

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Wed Sep 03 2008 - 09:24:54 EDT