From: Gastonguay, Marc <*marcg*>

Date: Tue, 20 Mar 2007 17:43:37 -0400

Hello Mark & nmusers,

I'm just catching up with the flurry of emails on this topic...

I don't think that anyone mentioned full model approaches to

covariate modeling, although we have discussed this topic in detail

in past nmusers threads. Frank Harrell's Regression Modeling

Strategies text (I'll second Tony's recommendation) advocates this

method as an alternative to stepwise methods when the purpose is to

estimate the effect of covariates. The text also includes a useful

discussion of the choice of modeling strategy as it relates to

modeling objectives (e.g. prediction, effect estimation, hypothesis

testing).

In our group, we routinely apply full model methods for population PK =

covariate modeling and have managed to make useful inferences about

covariate effects while avoiding stepwise methods and p-values

altogether. Some examples of this method will be presented at ASCPT

later this week.

I also agree with the sentiment expressed by several contributors

that we shouldn't be so concerned with finding the one perfect model. =

Instead, we should probably spend more time evaluating the impact of

model deficiencies on the intended model-based applications and

inferences.

In addition to Harrell's book, some relevant references are listed

below.

Best regards,

Marc

Marc R. Gastonguay, Ph.D.

Scientific Director, Metrum Institute [www.metruminstitute.org]

President & CEO, Metrum Research Group LLC [www.metrumrg.com]

Email: marcg

+1.860.735.7043

1. Ulrika Wählby, E. Niclas Jonsson and Mats O. Karlsson AAPS

PharmSci 2002; 4 (4) article 27 (http://www.aapspharmsci.org).

Comparison of Stepwise Covariate Model Building Strategies in

Population Pharmacokinetic-Pharmacodynamic Analysis ****(full model =

approach is described in Discussion section).

2. Steyerberg EW, Eijkemans MJ, Habbema JD. Stepwise selection in

small data sets: a simulation study of bias in logistic regression

analysis. J Clin Epidemiol. October 1999;52(10):935-942.

3. Harrell FE, Jr., Lee KL, Mark DB. Multivariable prognostic models: =

issues in developing models, evaluating assumptions and adequacy, and =

measuring and reducing errors. Stat Med. 1996;15(4):361-387.

4. Steyerberg EW, Eijkemans MJ, Harrell FE, Jr., Habbema JD.

Prognostic modelling with logistic regression analysis: a comparison

of selection and estimation methods in small data sets. Stat Med.

2000;19(8):1059-1079.

5. M.R. Gastonguay. A Full Model Estimation Approach for Covariate

Effects: Inference Based on Clinical Importance and Estimation

Precision. The AAPS Journal; 6(S1), Abstract W4354, 2004. (http://

metrumrg.com/publications/full_model.pdf)

6. Balaji Agoram; Anne C. Heatherington; Marc R. Gastonguay.

Development and Evaluation of a Population Pharmacokinetic-

Pharmacodynamic Model of Darbepoetin Alfa in Patients with Nonmyeloid =

Malignancies Undergoing Multicycle Chemotherapy. AAPS PharmSci Vol.: =

8, No.: 3, 2006

On Mar 19, 2007, at 3:34 PM, AJ Rossini wrote:

*> I'd highly recommend reading Frank Harrell's book on Regression
*

*> Modeling if
*

*> you think that stepwise regression makes any sense. While much of =
*

*> the book
*

*> applies to linear and generalized linear (i.e. categorical, etc)
*

*> regression
*

*> models, nonlinear models (and mixed effects models) would generally =
*

*> fall into
*

*> the "well, if the simple case was like that, it can't be any
*

*> simpler for the
*

*> harder cases..."... Frank demonstrates some of the reasons that p-
*

*> values
*

*> from models generated using stepwise modeling are fairly useless
*

*> (i.e. don't
*

*> follow the behavior you'd expect from p-values).
*

*>
*

*> The literature to start looking at would be modern variable selection
*

*> techniques for linear regression, i.e. work at Stanford Statistics
*

*> by Hastie,
*

*> Tibshirani, and their collaborators and former grad students
*

*> (LASSO, LARS,
*

*> elastic nets, and similar approaches).
*

*>
*

*> On Monday 19 March 2007 19:32, Mark Sale - Next Level Solutions wrote:
*

*>> Dear Colleagues,
*

*>> I've lately been reviewing the literature on model building/
*

*>> selection
*

*>> algorithms. I have been unable to find any even remotely rigorous
*

*>> discussion of the way we all build NONMEM models. The structural
*

*>> first, then variances/forward addition/backward elimination is
*

*>> generally mentioned in a number of places (Ene Ettes in Ann
*

*>> Pharmacother, 2004, Jaap Mandemas series on POP PK series J PK
*

*>> Biopharm
*

*>> in 1992, Jose Pinheiros paper from the Joint Stats meeting in 1994,
*

*>> Peter Bonates AAPS journal article in 2005, Mats Karlsons AAPS
*

*>> PharmSci, 2002, the FDA guidance on Pop PK). It is most explicitly
*

*>> stated in the NONMEM manuals (Vol 5, figure 11.1) - without any
*

*>> reference. From the NONMEM manuals it is reproduced in many =
*

courses,

*>> and has become axiomatic. I've looked at the stats literature on
*

*>> forward addition/backwards elimination in both linear and logistic
*

*>> regression, where it is at least formally discussed (with some
*

*>> disagreement about whether it is "correct"). But, I am unable to
*

*>> find
*

*>> any justification for the structural first, then covariates (drive by
*

*>> post-hoc plots), then variance effects approach we use (I'm sure many
*

*>> people will point out that it is not nearly that linear a process,
*

*>> although in figure 11.1, Vol 5 of the NONMEM manuals, it is
*

*>> depicted as
*

*>> a step-by-step algorithm, without any looping back). Can anyone
*

*>> point
*

*>> me to any rigorous discussion of this model building strategy?
*

*>>
*

*>> Mark Sale MD
*

*>> Next Level Solutions, LLC
*

*>> www.NextLevelSolns.com
*

*>
*

*> --
*

*> best,
*

*> -tony
*

*>
*

*> blindglobe *

*> Muttenz, Switzerland.
*

*> "Commit early,commit often, and commit in a repository from which
*

*> we can
*

*> easily
*

*> roll-back your mistakes" (AJR, 4Jan05).
*

Received on Tue Mar 20 2007 - 17:43:37 EDT

Date: Tue, 20 Mar 2007 17:43:37 -0400

Hello Mark & nmusers,

I'm just catching up with the flurry of emails on this topic...

I don't think that anyone mentioned full model approaches to

covariate modeling, although we have discussed this topic in detail

in past nmusers threads. Frank Harrell's Regression Modeling

Strategies text (I'll second Tony's recommendation) advocates this

method as an alternative to stepwise methods when the purpose is to

estimate the effect of covariates. The text also includes a useful

discussion of the choice of modeling strategy as it relates to

modeling objectives (e.g. prediction, effect estimation, hypothesis

testing).

In our group, we routinely apply full model methods for population PK =

covariate modeling and have managed to make useful inferences about

covariate effects while avoiding stepwise methods and p-values

altogether. Some examples of this method will be presented at ASCPT

later this week.

I also agree with the sentiment expressed by several contributors

that we shouldn't be so concerned with finding the one perfect model. =

Instead, we should probably spend more time evaluating the impact of

model deficiencies on the intended model-based applications and

inferences.

In addition to Harrell's book, some relevant references are listed

below.

Best regards,

Marc

Marc R. Gastonguay, Ph.D.

Scientific Director, Metrum Institute [www.metruminstitute.org]

President & CEO, Metrum Research Group LLC [www.metrumrg.com]

Email: marcg

+1.860.735.7043

1. Ulrika Wählby, E. Niclas Jonsson and Mats O. Karlsson AAPS

PharmSci 2002; 4 (4) article 27 (http://www.aapspharmsci.org).

Comparison of Stepwise Covariate Model Building Strategies in

Population Pharmacokinetic-Pharmacodynamic Analysis ****(full model =

approach is described in Discussion section).

2. Steyerberg EW, Eijkemans MJ, Habbema JD. Stepwise selection in

small data sets: a simulation study of bias in logistic regression

analysis. J Clin Epidemiol. October 1999;52(10):935-942.

3. Harrell FE, Jr., Lee KL, Mark DB. Multivariable prognostic models: =

issues in developing models, evaluating assumptions and adequacy, and =

measuring and reducing errors. Stat Med. 1996;15(4):361-387.

4. Steyerberg EW, Eijkemans MJ, Harrell FE, Jr., Habbema JD.

Prognostic modelling with logistic regression analysis: a comparison

of selection and estimation methods in small data sets. Stat Med.

2000;19(8):1059-1079.

5. M.R. Gastonguay. A Full Model Estimation Approach for Covariate

Effects: Inference Based on Clinical Importance and Estimation

Precision. The AAPS Journal; 6(S1), Abstract W4354, 2004. (http://

metrumrg.com/publications/full_model.pdf)

6. Balaji Agoram; Anne C. Heatherington; Marc R. Gastonguay.

Development and Evaluation of a Population Pharmacokinetic-

Pharmacodynamic Model of Darbepoetin Alfa in Patients with Nonmyeloid =

Malignancies Undergoing Multicycle Chemotherapy. AAPS PharmSci Vol.: =

8, No.: 3, 2006

On Mar 19, 2007, at 3:34 PM, AJ Rossini wrote:

courses,

Received on Tue Mar 20 2007 - 17:43:37 EDT