NONMEM Users Network Archive

Hosted by Cognigen

Re: General question on modeling

From: Gastonguay, Marc <marcg>
Date: Tue, 20 Mar 2007 17:43:37 -0400

Hello Mark & nmusers,
I'm just catching up with the flurry of emails on this topic...

I don't think that anyone mentioned full model approaches to
covariate modeling, although we have discussed this topic in detail
in past nmusers threads. Frank Harrell's Regression Modeling
Strategies text (I'll second Tony's recommendation) advocates this
method as an alternative to stepwise methods when the purpose is to
estimate the effect of covariates. The text also includes a useful
discussion of the choice of modeling strategy as it relates to
modeling objectives (e.g. prediction, effect estimation, hypothesis

In our group, we routinely apply full model methods for population PK =

covariate modeling and have managed to make useful inferences about
covariate effects while avoiding stepwise methods and p-values
altogether. Some examples of this method will be presented at ASCPT
later this week.

I also agree with the sentiment expressed by several contributors
that we shouldn't be so concerned with finding the one perfect model. =

Instead, we should probably spend more time evaluating the impact of
model deficiencies on the intended model-based applications and

In addition to Harrell's book, some relevant references are listed

Best regards,

Marc R. Gastonguay, Ph.D.
Scientific Director, Metrum Institute []
President & CEO, Metrum Research Group LLC []
Email: marcg

1. Ulrika Wählby, E. Niclas Jonsson and Mats O. Karlsson AAPS
PharmSci 2002; 4 (4) article 27 (
Comparison of Stepwise Covariate Model Building Strategies in
Population Pharmacokinetic-Pharmacodynamic Analysis ****(full model =

approach is described in Discussion section).

2. Steyerberg EW, Eijkemans MJ, Habbema JD. Stepwise selection in
small data sets: a simulation study of bias in logistic regression
analysis. J Clin Epidemiol. October 1999;52(10):935-942.

3. Harrell FE, Jr., Lee KL, Mark DB. Multivariable prognostic models: =

issues in developing models, evaluating assumptions and adequacy, and =

measuring and reducing errors. Stat Med. 1996;15(4):361-387.

4. Steyerberg EW, Eijkemans MJ, Harrell FE, Jr., Habbema JD.
Prognostic modelling with logistic regression analysis: a comparison
of selection and estimation methods in small data sets. Stat Med.

5. M.R. Gastonguay. A Full Model Estimation Approach for Covariate
Effects: Inference Based on Clinical Importance and Estimation
Precision. The AAPS Journal; 6(S1), Abstract W4354, 2004. (http://

6. Balaji Agoram; Anne C. Heatherington; Marc R. Gastonguay.
Development and Evaluation of a Population Pharmacokinetic-
Pharmacodynamic Model of Darbepoetin Alfa in Patients with Nonmyeloid =

Malignancies Undergoing Multicycle Chemotherapy. AAPS PharmSci Vol.: =

8, No.: 3, 2006

On Mar 19, 2007, at 3:34 PM, AJ Rossini wrote:

> I'd highly recommend reading Frank Harrell's book on Regression
> Modeling if
> you think that stepwise regression makes any sense. While much of =

> the book
> applies to linear and generalized linear (i.e. categorical, etc)
> regression
> models, nonlinear models (and mixed effects models) would generally =

> fall into
> the "well, if the simple case was like that, it can't be any
> simpler for the
> harder cases..."... Frank demonstrates some of the reasons that p-
> values
> from models generated using stepwise modeling are fairly useless
> (i.e. don't
> follow the behavior you'd expect from p-values).
> The literature to start looking at would be modern variable selection
> techniques for linear regression, i.e. work at Stanford Statistics
> by Hastie,
> Tibshirani, and their collaborators and former grad students
> elastic nets, and similar approaches).
> On Monday 19 March 2007 19:32, Mark Sale - Next Level Solutions wrote:
>> Dear Colleagues,
>> I've lately been reviewing the literature on model building/
>> selection
>> algorithms. I have been unable to find any even remotely rigorous
>> discussion of the way we all build NONMEM models. The structural
>> first, then variances/forward addition/backward elimination is
>> generally mentioned in a number of places (Ene Ettes in Ann
>> Pharmacother, 2004, Jaap Mandemas series on POP PK series J PK
>> Biopharm
>> in 1992, Jose Pinheiros paper from the Joint Stats meeting in 1994,
>> Peter Bonates AAPS journal article in 2005, Mats Karlsons AAPS
>> PharmSci, 2002, the FDA guidance on Pop PK). It is most explicitly
>> stated in the NONMEM manuals (Vol 5, figure 11.1) - without any
>> reference. From the NONMEM manuals it is reproduced in many =
>> and has become axiomatic. I've looked at the stats literature on
>> forward addition/backwards elimination in both linear and logistic
>> regression, where it is at least formally discussed (with some
>> disagreement about whether it is "correct"). But, I am unable to
>> find
>> any justification for the structural first, then covariates (drive by
>> post-hoc plots), then variance effects approach we use (I'm sure many
>> people will point out that it is not nearly that linear a process,
>> although in figure 11.1, Vol 5 of the NONMEM manuals, it is
>> depicted as
>> a step-by-step algorithm, without any looping back). Can anyone
>> point
>> me to any rigorous discussion of this model building strategy?
>> Mark Sale MD
>> Next Level Solutions, LLC
> --
> best,
> -tony
> blindglobe
> Muttenz, Switzerland.
> "Commit early,commit often, and commit in a repository from which
> we can
> easily
> roll-back your mistakes" (AJR, 4Jan05).

Received on Tue Mar 20 2007 - 17:43:37 EDT

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to:

Once subscribed, you may contribute to the discussion by emailing: