From: Mats Karlsson <*mats.karlsson*>

Date: Fri, 14 Nov 2008 12:38:46 +0100

Nick,

Whenever the amount on information about a parameter is linked to its value

in an individual you can expect asymmetric shrinkage. For example with

standard sampling schemes for PK studies (barring BQL problems), there is

often more information about CL in subjects with high CL than low CL. You

can think of it as have a smaller extrapolated AUC beyond the last

observation. Similarly for example KA will generally be more informed if it

is slow than fast (in individuals with very fast absorption the first

observation may already be beyond the peak). Similarly with EC50 - with a

given concentration range subjects with low EC50 will have a better

characterized profile and more precise estimation of its EC50. As shrinkage

is linked to the information contained in an individual's data shrinkage

will be more pronounced at higher (or lower depending on situation and

parameter) values for individuals and asymmetric shrinkage will result.

Total shrinkage to zero you will only get when you have no individual

information, a situation where you obviously would not have an eta on the

parameter.

I agree with many of the statements. I don't make any decisions the p-values

but would be alerted by large deviations from zero, large being relative to

the magnitude of the variability of the corresponding omega-estimate.

However, even when I see such deviations, I try to check whether it could

come about due to asymmetric shrinkage. The way to do so is first to check

the shrinkage magnitude. If shrinkage is low is low, the deviation is

probably representing a true misfit of the model. If shrinkage is high, then

I may perform a simple simulation from the model parameters, then reestimate

etas using MAXEVAL=0 and check for the ETABAR in this fit. If it is close to

zero, again the high ETABAR in the original model is probably representing a

misfit. If it is of the same size and sign as the ETABAR for the original

data, I would conclude that the high ETABAR in the original fit was a

consequence of asymmetric shrinkage and I wouldn't be disturbed by it.

In my opinion, it would be better if ETABAR had represented the median ETA.

That is always expected to be close to zero for a well-behaved model. A

problem with that is that in the presence of shrinkage, the power to detect

misfit is diminished. Again, it just goes to show that in the presence of

shrinkage, all diagnostics based individual ETAS are less useful.

Best regards,

Mats

Mats Karlsson, PhD

Professor of Pharmacometrics

Dept of Pharmaceutical Biosciences

Uppsala University

Box 591

751 24 Uppsala Sweden

phone: +46 18 4714105

fax: +46 18 471 4003

-----Original Message-----

From: owner-nmusers

Behalf Of Nick Holford

Sent: Friday, November 14, 2008 1:11 AM

To: nmusers

Subject: Re: [NMusers] Very small P-Value for ETABAR

Jakob,

Thanks for some more info on this issue. I have seen work from Mats and

Rada that says ETABAR can be biased when there is a lot of shrinkage

even when the data is simulated and fitted with the correct model. Can

you confirm this and can you explain how it arises? In the worst case of

shrinkage then bias is impossible because all ETAs must be zero. So why

does it occur with non-zero shrinkage?

Nick

Ribbing, Jakob wrote:

*> Dear all,
*

*>
*

*> First of all, I am not sure that there is any assumption of etas having
*

*> a normal distribution when estimating a parametric model in NONMEM. The
*

*> variance of eta (OMEGA) does not carry an assumption of normality. I
*

*> believe that Stuart used to say the assumption of normality is only when
*

*> simulating. I guess the assumption also affects EBE:s unless the
*

*> individual information is completely dominating? If the assumption of
*

*> normality is wrong, the weighting of information may not be optimal, but
*

*> as long as the true distribution is symmetric the estimated parameters
*

*> are in principle correct (but again, the model may not be suitable for
*

*> simulation if the distributional assumption was wrong). I will be off
*

*> line for a few days, but I am sure somebody will correct me if I am
*

*> wrong about this.
*

*>
*

*> If etas are shrunk, you can not expect a normal distribution of that
*

*> (EBE) eta. That does not invalidate parameterization/distributional
*

*> assumptions. Trying other semi-parametric distributions or a
*

*> non-parametric distribution (or a mixture model) may give more
*

*> confidence in sticking with the original parameterization or else reject
*

*> it as inadequate. In the end, you may feel confident about the model
*

*> even if the EBE eta distribution is asymmetric and biased (I mentioned
*

*> two examples in my earlier posting).
*

*>
*

*> Connecting to how PsN may help in this case: http://psn.sourceforge.net/
*

*> In practice to evaluate shrinkage, you would simply give the command
*

*> (assuming the model file is called run1.mod):
*

*> execute --shrinkage run1.mod
*

*>
*

*> Another quick evaluation that can be made with this program is to
*

*> produce mirror plots (PsN links in nicely with Xpose for producing the
*

*> diagnostic plots):
*

*>
*

*> execute --mirror=3 run1.mod
*

*>
*

*> This will give you three simulation table files that have been derived
*

*> by simulating under the model and then fitting the simulated data using
*

*> the same model (using the design of the original data). If you see a
*

*> similar pattern in the mirror plots as in the original diagnostic plots,
*

*> this gives you more confidence in the model. That brings us back to
*

*> Leonids point about it being more useful to look at diagnostic plots
*

*> than eta bar.
*

*>
*

*> Wishing you a great weekend!
*

*>
*

*> Jakob
*

*>
*

*> -----Original Message-----
*

*> From: BAE, KYUN-SEOP
*

*> Sent: 13 November 2008 22:05
*

*> To: Ribbing, Jakob; XIA LI; nmusers *

*> Subject: RE: [NMusers] Very small P-Value for ETABAR
*

*>
*

*> Dear All,
*

*>
*

*> Realized etas (EBEs, MAPs) is estimated under the assumption of normal
*

*> distribution.
*

*> However, the resultant distribution of EBEs may not be normal or mean of
*

*> them may not be 0.
*

*> To pass t-test, one may use "CENTERING" option at $ESTIMATION.
*

*> But, this practice is discouraged by some (and I agree).
*

*>
*

*> Normal assumption cannot coerce the distribution of EBE to be normal,
*

*> and furthermore non-normal (and/or not-zero-mean) distribution of EBE
*

*> can be nature's nature.
*

*> One simple example is mixture population with polymorphism.
*

*>
*

*> If I could not get normal(?) EBEs even after careful examination of
*

*> covariate relationships as others suggested,
*

*> I would bear it and assume nonparametric distribution.
*

*>
*

*> Regards,
*

*>
*

*> Kyun-Seop
*

*> =====================
*

*> Kyun-Seop Bae MD PhD
*

*> Email: kyun-seop.bae *

*>
*

*> -----Original Message-----
*

*> From: owner-nmusers *

*> On Behalf Of Ribbing, Jakob
*

*> Sent: Thursday, November 13, 2008 13:19
*

*> To: XIA LI; nmusers *

*> Subject: RE: [NMusers] Very small P-Value for ETABAR
*

*>
*

*> Hi Xia,
*

*>
*

*> Just to clarify one thing (I agree with almost everything you said):
*

*>
*

*> The p-value indeed is related to the test of ETABAR=0. However, this is
*

*> not a test of normality, only a test that may reject the mean of the
*

*> etas being zero (H0). Therefore, shrinkage per se does not lead to
*

*> rejection of HO, as long as both tails of the eta distribution are
*

*> shrunk to a similar degree.
*

*>
*

*> I agree with the assumption of normality. This comes into play when you
*

*> simulate from the model and if you got the distribution of individual
*

*> parameters wrong, simulations may not reflect even the data used to fit
*

*> the model.
*

*>
*

*> Best Regards
*

*>
*

*> Jakob
*

*>
*

*> -----Original Message-----
*

*> From: owner-nmusers *

*> On Behalf Of XIA LI
*

*> Sent: 13 November 2008 20:31
*

*> To: nmusers *

*> Subject: Re: [NMusers] Very small P-Value for ETABAR
*

*>
*

*> Dear All,
*

*>
*

*> Just some quick statistical points...
*

*>
*

*> P value is usually associated with hypothesis test. As far as I know,
*

*> NONMEM assume normal distribution for ETA, ETA~N(0,omega), which means
*

*> the null hypothesis to test is H0: ETABAR=0. A small P value indicates a
*

*> significant test. You reject the null hypothesis.
*

*>
*

*> More...
*

*> As we all know, ETA is used to capture the variation among individual
*

*> parameters and model's unexplained error. We usually use the function
*

*> (or model) parameter=typical value*exp (ETA), which leads to a lognormal
*

*> distribution assumption for all fixed effect parameters (i.e., CL, V,
*

*> Ka, Ke...).
*

*>
*

*> By some statistical theory, the variation of individual parameter equals
*

*> a function of the typical value and the variance of ETA.
*

*>
*

*> VAR (CL) = typical value*exp (omega/2). NO MATH PLS!!
*

*>
*

*> If your typical value captures all overall patterns among patients
*

*> clearance, then ETA will have a nice symmetric normal distribution with
*

*> small variance. Otherwise, you leave too many patterns to ETA and will
*

*> see some deviation or shrinkage (whatever you call).
*

*>
*

*> Why adding covariates is a good way to deal with this situation? You
*

*> model become CL=typical value*exp (covariate)*exp (ETA). The variation
*

*> of individual parameter will be changed to:
*

*>
*

*> VAR (CL) = (typical value + covariate)*exp (omega/2)).
*

*>
*

*> You have one more item to capture the overall patterns, less leave to
*

*> ETA. So a 'good' covariate will reduce both the magnitude of omega and
*

*> ETA's deviation from normal.
*

*>
*

*> Understanding this is also useful when you are modeling BOV studies.
*

*> When you see variation of PK parameters decrease with time (or
*

*> occasions). Adding a covariate that make physiological sense and also
*

*> decrease with time may help your modeling.
*

*>
*

*> Best,
*

*> Xia
*

*> ======================================
*

*> Xia Li
*

*> Mathematical Science Department
*

*> University of Cincinnati
*

*>
*

--

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand

n.holford

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Fri Nov 14 2008 - 06:38:46 EST

Date: Fri, 14 Nov 2008 12:38:46 +0100

Nick,

Whenever the amount on information about a parameter is linked to its value

in an individual you can expect asymmetric shrinkage. For example with

standard sampling schemes for PK studies (barring BQL problems), there is

often more information about CL in subjects with high CL than low CL. You

can think of it as have a smaller extrapolated AUC beyond the last

observation. Similarly for example KA will generally be more informed if it

is slow than fast (in individuals with very fast absorption the first

observation may already be beyond the peak). Similarly with EC50 - with a

given concentration range subjects with low EC50 will have a better

characterized profile and more precise estimation of its EC50. As shrinkage

is linked to the information contained in an individual's data shrinkage

will be more pronounced at higher (or lower depending on situation and

parameter) values for individuals and asymmetric shrinkage will result.

Total shrinkage to zero you will only get when you have no individual

information, a situation where you obviously would not have an eta on the

parameter.

I agree with many of the statements. I don't make any decisions the p-values

but would be alerted by large deviations from zero, large being relative to

the magnitude of the variability of the corresponding omega-estimate.

However, even when I see such deviations, I try to check whether it could

come about due to asymmetric shrinkage. The way to do so is first to check

the shrinkage magnitude. If shrinkage is low is low, the deviation is

probably representing a true misfit of the model. If shrinkage is high, then

I may perform a simple simulation from the model parameters, then reestimate

etas using MAXEVAL=0 and check for the ETABAR in this fit. If it is close to

zero, again the high ETABAR in the original model is probably representing a

misfit. If it is of the same size and sign as the ETABAR for the original

data, I would conclude that the high ETABAR in the original fit was a

consequence of asymmetric shrinkage and I wouldn't be disturbed by it.

In my opinion, it would be better if ETABAR had represented the median ETA.

That is always expected to be close to zero for a well-behaved model. A

problem with that is that in the presence of shrinkage, the power to detect

misfit is diminished. Again, it just goes to show that in the presence of

shrinkage, all diagnostics based individual ETAS are less useful.

Best regards,

Mats

Mats Karlsson, PhD

Professor of Pharmacometrics

Dept of Pharmaceutical Biosciences

Uppsala University

Box 591

751 24 Uppsala Sweden

phone: +46 18 4714105

fax: +46 18 471 4003

-----Original Message-----

From: owner-nmusers

Behalf Of Nick Holford

Sent: Friday, November 14, 2008 1:11 AM

To: nmusers

Subject: Re: [NMusers] Very small P-Value for ETABAR

Jakob,

Thanks for some more info on this issue. I have seen work from Mats and

Rada that says ETABAR can be biased when there is a lot of shrinkage

even when the data is simulated and fitted with the correct model. Can

you confirm this and can you explain how it arises? In the worst case of

shrinkage then bias is impossible because all ETAs must be zero. So why

does it occur with non-zero shrinkage?

Nick

Ribbing, Jakob wrote:

--

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand

n.holford

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Fri Nov 14 2008 - 06:38:46 EST