From: Ribbing, Jakob <*Jakob.Ribbing*>

Date: Fri, 14 Nov 2008 08:48:07 -0000

Nick,

The only way I can see ETABAR being biased when fitting the correct

model, is due to asymmetric shrinkage, i.e. that the distribution of EBE

etas is shrunk more in one tail than the other so that the EBE-eta

distribution becomes non-symmetric.

A situation where I would expect this to happen is when putting an "eta

on epsilon" (see ref below). This is a great and simple way of handling

that subjects have different intra-individual error magnitude (instead

of just assuming the same SIGMA for all). In practice, you multiply

whatever the model weight (W) is by e.g. exp(eta) to incorporate eta on

epsilon. This is a simple way of accounting for e.g. that some subjects

are more compliant than others (compliant with therapy, fasting and

other prohibited/compulsory activities during the study).

Assuming that data is not extremely sparse: For subjects where the eta

is highly positive, there will be evidence of them having a higher

variability in the intra-individual error, since their observations

otherwise will become highly unlikely (epsilons which are extremely

positive and negative, in comparison to the value of SIGMA). The eta for

these subject will only be shrunk to a small degree. For the compliant

subject, eps is small (close to zero) for all observations and

consequently, these observations are likely regardless of if the

intra-individual error magnitude is typical or smaller. The eta on these

subjects will shrink from the true (highly negative) eta towards zero.

In consequence, ETABAR can be expected to be positive. This asymmetric

shrinkage does not invalidate the model and it may work great both for

fitting your data and simulate from the model.

Other examples of asymmetric shrinkage may be if there is a continuum of

EC50 values but many subjects where not administered doses high enough

to see a profound effect (all subjects received a low dose so that

drug-effects below Emax have been observed in all): For subjects with

high EC50, that did not receive a high dose, there is no clear effect at

all and the very high eta on EC50 will be shrunk a bit towards zero. For

subjects with low or normal EC50 there will be information in the data

to determine the correct EC50 without shrinkage. The EBE eta

distribution will be skewed to the left, e.g. ranging from -4 to 2, but

still with the median around 0. The model may still be fine, if

alternative parameterisations do not fit the data better.

Best Regards

Jakob

J Pharmacokinet Biopharm. 1995 Dec;23(6):651-72.

Three new residual error models for population PK/PD analyses.Karlsson

MO, Beal SL, Sheiner LB.

Department of Pharmacy, School of Pharmacy, University of California,

San Francisco 94143-0626, USA.

Residual error models, traditionally used in population pharmacokinetic

analyses, have been developed as if all sources of error have properties

similar to those of assay error. Since assay error often is only a minor

part of the difference between predicted and observed concentrations,

other sources, with potentially other properties, should be considered.

We have simulated three complex error structures. The first model

acknowledges two separate sources of residual error, replication error

plus pure residual (assay) error. Simulation results for this case

suggest that ignoring these separate sources of error does not adversely

affect parameter estimates. The second model allows serially correlated

errors, as may occur with structural model misspecification. Ignoring

this error structure leads to biased random-effect parameter estimates.

A simple autocorrelation model, where the correlation between two errors

is assumed to decrease exponentially with the time between them,

provides more accurate estimates of the variability parameters in this

case. The third model allows time-dependent error magnitude. This may be

caused, for example, by inaccurate sample timing. A time-constant error

model fit to time-varying error data can lead to bias in all population

parameter estimates. A simple two-step time-dependent error model is

sufficient to improve parameter estimates, even when the true time

dependence is more complex. Using a real data set, we also illustrate

the use of the different error models to facilitate the model building

process, to provide information about error sources, and to provide more

accurate parameter estimates.

-----Original Message-----

From: owner-nmusers

On Behalf Of Nick Holford

Sent: 14 November 2008 00:11

To: nmusers

Subject: Re: [NMusers] Very small P-Value for ETABAR

Jakob,

Thanks for some more info on this issue. I have seen work from Mats and

Rada that says ETABAR can be biased when there is a lot of shrinkage

even when the data is simulated and fitted with the correct model. Can

you confirm this and can you explain how it arises? In the worst case of

shrinkage then bias is impossible because all ETAs must be zero. So why

does it occur with non-zero shrinkage?

Nick

Ribbing, Jakob wrote:

*> Dear all,
*

*>
*

*> First of all, I am not sure that there is any assumption of etas
*

having

*> a normal distribution when estimating a parametric model in NONMEM.
*

The

*> variance of eta (OMEGA) does not carry an assumption of normality. I
*

*> believe that Stuart used to say the assumption of normality is only
*

when

*> simulating. I guess the assumption also affects EBE:s unless the
*

*> individual information is completely dominating? If the assumption of
*

*> normality is wrong, the weighting of information may not be optimal,
*

but

*> as long as the true distribution is symmetric the estimated parameters
*

*> are in principle correct (but again, the model may not be suitable for
*

*> simulation if the distributional assumption was wrong). I will be off
*

*> line for a few days, but I am sure somebody will correct me if I am
*

*> wrong about this.
*

*>
*

*> If etas are shrunk, you can not expect a normal distribution of that
*

*> (EBE) eta. That does not invalidate parameterization/distributional
*

*> assumptions. Trying other semi-parametric distributions or a
*

*> non-parametric distribution (or a mixture model) may give more
*

*> confidence in sticking with the original parameterization or else
*

reject

*> it as inadequate. In the end, you may feel confident about the model
*

*> even if the EBE eta distribution is asymmetric and biased (I mentioned
*

*> two examples in my earlier posting).
*

*>
*

*> Connecting to how PsN may help in this case:
*

http://psn.sourceforge.net/

*> In practice to evaluate shrinkage, you would simply give the command
*

*> (assuming the model file is called run1.mod):
*

*> execute --shrinkage run1.mod
*

*>
*

*> Another quick evaluation that can be made with this program is to
*

*> produce mirror plots (PsN links in nicely with Xpose for producing the
*

*> diagnostic plots):
*

*>
*

*> execute --mirror=3 run1.mod
*

*>
*

*> This will give you three simulation table files that have been derived
*

*> by simulating under the model and then fitting the simulated data
*

using

*> the same model (using the design of the original data). If you see a
*

*> similar pattern in the mirror plots as in the original diagnostic
*

plots,

*> this gives you more confidence in the model. That brings us back to
*

*> Leonids point about it being more useful to look at diagnostic plots
*

*> than eta bar.
*

*>
*

*> Wishing you a great weekend!
*

*>
*

*> Jakob
*

*>
*

*> -----Original Message-----
*

*> From: BAE, KYUN-SEOP
*

*> Sent: 13 November 2008 22:05
*

*> To: Ribbing, Jakob; XIA LI; nmusers *

*> Subject: RE: [NMusers] Very small P-Value for ETABAR
*

*>
*

*> Dear All,
*

*>
*

*> Realized etas (EBEs, MAPs) is estimated under the assumption of normal
*

*> distribution.
*

*> However, the resultant distribution of EBEs may not be normal or mean
*

of

*> them may not be 0.
*

*> To pass t-test, one may use "CENTERING" option at $ESTIMATION.
*

*> But, this practice is discouraged by some (and I agree).
*

*>
*

*> Normal assumption cannot coerce the distribution of EBE to be normal,
*

*> and furthermore non-normal (and/or not-zero-mean) distribution of EBE
*

*> can be nature's nature.
*

*> One simple example is mixture population with polymorphism.
*

*>
*

*> If I could not get normal(?) EBEs even after careful examination of
*

*> covariate relationships as others suggested,
*

*> I would bear it and assume nonparametric distribution.
*

*>
*

*> Regards,
*

*>
*

*> Kyun-Seop
*

*> =====================
*

*> Kyun-Seop Bae MD PhD
*

*> Email: kyun-seop.bae *

*>
*

*> -----Original Message-----
*

*> From: owner-nmusers *

[mailto:owner-nmusers

*> On Behalf Of Ribbing, Jakob
*

*> Sent: Thursday, November 13, 2008 13:19
*

*> To: XIA LI; nmusers *

*> Subject: RE: [NMusers] Very small P-Value for ETABAR
*

*>
*

*> Hi Xia,
*

*>
*

*> Just to clarify one thing (I agree with almost everything you said):
*

*>
*

*> The p-value indeed is related to the test of ETABAR=0. However, this
*

is

*> not a test of normality, only a test that may reject the mean of the
*

*> etas being zero (H0). Therefore, shrinkage per se does not lead to
*

*> rejection of HO, as long as both tails of the eta distribution are
*

*> shrunk to a similar degree.
*

*>
*

*> I agree with the assumption of normality. This comes into play when
*

you

*> simulate from the model and if you got the distribution of individual
*

*> parameters wrong, simulations may not reflect even the data used to
*

fit

*> the model.
*

*>
*

*> Best Regards
*

*>
*

*> Jakob
*

*>
*

*> -----Original Message-----
*

*> From: owner-nmusers *

[mailto:owner-nmusers

*> On Behalf Of XIA LI
*

*> Sent: 13 November 2008 20:31
*

*> To: nmusers *

*> Subject: Re: [NMusers] Very small P-Value for ETABAR
*

*>
*

*> Dear All,
*

*>
*

*> Just some quick statistical points...
*

*>
*

*> P value is usually associated with hypothesis test. As far as I know,
*

*> NONMEM assume normal distribution for ETA, ETA~N(0,omega), which means
*

*> the null hypothesis to test is H0: ETABAR=0. A small P value =
*

indicates

a

*> significant test. You reject the null hypothesis.
*

*>
*

*> More...
*

*> As we all know, ETA is used to capture the variation among individual
*

*> parameters and model's unexplained error. We usually use the function
*

*> (or model) parameter=typical value*exp (ETA), which leads to a
*

lognormal

*> distribution assumption for all fixed effect parameters (i.e., CL, V,
*

*> Ka, Ke...).
*

*>
*

*> By some statistical theory, the variation of individual parameter
*

equals

*> a function of the typical value and the variance of ETA.
*

*>
*

*> VAR (CL) = typical value*exp (omega/2). NO MATH PLS!!
*

*>
*

*> If your typical value captures all overall patterns among patients
*

*> clearance, then ETA will have a nice symmetric normal distribution
*

with

*> small variance. Otherwise, you leave too many patterns to ETA and will
*

*> see some deviation or shrinkage (whatever you call).
*

*>
*

*> Why adding covariates is a good way to deal with this situation? You
*

*> model become CL=typical value*exp (covariate)*exp (ETA). The =
*

variation

*> of individual parameter will be changed to:
*

*>
*

*> VAR (CL) = (typical value + covariate)*exp (omega/2)).
*

*>
*

*> You have one more item to capture the overall patterns, less leave to
*

*> ETA. So a 'good' covariate will reduce both the magnitude of omega and
*

*> ETA's deviation from normal.
*

*>
*

*> Understanding this is also useful when you are modeling BOV studies.
*

*> When you see variation of PK parameters decrease with time (or
*

*> occasions). Adding a covariate that make physiological sense and also
*

*> decrease with time may help your modeling.
*

*>
*

*> Best,
*

*> Xia
*

*> =
*

==========================

=============

*> Xia Li
*

*> Mathematical Science Department
*

*> University of Cincinnati
*

*>
*

--

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New

Zealand

n.holford

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Fri Nov 14 2008 - 03:48:07 EST

Date: Fri, 14 Nov 2008 08:48:07 -0000

Nick,

The only way I can see ETABAR being biased when fitting the correct

model, is due to asymmetric shrinkage, i.e. that the distribution of EBE

etas is shrunk more in one tail than the other so that the EBE-eta

distribution becomes non-symmetric.

A situation where I would expect this to happen is when putting an "eta

on epsilon" (see ref below). This is a great and simple way of handling

that subjects have different intra-individual error magnitude (instead

of just assuming the same SIGMA for all). In practice, you multiply

whatever the model weight (W) is by e.g. exp(eta) to incorporate eta on

epsilon. This is a simple way of accounting for e.g. that some subjects

are more compliant than others (compliant with therapy, fasting and

other prohibited/compulsory activities during the study).

Assuming that data is not extremely sparse: For subjects where the eta

is highly positive, there will be evidence of them having a higher

variability in the intra-individual error, since their observations

otherwise will become highly unlikely (epsilons which are extremely

positive and negative, in comparison to the value of SIGMA). The eta for

these subject will only be shrunk to a small degree. For the compliant

subject, eps is small (close to zero) for all observations and

consequently, these observations are likely regardless of if the

intra-individual error magnitude is typical or smaller. The eta on these

subjects will shrink from the true (highly negative) eta towards zero.

In consequence, ETABAR can be expected to be positive. This asymmetric

shrinkage does not invalidate the model and it may work great both for

fitting your data and simulate from the model.

Other examples of asymmetric shrinkage may be if there is a continuum of

EC50 values but many subjects where not administered doses high enough

to see a profound effect (all subjects received a low dose so that

drug-effects below Emax have been observed in all): For subjects with

high EC50, that did not receive a high dose, there is no clear effect at

all and the very high eta on EC50 will be shrunk a bit towards zero. For

subjects with low or normal EC50 there will be information in the data

to determine the correct EC50 without shrinkage. The EBE eta

distribution will be skewed to the left, e.g. ranging from -4 to 2, but

still with the median around 0. The model may still be fine, if

alternative parameterisations do not fit the data better.

Best Regards

Jakob

J Pharmacokinet Biopharm. 1995 Dec;23(6):651-72.

Three new residual error models for population PK/PD analyses.Karlsson

MO, Beal SL, Sheiner LB.

Department of Pharmacy, School of Pharmacy, University of California,

San Francisco 94143-0626, USA.

Residual error models, traditionally used in population pharmacokinetic

analyses, have been developed as if all sources of error have properties

similar to those of assay error. Since assay error often is only a minor

part of the difference between predicted and observed concentrations,

other sources, with potentially other properties, should be considered.

We have simulated three complex error structures. The first model

acknowledges two separate sources of residual error, replication error

plus pure residual (assay) error. Simulation results for this case

suggest that ignoring these separate sources of error does not adversely

affect parameter estimates. The second model allows serially correlated

errors, as may occur with structural model misspecification. Ignoring

this error structure leads to biased random-effect parameter estimates.

A simple autocorrelation model, where the correlation between two errors

is assumed to decrease exponentially with the time between them,

provides more accurate estimates of the variability parameters in this

case. The third model allows time-dependent error magnitude. This may be

caused, for example, by inaccurate sample timing. A time-constant error

model fit to time-varying error data can lead to bias in all population

parameter estimates. A simple two-step time-dependent error model is

sufficient to improve parameter estimates, even when the true time

dependence is more complex. Using a real data set, we also illustrate

the use of the different error models to facilitate the model building

process, to provide information about error sources, and to provide more

accurate parameter estimates.

-----Original Message-----

From: owner-nmusers

On Behalf Of Nick Holford

Sent: 14 November 2008 00:11

To: nmusers

Subject: Re: [NMusers] Very small P-Value for ETABAR

Jakob,

Thanks for some more info on this issue. I have seen work from Mats and

Rada that says ETABAR can be biased when there is a lot of shrinkage

even when the data is simulated and fitted with the correct model. Can

you confirm this and can you explain how it arises? In the worst case of

shrinkage then bias is impossible because all ETAs must be zero. So why

does it occur with non-zero shrinkage?

Nick

Ribbing, Jakob wrote:

having

The

when

but

reject

http://psn.sourceforge.net/

using

plots,

of

[mailto:owner-nmusers

is

you

fit

[mailto:owner-nmusers

indicates

a

lognormal

equals

with

variation

==========================

=============

--

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New

Zealand

n.holford

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Fri Nov 14 2008 - 03:48:07 EST