From: Gastonguay, Marc <*marcg*>

Date: Fri, 21 Aug 2009 12:10:13 -0400

Hello Jakob, et al.

I would agree that individuals who do not contribute data to the

estimation of a particular element of OMEGA should be excluded from

the ETA-shrinkage calculation or ETA-based diagnostics. I think that

using individual ETA=0 as the filtering criterion may be a reasonable =

thing to do when OMEGA is DIAGONAL (e.g. all off-diagonal elements are =

zero), but this practice could be misleading when covariance in the

inter-individual random effects exists (e.g. OMEGA BLOCK(n)).

For example, consider a population PK model simultaneously

incorporating parent and metabolite data. Also imagine that the OMEGA =

matrix is constructed to allow covariance between ETA[parent CL] and

ETA[metabolite CL]. If the correlation between these ETAs is non-zero, =

it is possible that individuals who are entirely missing metabolite

data will still have a non-zero ETA[metabolite CL] estimate. This is

because the expected value for that ETA should be driven by the

covariance structure in OMEGA. Although this ETA estimate is non-zero, =

it is shrunken toward the population expected value, and may

contribute to a biased shrinkage calculation and/or diagnostics.

To avoid both this situation and the issue that Douglas raised, it is =

preferable to filter ETAs based on design factors rather than

automatically based on individual ETA=0.

Having said all this, I'm not sure how important this particular

source of bias in the ETA-shrinkage calculation is anyway. There are

other potential biases in this calculation, including:

1. Bias in the population estimates of OMEGA variance elements- It's =

not uncommon for these terms to be over-estimated, which may lead to

an artificial apparent shrinkage (the calculation for ETA shrinkage

uses estimated variance in the denominator).

2. Bias in the observed sample SD of individual ETAs due to

insufficient sample size- Biased shrinkage estimates may result from

biased sample SD (used in the numerator of the shrinkage calculation), =

particularly in small data sets.

I think the take-home message is that ETA-based diagnostics (and

diagnostics of the diagnostics) can be useful, but should be

considered in the context of the design and potential biases.

Best regards,

Marc

Marc R. Gastonguay, Ph.D. < marcg

President & CEO, Metrum Research Group LLC < metrumrg.com >

Scientific Director, Metrum Institute < metruminstitute.org >

2 Tunxis Rd, Suite 112, Tariffville, CT 06081 Direct:

+1.860.670.0744 Main: +1.860.735.7043 Fax: +1.860.760.6014

On Aug 21, 2009, at 9:12 AM, Ribbing, Jakob wrote:

*> Hi Douglas,
*

*>
*

*> This has been a concern for me as well, although I do not know if
*

*> this ever happens(?). For the automatic (generic scripts) exclusion =
*

*> of etas that I use for eta-diagnostics, I tend to exclude a group
*

*> (e.g. each dose or dose-study combination) if all subjects have
*

*> eta=0 in that group. This would for example exclude IOV-eta3 from a =
*

*> study that only hade two occasions, or the placebo group(s) for etas =
*

*> on drug effect. I feel safe with that exclusion for my diagnostics. =
*

*> If I had to make the choice between excluding all etas that are
*

*> exactly equal to zero or none at all, I would more trust diagnostics =
*

*> after exclusion.
*

*>
*

*> Jakob
*

*>
*

*> From: Eleveld, DJ [mailto:d.j.eleveld *

*> Sent: 21 August 2009 13:57
*

*> To: Ribbing, Jakob; Pyry Välitalo; nmusers *

*> Subject: RE: [NMusers] Calculating shrinkage when some etas are zero
*

*>
*

*> Hi Pyry and Jacob,
*

*>
*

*> If you exclude zero etas then what happens to infomative individuals =
*

*> who just happen to have the population typical values?
*

*> This approch would exclude these individuals when trying to indicate =
*

*> how informative an estimation is about a parameter.
*

*> I know this is unlikely, but it is possible.
*

*>
*

*> The etas just tell what value is estimated, its not the whole story =
*

*> about how infomative an estimation is. I dont think you can do
*

*> this without considering how 'certian' you are of each of those eta =
*

*> values.
*

*>
*

*> Douglas Eleveld
*

*>
*

*> Van: owner-nmusers *

*> Verzonden: vr 21-8-2009 12:26
*

*> Aan: Pyry Välitalo; nmusers *

*> Onderwerp: RE: [NMusers] Calculating shrinkage when some etas are zero
*

*>
*

*> Hi Pyry,
*

*>
*

*> Yes, when calculating shrinkage or looking at eta-diagnostic plots
*

*> it is often better to exclude etas from subjects that has no
*

*> information on that parameter at all. For a PK model we would not
*

*> include subjects that were only administered placebo (if PK is
*

*> exogenous compound). In the same manner placebo subjects are not
*

*> informative on the drug-effects parameters of a (PK-)PD model. These =
*

*> subjects have informative etas for the placebo-part of the PD model, =
*

*> but not on the drug-effects (etas on Emax, ED50, etc.). For any eta- =
*

*> diagnostics you can removed these etas based on design (placebo
*

*> subject, IV dosing, et c) or the empirical-Bayes estimate of eta
*

*> being zero.
*

*>
*

*> Cheers
*

*>
*

*> Jakob
*

*>
*

*> From: owner-nmusers *

[mailto:owner-nmusers

*> ] On Behalf Of Pyry Välitalo
*

*> Sent: 21 August 2009 10:45
*

*> To: nmusers *

*> Subject: [NMusers] Calculating shrinkage when some etas are zero
*

*>
*

*> Hi all,
*

*>
*

*> I saw this snippet of information on PsN-general mailing list.
*

*>
*

*> Kajsa Harling wrote in PsN-general:
*

*> "I talked to the experts here about shrinkage. Apparently, sometimes =
*

*> an
*

*> individual's eta may be exactly 0 (no effect, placebo, you probably
*

*> understand this better than I do). These zeros should not be
*

*> included in
*

*> the shrinkage calculation, but now they are (erroneously) in PsN."
*

*>
*

*> This led me to wonder about the calculation of shrinkage. I decided =
*

*> to post here on nmusers, because my question mainly relates to
*

*> NONMEM. I could not find previous discussions about this topic
*

*> exactly.
*

*>
*

*> As I understand, if a parameter with BSV is not used by some
*

*> individuals, the etas for these individuals will be set to zero. An =
*

*> example would be a dataset with IV and oral dosing data. If oral
*

*> absorption rate constant KA with BSV is estimated for this data,
*

*> then all eta(KA) values for IV dosing group will be zero.
*

*>
*

*> The shrinkage of etas is calculated as
*

*> 1-sd(etas)/omega
*

*> If the etas that equal exactly zero would have to be removed from
*

*> this equation then it would mean that NONMEM estimates the omega
*

*> based on only those individuals who need it for the parameter in
*

*> question, e.g. the omega(KA) would be estimated only based on the
*

*> oral dosing group. Is this a correct interpretation for the
*

*> rationale to leave out zero etas?
*

*>
*

*> I guess the inclusion of zero etas into shrinkage calculations
*

*> significantly increases the estimate of shrinkage because the zero
*

*> etas always reduce the sd(etas). As a practical example, suppose a
*

*> dataset of 20 patients with oral and 20 patients with IV
*

*> administration. Suppose NONMEM estimates an omega of 0.4 for BSV of =
*

*> KA. Suppose the sd(etas) for oral group is 0.3 and thus sd(etas) for =
*

*> all patients is 0.3/sqrt(2) since the etas in IV group for KA are
*

*> zero.
*

*> Thus, as far as I know, PsN would currently calculate a shrinkage of =
*

*> 1-(0.3/sqrt(2))/0.4=0.47.
*

*> Would it be more appropriate to manually calculate a shrinkage of
*

*> 1-0.3/0.4=0.25 instead?
*

*>
*

*> All comments much appreciated.
*

*>
*

*> Kind regards,
*

*> Pyry
*

*>
*

*>
*

*>
*

*> Kajsa Harling wrote:
*

*> Dear Ethan,
*

*>
*

*> I have also been away for a while, thank you for your patience.
*

*>
*

*> I talked to the experts here about shrinkage. Apparently, sometimes an
*

*> individual's eta may be exactly 0 (no effect, placebo, you probably
*

*> understand this better than I do). These zeros should not be
*

*> included in
*

*> the shrinkage calculation, but now they are (erroneously) in PsN.
*

*>
*

*> Does this explain the discrepancy?
*

*>
*

*> Then, the heading shrinkage_wres is incorrect, it should say
*

*> shrinkage_iwres (or eps) they say.
*

*>
*

*> Comments are fine as long as they do not have commas in them. But this
*

*> is fixed in the latest release.
*

*>
*

*> Best regards,
*

*> Kajsa
*

*>
*

Received on Fri Aug 21 2009 - 12:10:13 EDT

Date: Fri, 21 Aug 2009 12:10:13 -0400

Hello Jakob, et al.

I would agree that individuals who do not contribute data to the

estimation of a particular element of OMEGA should be excluded from

the ETA-shrinkage calculation or ETA-based diagnostics. I think that

using individual ETA=0 as the filtering criterion may be a reasonable =

thing to do when OMEGA is DIAGONAL (e.g. all off-diagonal elements are =

zero), but this practice could be misleading when covariance in the

inter-individual random effects exists (e.g. OMEGA BLOCK(n)).

For example, consider a population PK model simultaneously

incorporating parent and metabolite data. Also imagine that the OMEGA =

matrix is constructed to allow covariance between ETA[parent CL] and

ETA[metabolite CL]. If the correlation between these ETAs is non-zero, =

it is possible that individuals who are entirely missing metabolite

data will still have a non-zero ETA[metabolite CL] estimate. This is

because the expected value for that ETA should be driven by the

covariance structure in OMEGA. Although this ETA estimate is non-zero, =

it is shrunken toward the population expected value, and may

contribute to a biased shrinkage calculation and/or diagnostics.

To avoid both this situation and the issue that Douglas raised, it is =

preferable to filter ETAs based on design factors rather than

automatically based on individual ETA=0.

Having said all this, I'm not sure how important this particular

source of bias in the ETA-shrinkage calculation is anyway. There are

other potential biases in this calculation, including:

1. Bias in the population estimates of OMEGA variance elements- It's =

not uncommon for these terms to be over-estimated, which may lead to

an artificial apparent shrinkage (the calculation for ETA shrinkage

uses estimated variance in the denominator).

2. Bias in the observed sample SD of individual ETAs due to

insufficient sample size- Biased shrinkage estimates may result from

biased sample SD (used in the numerator of the shrinkage calculation), =

particularly in small data sets.

I think the take-home message is that ETA-based diagnostics (and

diagnostics of the diagnostics) can be useful, but should be

considered in the context of the design and potential biases.

Best regards,

Marc

Marc R. Gastonguay, Ph.D. < marcg

President & CEO, Metrum Research Group LLC < metrumrg.com >

Scientific Director, Metrum Institute < metruminstitute.org >

2 Tunxis Rd, Suite 112, Tariffville, CT 06081 Direct:

+1.860.670.0744 Main: +1.860.735.7043 Fax: +1.860.760.6014

On Aug 21, 2009, at 9:12 AM, Ribbing, Jakob wrote:

[mailto:owner-nmusers

Received on Fri Aug 21 2009 - 12:10:13 EDT