From: Mats Karlsson <*mats.karlsson*>

Date: Tue, 1 Jun 2010 07:48:05 +0200

Nick,

In your example you simulate and estimate only one random effect. As we =

are talking about situations where we have several levels of random =

effects, I redid you example with a full model which simulates and =

estimates with 1 THETA, 1 OMEGA and 1 SIGMA. The data were still made =

highly informative about ETAs (low shrinkage). Two observations each in =

100 subjects. When simulating with a uniform ETA distribution, =

estimation resulted in biased parameters (see below). When simulating =

with a normal ETA distribution of the same variance, estimation =

resulted in unbiased parameters (see below).

What you would want to do for the case where data is simulated with a =

uniform distribution, is a transform that makes the uniform to a normal. =

I don’t know of a transform that can be implemented into =

estimation that does this, but I implemented a logit transformation (see =

below) which decreased OFV by about 50 units. Also, see below.

$PROB EBE

$INPUT ID DV UNIETA

$DATA uni2.csv; 100 subjects with 2 obs each

$THETA 5 ; HILL

$OMEGA 0.083333333 ; PPV_HILL = 1/12

$SIGMA 0.0001 ; EPS1

$SIM (1234) (5678 UNIFORM) NSUB=10

$EST METHOD=COND MAX=9990 SIG=3 PRINT=1 ;MSFO=msf

$PRED

IF (ICALL.EQ.4) THEN

IF (NEWIND.LE.1) THEN

CALL RANDOM(2,R)

UNIETA=R-0.5 ; U(-0.5,0.5) mean=0, variance=1/12

HILL=THETA(1)*EXP(UNIETA)

Y=1.1**HILL/(1.1**HILL+1)+EPS(1)

ENDIF

ELSE

HILL=THETA(1)*EXP(ETA(1))

Y=1.1**HILL/(1.1**HILL+1) + EPS(1)

ENDIF

REP=IREP

$TABLE ID REP HILL UNIETA ETA(1) Y

ONEHEADER NOPRINT FILE=uni2.fit

simulated normal =

simulated uniform

TRUE 5 0.001 =

0.083333 5 0.001 =

0.083333

Average 5.040598 9.63339E-05 0.078359 =

4.965324 1E-08 0.094581

SD 0.068156 7.00218E-06 =

0.005206 0.08444 1.74E-24 0.001808

LGPAR1 = THETA(2)

LGPAR2 = THETA(3)

PHI = LOG(LGPAR1/(1-LGPAR1))

PAR1 = EXP(PHI+ETA(1))

ETATR = (PAR1/(1+PAR1)-LGPAR1)*LGPAR2

HILL=THETA(1)*EXP(ETATR)

Y=1.1**HILL/(1.1**HILL+1) + EPS(1)

Best regards,

Mats

Mats Karlsson, PhD

Professor of Pharmacometrics

Dept of Pharmaceutical Biosciences

Uppsala University

Box 591

751 24 Uppsala Sweden

phone: +46 18 4714105

fax: +46 18 471 4003

From: owner-nmusers

On Behalf Of Nick Holford

Sent: Monday, May 31, 2010 9:28 PM

To: nmusers

Cc: 'Marc Lavielle'

Subject: Re: [NMusers] distribution assumption of Eta in NONMEM

Mats,

I agree that trying to learn anything from the EBE distribution is a =

largely uninformative activity. If the shrinkage is too small then the =

EBEs are driven primarily by the data distribution (which is probably =

happening in the example I reported) while if they are too big they =

shrink to the population mean (with no information about the =

distribution at the limit). NONMEM7 claims that the ETA shrinkage for =

this experiment is about -40% (whatever that might mean).

I didn’t suggest that learning from the EBE distribution in =

general is a bad idea, I just didn’t understand how your example =

showed your point.

It can be informative to inspect EBEs. Just like you did in your =

example, you learnt that the underlying parameters were more uniformly =

than normally distributed. The distribution of the parameters is often =

we want to know something about whenever possible. (Maybe you agree =

since you think it is only “largely” uninformative).

What do I think this exercise contributes? Well it seems (with this =

rather limited example) to show that NONMEM FOCE can estimate the =

variance of the true ETA distribution (and also a non-linear fixed =

effect parameter) quite accurately even if the true ETA distribution is =

clearly not normal and no normalizing transformation is used.

I think your example is too simple with simulating and estimating only =

one level of random effects.

It has also made me think more carefully what I mean by ETA. There is =

the true ETA used in a simulation that represents the random deviation =

of a parameter from the population value and that random deviation might =

arise from many different distributions. There is the value of NONMEM's =

ETA variable which arises from a distribution defined with mean zero and =

variance OMEGA but without any assumption about its distribution being =

normal when used for estimation (according to Stuart). And finally there =

is some transformed value of NONMEM's ETA variable which influences the =

objective function.

Which of these kinds of ETA were you referring to when you wrote this?

" If you use a method like FOCE, and try different transformation of you =

parameters, you will find that OFV will be lowest and other goodness of =

fit best, when the transformation is such that ETA is normally =

distributed. "

I don’t think that ETA of the 2nd type exist. I don’t =

understand why you say “transformed” for the third type, =

so I won’t go for that one either. I talk about the deviations =

between the typical parameter value and the individual parameters under =

the model.

Are you saying that in my simulation experiment in which the true ETAs =

are known to be uniform that if I apply a transformation involving the =

NONMEM ETA(*) variable which makes the transformed random effect =

normally distributed then the OFV cannot be made lower? So if I try =

different transformations and find the transformation with the lowest =

OFV and if I know what distribution this transformation of the true ETA =

turns into a normal distribution that I can then learn the nature of the =

true ETA distribution?

Do you know what transformation can I apply with NM-TRAN that will =

transform a function of uniform ETA(*) into a normal distribution? =

Implementations of things like the Box-Mueller transform require the use =

of additional random uniform numbers so that won't work for estimation =

e.g. =

http://stackoverflow.com/questions/75677/converting-a-uniform-distributio=

n-to-a-normal-distribution

No, so if you want to simulate – reestimate to understand this =

better, I suggest you choose an underlying distribution that has a =

simpler transform to the normal.

Nick

Mats Karlsson wrote:

Nick,

It has been showed over and over again that empirical Bayes estimates, =

when individual data is rich, will resemble the true individual =

parameter regardless of the underlying distribution. Therefore I =

don’t understand what you think this exercise contributes.

Best regards,

Mats

Mats Karlsson, PhD

Professor of Pharmacometrics

Dept of Pharmaceutical Biosciences

Uppsala University

Box 591

751 24 Uppsala Sweden

phone: +46 18 4714105

fax: +46 18 471 4003

From: owner-nmusers

On Behalf Of Nick Holford

Sent: Monday, May 31, 2010 6:05 PM

To: nmusers

Cc: 'Marc Lavielle'

Subject: Re: [NMusers] distribution assumption of Eta in NONMEM

Hi,

I tried to see with brute force how well NONMEM can produce an empirical =

Bayes estimate when the ETA used for simulation is uniform. I attempted =

to stress NONMEM with a non-linear problem (the average DV is 0.62). The =

mean estimate of OMEGA(1) was 0.0827 compared with the theoretical value =

of 0.0833.

The distribution of 1000 EBEs of ETA(1) looked much more uniform than =

normal.

Thus FOCE show no evidence of normality being imposed on the EBEs.

$PROB EBE

$INPUT ID DV UNIETA

$DATA uni1.csv ; 100 subjects with 1 obs each

$THETA 5 ; HILL

$OMEGA 0.083333333 ; PPV_HILL = 1/12

$SIGMA 0.000001 FIX ; EPS1

$SIM (1234) (5678 UNIFORM) NSUB=10

$EST METHOD=COND MAX=9990 SIG=3

$PRED

IF (ICALL.EQ.4) THEN

IF (NEWIND.LE.1) THEN

CALL RANDOM(2,R)

UNIETA=R-0.5 ; U(-0.5,0.5) mean=0, variance=1/12

HILL=THETA(1)*EXP(UNIETA)

Y=1.1**HILL/(1.1**HILL+1)

ENDIF

ELSE

HILL=THETA(1)*EXP(ETA(1))

Y=1.1**HILL/(1.1**HILL+1) + EPS(1)

ENDIF

REP=IREP

$TABLE ID REP HILL UNIETA ETA(1) Y

ONEHEADER NOPRINT FILE=uni.fit

I realized after a bit more thought that my suggestion to transform the =

eta value for estimation wasn't rational so please ignore that senior =

moment in my earlier email on this topic.

Nick

--

Nick Holford, Professor Clinical Pharmacology

Dept Pharmacology & Clinical Pharmacology

University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand

tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53

email: n.holford

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

--

Nick Holford, Professor Clinical Pharmacology

Dept Pharmacology & Clinical Pharmacology

University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand

tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53

email: n.holford

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Tue Jun 01 2010 - 01:48:05 EDT

Date: Tue, 1 Jun 2010 07:48:05 +0200

Nick,

In your example you simulate and estimate only one random effect. As we =

are talking about situations where we have several levels of random =

effects, I redid you example with a full model which simulates and =

estimates with 1 THETA, 1 OMEGA and 1 SIGMA. The data were still made =

highly informative about ETAs (low shrinkage). Two observations each in =

100 subjects. When simulating with a uniform ETA distribution, =

estimation resulted in biased parameters (see below). When simulating =

with a normal ETA distribution of the same variance, estimation =

resulted in unbiased parameters (see below).

What you would want to do for the case where data is simulated with a =

uniform distribution, is a transform that makes the uniform to a normal. =

I don’t know of a transform that can be implemented into =

estimation that does this, but I implemented a logit transformation (see =

below) which decreased OFV by about 50 units. Also, see below.

$PROB EBE

$INPUT ID DV UNIETA

$DATA uni2.csv; 100 subjects with 2 obs each

$THETA 5 ; HILL

$OMEGA 0.083333333 ; PPV_HILL = 1/12

$SIGMA 0.0001 ; EPS1

$SIM (1234) (5678 UNIFORM) NSUB=10

$EST METHOD=COND MAX=9990 SIG=3 PRINT=1 ;MSFO=msf

$PRED

IF (ICALL.EQ.4) THEN

IF (NEWIND.LE.1) THEN

CALL RANDOM(2,R)

UNIETA=R-0.5 ; U(-0.5,0.5) mean=0, variance=1/12

HILL=THETA(1)*EXP(UNIETA)

Y=1.1**HILL/(1.1**HILL+1)+EPS(1)

ENDIF

ELSE

HILL=THETA(1)*EXP(ETA(1))

Y=1.1**HILL/(1.1**HILL+1) + EPS(1)

ENDIF

REP=IREP

$TABLE ID REP HILL UNIETA ETA(1) Y

ONEHEADER NOPRINT FILE=uni2.fit

simulated normal =

simulated uniform

TRUE 5 0.001 =

0.083333 5 0.001 =

0.083333

Average 5.040598 9.63339E-05 0.078359 =

4.965324 1E-08 0.094581

SD 0.068156 7.00218E-06 =

0.005206 0.08444 1.74E-24 0.001808

LGPAR1 = THETA(2)

LGPAR2 = THETA(3)

PHI = LOG(LGPAR1/(1-LGPAR1))

PAR1 = EXP(PHI+ETA(1))

ETATR = (PAR1/(1+PAR1)-LGPAR1)*LGPAR2

HILL=THETA(1)*EXP(ETATR)

Y=1.1**HILL/(1.1**HILL+1) + EPS(1)

Best regards,

Mats

Mats Karlsson, PhD

Professor of Pharmacometrics

Dept of Pharmaceutical Biosciences

Uppsala University

Box 591

751 24 Uppsala Sweden

phone: +46 18 4714105

fax: +46 18 471 4003

From: owner-nmusers

On Behalf Of Nick Holford

Sent: Monday, May 31, 2010 9:28 PM

To: nmusers

Cc: 'Marc Lavielle'

Subject: Re: [NMusers] distribution assumption of Eta in NONMEM

Mats,

I agree that trying to learn anything from the EBE distribution is a =

largely uninformative activity. If the shrinkage is too small then the =

EBEs are driven primarily by the data distribution (which is probably =

happening in the example I reported) while if they are too big they =

shrink to the population mean (with no information about the =

distribution at the limit). NONMEM7 claims that the ETA shrinkage for =

this experiment is about -40% (whatever that might mean).

I didn’t suggest that learning from the EBE distribution in =

general is a bad idea, I just didn’t understand how your example =

showed your point.

It can be informative to inspect EBEs. Just like you did in your =

example, you learnt that the underlying parameters were more uniformly =

than normally distributed. The distribution of the parameters is often =

we want to know something about whenever possible. (Maybe you agree =

since you think it is only “largely” uninformative).

What do I think this exercise contributes? Well it seems (with this =

rather limited example) to show that NONMEM FOCE can estimate the =

variance of the true ETA distribution (and also a non-linear fixed =

effect parameter) quite accurately even if the true ETA distribution is =

clearly not normal and no normalizing transformation is used.

I think your example is too simple with simulating and estimating only =

one level of random effects.

It has also made me think more carefully what I mean by ETA. There is =

the true ETA used in a simulation that represents the random deviation =

of a parameter from the population value and that random deviation might =

arise from many different distributions. There is the value of NONMEM's =

ETA variable which arises from a distribution defined with mean zero and =

variance OMEGA but without any assumption about its distribution being =

normal when used for estimation (according to Stuart). And finally there =

is some transformed value of NONMEM's ETA variable which influences the =

objective function.

Which of these kinds of ETA were you referring to when you wrote this?

" If you use a method like FOCE, and try different transformation of you =

parameters, you will find that OFV will be lowest and other goodness of =

fit best, when the transformation is such that ETA is normally =

distributed. "

I don’t think that ETA of the 2nd type exist. I don’t =

understand why you say “transformed” for the third type, =

so I won’t go for that one either. I talk about the deviations =

between the typical parameter value and the individual parameters under =

the model.

Are you saying that in my simulation experiment in which the true ETAs =

are known to be uniform that if I apply a transformation involving the =

NONMEM ETA(*) variable which makes the transformed random effect =

normally distributed then the OFV cannot be made lower? So if I try =

different transformations and find the transformation with the lowest =

OFV and if I know what distribution this transformation of the true ETA =

turns into a normal distribution that I can then learn the nature of the =

true ETA distribution?

Do you know what transformation can I apply with NM-TRAN that will =

transform a function of uniform ETA(*) into a normal distribution? =

Implementations of things like the Box-Mueller transform require the use =

of additional random uniform numbers so that won't work for estimation =

e.g. =

http://stackoverflow.com/questions/75677/converting-a-uniform-distributio=

n-to-a-normal-distribution

No, so if you want to simulate – reestimate to understand this =

better, I suggest you choose an underlying distribution that has a =

simpler transform to the normal.

Nick

Mats Karlsson wrote:

Nick,

It has been showed over and over again that empirical Bayes estimates, =

when individual data is rich, will resemble the true individual =

parameter regardless of the underlying distribution. Therefore I =

don’t understand what you think this exercise contributes.

Best regards,

Mats

Mats Karlsson, PhD

Professor of Pharmacometrics

Dept of Pharmaceutical Biosciences

Uppsala University

Box 591

751 24 Uppsala Sweden

phone: +46 18 4714105

fax: +46 18 471 4003

From: owner-nmusers

On Behalf Of Nick Holford

Sent: Monday, May 31, 2010 6:05 PM

To: nmusers

Cc: 'Marc Lavielle'

Subject: Re: [NMusers] distribution assumption of Eta in NONMEM

Hi,

I tried to see with brute force how well NONMEM can produce an empirical =

Bayes estimate when the ETA used for simulation is uniform. I attempted =

to stress NONMEM with a non-linear problem (the average DV is 0.62). The =

mean estimate of OMEGA(1) was 0.0827 compared with the theoretical value =

of 0.0833.

The distribution of 1000 EBEs of ETA(1) looked much more uniform than =

normal.

Thus FOCE show no evidence of normality being imposed on the EBEs.

$PROB EBE

$INPUT ID DV UNIETA

$DATA uni1.csv ; 100 subjects with 1 obs each

$THETA 5 ; HILL

$OMEGA 0.083333333 ; PPV_HILL = 1/12

$SIGMA 0.000001 FIX ; EPS1

$SIM (1234) (5678 UNIFORM) NSUB=10

$EST METHOD=COND MAX=9990 SIG=3

$PRED

IF (ICALL.EQ.4) THEN

IF (NEWIND.LE.1) THEN

CALL RANDOM(2,R)

UNIETA=R-0.5 ; U(-0.5,0.5) mean=0, variance=1/12

HILL=THETA(1)*EXP(UNIETA)

Y=1.1**HILL/(1.1**HILL+1)

ENDIF

ELSE

HILL=THETA(1)*EXP(ETA(1))

Y=1.1**HILL/(1.1**HILL+1) + EPS(1)

ENDIF

REP=IREP

$TABLE ID REP HILL UNIETA ETA(1) Y

ONEHEADER NOPRINT FILE=uni.fit

I realized after a bit more thought that my suggestion to transform the =

eta value for estimation wasn't rational so please ignore that senior =

moment in my earlier email on this topic.

Nick

--

Nick Holford, Professor Clinical Pharmacology

Dept Pharmacology & Clinical Pharmacology

University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand

tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53

email: n.holford

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

--

Nick Holford, Professor Clinical Pharmacology

Dept Pharmacology & Clinical Pharmacology

University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand

tel:+64(9)923-6730 fax:+64(9)373-7090 mobile:+64(21)46 23 53

email: n.holford

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Tue Jun 01 2010 - 01:48:05 EDT