From: Mats Karlsson <*mats.karlsson*>

Date: Sun, 30 May 2010 08:40:19 +0200

Dear Douglas and all,

We always have some knowledge about our parameter distribution. It comes =

from two sources: prior information and the data, under the model. Prior =

information almost always tell us that parameters must be non-normally =

distributed. That’s why we enforce different types of fixed =

transformations. Usually exponential transformation for parameters that =

has to be non-negative and logit transformation for fractions and =

probabilities. We then often have introduced what prior knowledge we =

have regarding the shape of the distribution. However, also our data =

contain information about the parameter distribution under the model we =

choose and one distribution may describe data better than another. We =

can explore this by choosing different fixed transformation. We may also =

allow the data to speak to the shape of the distribution as part of the =

estimation process. The latter approach was introduced into our field by =

Davidian&Gallant (J Pharmacokinet Biopharm. 1992 Oct;20(5):529-56) using =

polynomials and a specialized software. We recently explored other =

transformation that could be easily introduced into NONMEM and other =

standard programs (Petersson et al., Pharm Res. 2009 Sep;26(9):2174-85). =

If you want to explore deviations from normality under your fixed =

transformation, these semi-parametric* methods may be a good =

alternative. Below is code for a simple box-cox transformation on top =

of a fixed exponential transformation. Positive values of SHP indicates =

right-skewed distribution (compared to a exponential transformation), =

negative a left-skewed. If the transformation offers no improvement in =

fit over an exponential distribution, the goodness-of-fit will be =

similar to that of a simpler model (CL=THETA(1)*EXP(ETA(1))).

SHP = THETA(2)

TETA = ((EXP(ETA(1))**SHP-1)/SHP

CL = THETA(1)*EXP(TETA)

(Semi-parametric is the traditionally used word for these methods, it =

probably comes from the fact that it lies between the standard =

parametric methods where the shape is prescribed by the model, and =

non-parametric methods where very little distributional assumption is =

being made. Semi-parametric methods are essentially parametric but =

parameters are estimated that relates not just the magnitude, but also =

the shape of the distribution.)

Best regards,

Mats

Mats Karlsson, PhD

Professor of Pharmacometrics

Dept of Pharmaceutical Biosciences

Uppsala University

Box 591

751 24 Uppsala Sweden

phone: +46 18 4714105

fax: +46 18 471 4003

From: owner-nmusers

On Behalf Of Eleveld, DJ

Sent: Sunday, May 30, 2010 1:20 AM

To: Nick Holford; nmusers

Cc: Marc Lavielle

Subject: RE: [NMusers] distribution assumption of Eta in NONMEM

I'd like to interject a slightly different point of view to the =

distributional assumption question here.

When I hear people speak in terms of the “distribution =

assumptions of some estimation method” I think its easy for =

people to jump to the conclusion that the normal distribution assumption =

is just one of many possible, equally justifiable distributional =

assumptions that could potentially be made. And that if the normal =

distribution is the “wrong” one then the results from =

such an estimation method would be “wrong”. This is =

what I used to think, but now I believe this is wrong and I'd like to =

help others from wasting as much time thinking along this path, as I =

have.

From information theory, information is gained when entropy decreases. =

So if you have data from some unknown distribution and if you must make =

some distribution assumption in order to analyze the data, you should =

choose the highest entropy distribution you can. This insures that your =

initial assumptions, the ones you do before you actually consider your =

data, are the most uninformative you can make. This is the principle of =

Maximum Entropy which is related to Principle of Indifference and the =

Principle of Insufficient Reason.

A normal distribution has the highest entropy of all real-valued =

distributions that share the same mean and standard deviation. So if =

you assume your data has some true SD, then the best distribution to =

assume would be normal distribution. So we should not think of the =

normal distribution assumption as one of many equally justifiable =

choices, it is really the “least-bad” assumption we can =

make when we do not know the true distribution. Even if normal is the =

“wrong” distribution, it still remains the =

“best”, by virtue of being the =

“least-bad”, because it is the most uninformative =

assumption that can be made (assuming a some finite true variance).

In the real-word we never know the true distribution and so it makes =

sense to always assume a normal distribution unless we have some =

scientifically justifiable reason to believe that some other =

distribution assumption would be advantageous.

The Cauchy distribution is a different animal though since its has an =

infinite variance, and is therefore an even weaker assumption than the =

finite true SD of a normal distribution. It would possibly be even =

better than a normal distribution because its entropy is even higher =

(comparing the standard Cauchy and standard normal). It would be very =

interesting if Cauchy distributions could be used in NONMEM. Actually, =

the ratio of two N(0,1) random variables is Cauchy distributed. Maybe =

this property could be used trick NONMEM into making a Cauchy (or =

nearly-Cauchy) distributed random variable?

Douglas Eleveld

_____

De inhoud van dit bericht is vertrouwelijk en alleen bestemd voor de =

geadresseerde(n). Anderen dan de geadresseerde(n) mogen geen gebruik =

maken van dit bericht, het niet openbaar maken of op enige wijze =

verspreiden of vermenigvuldigen. Het UMCG kan niet aansprakelijk gesteld =

worden voor een incomplete aankomst of vertraging van dit verzonden =

bericht.

The contents of this message are confidential and only intended for the =

eyes of the addressee(s). Others than the addressee(s) are not allowed =

to use this message, to make it public or to distribute or multiply this =

message in any way. The UMCG cannot be held responsible for incomplete =

reception or delay of this transferred message.

Received on Sun May 30 2010 - 02:40:19 EDT

Date: Sun, 30 May 2010 08:40:19 +0200

Dear Douglas and all,

We always have some knowledge about our parameter distribution. It comes =

from two sources: prior information and the data, under the model. Prior =

information almost always tell us that parameters must be non-normally =

distributed. That’s why we enforce different types of fixed =

transformations. Usually exponential transformation for parameters that =

has to be non-negative and logit transformation for fractions and =

probabilities. We then often have introduced what prior knowledge we =

have regarding the shape of the distribution. However, also our data =

contain information about the parameter distribution under the model we =

choose and one distribution may describe data better than another. We =

can explore this by choosing different fixed transformation. We may also =

allow the data to speak to the shape of the distribution as part of the =

estimation process. The latter approach was introduced into our field by =

Davidian&Gallant (J Pharmacokinet Biopharm. 1992 Oct;20(5):529-56) using =

polynomials and a specialized software. We recently explored other =

transformation that could be easily introduced into NONMEM and other =

standard programs (Petersson et al., Pharm Res. 2009 Sep;26(9):2174-85). =

If you want to explore deviations from normality under your fixed =

transformation, these semi-parametric* methods may be a good =

alternative. Below is code for a simple box-cox transformation on top =

of a fixed exponential transformation. Positive values of SHP indicates =

right-skewed distribution (compared to a exponential transformation), =

negative a left-skewed. If the transformation offers no improvement in =

fit over an exponential distribution, the goodness-of-fit will be =

similar to that of a simpler model (CL=THETA(1)*EXP(ETA(1))).

SHP = THETA(2)

TETA = ((EXP(ETA(1))**SHP-1)/SHP

CL = THETA(1)*EXP(TETA)

(Semi-parametric is the traditionally used word for these methods, it =

probably comes from the fact that it lies between the standard =

parametric methods where the shape is prescribed by the model, and =

non-parametric methods where very little distributional assumption is =

being made. Semi-parametric methods are essentially parametric but =

parameters are estimated that relates not just the magnitude, but also =

the shape of the distribution.)

Best regards,

Mats

Mats Karlsson, PhD

Professor of Pharmacometrics

Dept of Pharmaceutical Biosciences

Uppsala University

Box 591

751 24 Uppsala Sweden

phone: +46 18 4714105

fax: +46 18 471 4003

From: owner-nmusers

On Behalf Of Eleveld, DJ

Sent: Sunday, May 30, 2010 1:20 AM

To: Nick Holford; nmusers

Cc: Marc Lavielle

Subject: RE: [NMusers] distribution assumption of Eta in NONMEM

I'd like to interject a slightly different point of view to the =

distributional assumption question here.

When I hear people speak in terms of the “distribution =

assumptions of some estimation method” I think its easy for =

people to jump to the conclusion that the normal distribution assumption =

is just one of many possible, equally justifiable distributional =

assumptions that could potentially be made. And that if the normal =

distribution is the “wrong” one then the results from =

such an estimation method would be “wrong”. This is =

what I used to think, but now I believe this is wrong and I'd like to =

help others from wasting as much time thinking along this path, as I =

have.

From information theory, information is gained when entropy decreases. =

So if you have data from some unknown distribution and if you must make =

some distribution assumption in order to analyze the data, you should =

choose the highest entropy distribution you can. This insures that your =

initial assumptions, the ones you do before you actually consider your =

data, are the most uninformative you can make. This is the principle of =

Maximum Entropy which is related to Principle of Indifference and the =

Principle of Insufficient Reason.

A normal distribution has the highest entropy of all real-valued =

distributions that share the same mean and standard deviation. So if =

you assume your data has some true SD, then the best distribution to =

assume would be normal distribution. So we should not think of the =

normal distribution assumption as one of many equally justifiable =

choices, it is really the “least-bad” assumption we can =

make when we do not know the true distribution. Even if normal is the =

“wrong” distribution, it still remains the =

“best”, by virtue of being the =

“least-bad”, because it is the most uninformative =

assumption that can be made (assuming a some finite true variance).

In the real-word we never know the true distribution and so it makes =

sense to always assume a normal distribution unless we have some =

scientifically justifiable reason to believe that some other =

distribution assumption would be advantageous.

The Cauchy distribution is a different animal though since its has an =

infinite variance, and is therefore an even weaker assumption than the =

finite true SD of a normal distribution. It would possibly be even =

better than a normal distribution because its entropy is even higher =

(comparing the standard Cauchy and standard normal). It would be very =

interesting if Cauchy distributions could be used in NONMEM. Actually, =

the ratio of two N(0,1) random variables is Cauchy distributed. Maybe =

this property could be used trick NONMEM into making a Cauchy (or =

nearly-Cauchy) distributed random variable?

Douglas Eleveld

_____

De inhoud van dit bericht is vertrouwelijk en alleen bestemd voor de =

geadresseerde(n). Anderen dan de geadresseerde(n) mogen geen gebruik =

maken van dit bericht, het niet openbaar maken of op enige wijze =

verspreiden of vermenigvuldigen. Het UMCG kan niet aansprakelijk gesteld =

worden voor een incomplete aankomst of vertraging van dit verzonden =

bericht.

The contents of this message are confidential and only intended for the =

eyes of the addressee(s). Others than the addressee(s) are not allowed =

to use this message, to make it public or to distribute or multiply this =

message in any way. The UMCG cannot be held responsible for incomplete =

reception or delay of this transferred message.

Received on Sun May 30 2010 - 02:40:19 EDT