From: Bob Leary <*bleary*>

Date: Fri, 19 Jun 2009 16:08:00 -0400

Steve -

I agree with you that adding addtional data (in this case adding a =

sparse data set to a dense data set)

ideally should result in better (more precise) estimates when the =

individuals from the two data sets

are exchangeable, but only assuming the underlying estimation =

methodology is well-behaved for

both the sparse and dense data. In the real world, and in particular

with the FOCE approximation, sparse data may not be well estimated by =

FOCE and merging sparse data with

dense data may contaminate the dense data rather than enhance it.

As an example, I ran a simulation using an Emax Hill coefficient model =

with gamma=4.5 (this is the same

model that appears in Mats' 2007 Pharm Res paper Hooker et al, =

"Conditional Weighted Residuals (CWRES):

A Diagostic For the FOCE Method)". Using dense data (25 observations =

per subject) and 200 subjects

simulated from the true model, all parameters all well estimated. In =

particular, gamma is estimated at

4.38 (std err = 0.069)

For an equivalent amount of sparse data (2000 subjects, average of 2.5 =

obs/susbject, also simulated from the same

model and the same design except that observations were removed at =

random with a 90% probability of removal for

any given observation), the FOCE estimate of gamma is 3.51 (std err = =

0.060)

(all other parameters are reasonably estimated by the sparse data).

When the data sets are combined, the gamma estimate is 3.69 (std err =

=0.104 ) . Thus merging the dense data with

the sparse data has resulted in a good estimate being converted to a =

relatively poor one. Moreover,

the std error (albeit from a Hessian based computation) has increased in =

the merged set relative to both

separate individual sets.

I agree with Jurgen that nonparametric estimation is much less =

susceptible to this contamination

effect. For the merged data set, the nonparametric

method will likely simply use the sharply defined support points from =

the dense data, (assuming there are a reasonable number

of these and they cover the region of interest). Individuals from the =

dense data set will be modeled with very large

probabilities associated with their single corresponding support point, =

while individuals from the sparse set will

have probabilities spread out over several supports.

Bob Leary

Robert H. Leary, PhD

Fellow

Pharsight - A Certara(tm) Company

5625 Dillard Dr., Suite 205

Cary, NC 27511

Phone/Voice Mail: (919) 852-4625, Fax: (919) 859-6871

Email: bleary

This email message (including any attachments) is for the sole use of =

the intended recipient and may contain confidential and proprietary =

information. Any disclosure or distribution to third parties that is =

not specifically authorized by the sender is prohibited. If you are not =

the intended recipient, please contact the sender by reply email and =

destroy all copies of the original message.

-----Original Message-----

From: owner-nmusers

[mailto:owner-nmusers

Sent: Thursday, June 18, 2009 2:4 AM

To: Mats Karlsson; 'Ribbing, Jakob'; 'Ethan Wu'; 'Jurgen Bulitta'; =

nmusers

Cc: 'Roger Jelliffe'; 'Neely, Michael'

Subject: RE: [NMusers] estimating Ka from dataset combining rich sample =

study and sparse sampling study

Dear Ethan

I concur with Mats's comments below.

As a note, from a design perspective adding additional data to an =

experiment cannot result in less precise parameter estimates under the =

assumption that the individuals from the two data sets are exchangeable. =

Under this assumption therefore the Sparse data should merely add =

information to the Rich data. That the Sparse data is affecting the =

parameter estimates from the Rich data suggests that the two data sets =

are not exchangeable (different centre, different assay, different =

covariates ...).

Another possible way to investigate the differences between the two data =

sets would be to analyse them sequentially, perhaps with consideration =

for using the analysis from the Rich data as an informative prior for =

the analysis of the Sparse data and see where this leads you.

Kind regards

Steve

--

Professor Stephen Duffull

Chair of Clinical Pharmacy

School of Pharmacy

University of Otago

PO Box 913 Dunedin

New Zealand

E: <mailto:stephen.duffull

P: +64 3 479 5044

F: +64 3 479 7034

Design software: <http://www.winpopt.com> www.winpopt.com

From: owner-nmusers

On Behalf Of Mats Karlsson

Sent: Thursday, 18 June 2009 9:17 a.m.

To: 'Ribbing, Jakob'; 'Ethan Wu'; 'Jurgen Bulitta'; =

nmusers

Cc: 'Roger Jelliffe'; 'Neely, Michael'

Subject: RE: [NMusers] estimating Ka from dataset combining rich sample =

study and sparse sampling study

Dear Ethan,

Variances estimated to be zero may result from fixing off-diagonal =

variances to zero (i.e. not using BLOCKs in IIV). Here, however, it may =

be that there are systematic differences between the sparse and the rich =

data experiments. Maybe fasting/fed status or something else is =

different. If the fit to the rich data is markedly worse when including =

the rich data, at least one parameter is different between the two =

situations. I would explore what parameter(s) that would be. In addition =

to Jakob's suggestions below, the two data sets together may indicate a =

more complex structural model that a single profile indicated. Maybe you =

need to go to a two-compartment for example.

Best regards,

Mats

Mats Karlsson, PhD

Professor of Pharmacometrics

Dept of Pharmaceutical Biosciences

Uppsala University

Box 591

751 24 Uppsala Sweden

phone: +46 18 4714105

fax: +46 18 471 4003

From: owner-nmusers

On Behalf Of Ribbing, Jakob

Sent: Wednesday, June 17, 2009 10:43 PM

To: Ethan Wu; Jurgen Bulitta; nmusers

Cc: Roger Jelliffe; Neely, Michael

Subject: RE: [NMusers] estimating Ka from dataset combining rich sample =

study and sparse sampling study

Hi Ethan,

If OMEGA(?) for KA is drastically reduced when including the sparse =

data, then something is wrong with your model and in this case it is not =

the estimation method or assumption on distribution of individual =

parameter). Eta-shrinkage would not drastically reduce the estimate of =

OMEGA, since this estimate is driven by the subjects/studies which =

contain information on the parameter.

If the sparse data is multiple dosing it may be that KA is variable =

between occasions, rather than between subjects (assuming the sparse =

data contain some information on KA). Or if the sparse data is from a =

less well-controlled study or a different population, it may be that =

increased IIV in other parts of the model (e.g. OMEGA on V) is making =

IIV in KA appear low for the rich study, when fitting the two studies =

together. If you get the covariate model in place this problem will be =

solved. For the simple model you have it should be quick to start out =

assuming that most parameters (THETAs and OMEGAs) are different between =

the two studies and then reduce down to a model which is stable and =

parsimonious. Obviously, if you eventually can explain the differences =

using more mechanistic covariates than study number that is of more use.

Cheers

Jakob

Received on Fri Jun 19 2009 - 16:08:00 EDT

Date: Fri, 19 Jun 2009 16:08:00 -0400

Steve -

I agree with you that adding addtional data (in this case adding a =

sparse data set to a dense data set)

ideally should result in better (more precise) estimates when the =

individuals from the two data sets

are exchangeable, but only assuming the underlying estimation =

methodology is well-behaved for

both the sparse and dense data. In the real world, and in particular

with the FOCE approximation, sparse data may not be well estimated by =

FOCE and merging sparse data with

dense data may contaminate the dense data rather than enhance it.

As an example, I ran a simulation using an Emax Hill coefficient model =

with gamma=4.5 (this is the same

model that appears in Mats' 2007 Pharm Res paper Hooker et al, =

"Conditional Weighted Residuals (CWRES):

A Diagostic For the FOCE Method)". Using dense data (25 observations =

per subject) and 200 subjects

simulated from the true model, all parameters all well estimated. In =

particular, gamma is estimated at

4.38 (std err = 0.069)

For an equivalent amount of sparse data (2000 subjects, average of 2.5 =

obs/susbject, also simulated from the same

model and the same design except that observations were removed at =

random with a 90% probability of removal for

any given observation), the FOCE estimate of gamma is 3.51 (std err = =

0.060)

(all other parameters are reasonably estimated by the sparse data).

When the data sets are combined, the gamma estimate is 3.69 (std err =

=0.104 ) . Thus merging the dense data with

the sparse data has resulted in a good estimate being converted to a =

relatively poor one. Moreover,

the std error (albeit from a Hessian based computation) has increased in =

the merged set relative to both

separate individual sets.

I agree with Jurgen that nonparametric estimation is much less =

susceptible to this contamination

effect. For the merged data set, the nonparametric

method will likely simply use the sharply defined support points from =

the dense data, (assuming there are a reasonable number

of these and they cover the region of interest). Individuals from the =

dense data set will be modeled with very large

probabilities associated with their single corresponding support point, =

while individuals from the sparse set will

have probabilities spread out over several supports.

Bob Leary

Robert H. Leary, PhD

Fellow

Pharsight - A Certara(tm) Company

5625 Dillard Dr., Suite 205

Cary, NC 27511

Phone/Voice Mail: (919) 852-4625, Fax: (919) 859-6871

Email: bleary

This email message (including any attachments) is for the sole use of =

the intended recipient and may contain confidential and proprietary =

information. Any disclosure or distribution to third parties that is =

not specifically authorized by the sender is prohibited. If you are not =

the intended recipient, please contact the sender by reply email and =

destroy all copies of the original message.

-----Original Message-----

From: owner-nmusers

[mailto:owner-nmusers

Sent: Thursday, June 18, 2009 2:4 AM

To: Mats Karlsson; 'Ribbing, Jakob'; 'Ethan Wu'; 'Jurgen Bulitta'; =

nmusers

Cc: 'Roger Jelliffe'; 'Neely, Michael'

Subject: RE: [NMusers] estimating Ka from dataset combining rich sample =

study and sparse sampling study

Dear Ethan

I concur with Mats's comments below.

As a note, from a design perspective adding additional data to an =

experiment cannot result in less precise parameter estimates under the =

assumption that the individuals from the two data sets are exchangeable. =

Under this assumption therefore the Sparse data should merely add =

information to the Rich data. That the Sparse data is affecting the =

parameter estimates from the Rich data suggests that the two data sets =

are not exchangeable (different centre, different assay, different =

covariates ...).

Another possible way to investigate the differences between the two data =

sets would be to analyse them sequentially, perhaps with consideration =

for using the analysis from the Rich data as an informative prior for =

the analysis of the Sparse data and see where this leads you.

Kind regards

Steve

--

Professor Stephen Duffull

Chair of Clinical Pharmacy

School of Pharmacy

University of Otago

PO Box 913 Dunedin

New Zealand

E: <mailto:stephen.duffull

P: +64 3 479 5044

F: +64 3 479 7034

Design software: <http://www.winpopt.com> www.winpopt.com

From: owner-nmusers

On Behalf Of Mats Karlsson

Sent: Thursday, 18 June 2009 9:17 a.m.

To: 'Ribbing, Jakob'; 'Ethan Wu'; 'Jurgen Bulitta'; =

nmusers

Cc: 'Roger Jelliffe'; 'Neely, Michael'

Subject: RE: [NMusers] estimating Ka from dataset combining rich sample =

study and sparse sampling study

Dear Ethan,

Variances estimated to be zero may result from fixing off-diagonal =

variances to zero (i.e. not using BLOCKs in IIV). Here, however, it may =

be that there are systematic differences between the sparse and the rich =

data experiments. Maybe fasting/fed status or something else is =

different. If the fit to the rich data is markedly worse when including =

the rich data, at least one parameter is different between the two =

situations. I would explore what parameter(s) that would be. In addition =

to Jakob's suggestions below, the two data sets together may indicate a =

more complex structural model that a single profile indicated. Maybe you =

need to go to a two-compartment for example.

Best regards,

Mats

Mats Karlsson, PhD

Professor of Pharmacometrics

Dept of Pharmaceutical Biosciences

Uppsala University

Box 591

751 24 Uppsala Sweden

phone: +46 18 4714105

fax: +46 18 471 4003

From: owner-nmusers

On Behalf Of Ribbing, Jakob

Sent: Wednesday, June 17, 2009 10:43 PM

To: Ethan Wu; Jurgen Bulitta; nmusers

Cc: Roger Jelliffe; Neely, Michael

Subject: RE: [NMusers] estimating Ka from dataset combining rich sample =

study and sparse sampling study

Hi Ethan,

If OMEGA(?) for KA is drastically reduced when including the sparse =

data, then something is wrong with your model and in this case it is not =

the estimation method or assumption on distribution of individual =

parameter). Eta-shrinkage would not drastically reduce the estimate of =

OMEGA, since this estimate is driven by the subjects/studies which =

contain information on the parameter.

If the sparse data is multiple dosing it may be that KA is variable =

between occasions, rather than between subjects (assuming the sparse =

data contain some information on KA). Or if the sparse data is from a =

less well-controlled study or a different population, it may be that =

increased IIV in other parts of the model (e.g. OMEGA on V) is making =

IIV in KA appear low for the rich study, when fitting the two studies =

together. If you get the covariate model in place this problem will be =

solved. For the simple model you have it should be quick to start out =

assuming that most parameters (THETAs and OMEGAs) are different between =

the two studies and then reduce down to a model which is stable and =

parsimonious. Obviously, if you eventually can explain the differences =

using more mechanistic covariates than study number that is of more use.

Cheers

Jakob

Received on Fri Jun 19 2009 - 16:08:00 EDT