From: Nick Holford <*n.holford*>

Date: Sat, 31 Oct 2009 11:43:09 +1300

Hi,

Thuy shows an example of using IMP to calculate OBJ with a value of

NITER=300. In the intro7.pdf (p 16) there is this example with NITER=5:

"After the analysis, suitable objective function for hypothesis testing

and standard errors can be obtained by importance sampling at the final

population parameter values. Thus, one could issue this sequence of

commands:

$EST METHOD=SAEM NBURN=2000 NITER=1000

$EST METHOD=IMP EONLY=1 ISAMPLE=1000 NITER=5"

On p 30 there is an example with NITER=10:

"Importance sampling for 10 iterations, expectation step only (this

evaluates OBJF without moving population parameters). Note that SIGL=7

that was set for the previous $EST command is assumed for this $EST

command as well

$EST METHOD=IMP INTERACTION ISAMPLE=1000 NITER=10 EONLY=1"

And on p 31 there is another example which now has NITER=5 again.

"$EST METHOD=SAEM NBURN=200 NITER=500 PRINT=100

; Calculate OBJF by importance sampling

$EST METHOD=IMP EONLY=1 NITER=5 ISAMPLE=3000 PRINT=1"

Intro7.pdf discusses the use of ISAMPLE on p 14 and EONLY and NITER on p15:

"ISAMPLE

Sets number of random samples per subject used for expectation step

(default 300). Usually 300 is sufficient, but may require 1000-3000 for

very sparse data, and when desiring objective function evaluation with

low Monte Carlo noise.

EONLY=1

Evaluate the objective function by performing only the expectation step,

without advancing the population parameters (default is 0, population

parameters are updated). When this method is used, NITER should equal 5

to 10, to allow proposal density to improve with each iteration, since

mean and variance of parameters of normal or t distribution proposal

density are obtained from the previous iteration. Also it is good to get

several objective function values to assess the Monte Carlo noise in it."

I tried one test problem which ran FOCE then SAEM then IMP with

different values for NITER in the IMP step e.g. here with NITER=5:

$EST METHOD=COND INTER MAX=9990 NSIG=3 SIGL=9 PRINT=20 NOABORT

$EST METHOD=SAEM NBURN=500 NITER=200 PRINT=100

$EST METHOD=IMP EONLY=1 ISAMPLE=1000 NITER=5

With NITER=0 and 1 the OBJ was clearly different but with NITER=5 the

OBJ was very similar (but not identical) to the other IMP runs and to

the FOCE run. As might be expected the run times get longer with larger

NITER values so it seems that NITER=5 is an empirical compromise between

estimation of OBJ and run time. It also illustrates the "Monte Carlo

noise" in the OBJ estimate.

NITER for IMP Obj Meth Seconds

- 217.997 FOCE 0.91

0 225.403 IMP 6.70

1 223.467 IMP 7.98

5 217.639 IMP 8.36

10 217.017 IMP 10.31

100 217.469 IMP 39.03

500 217.399 IMP 168.08

So, if you have nothing much to do (Thuy <grin>) then set NITER=300 or

more but if you want to get the OBJ value more quickly than a value of 5

would seem to be good enough.

Best wishes,

Nick

Vu, Thuy wrote:

*> Dear NM7 users,
*

*>
*

*> According to the examples in the users guide, the IMP step followed
*

*> the SAEM step with EONLY=1 to evaluate the objective function. In all
*

*> of my models (for the same data set), I have these two $EST steps:
*

*> $EST METHOD=SAEM INTER FILE=saem.txt NBURN=3000 NITER=2000 PRINT=5
*

*> CTYPE=3 GRD=DDDDDDDDDSSDDD SEED=20091013 ISAMPLE=2
*

*> $EST METHOD=IMP EONLY=1 FILE=imp.txt NITER=300 ISAMPLE=3000 PRINT=1
*

*>
*

*> I
*

*>
*

--

Nick Holford, Professor Clinical Pharmacology

Dept Pharmacology & Clinical Pharmacology

University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand

n.holford

mobile: +64 21 46 23 53

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Fri Oct 30 2009 - 18:43:09 EDT

Date: Sat, 31 Oct 2009 11:43:09 +1300

Hi,

Thuy shows an example of using IMP to calculate OBJ with a value of

NITER=300. In the intro7.pdf (p 16) there is this example with NITER=5:

"After the analysis, suitable objective function for hypothesis testing

and standard errors can be obtained by importance sampling at the final

population parameter values. Thus, one could issue this sequence of

commands:

$EST METHOD=SAEM NBURN=2000 NITER=1000

$EST METHOD=IMP EONLY=1 ISAMPLE=1000 NITER=5"

On p 30 there is an example with NITER=10:

"Importance sampling for 10 iterations, expectation step only (this

evaluates OBJF without moving population parameters). Note that SIGL=7

that was set for the previous $EST command is assumed for this $EST

command as well

$EST METHOD=IMP INTERACTION ISAMPLE=1000 NITER=10 EONLY=1"

And on p 31 there is another example which now has NITER=5 again.

"$EST METHOD=SAEM NBURN=200 NITER=500 PRINT=100

; Calculate OBJF by importance sampling

$EST METHOD=IMP EONLY=1 NITER=5 ISAMPLE=3000 PRINT=1"

Intro7.pdf discusses the use of ISAMPLE on p 14 and EONLY and NITER on p15:

"ISAMPLE

Sets number of random samples per subject used for expectation step

(default 300). Usually 300 is sufficient, but may require 1000-3000 for

very sparse data, and when desiring objective function evaluation with

low Monte Carlo noise.

EONLY=1

Evaluate the objective function by performing only the expectation step,

without advancing the population parameters (default is 0, population

parameters are updated). When this method is used, NITER should equal 5

to 10, to allow proposal density to improve with each iteration, since

mean and variance of parameters of normal or t distribution proposal

density are obtained from the previous iteration. Also it is good to get

several objective function values to assess the Monte Carlo noise in it."

I tried one test problem which ran FOCE then SAEM then IMP with

different values for NITER in the IMP step e.g. here with NITER=5:

$EST METHOD=COND INTER MAX=9990 NSIG=3 SIGL=9 PRINT=20 NOABORT

$EST METHOD=SAEM NBURN=500 NITER=200 PRINT=100

$EST METHOD=IMP EONLY=1 ISAMPLE=1000 NITER=5

With NITER=0 and 1 the OBJ was clearly different but with NITER=5 the

OBJ was very similar (but not identical) to the other IMP runs and to

the FOCE run. As might be expected the run times get longer with larger

NITER values so it seems that NITER=5 is an empirical compromise between

estimation of OBJ and run time. It also illustrates the "Monte Carlo

noise" in the OBJ estimate.

NITER for IMP Obj Meth Seconds

- 217.997 FOCE 0.91

0 225.403 IMP 6.70

1 223.467 IMP 7.98

5 217.639 IMP 8.36

10 217.017 IMP 10.31

100 217.469 IMP 39.03

500 217.399 IMP 168.08

So, if you have nothing much to do (Thuy <grin>) then set NITER=300 or

more but if you want to get the OBJ value more quickly than a value of 5

would seem to be good enough.

Best wishes,

Nick

Vu, Thuy wrote:

--

Nick Holford, Professor Clinical Pharmacology

Dept Pharmacology & Clinical Pharmacology

University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand

n.holford

mobile: +64 21 46 23 53

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Fri Oct 30 2009 - 18:43:09 EDT