From: Nick Holford <*n.holford*>

Date: Fri, 17 Oct 2008 09:51:49 +1300

Jun,

VPCs are created using simulation alone (i.e. without estimation)

(Karlsson & Holford 2008)

If you do simulation followed by estimation this is called parametric

bootstrapping (nmgosim in WFN). An alternative bootstrap method is

called the non-parametric bootstrap which samples observations from the

original data set (nmbs in WFN).

The non-parametric bootstrap is probably more robust than the parametric

bootstrap if you have covariates in your model because it is tricky to

simulate correlated covariate distributions.

The distribution of the THETA, OMEGA, SIGMA parameter estimates obtained

from fittting many bootstrap replications can be used to generate

confidence intervals on parameters and explore other properties of the

parameter uncertainty.

The bootstrap confidence intervals are more realistic than those

computed from asymptotic standard errors because they dont assume

normality of the uncertainty. The mean of the bootstrap distribution is

probably a more robust estimate of the true population mean than the

point estimate obtained from the original data.

WFN gives you all the parameter estimates from parametric or

non-parametric bootstraps in a tab delimited file that can be easily

read into Excel or other packages to examine the results and compute

statistics from the distributions.

If you want help with getting other methods to extract parameters from

NONMEM you need to learn to give useful information about what you did,

what went wrong, what the EXACT error messages or unexpected results

were. It may be old stuff but most nmusers are not mind readers :-)

Karlsson MO, Holford NHG. A Tutorial on Visual Predictive Checks. PAGE

17 (2008) Abstr 1434 [wwwpage-meetingorg/?abstract=1434]. 2008.

Jun Shen wrote:

*> Dear Elodie/Steve/Nick,
*

*>
*

*> Thank you for your replies. So we can either simulate or bootstrap to
*

*> perform a visual predictive check. I just downloaded a WFN package
*

*> (developed by Nick) which is very convenient. By doing $Sim and $Est
*

*> we can get simulated data. Now I just wonder since we also get
*

*> estimated parameters (THETA, ETA, SIGMA) from simulation or bootstrap,
*

*> has anyone ever used these parameters? I mean in addition to comparing
*

*> the percentiles of the DV can we extract any useful information by
*

*> looking at those parameters? Something like, if the estimated
*

*> parameters from real dataset fall within the 95% intervals of
*

*> simulated or bootstraped data? Does such comparison make any sense?
*

*>
*

*> A following question is how we export those parameters to a table. I
*

*> know I bring up old stuff. Alison and other people have posted codes.
*

*> But these codes do not seem to work for me. I wonder if anyone has
*

*> successfully used these codes.
*

*>
*

*> Attached codes I found in previous discussions.
*

*>
*

*> SUBROUTINE INFN (ICALL,THETA,DATREC,INDXS,NEWIND)
*

*> DIMENSION THETA(*),DATREC(*),INDXS(*)
*

*> DOUBLE PRECISION THETA
*

*>
*

*> IF (ICALL.EQ.3) THEN
*

*> DO WHILE(DATA)
*

*> IF (NEWIND.LE.1) WRITE (50,*) ETA
*

*> ENDDO
*

*> WRITE (51,*) OBJECT
*

*> WRITE (52,*) THETA
*

*> WRITE (53,*) SETHET
*

*> WRITE (54,*) OMEGA(BLOCK)
*

*> WRITE (55,*) SEOMEG(BLOCK)
*

*> WRITE (56,*) SIGMA(BLOCK)
*

*> WRITE (57,*) SESIGM(BLOCK)
*

*> WRITE (58,*) IERE,IERC
*

*> ENDIF
*

*>
*

*>
*

*> On Tue, Oct 14, 2008 at 8:26 PM, Nick Holford
*

*> <n.holford *

*>
*

*> Jun,
*

*>
*

*> In addition to Elodie's clear explanation of the basic process of
*

*> using $SIM and $EST in the same control stream you may wish to
*

*> know that you don't have to use the same parameters for simulation
*

*> as those used for estimation. The ICALL variable has a value of 4
*

*> when NONMEM is simulating so you can do this:
*

*>
*

*> $THETA
*

*> 1 FIX ; sim_CL
*

*> 10 ; est_CL
*

*>
*

*> $PK
*

*>
*

*> IF (ICALL.EQ.4) THEN ; for simulation
*

*> CL=THETA(1)
*

*> ELSE ; for estimation
*

*> CL=THETA(2)
*

*> ENDIF
*

*>
*

*> Nick
*

*>
*

*>
*

*> Elodie Plan wrote:
*

*>
*

*>
*

*> Dear Jun,
*

*>
*

*> When $SIM and $EST on the same model file, the simulation will
*

*> be run first, based on initial values, and then, afterwards,
*

*> the estimation of the simulated data will be done, that's it,
*

*> no further re-simulation based on the estimated parameters
*

*> will follow.
*

*>
*

*> So what you should do is first to analyze your observed data
*

*> in an estimation model file, and then report your estimated
*

*> parameters as initial values to run your simulation-estimation
*

*> study; this allows you to compare estimates of real data to
*

*> estimates of simulated data, so to check simulation properties
*

*> of your model.
*

*>
*

*> I hope this helps,
*

*>
*

*> Elodie
*

*>
*

*> / Elodie Plan, PharmD, MSc, ///
*

*>
*

*> / PhDstudent// ///
*

*>
*

*> /*****************************************///
*

*>
*

*> /Div. of Pharmacokinetics and Drug Therapy,
*

*> Department of Pharmaceutical Biosciences,
*

*> Faculty of Pharmacy, //Uppsala University/
*

*>
*

*> / PO - Box 591 - 751 24 //Uppsala - SWEDEN/
*

*>
*

*> /Office +46 18 4714385 - Fax +46 18 4714003
*

*> ------------------------------------------------------------///
*

*>
*

*> *From:* owner-nmusers *

*> <mailto:owner-nmusers *

*> [mailto:owner-nmusers *

*> <mailto:owner-nmusers *

*> *Sent:* Wednesday, October 15, 2008 1:01 AM
*

*> *To:* nmusers *

*>
*

*> *Subject:* [NMusers] How $simulation work with $estimation
*

*>
*

*> Dear NMusers,
*

*>
*

*> I wonder how does $Simulation work with $Estimation in NONMEM
*

*> exactly?
*

*> The manual says, NONMEM will simulate DVs and replace the
*

*> original DVs
*

*> based on the parameter initial values. But the initial values
*

*> are not final
*

*> estimates. Does the simulation based on initial values make
*

*> sense? Do
*

*> $Estimation and $Simulation run alternatively? The $Simulation
*

*> generates a
*

*> set of data based on which the parameters are estimated? And
*

*> then the
*

*> predictions are made on the estimated parameters? A little
*

*> confused.
*

*>
*

*> Appreciate any comment.
*

*>
*

*> Jun
*

*>
*

*>
*

*> --
*

*> Nick Holford, Dept Pharmacology & Clinical Pharmacology
*

*> University of Auckland, 85 Park Rd, Private Bag 92019, Auckland,
*

*> New Zealand
*

*> n.holford *

*> tel:+64(9)923-6730 fax:+64(9)373-7090
*

*> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
*

*>
*

*>
*

--

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand

n.holford

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Thu Oct 16 2008 - 16:51:49 EDT

Date: Fri, 17 Oct 2008 09:51:49 +1300

Jun,

VPCs are created using simulation alone (i.e. without estimation)

(Karlsson & Holford 2008)

If you do simulation followed by estimation this is called parametric

bootstrapping (nmgosim in WFN). An alternative bootstrap method is

called the non-parametric bootstrap which samples observations from the

original data set (nmbs in WFN).

The non-parametric bootstrap is probably more robust than the parametric

bootstrap if you have covariates in your model because it is tricky to

simulate correlated covariate distributions.

The distribution of the THETA, OMEGA, SIGMA parameter estimates obtained

from fittting many bootstrap replications can be used to generate

confidence intervals on parameters and explore other properties of the

parameter uncertainty.

The bootstrap confidence intervals are more realistic than those

computed from asymptotic standard errors because they dont assume

normality of the uncertainty. The mean of the bootstrap distribution is

probably a more robust estimate of the true population mean than the

point estimate obtained from the original data.

WFN gives you all the parameter estimates from parametric or

non-parametric bootstraps in a tab delimited file that can be easily

read into Excel or other packages to examine the results and compute

statistics from the distributions.

If you want help with getting other methods to extract parameters from

NONMEM you need to learn to give useful information about what you did,

what went wrong, what the EXACT error messages or unexpected results

were. It may be old stuff but most nmusers are not mind readers :-)

Karlsson MO, Holford NHG. A Tutorial on Visual Predictive Checks. PAGE

17 (2008) Abstr 1434 [wwwpage-meetingorg/?abstract=1434]. 2008.

Jun Shen wrote:

--

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand

n.holford

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Thu Oct 16 2008 - 16:51:49 EDT