From: Nick Holford <*n.holford*>

Date: Fri, 10 Jul 2009 20:21:52 +0200

Jian,

Thanks for this suggestion. However, as pointed out by Marc Gastonguay

and Juan-Jo Perez Ruixo there are likely to be more serious problems

when assuming that the uncertainty is a multivariate normal distribution.

I personally have little faith in using standard errors, because to be

useful they require the assumption of a normal distribution. Empirical

studies of non-linear model parameters show that they are typically

asymmetrical and non-normal e.g. see Holford & Peace 1992 using

likelihood profiling and Matthews et al. 2004 using bootstraps.

It takes substantial effort to evaluate if the uncertainty distributions

might be close to being normal. The same effort can produce more robust

posterior distributions which can then be used directly.

Best wishes,

Nick

Holford NHG, Peace KE. Results and validation of a population

pharmacodynamic model for cognitive effects in Alzheimer patients

treated with tacrine. Proc Natl Acad Sci U S A. 1992;89(23):11471-5.

Matthews I, Kirkpatrick C, Holford NHG. Quantitative justification for

target concentration intervention - Parameter variability and predictive

performance using population pharmacokinetic models for aminoglycosides.

British Journal of Clinical Pharmacology. 2004;58(1):8-19.

Jian Xu wrote:

*> Hi, Nick,
*

*>
*

*> This may work. In order to account for uncertainty to the random
*

*> effect parameters, we can fix OMEGA and SIGMA to 1, and reparametrize
*

*> them as THETA.
*

*>
*

*> Your example modified:
*

*>
*

*> $SIM (20090709) ONLYSIM SUBPROBLEMS=100
*

*> ; estimates of THETA and OMEGA from previous run
*

*> $THETA
*

*> 1 ; POP_CL theta1
*

*> 10 ; POP_V theta2
*

*> 0.2 ; Coefficient on CL ETA theta3
*

*> 0.5 ; Coefficient on V ETA theta4
*

*> 0.3 ; PROP_RV theta5
*

*> $OMEGA
*

*> 1 ;FIX ; PPV_CL eta1
*

*> 1 ;FIX; PPV_V eta2
*

*> $SIGMA
*

*> 1 ;FIX; PROP RV eps1
*

*> ;variance-covariance matrix of the THETA estimates from previous run
*

*> $OMEGA BLOCK(5)
*

*> 0.2 ; UNC_POP_CL eta3
*

*> 0.1 3 ; UNC_POP_V eta 4
*

*> 0.08 0.2 0.7 ; UNC Coef_CL eta5
*

*> 0.01 0.02 0.02 0.03 ; UNC Coef_V eta6
*

*> 0.02 0.04 0.005 0.01 0.07 ;UNC PROP RV eta7
*

*> $PK
*

*> ; get CL and V uncertainties
*

*> IF (NEWIND.EQ.0) THEN ; do this just once per subproblem
*

*> UNCCL=THETA(1)+ETA(3)
*

*> UNCV=THETA(2)+ETA(4)
*

*> UNCLE=THETA(3)+ETA(5)
*

*> UNVE=THETA(4)+ ETA(6)
*

*> UNRV=THETA(5)+ETA(7)
*

*> ENDIF
*

*> CL=UNCCL*EXP(UNCLE*ETA(1)) ; with uncertainty for CL and OMEGA
*

*> V =UNCV*EXP(UNVE*ETA(2)) ; with uncertainty for V and OMEGA
*

*> PROP=UNRV
*

*> $ERROR
*

*> IPRE=F
*

*> Y=F*(1+PROP*EPS(1))
*

*>
*

*> Cheers,
*

*>
*

*> Jian
*

*>
*

*> ------------------------------------------------------------------------
*

*> *From:* Nick Holford <n.holford *

*> *To:* nmusers <nmusers *

*> *Sent:* Friday, July 10, 2009 3:34:50 AM
*

*> *Subject:* Re: AW: [NMusers] Simulations with/without residual error
*

*>
*

*> Andreas,
*

*>
*

*> Thanks for your comments. I am sorry I did not explain everything that
*

*> NONMEM was doing in this simple example. It does not shave my face (!)
*

*> but it does recognize the covariance between the uncertainty estimates
*

*> UNC_POP_CL and UNC_POP_V when the values of ETA(3) and ETA(4) are
*

*> sampled -- so the covariance of 0.1 in the OMEGA block defining
*

*> parameter uncertainty is not ignored. NONMEM is doing exactly the same
*

*> thing you describe in R -- it is sampling from multivariate normal
*

*> distributions.
*

*>
*

*> The code I gave was just a simple example showing the idea. Of course,
*

*> you can include the full variance-covariance matrix of the estimate
*

*> from a previous run including the uncertainties in THETA, OMEGA and
*

*> SIGMA (and their correlations).
*

*>
*

*> You can also apply the uncertainties to the random effect parameters,
*

*> OMEGA and SIGMA, but it may not be so simple as for THETA. I
*

*> personally have no experience of this. I am sure there are others who
*

*> have done it who may wish to comment.
*

*>
*

*> Best wishes,
*

*>
*

*> Nick
*

*>
*

*>
*

*> andreas.krause *

*> >
*

*> > Nick,
*

*> >
*

*> > your example shows there is almost nothing you can not do with
*

*> nonmem (maybe except shaving your face).
*

*> > On the other hand, even in the simple two parameter example you have
*

*> off-diagonal covariance terms.
*

*> > In your example code the value of 0.1 in the $OMEGA block seems
*

*> ignored (covariance pop CL and pop Vol).
*

*> >
*

*> > There would typically also be covariances between the pop parameters
*

*> and the OMEGA and SIGMA blocks.
*

*> > The latter are often small compared to other variance terms but the
*

*> proper way would be to draw from the full variance-covariance matrix.
*

*> > For now it seems best to draw multivariate Normals with full
*

*> covariance matrices in some other environment like R and write the
*

*> generated population parameters to nonmem control streams.
*

*> > Unless you find a way again of doing it all in nonmem.
*

*> >
*

*> > Best regards,
*

*> >
*

*> > Andreas
*

*> >
*

*> > PS. Specifying the variance-covariance matrix to use it with $SIM
*

*> might actually be a good candidate for the to-do list for nonmem VIII.
*

*> >
*

*> >
*

*> > -----
*

*> >
*

*> > Andreas Krause, PhD
*

*> > Lead Scientist Modeling and Simulation
*

*> >
*

*> > Actelion Pharmaceuticals Ltd / Gewerbestrasse 16 / CH-4123 Allschwil
*

*> / Switzerland
*

*> > andreas.krause *

*> www.actelion.com <http://www.actelion.com>
*

*> >
*

*> >
*

*> >
*

*> > *Nick Holford <n.holford *

*> <mailto:n.holford *

*> > Sent by: owner-nmusers *

*> <mailto:owner-nmusers *

*> >
*

*> > 07/09/2009 11:51 AM
*

*> >
*

*> >
*

*> > To
*

*> > nmusers <nmusers *

*> > cc
*

*> >
*

*> > Subject
*

*> > Re: AW: [NMusers] Simulations with/without residual error
*

*> >
*

*> >
*

*> >
*

*> >
*

*> >
*

*> >
*

*> >
*

*> >
*

*> >
*

*> > Andreas K,
*

*> >
*

*> > It is not strictly true to say you cannot specify the parameter
*

*> > uncertainties from a previous run to be included in a simulation.
*

*> >
*

*> > If you take the variance-covariance matrix of the estimate from a
*

*> > previous run ('the uncertainty matrix') you can add it as an additional
*

*> > OMEGA matrix and use it to obtain parameter samples with uncertainty.
*

*> >
*

*> > e.g. with a very simple example with just two parameters. This will
*

*> > simulate 100 data sets and uncertainty to the THETA values for CL and V.
*

*> >
*

*> > $SIM (20090709) ONLYSIM SUBPROBLEMS=100
*

*> > ; estimates of THETA and OMEGA from previous run
*

*> > $THETA
*

*> > 1 ; POP_CL theta1
*

*> > 10 ; POP_V theta2
*

*> > $OMEGA
*

*> > 0.5 ; PPV_CL eta1
*

*> > 0.5 ; PPV_V eta2
*

*> > ;variance-covariance matrix of the THETA estimates from previous run
*

*> > $OMEGA BLOCK(2)
*

*> > 0.2 ; UNC_POP_CL eta3
*

*> > 0.1 3 ; UNC_POP_V eta 4
*

*> >
*

*> > $PK
*

*> > ; get CL and V uncertainties
*

*> > IF (NEWIND.EQ.0) THEN ; do this just once per subproblem
*

*> > UNCCL=THETA(1)+ETA(3)
*

*> > UNCV=THETA(2)+ETA(4)
*

*> > ENDIF
*

*> > CL=UNCCL*EXP(ETA(1)) ; with uncertainty for CL
*

*> > V =UNCV*EXP(ETA(2)) ; with uncertainty for V
*

*> > ...
*

*> >
*

*> > Nick
*

*> >
*

*> >
*

*> >
*

*> > andreas.krause *

*> > > Andreas,
*

*> > >
*

*> > > I guess you are hinting at the difference between simulation of a
*

*> large
*

*> > > population and simulation of a study.
*

*> > >
*

*> > > The latter incorporates the added uncertainty of the parameter
*

*> estimates,
*

*> > > as you point out.
*

*> > > You would simulate the population parameters with their
*

*> uncertainties first
*

*> > > (from the "big covariance matrix" in nonmem) and then simulate the
*

*> study
*

*> > > with these sampled population parameters (both steps many times).
*

*> > > Nonmem can only do the latter directly since you cannot specify the
*

*> > > parameter uncertainties from a previous run to be included in the
*

*> > > simulation.
*

*> > > It is fairly straightforward though since the matrix reflects a
*

*> > > multivariate Normal distribution.
*

*> > >
*

*> > > Andreas
*

*> > >
*

*> > > -----
*

*> > >
*

*> > > Andreas Krause, PhD
*

*> > > Lead Scientist Modeling and Simulation
*

*> > >
*

*> > > Actelion Pharmaceuticals Ltd / Gewerbestrasse 16 / CH-4123 Allschwil /
*

*> > > Switzerland
*

*> > > andreas.krause *

*> www.actelion.com <http://www.actelion.com>
*

*> > >
*

*> > >
*

*> > >
*

*> > > -----owner-nmusers *

*> <mailto:-----owner-nmusers *

*> > >
*

*> > >
*

*> > > To: <nmusers *

*> > > From: "andreas lindauer" <lindauer *

*> <mailto:lindauer *

*> > > Sent by: owner-nmusers *

*> <mailto:owner-nmusers *

*> > > Date: 2009-07-09 09:42
*

*> > > Subject: AW: [NMusers] Simulations with/without residual error
*

*> > >
*

*> > > Nick,
*

*> > > Thank you very much for your comments.
*

*> > > Indeed for VPC et al. i always simulate with residual error.
*

*> > > I understand that when one wants to simulate the 'true' value residual
*

*> > > error
*

*> > > is not needed. But what if one wants to simulate 'real' values
*

*> which will
*

*> > > be
*

*> > > observed in a future study. For example, you have a PK/PD model for an
*

*> > > anti-hypertensive drug and want to predict how many subjects will
*

*> attain a
*

*> > > blood pressure below a pre-defined value. Wouldn't a simulation
*

*> without
*

*> > > residual error result in an overoptimistic prediction because in
*

*> reality
*

*> > > blood pressure is measured with error?
*

*> > > On the other hand, the estimated residual error does not only reflect
*

*> > > measurement error but also model misspecification etc.. So, might
*

*> it be an
*

*> > > option to simulate not with the estimated residual error but
*

*> rather with a
*

*> > > residual error set to the imprecision of the measurement method?
*

*> > > Best regards, Andreas.
*

*> > >
*

*> > >
*

*> > > .
*

*> > >
*

*> > > -----Ursprüngliche Nachricht-----
*

*> > > Von: owner-nmusers *

*> <mailto:owner-nmusers *

*> [mailto:owner-nmusers *

*> <mailto:owner-nmusers *

*> > > Auftrag von Nick Holford
*

*> > > Gesendet: Mittwoch, 8. Juli 2009 15:39
*

*> > > An: nmusers
*

*> > > Betreff: Re: [NMusers] Simulations with/without residual error
*

*> > >
*

*> > > Andreas,
*

*> > >
*

*> > > My suggestion:
*

*> > >
*

*> > > If you want to compare your simulations with actual observations then
*

*> > > you should include residual error in the simulation. The observations
*

*> > > will include noise as well as the 'true' value so in order to compare
*

*> > > observations with simulated observations you need the residual error.
*

*> > >
*

*> > > If you want to use the simulation to describe the 'true' value
*

*> then dont
*

*> > > include the residual error. Residual error is assumed to have a
*

*> mean of
*

*> > > zero around the 'true' value so there is no point in adding this
*

*> kind of
*

*> > > noise if you are trying to predict the 'true' value.
*

*> > >
*

*> > > Your examples suggest to me that you are trying to predict the 'true'
*

*> > > value -- not trying to match simulations directly with measured
*

*> values.
*

*> > > If my guess is correct then you dont need to include residual error.
*

*> > >
*

*> > > However, if you are using simulations for some kind of predictive
*

*> check
*

*> > > (visual, numerical, statistical) that will be compared to distribution
*

*> > > statistics of the observations then you should include residual error.
*

*> > >
*

*> > > Nick
*

*> > >
*

*> > > andreas lindauer wrote:
*

*> > > >> Dear NMUSERS,
*

*> > >>
*

*> > >>
*

*> > >>
*

*> > >> The recent discussion about simulation with a nonparametric method
*

*> > >> brought a general question concerning monte-carlo simulations into my
*

*> > >> mind. When should simulations be performed with residual error and
*

*> > >> when not. I am especially interested in comments regarding the
*

*> > >> following scenarios when the result of the simulation should be
*

*> > >> reported as mean or median and 90% prediction interval:
*

*> > >>
*

*> > >> 1. Simulated response at a particular time point (eg. Trough values)
*

*> > >>
*

*> > >> 2. Simulated response at a particular time point (x) relative to
*

*> > >> baseline response (IPRED(t=x)/IPRED(t=0) vs. DV(t=x)/DV(t=0) )
*

*> > >>
*

*> > >> 3. Simulated time of maximal response (eg. Tmax)
*

*> > >>
*

*> > >>
*

*> > >>
*

*> > >>
*

*> > >>
*

*> > >> Thanks and best regards, Andreas.
*

*> > >>
*

*> > >>
*

*> > >>
*

*> > >>
*

*> > >>
*

*> > >> ____________________________
*

*> > >>
*

*> > >>
*

*> > >>
*

*> > >> Andreas Lindauer
*

*> > >>
*

*> > >>
*

*> > >>
*

*> > >> Department of Clinical Pharmacy
*

*> > >>
*

*> > >> Institute of Pharmacy
*

*> > >>
*

*> > >> University of Bonn
*

*> > >>
*

*> > >> An der Immenburg 4
*

*> > >>
*

*> > >> D-53121 Bonn
*

*> > >>
*

*> > >>
*

*> > >>
*

*> > >> phone: + 49 228 73 5781
*

*> > >>
*

*> > >> fax: + 49 228 73 9757
*

*> > >>
*

*> > >>
*

*> > >>
*

*> > >> >
*

*> > > --
*

*> > > Nick Holford, Professor Clinical Pharmacology
*

*> > > Dept Pharmacology & Clinical Pharmacology
*

*> > > University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New
*

*> > > Zealand
*

*> > > n.holford *

*> tel:+64(9)923-6730 fax:+64(9)373-7090
*

*> > > mobile: +33 64 271-6369 (Apr 6-Jul 20 2009)
*

*> > > http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
*

*> > >
*

*> > >
*

*> > >
*

*> > > The information of this email and in any file transmitted with it
*

*> is strictly confidential and may be legally privileged.
*

*> > > It is intended solely for the addressee. If you are not the
*

*> intended recipient, any copying, distribution or any other use of this
*

*> email is prohibited and may be unlawful. In such case, you should
*

*> please notify the sender immediately and destroy this email.
*

*> > > The content of this email is not legally binding unless confirmed
*

*> by letter.
*

*> > > Any views expressed in this message are those of the individual
*

*> sender, except where the message states otherwise and the sender is
*

*> authorised to state them to be the views of the sender's company. For
*

*> further information about Actelion please see our website at
*

*> http://www.actelion.com
*

*> > >
*

*> > >
*

*> > -- Nick Holford, Professor Clinical Pharmacology
*

*> > Dept Pharmacology & Clinical Pharmacology
*

*> > University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New
*

*> Zealand
*

*> > n.holford *

*> tel:+64(9)923-6730 fax:+64(9)373-7090
*

*> > mobile: +33 64 271-6369 (Apr 6-Jul 20 2009)
*

*> > http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
*

*> >
*

*> >
*

*> >
*

*> > The information of this email and in any file transmitted with it is
*

*> strictly confidential and may be legally privileged.
*

*> > It is intended solely for the addressee. If you are not the intended
*

*> recipient, any copying, distribution or any other use of this email is
*

*> prohibited and may be unlawful. In such case, you should please notify
*

*> the sender immediately and destroy this email.
*

*> > The content of this email is not legally binding unless confirmed by
*

*> letter.
*

*> > Any views expressed in this message are those of the individual
*

*> sender, except where the message states otherwise and the sender is
*

*> authorised to state them to be the views of the sender's company. For
*

*> further information about Actelion please see our website at
*

*> http://www.actelion.com
*

*>
*

*> -- Nick Holford, Professor Clinical Pharmacology
*

*> Dept Pharmacology & Clinical Pharmacology
*

*> University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New
*

*> Zealand
*

*> n.holford *

*> tel:+64(9)923-6730 fax:+64(9)373-7090
*

*> mobile: +33 64 271-6369 (Apr 6-Jul 20 2009)
*

*> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
*

*>
*

*>
*

*>
*

*>
*

--

Nick Holford, Professor Clinical Pharmacology

Dept Pharmacology & Clinical Pharmacology

University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand

n.holford

mobile: +33 64 271-6369 (Apr 6-Jul 20 2009)

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Fri Jul 10 2009 - 14:21:52 EDT

Date: Fri, 10 Jul 2009 20:21:52 +0200

Jian,

Thanks for this suggestion. However, as pointed out by Marc Gastonguay

and Juan-Jo Perez Ruixo there are likely to be more serious problems

when assuming that the uncertainty is a multivariate normal distribution.

I personally have little faith in using standard errors, because to be

useful they require the assumption of a normal distribution. Empirical

studies of non-linear model parameters show that they are typically

asymmetrical and non-normal e.g. see Holford & Peace 1992 using

likelihood profiling and Matthews et al. 2004 using bootstraps.

It takes substantial effort to evaluate if the uncertainty distributions

might be close to being normal. The same effort can produce more robust

posterior distributions which can then be used directly.

Best wishes,

Nick

Holford NHG, Peace KE. Results and validation of a population

pharmacodynamic model for cognitive effects in Alzheimer patients

treated with tacrine. Proc Natl Acad Sci U S A. 1992;89(23):11471-5.

Matthews I, Kirkpatrick C, Holford NHG. Quantitative justification for

target concentration intervention - Parameter variability and predictive

performance using population pharmacokinetic models for aminoglycosides.

British Journal of Clinical Pharmacology. 2004;58(1):8-19.

Jian Xu wrote:

--

Nick Holford, Professor Clinical Pharmacology

Dept Pharmacology & Clinical Pharmacology

University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand

n.holford

mobile: +33 64 271-6369 (Apr 6-Jul 20 2009)

http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford

Received on Fri Jul 10 2009 - 14:21:52 EDT