NONMEM Users Network Archive

Hosted by Cognigen

Re: Funny behaviour with MCETA>1 and parallel computation

From: Paolo Denti <paolo.denti>
Date: Wed, 30 Jul 2014 09:53:50 +0200

Hi Bob,
thanks for the prompt response and suggestions.

I have tried to implement with RANMETHOD=P and increasing MCETA to 1000,=
 
but unfortunately without success.
Even using MCETA=1000, the result is the same: when I use the parallel
computation feature, it performs worse than using MCETA=0. With one
processor, things work as expected.

Maybe I did not explain the situation well. The issue is not that the
optimisation takes a slightly different path and it reaches a different
minimum, that would not surprise me, as I know different rounding and
other random factor can influence that.
The problem here is that when using parallel computation even on the
first iteration (using MAXEVAL=0) MCETA>1 gives a worse OFV that
MCETA=0. This still does not make any sense to me, irrespectively of
random number generators, numerical approximation,etc. My understanding
is that, in each individual, NONMEM will try 0 and other initial
estimates to find the optimal ETAs, and then it will choose the solution
giving the lowest individual OFV. Even if this is done with different
seeds and on different CPUs, they will all try 0, so whatever MCETA=0
gives out, it should be the upper bound for the OFV for that individual.
Then all these individual OFVs are summed together to find the total
OFV. Since NONMEM is trying 0 in each subject - plus other random values
that may vary - it should at least be able to use those results. In each
individual it can only do better by trying extra values, and if all the
individual OFVs are lower or at worst the same as the ones provided by
MCETA=0, then the total can only be better.

Am I missing something? Is NONMEM maybe minimising only some other
individual likelihood in the MAP step, and that does not coincide with
the individual OFV?

To better understand I have looked at the values of individual OFVs in
the two runs (MCETA=1000 and MCETA=0, both with parallel computaion) an=
d
all the tables are exactly the same cell by cell (to the 5th digit or
so), except for the records of that outlying individual. In spite of
trying 1000 initial estimates (including 0) MCETA=1000 gets the
individual ETA for that subject wrong, and gives a worse iOFV than
MCETA=0. And it's not a matter of numerical noise, the individual OFV is=
 
100 points worse and the individual parameters are very different..

MCETA=0 is a sub-case of MECTA>1, so MCETA>1 should not do any worse. I=
 
have tried and re-tried, and it is unlikely that the estimator was
unlucky all the times with that subject, even trying 1000 initial
estimates..

I don't know how to explain this. But maybe I am misunderstanding how
this works?
Any explanation?

Thank you,
Paolo








On 2014/07/29 22:45, Bauer, Robert wrote:
> Paolo:
> MCETA=5 is a small number of random samples of etas to test, and when=
 
> you parallelize, your random seed pattern changes, unless, you include
> $EST RANMETHOD=P. Yes, sometimes a worse result follows if a lower OBJ=
 
> is chosen, yet those etas that gave a lower OBJ offered a worse path
> than when eta=0. I suggest the following:
>
> $EST METHOD=1 INTERACTION MAXEVAL=0 MCETA=1000 RANMETHOD=P ... ; =
spend
> a lot of random samples doing a thorough search with MCETA=1000, but
> it takes time, so do it just for the first iteration.
> $EST METHOD=1 INTERACTION MAXEVAL=9999 MCETA=5 RANMETHOD=P ... ;=
 
> remaining iterations use MCETA more lightly.
>
>
>
> Robert J. Bauer, Ph.D.
> Vice President, Pharmacometrics, R&D
> ICON Development Solutions
> 7740 Milestone Parkway
> Suite 150
> Hanover, MD 21076
> Tel: (215) 616-6428
> Mob: (925) 286-0769
> Email: Robert.Bauer
> Web: www.iconplc.com <http://www.iconplc.com>
>
> -----Original Message-----
> From: owner-nmusers
> <mailto:owner-nmusers
> [mailto:owner-nmusers
> Sent: Tuesday, July 29, 2014 3:54 PM
> To: nmusers
> Subject: [NMusers] Funny behaviour with MCETA>1 and parallel computation
>
> Dear all,
> I am working with FOCE-INTERACTION and a PK model with some issues in
> optimising the individual ETAs.
> A couple of subjects in the dataset are very fast absorbers and
> sometimes NONMEM fails to find the optimal value for their individual
> ETAs. This results in the OFV randomly jumping up a lot (~100 points)
> when I restart the model from the final estimates of a previous run.
> The model may sometimes find its way back to the same minimum, but not
> always.
>
> So I decided to use the new feature MCETA, which is supposed to
> address these kinds of issues by trying several randomly generated
> sets of initial ETAs, increasing the chance of finding the best value
> of individual ETA for each subject.
>
> I first tested the new feature on a single CPU and everything worked as
> expected: using MCETA=0 (the default) the OFV jumps up after
> restarting with the final values, while with MCETA=5 the problem is
> solved and the OFV is stable.
>
> I then ran the same test using the parallel computation feature to
> speed it up, but I noticed that here the OFV was jumping up anyway.
> Not only, using MCETA=5 was actually giving a WORSE OFV than the
> default value of MCETA=0, which is indeed giving the same result
> irrespectively of whether I use the parallel computation or not.
> I have retried several times to make sure I hadn't made any mistakes
> and when I use MCETA>1 and the parallel computation feature, it
> sometimes perform worse than the default MCETA=0. I tried other values=
 
> of MCETA too, but same story.
> When I say that the OFV goes up, I am referring to the first
> iteration, so I have replicated these results using MAXEVAL=0.
>
> Any suggestion about what may be going on? My understanding is that
> using MCETA>1 CANNOT be any worse than the default behaviour (MECTA=0)=
 
> of just using 0 as initial value. The guide says NONMEM is supposed to
> try both 0 AND some other values as initial estimates, and choose in
> every subject the option that minimises the OFV. So in the worst case
> scenario NONMEM will just use 0 and I will have wasted some electrons.
> Am I misunderstanding something?
>
> Sorry for the lengthy email and thanks for any input on this.
> Paolo
>
>
> --
> ------------------------------------------------
> Paolo Denti, PhD
> Pharmacometrics Group
> Division of Clinical Pharmacology
> Department of Medicine
> University of Cape Town
>
> K45 Old Main Building
> Groote Schuur Hospital
> Observatory, Cape Town
> 7925 South Africa
> phone: +27 21 404 7719
> fax: +27 21 448 1989
> email: paolo.denti
> ------------------------------------------------
>
> ________________________________
> UNIVERSITY OF CAPE TOWN
>
> This e-mail is subject to the UCT ICT policies and e-mail disclaimer
> published on our website at
> http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable
> from +27 21 650 9111. This e-mail is intended only for the person(s)
> to whom it is addressed. If the e-mail has reached you in error,
> please notify the author. If you are not the intended recipient of the
> e-mail you may not use, disclose, copy, redirect or print the content.
> If this e-mail is not related to the business of UCT it is sent by the
> sender in the sender's individual capacity.
>
>
> ICON plc made the following annotations.
> -------------------------------------------------------------------------=
-----
>
> This e-mail transmission may contain confidential or legally
> privileged information that is intended only for the individual or
> entity named in the e-mail address. If you are not the intended
> recipient, you are hereby notified that any disclosure, copying,
> distribution, or reliance upon the contents of this e-mail is strictly
> prohibited. If you have received this e-mail transmission in error,
> please reply to the sender, so that ICON plc can arrange for proper
> delivery, and then please delete the message.
>
> Thank You,
>
> ICON plc
> South County Business Park
> Leopardstown
> Dublin 18
> Ireland
> Registered number: 145835
>

--
------------------------------------------------
Paolo Denti, PhD
Pharmacometrics Group
Division of Clinical Pharmacology
Department of Medicine
University of Cape Town

K45 Old Main Building
Groote Schuur Hospital
Observatory, Cape Town
7925 South Africa
phone: +27 21 404 7719
fax: +27 21 448 1989
email: paolo.denti
------------------------------------------------




Received on Wed Jul 30 2014 - 03:53:50 EDT

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to: nmusers-request@iconplc.com.

Once subscribed, you may contribute to the discussion by emailing: nmusers@globomaxnm.com.