NONMEM Users Network Archive

Hosted by Cognigen

Re: Validation Strategy for NONMEM

From: Jun Shen <jun.shen.ut>
Date: Tue, 21 Oct 2008 17:36:23 -0500

Just to add a bit on the point Nick has brought up. Even if NONMEM can be
validated somehow (the only way I see is someone develops a validation
suite) there is still no guarantee to obtain the "right" results. Because
both algorithm (softwares) and human inputs (e.g. model construction,
initial values) contribute to the NONMEM results. A validation is just to
demonstrate the software is doing what it is supposed to do.

Jun
On Tue, Oct 21, 2008 at 4:11 PM, Nick Holford <n.holford

> Tony,
>
> To expand a bit on your useful validation definition.
>
> Making sure computational results are reproducible does not mean they are
> correct. I believe that most NONMEM analyses cannot be proven to be correct
> therefore the validation efforts only assure reproducibility. Whether this
> makes the validation useful or useless is debatable.
>
> Nick
>
>
>
>
> A.J. Rossini wrote:
>
>> There is useless validation, and then there is useful validation. The
>> latter is about making sure your computational results are
>> reproducible, the former is about making sure that your documentation
>> can be photocopied. It's sort of the same thing, if you aren't a
>> modeler.
>>
>> Unfortunately, most on this list tend to be modelers.
>>
>> On Tue, Oct 21, 2008 at 7:37 PM, <Michael.J.Fossler
>>
>>
>>> After reading this, it is no wonder scientific productivity is at an
>>> all-time low. Imagine if Marie Curie had to qualify her radium-purifying
>>> equipment, or if Alexander Fleming had to validate his petri dishes
>>> before
>>> culturing Penecillium. One day scientists are going to push back against
>>> these IT people, who just make busy work for everyone.
>>>
>>>
>>>
>>> Joachim.Grevel
>>> Sent by: owner-nmusers
>>>
>>> 21-Oct-2008 03:18
>>>
>>>
>>> To
>>> nmusers
>>> cc
>>> Subject
>>> Re: [NMusers] Validation Strategy for NONMEM
>>>
>>>
>>>
>>>
>>>
>>> Dear Mark,
>>>
>>> I am engaged in this question since the beginning of this year (not
>>> finished
>>> yet), and I am happy to share some basics of my experiences:
>>>
>>> 1. It is important to specify where the data for NONMEM analysis
>>> come
>>> from. If they come from a GCP source and are already QA-ed when they
>>> arrive
>>> at the doorstep of NONMEM then your system will only subject to GCP
>>> regulation. Otherwise, you also have to comply with GLP.
>>>
>>> 2. There is no way around what is called here a 'Validation Plan'
>>> and
>>> a 'Risk Analysis'. These documents will trigger a slate of other
>>> documents
>>> (in our case here about 15) which describe Installation, Installation
>>> Validation, Qualification of Users, Modeling Strategy, Review Processes,
>>> System Life Cycle Management etc.
>>>
>>> 3. We found it useful to differentiate between 'Exploratory Work'
>>> and
>>> 'Submission Work'.
>>>
>>> 4. Before you worry about passing inspection by the FDA, you need
>>> to
>>> worry about passing inspection of your own company QA officers.
>>>
>>> 5. Just installing NONMEM with NMQual does not render you new
>>> system
>>> 'validated' or 'qualified'. Here my apologies to the excellent folks at
>>> Metrum, but for various reasons, we ended up not using NMQual.
>>>
>>> 6. You have to know what you are trying to build before you
>>> concern
>>> yourself about QA processes. A number of separate installations on PCs
>>> linked to a file server is a different animal from a server-based
>>> installation with a grid engine.
>>>
>>> 7. It all takes more time than you think: make generous budgets
>>> and
>>> time lines.
>>>
>>> I hope I helped more than I confused,
>>>
>>> Joachim
>>>
>>> __________________________________________
>>> Joachim GREVEL, Ph.D.
>>> MERCK SERONO International S.A.
>>> Exploratory Medicine
>>> 1202 Geneva
>>> Tel: +41.22.414.4751
>>> Fax: +41.22.414.3059
>>> Email: joachim.grevel
>>>
>>>
>>>
>>> "Vilicich, Mark" <Mark.Vilicich
>>> Sent by: owner-nmusers
>>>
>>> 10/17/2008 10:08 PM
>>>
>>> To
>>> <nmusers
>>> cc
>>> Subject
>>> [NMusers] Validation Strategy for NONMEM
>>>
>>>
>>>
>>>
>>>
>>>
>>> Dear All,
>>>
>>> I am interested in perspectives on strategies for "validating" NONMEM.
>>> Also,
>>> experiences from or with the FDA since the FDA is: a key user, customer
>>> of
>>> analysis and auditor of NONMEM use in the industry. Without a large
>>> nonmem
>>> staff here, the challenge I see is in scaling the validation strategy to
>>> provide the most efficient environment for doing analysis that is
>>> defensible
>>> to both internal and external audits based on the associated GxP risk
>>> level.
>>>
>>> Below are the concepts I've cobbled together, though instead of my
>>> reinventing the wheel I appreciate anything you could share. Any and all
>>> gems of insight you can share whether it regard the big picture or some
>>> detailed specifics, IT centric or business process related. You may send
>>> them back to the listserver or me directly as you feel appropriate.
>>>
>>> Details:
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>> From searching the archives and other random bits of knowledge on NONMEM,
>>> part of the validation strategy is to recognize that NONMEM is not to be
>>> literally validated. NONMEM may be considered more of a development
>>> environment, optimized for developing specialized forms of complex
>>> analysis
>>> and modeling. As a development platform, an approach could be that NONMEM
>>> itself is qualified and each specific analysis is validated individually.
>>>
>>> To support establishing a defensible NONMEM environment, I've also read
>>> discussions on integrating common software development best practices
>>> such
>>> as version control of the "programming" of nonmem, NMQual and other
>>> commercial and custom tools for capturing all the metadata related to
>>> running a specific NONMEM job. These themes support defining the state of
>>> the NONMEM environment and ability to reproduce the outcomes.
>>>
>>> Also, reading in the archives about the differences in the numeric
>>> outcomes
>>> of NONMEM analyses based on the hardware platform, etc. are helpful to
>>> know
>>> up front and to consider in the validation strategy so it is not destined
>>> to
>>> failure if the target environment is multiplatform or otherwise complex.
>>>
>>> Gaps noticed/topics not discussed:
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>> Is there opportunity in looking at the risk based approached sanctioned
>>> by
>>> the FDA a few years ago that would make the total validation deliverable,
>>> including both the application and the model development process, more
>>> lean
>>> and targeted at the primary risk targets?
>>>
>>> Does this scientific software environment lend itself to use of modern
>>> agile
>>> software development methodologies that go far beyond basic iterative
>>> approaches. These methodologies are being used in software development
>>> for
>>> the regulated/GxP industry.
>>>
>>> I've seen the excellent presentation from 2004 that Joga Gobburu from the
>>> FDA gave, seems like there has been some progression of thought or
>>> actions
>>> on the proposals included there. Any references to follow-up information
>>> on
>>> it would be helpful?
>>>
>>> Regards ,
>>>
>>> Mark Vilicich
>>> Early Development
>>> mark.vilicich
>>>
>>> ________________________________
>>>
>>> This message and any attachment are confidential, may be privileged or
>>> otherwise protected from disclosure and are intended only for use by the
>>> addressee(s) named herein. If you are not the intended recipient, you
>>> must
>>> not copy this message or attachment or disclose the contents to any other
>>> person. If you have received this transmission in error, please notify
>>> the
>>> sender immediately and delete the message and any attachment from your
>>> system. Merck Serono does not accept liability for any omissions or
>>> errors
>>> in this message which may arise as a result of E-Mail-transmission or for
>>> damages resulting from any unauthorized changes of the content of this
>>> message and any attachment thereto. If verification is required, please
>>> request a hard-copy version. Merck Serono does not guarantee that this
>>> message is free of viruses and does not accept liability for any damages
>>> caused by any virus transmitted therewith.
>>>
>>>
>>
>>
>>
>>
>>
>
> --
> Nick Holford, Dept Pharmacology & Clinical Pharmacology
> University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New
> Zealand
> n.holford
> http://www.fmhs.auckland.ac.nz/sms/pharmacology/holford
>
>

Received on Tue Oct 21 2008 - 18:36:23 EDT

The NONMEM Users Network is maintained by ICON plc. Requests to subscribe to the network should be sent to: nmusers-request@iconplc.com.

Once subscribed, you may contribute to the discussion by emailing: nmusers@globomaxnm.com.