Webinar GXP System Validation

1
WEBINAR
How to Validate Computerized GxP Systems:

Is your documentation accurate, comprehensive and
accessible?
Presented by Chrysa Plagiannos

Senior Validation & Compliance Analyst
Montrium
© Montrium Inc. 2016
Your Presenter
2
Chrysa Plagiannos
Senior Validation & Compliance Analyst – Montrium
Educational Background:
• Degree in Chemical Engineering from McGill University
Career Overview:
• 13 years experience in pharmaceutical industry
• Extensive validation experience, including validation of manufacturing processes,
equipment, facilities, and computerized systems
• Experience in QA functions (approval of controlled documents, complaint
management)
• Experience in Engineering functions (equipment selection and purchasing,
investigation reporting)

About Montrium
3
• Montrium is a technology company focused on the delivery of enterprise content

management solutions and professional services for the Life Sciences industry.
• Montrium’s ECM platform, Montrium Connect, consists of solutions for Electronic Trial
Master Files, Regulatory Document Management & Submission Planning and Quality
Management.
• We offer also services in validation, quality assurance, auditing and strategy
surrounding technologies.
• Montrium was founded in 2005 and is headquartered in Montreal.
• Montrium works with pharmaceutical companies in North America, Europe and Asia.

Today’s Focus
4
1. Introduction 3. Executing Test Scripts

• What is a GxP System? • Setting up Prerequisites
• What is an Electronic Record? • Good Documentation Practices
• Why is Testing Important? • Creating Screen Captures
• Validation Terminology • Documenting Non-Conformances
2. Preparation for Testing 4. Traceability & Reporting
• Planning 5. Conclusion & Recommendations
• Defining the scope of testing
• Tips for writing test scripts
• Assembling a Team

5
1
Introduction
• What is a GxP System?
• What is an Electronic Record?
• Why is Testing Important?
• Validation Terminology

What is a GxP System?
6
• GxP: Set of compliance regulations including but not limited to, Good Laboratory
Practice (GLP), Good Manufacturing Practice (GMP), and Good Clinical Practice
(GCP).
Examples of computerized systems used for GxP activities :
- Systems used to generate electronic records (ex. training records,

complaint files, change requests, incident reports).
- Systems used to store/ manage data that is subject to regulations, such

as adverse event reports and raw data obtained during laboratory and
production activities

What is an Electronic Record?
7
Definition as per 21 CFR Part 11.3:

“Electronic record means any combination of text, graphics, data, audio, pictorial, or
other information representation in digital form that is created, modified, maintained,
archived, retrieved, or distributed by a computer system.”

Why is Testing Important?
8
Reason 1: Validation provides documented evidence that a system is fit for its
intended use.
• System requirements provide an objective standard to which the system is tested
during validation.
• System functionality is tested by executing test scripts designed to demonstrate
that the requirements were met.
• Requirements are based on:
– Regulatory requirements
– End-user/ business needs
– Risk assessment/ mitigation

9
Reason 2: Validation documents are auditable records.

• Inspectors are looking for:
– Documented evidence that validation activities were performed in accordance with approved
procedures
– Readily accessible records (can be produced in a timely manner)
– Proof of sufficient testing, based on intended use and potential risk
• Scope of validation should be tied to the risk the system poses to:
– Patient safety
– Product quality
– Data integrity

10
Reason 3: Electronic records managed by GxP systems can be auditable

records.
• These records will also be presented to Inspectors/ Auditors
• Examples of records
– Complaint files
– Certificates of Analysis
– Standard Operating Procedures (SOP)
• Failure to perform adequate testing could lead to:
– Inability to produce documents upon request
– Loss of critical data
– Compliance violations ex. missed regulatory filing deadline

11
• Real-life Example: Warning Letter

12
• Summary of Issues Identified in the Warning Letter:

– System not fully validated upon release
– Critical issues found with interim report used to release system:
• Inadequate training
• Incomplete documentation (SOPs and work instructions)
• Inaccurate data migration of legacy database
– Technical issues with system also identified:
• Inaccurate dates captured in system
• Follow-up information recorded as new event
– System deficiencies led to inaccurate assessments of adverse events and untimely submission
of alerts

Validation Terminology
13
• Qualification: Process of demonstrating whether an entity is capable of fulfilling specified

requirements.
• Validation: Process of establishing documented evidence which provides a high degree

of assurance that a specific process will consistently produce a product meeting its
predetermined specifications and quality attributes
• Note: Qualification is part of validation.
• Visit Montrium blog for more information

Validation Terminology
14

Types of Testing
15
• Verification that the appropriate system was delivered and that it is installed correctly (i.e. in a
manner consistent with requirements).
• Ex. IQ
• Verification that the system is capable of consistently operating in accordance with established
functional specifications.
• Ex. OQ
• Verification that the system consistently operates in accordance with established requirements in
its typical operating environment and is fit for its intended use.
• Ex. UAT, PQ
• Note: Terminology can vary from organization to organization. What’s important is to ensure that
the system is thoroughly tested regardless of the term used to describe the testing.

16
2 Preparation for Testing

•
•
•
Planning
Defining the scope of testing
Tips for writing test scripts
• Assembling a Team

Validation Planning
17
• Validation Plans should be formally documented

• Types of Planning Documents:
– Validation Master Plan
– Qualification/ Validation Plan(s)
Stage 2:
Application (s) Verified by Validation Plan(s) for
Application
Governed by
Validation Master
Plan
Infrastructure Stage 1:
• Physical Hardware Verified by Infrastructure
• Virtual Machines Qualification Plan
GxP System
Validation Planning
18
• Information to be included in the Planning Document(s):

– Scope of qualification/ validation activities
– Project roles and responsibilities
– Required deliverables
– Required procedural controls
• For simple systems with few requirements, validation deliverables can be combined (ex. combined
IQOQ Protocol).
• Important: Conduct validation activities in accordance with the Plan. If the sequence of activities
outlined in the plan is not followed, document any outstanding issues along with the justification
for proceeding.

Scope Definition
19
• Ensure that the scope is clearly defined.
• For software, it is important to understand the system’s core functionality to be

able to define the scope of testing

Scope Definition (GAMP®5)
20
• GAMP®5 methodology can be used to define the scope

• What is GAMP®5?
– Guidance documentation which provides a risk-based approach to computer system
validation
– A system is assigned to a category based on its intended use and complexity.
• Determine the system Software Category as per GAMP®5
• This category is based on the extent to which the system is customized since the
risk of failure and defects generally increases as the complexity of the system
increases
• Rule of Thumb: Increased customization leads to more extensive testing

Where to Test
21
• Testing is often performed in multiple environments

• It is recommended to perform testing in three different environments:
– Development: Serves as a non-controlled, pre-QA environment for informal testing
– Test (QA): Serves as a controlled, pre-Production environment for testing and training
– Production: Serves as a controlled operational environment for day-to-day activities
• It is not necessary to perform the same tests in all three environments. Where
testing is done is determined using a risk-based approach

Advantages of Testing in Multiple Environments
22
• Prevent the transfer of issues to Production environment

• Avoid creating dummy data within the Production environment
• Testing that may be destructive to the Production environment can be performed
elsewhere (ex. infrastructure stress testing)
• Reduced system downtime
• Reduced maintenance and support costs

Test Scripts: Basic Characteristics
23
• Test Script: Used to define and document testing activity

• Traceable to requirement(s).
• If the test passes, the associated requirements have been met.
• Main Test Script Components:
– Test Objective
– Acceptance Criteria
– Prerequisites
– Test Instructions (Step-by-Step)
– Expected Results
– Designated spaces for recording what was observed

Test Scripts: Recording Results
24
• Data recorded by testers:

– Actual Results
– Identification of testing sequence (ex. run number)
– Overall Result (i.e. Pass, Fail, Conditional Pass, Not Performed)
– Cross-references to attachments
– References to any non-conformances
– Tester identification information (ex. name/ initials)
– Date actions were performed

Example: Test Script
25
Run
Number
Actual
Result Tester
Name
and Date
Reference to
Prerequisites
Step by Overall
Step Reference Result
Instructions supporting
docs

Test Scripts: Recording Results
26
It is recommended that Actual Results be written in full i.e. document exactly what was observed,
instead of simply writing “As Expected”
• Why? : Testers are more likely to detect/ report issues when additional details are
provided.
Poll: How does your organization record actual

results during test script execution?
Option 1: Actual results are written in full

or
Option 2: Results are documented as ‘as expected’
or ‘not as expected’

27
Characteristics of Well-Written Test Scripts
• Clearly defined prerequisites
• Concise, reproducible test instructions
• Fields available for recording data
• Precise expected results
• Results independently review

How to Record Results? 28
Electronic, Paper or Hybrid
• Execution can be performed in the following ways:

– Electronically: Results are recorded electronically, use of e-signatures
– On Paper: All results documented by hand
– Hybrid Approach: Results documented electronically, printed and signed off
with wet-ink signature
• Executing electronically means that the executed test script is
considered an Electronic Record
• Electronic execution process must be validated to ensure that the
solution is fit for use and compliant with applicable regulations

Advantages to Executing Test Scripts Electronically 30
• Managing documents electronically

– Trace tests electronically to requirements
• Accessibility of Records
– Remote access to documentation
– Can lead to reduced approval times
• Secure Access
– Can assign permissions to electronic documents
• Save Time
– Easier to fill out (type results, use of drop-down menus)
• Save Money
– No costs related to printing, scanning and storage of physical documents

Review of Test Results
31
• Test results are independently reviewed (by a member of the validation team
other than the tester).
• Provides assurance that execution was done properly and facilitates the reporting
of any issues.
• This review involves:
– Ensuring that executed test script is complete
– Verifying that all appropriate documentation has been generated
– Checking cross-references to supporting documents
– Confirming the overall result

Time to Assemble Your Testing Team
32
• Choose testers: Review the testing

documentation and determine how many
people are required for testing
• A good tester should:
– Understand validation and quality principles
– Be knowledgeable of good documentation
practices
– Be familiar with the system being tested
– Be aware of the purpose of testing
• Select testers based on the type of testing
being performed

Who Should You Recruit?
33
Type of Testing Who should be involved? Advantages

Technical Testing Involve technical personnel: • Leverage system
(ex. IQ) • System Administrators knowledge
• IT Professionals • Subject Matter
Experts (SME) to
evaluate issues
Business Process Involve end-users: • Ability to assess

Testing (ex. UAT) • Training Managers system suitability
• Investigators • Training opportunity
• QA

Train Your Testing Team
34
• Ensure testers have been adequately trained prior to participating in validation

activities
• Especially important when involving non-validation personnel (i.e. IT, end-users)
• Conduct training in accordance with approved procedures
• Successful completion of training must be documented
• There are 2 levels of training:
– System Training
• Ex. Training on how to use the system
– Validation Training
• Ex. Training on validation policy/ procedures, documenting non-conformance, good documentation
practices etc.

36
3
Executing Test Scripts
• Setting up Prerequisites
• Good Documentation Practices
• Creating Screen Captures
• Documenting Non-Conformances

Preparing Prerequisites
37
• Prerequisites: Conditions that must be satisfied prior to executing a test script

• Recorded within the executed test script document
• Examples of Prerequisites:
– Definition of technical components, such as server names, IP addresses
– Presence of data that allows for a particular scenario to be tested
• Tips:
– Gather supporting documentation, such as user guides
– Create dummy data before starting to test

Example of Prerequisites
38

39
Good Documentation Practices
• Governed by internal procedures.

• Ensure that testers have been trained prior to execution.
• Principles of Good Documentation Practices:
– Accurate
– Legible
– Contemporaneous
– Original
– Attributable

Annotations: Correcting Text
40
• Any correction to preprinted text may be made only if it does not alter the
original intent of the test.
• Corrections to preprinted or handwritten text may be made by crossing out the
incorrect information with a single line stroke and without obscuring the original
information.
• The correct information may be entered below, above, or next to the error.
• If additional space is required, use a numbered footnote and record the correct
information in the page margin.
• All corrections must be initialed and dated

Annotations: What Not to Do
41
Correction Error in Annotation

This result is bad. good Correction not initialed/ dated.
This result is bad. good Original text is obscured.
CP 15-JUN-2016
This result is bad. No correction was made.

CP 15-JUN-2016
This result is bad. Good Illegible correction.
CP 15-JUN-2016

Annotations: Best Practices
42
Correction
This result is bad. good 1
This result is bad. good CP 15-JUN-2016
1 Entry Error. CP 15-JUN-2016

When is an Annotation Allowed?
43
• When there is no impact on the original intent of the test
Test Execution Instructions & Results

Step Initial/
Step Instructions Expected Result Actual Result
Status Date
12. Verify that the Training Report The Training Result:
displays training requirements. Report displays
a) Navigate to the Training training
______________
Report. requirements.
Attachment:
b) Verify that training
requirements are displayed
for the Trainee identified in ______________
prerequisite PRQ-1. PRQ-2 Validation Non-
CP 15-JUN-2016 Conformance:

When Are Annotations Not Allowed?
44
• When a correction would alter the original intent of the test

• Should be addressed via a non-conformance report
Test Execution Instructions & Results
Step Initial/
Step Instructions Expected Result Actual Result
Status Date
13. Verify that a task is assigned to the A task is assigned Result:
Trainee. to the Training
a) From the Training Manager.
______________
Management application,
navigate to the Task List. Attachment:
b) Verify that a new task was
assigned and that it is _______________
associated to the Training Validation Non-
Course identified in Conformance:
prerequisite PRQ-5.
c) Generate a screen capture.

Why Are Screen Captures Important?
45
• Screen captures provide evidence of what was observed in the system during
testing (typically included in attachments)
• Used to corroborate the testers’ findings i.e. show whether the system’s state
matches the expected result
• While not mandatory, they facilitate review by third party (QA, Auditor/ Inspector)

When are Screen Captures Necessary?
46
• Proving Step: Demonstrates that the test objective was met

• Non-Proving Step: Intermediary step that allows the tester to perform a proving step (no screen capture required)

Tips for Generating Screen Captures
47
• Make sure screen captures depict the elements being verified

• Take screen captures only when necessary (proving vs non-proving steps)
• Ensure that screen captures are legible especially when printed
• When possible, screen captures should include test conditions which prove that
the test was properly executed (ex. URL, login names)

Screen Captures: What Not To Do
48
• Verification: Email attachment of Non-Conformance document was received by

Tester3 QA
• Issues with this Screen Capture:

– Does not show email recipient name, i.e. Tester3 QA
– Attachment information is not visible

Screen Captures: What Not To Do
49
• Verification: Email attachment of Non-Conformance document was received by

Tester3 QA
• Issues with this Screen Capture:

– Relevant information is not visible
– Image distorted by resizing (illegible)
– Image of entire screen is not necessary in this case

Screen Captures: Best Practices
50
• Verification: Email attachment of non-conformance document was received by

Tester3 QA
• Why is this an appropriate screen capture?

– Relevant information is visible
– Use of highlight to draw attention to elements verified (if your procedures permit the use of
highlight)
– Image is not distorted
What are Non-Conformances?
51
• Non-conformances: Event that prevents test acceptance criteria from being met.
• Terminology can vary. Other commonly used terms are:
– Deviation
– Exception
– Test Incident
• Examples of non-conformances:
– Discrepancy between expected and actual result
– Errors that occurred during execution
– Changes to test methodology resulting in test’s intent being altered

Poll Question:
52
What does your organization call non-conformances?
• Non-Conformances
• Deviations
• Exceptions
• Test Incidents
• Other

53
Documenting Non-Conformances
• A non-conformance report should:

– Be clear and concise
– Assess impact (blocking vs non-blocking issue)
– Pinpoint root cause(s)
– Incorporate SME input
– Propose appropriate corrective action(s)
– Provide a rationale for actions taken
– Avoid overly technical language, define all abbreviations
– Include supporting documentation when necessary

Resolving Non-Conformances (Step-by-Step Approach)
54
• Step 1: Provide a detailed description of the non-conformance

• Step 2: Determine the root cause of the issue reported
– Examples of possible root causes:
• System configuration error
• System deficiency
• Execution error
• Protocol generation error
• Step 3: Identify corrective actions (approved prior to implementation)
• Step 4: Implement corrective actions and provide documented evidence that the
issue is resolved

Example: Non-Conformance Description
55

Example: Non-Conformance Investigation
56

Example: Non-Conformance Corrective Action/ 57
QA Approval

58
4 Traceability and Reporting

Traceability
59
• Provide a link between system requirements and the test scripts/procedural controls that
verify them.
• Reminder: System functionality is tested by executing test scripts designed to
demonstrate that the requirements were met.
• Establishing traceability provides proof that the system functions as intended.
• May be presented in a Traceability Matrix or for simple systems (few requirements) can be
documented within another validation deliverable.
• A requirement may not be met via testing alone. Procedural controls are often necessary.
• If a requirement is not explicitly tested, a rationale should be provided (risk-based
approach).

Example: Traceability Matrix
60

Summary Report
61
• Issued at the end of the validation.

• Main components:
– Identification of all deliverables (Document ID, approval date)
– Test Result Summary
– Discussion of any non-conformances encountered during testing
– Statement of system acceptance (specify any system limitations)
• Report is approved by stakeholders.
• Possible stakeholders (depends on type of validation done):
– Process Owner
– System Owner
– QA

Conclusions and Recommendations
62
• Proper planning is key

• Team Work
– Involve stakeholders
– Pick the right people for each phase of the project
• Well-written test scripts lead to smoother test executions
• Document any testing issues via non-conformance reports
• Be sure to establish traceability
• Prepare a Summary Report (often reviewed by Inspectors)
• If it’s not documented, it didn’t happen!

63
Thank you for your attention!
Questions
?
cplagiannos@montrium.com

64
Have a question? Get in touch!
Schedule a call with one of our validation experts
Montrium Inc. Montrium S.A.

507 Place d’Armes, Suite 1050 2A, rue Alophe Diederich
Montreal (QC) L-5820 Fentange
H2Y 2W8 Luxembourg
Canada +352.20.88.01.30
+1 (514) 223-9153
info@montrium.com
www.montrium.com

Webinar GXP System Validation

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Webinar GXP System Validation

Uploaded by

Copyright:

Available Formats

1

How to Validate Computerized GxP Systems:

Presented by Chrysa Plagiannos

© Montrium Inc. 2016

• Montrium is a technology company focused on the delivery of enterprise content

© Montrium Inc. 2016

1. Introduction 3. Executing Test Scripts

© Montrium Inc. 2016

© Montrium Inc. 2016

Examples of computerized systems used for GxP activities :

- Systems used to generate electronic records (ex. training records,

- Systems used to store/ manage data that is subject to regulations, such

© Montrium Inc. 2016

Definition as per 21 CFR Part 11.3:

© Montrium Inc. 2016

© Montrium Inc. 2016

Reason 2: Validation documents are auditable records.

© Montrium Inc. 2016

Reason 3: Electronic records managed by GxP systems can be auditable

© Montrium Inc. 2016

• Real-life Example: Warning Letter

© Montrium Inc. 2016

• Summary of Issues Identified in the Warning Letter:

© Montrium Inc. 2016

• Qualification: Process of demonstrating whether an entity is capable of fulfilling specified

• Validation: Process of establishing documented evidence which provides a high degree

• Note: Qualification is part of validation.

• Visit Montrium blog for more information

© Montrium Inc. 2016

© Montrium Inc. 2016

© Montrium Inc. 2016

2 Preparation for Testing

© Montrium Inc. 2016

• Validation Plans should be formally documented

• Information to be included in the Planning Document(s):

© Montrium Inc. 2016

• Ensure that the scope is clearly defined.

• For software, it is important to understand the system’s core functionality to be

© Montrium Inc. 2016

• GAMP®5 methodology can be used to define the scope

© Montrium Inc. 2016

• Testing is often performed in multiple environments

© Montrium Inc. 2016

• Prevent the transfer of issues to Production environment

© Montrium Inc. 2016

• Test Script: Used to define and document testing activity

© Montrium Inc. 2016

• Data recorded by testers:

© Montrium Inc. 2016

© Montrium Inc. 2016

Poll: How does your organization record actual

Option 1: Actual results are written in full

© Montrium Inc. 2016

• Clearly defined prerequisites

• Concise, reproducible test instructions

• Fields available for recording data

• Precise expected results

• Results independently review

© Montrium Inc. 2016

Electronic, Paper or Hybrid

• Execution can be performed in the following ways:

© Montrium Inc. 2016

• Managing documents electronically