Clinical Evaluation 101A Key Requirement to Gaining Access to the European Market

Suzanne Halliday BSI Healthcare 16.June.2011

Course Description: Clinical evaluation is the assessment and analysis of clinical data pertaining to a medical device in order to verify the clinical safety and performance of the medical device. In this session you will gain a basic understanding of Clinical Evaluation process which covers the totality of information used to demonstrate compliance and can include: Clinical Trial, Literature Review and Product Performance History.

Learning Objectives: Understand the European Regulatory Framework Definition of Clinical Data Learn the Clinical Evaluation Process How the scope of data required is proportional to the Risk of the Device Obtain a listing of useful resources

MedDev 2.7.1 Dec 2009

http://ec.europa.eu/consumers/sectors/medical-devices/documents/guidelines/index_en.htm

MedDev 2.7.1 GHTF N2R8

New EU additions Clause 10 and Appendix F The role of the Notified Body in the assessment of clinical data:

10.1. Examination of a design dossier or type examination dossier 10.1.1 Decision-making by the Notified Body 10.1.2 The report of the Notified Body 10.2. Evaluation as part of quality system related procedures 10.2.1 Review of the manufacturers procedures 10.2.2 Review of samples and their technical documentation 10.3. Notified Body Specific Procedures and Expertise

Design Dossier (Annex II.4 & III) verification of:

1. Followed relevant Procedures, Standards & Guidance. 2. Verified Device Characteristics & Performance. 3. Completed Risk Analysis Severity, Probability & Multidisciplinary Approach. 4. Identified, Appraised, Analysed & Assessed Clinical Data. 5. Addressed Relevance, Equivalence, Similarity & Limitations of Clinical Data.

Design Dossier (Annex II.4 & III) verification of:

6. Safety, Performance & Risk-Benefit ERs 1, 3 & 6. 7. Demonstrated Conformity with Relevant ERs. 8. Provided relevant Clinical Investigation documents CIP, Amendments, Ethics, Informed Consent, Case Report Forms, Regulatory Authority approval and Final Signed & Dated Report. 9. Provided Conclusions and Justifications. 10. Justified appropriateness of Post Market Surveillance including Post Market Clinical Follow-up.

1. Followed relevant Procedures, Standards & Guidance

Verify relevant procedures followed NB report should cover procedures and use of harmonised standards

GHTF N2R8 / MedDev 2.7.1 / * ISO 14155 (2011)

SystematicReview GHTF N2R8 (2007) MedDev 2.7.1 (2009)

1. Objectives Safety Performance SubstantiatedClaims 2. Identify 3. Select SimilarorEquivalent 4. Appraise/Review/Analyse/Weight 5. CriticallyEvaluate 6. ClinicalEvaluation/RiskManagement Relationship 7. InclusionofPMSInformation 8. Conclusions 9. References 10. Qualifications

Clinical Investigations See #8.

2. Verified Device Characteristics & Performance.

Verify characteristics and performance Evaluate validation of clinical claims NB report should cover description and product specification

Primary total hip replacement surgery: a systematic review of outcomes 1998 Health Technology Assessment NHS R&D HTA Programme 11

Primary total hip replacement surgery: a systematic review of outcomes 1998 Health Technology Assessment NHS R&D HTA Programme 12

13

Charnley, Mller, PCA, Ring, McKee-Farrar, Harris-Galante, Stanmore, Charnley-Mller, Lubinus, Exeter

* Different Design Characteristics * Different Clinical Performance

3. Completed Risk Analysis Severity, Probability & Multidisciplinary Approach

Verify risk analysis, estimates of side effects and involvement of clinical expertise in risk analysis NB report should cover risks identified by hazards, severity of hazards and probability of occurrence of harm

15

Prepare risk management report (8)

Review production & postproduction information (9)

Is reassessment of risk necessary? (9) Yes No

EN ISO 14971:2009

16

Knees Fractures 2.0% 3.0% ----5.2% 5.0% --1.0% 1.0% ---

Knees Loosenings 25.0% 10.0% 27.0% --36.5% 33.0% --39.2% 38.0% ---

Probability of Occurrence

17

Functional Area Information Systems Development Engineer Research Clinical Marketing Regulatory

Signature

Date 11/October/2008 11/October/2008 11/October/2008 12/October/2008 11/October/2008 12/October/2008

Multidisciplinary Approach

4. Identified, appraised, analysed & assessed Clinical Data

Verify use of MedDev 2.7.1 NB report should cover complete and adequate documentation per MedDev 2.7.1

Unicompartmental Knee Arthroplasty for Unicompartmental Osteoarthritis: A Systematic Review. 2005 ASERNIP S 19

Identify

Unicompartmental Knee Arthroplasty for Unicompartmental Osteoarthritis: A Systematic Review. 2005 ASERNIP S 20

Include

Unicompartmental Knee Arthroplasty for Unicompartmental Osteoarthritis: A Systematic Review. 2005 ASERNIP S 21

Exclude

22

Definitions
1. Appraise to assess merits or quality. 2. Analyse to examine something in great detail in order to understand it. 3. Assess to examine something in order to evaluate it.

Unicompartmental Knee Arthroplasty for Unicompartmental Osteoarthritis: A Systematic Review. 2005 ASERNIP S 23

Appraise

JBJS Levels of Evidence

24

Types of Studies Therapeutic Studies Investigating the Results of Treatment Prognostic Studies Investigating the Effect of a Patient Characteristic on the Outcome of Disease
High-quality prospective study (all patients were enrolled at the same point in their disease with 80% followup of enrolled patients) Systematic review of Level-I studies Retrospective study Untreated controls from a randomized controlled trial Lesser-quality prospective study (e.g., patients enrolled at different points in their disease or <80% followup) Systematic review of Level-II studies

Diagnostic Studies Investigating a Diagnostic Test

Economic and Decision Analyses Developing an Economic or Decision Model

Sensible costs and alternatives; values obtained from many studies; multiway sensitivity analyses Systematic review of Level-I studies

Level I

High-quality randomized controlled trial with statistically significant difference or no statistically significant difference but narrow confidence intervals Systematic review of Level-I randomized controlled trials (and study results were homogeneous) Lesser-quality randomized controlled trial (e.g., <80% follow-up, no blinding, or improper randomization) Prospective comparative study Systematic review of Level-II studies or Level-I studies with inconsistent results

Testing of previously developed diagnostic criteria in series of consecutive patients (with universally applied reference "gold" standard) Systematic review of Level-I studies Development of diagnostic criteria on basis of consecutive patients (with universally applied reference "gold" standard) Systematic review of Level-II studies

Level II

Sensible costs and alternatives; values obtained from limited studies; multiway sensitivity analyses Systematic review of Level-II studies

JBJS Levels of Evidence

25

Types of Studies Therapeutic Studies Investigating the Results of Treatment Prognostic Studies Investigating the Effect of a Patient Characteristic on the Outcome of Disease
Case-control study

Diagnostic Studies Investigating a Diagnostic Test

Economic and Decision Analyses Developing an Economic or Decision Model

Analyses based on limited alternatives and costs; poor estimates Systematic review of Level-III studies

Level III

Case-control study Retrospective comparative study Systematic review of LevelIII studies

Study of nonconsecutive patients (without consistently applied reference "gold" standard) Systematic review of Level-III studies Case-control study Poor reference standard Expert opinion

Level IV

Case series

Case series

No sensitivity analyses

Level V

Expert opinion

Expert opinion

Expert opinion

Unicompartmental Knee Arthroplasty for Unicompartmental Osteoarthritis: A Systematic Review. 2005 ASERNIP S 26

Analyse

Unicompartmental Knee Arthroplasty for Unicompartmental Osteoarthritis: A Systematic Review. 2005 ASERNIP S 27

Assess

5. Addressed relevance, equivalence / similarity & limitations of Clinical Data

Verify relevance and inclusion of safety, performance and risks<benefits NB report should cover suitability of data, limitations of data and validity of any justifications

State of the Science on Implant Dentistry. 2006 The International Journal of Oral & Maxillofacial Implants 29

Relevance

30

Implants ?????

Equivalence / Similarity

Checklist 31

Equivalent or Similar:
Clinical: same clinical condition or purpose same site in the body similar population (including age, anatomy, physiology) similar relevant critical performance for specific intended use

Technical: similar conditions of use similar specifications and properties similar design similar principles of operation

Biological: same materials in contact with the same tissues or body fluids

32

Implants ?????

Limitations

6. Addressed safety, performance & risk-benefit ERs 1, 3 & 6

Verify relevance and inclusion of safety, performance and risks<benefits NB report should cover suitability of data, limitations of data and validity of any justifications

Evidence Report/Technology Assessment Total Knee Replacement. 2003 Agency for Healthcare Research and Quality. 34

Demographics

Evidence Report/Technology Assessment Total Knee Replacement. 2003 Agency for Healthcare Research and Quality. 35

Complications Safety

Evidence Report/Technology Assessment Total Knee Replacement. 2003 Agency for Healthcare Research and Quality. 36

Revisions Safety

Evidence Report/Technology Assessment Total Knee Replacement. 2003 Agency for Healthcare Research and Quality. 37

KSS Performance

Evidence Report/Technology Assessment Total Knee Replacement. 2003 Agency for Healthcare Research and Quality. 38

HSS Performance

39

RISKS BENEFITS

Risks < Benefits

7. Demonstrated conformity with relevant ERs

Verify data meets relevant ERs NB report should cover proof of clinical performance and expected benefits

41

93/42/EEC as amended by 2007/47/EC:

Annex X 1.1 As a general rule, confirmation of conformity with the requirements concerning the characteristics and performances referred to in ER1 and ER3, under the normal conditions of use of the device, and the evaluation of the side-effects and of the acceptability of the benefit/risk ratio referred to in ER 6, must be based on clinical data.

Intra-articular visco supplementation, oral glucosamine or chondroitin and arthroscopic debridement - treatment of OA. 2007 Agency for Healthcare Research and Quality 42

ER#1 & 3 safety & performance

ER#4 lifetime

ER#2 state of the art

Intra-articular visco supplementation, oral glucosamine or chondroitin and arthroscopic debridement - treatment of OA. 2007 Agency for Healthcare Research and Quality 43

Intra-articular visco supplementation, oral glucosamine or chondroitin and arthroscopic debridement - treatment of OA. 2007 Agency for Healthcare Research and Quality 44

Follow through to ER#13.

8. Provided relevant Clinical Investigation documents

Verify relevant procedures followed, use of harmonised standards, compliance with regulatory and ethical requirements NB report should cover proof of clinical performance intended

Where a clinical investigation has been carried out it is expected that documentation will be available for consideration. Expected to be designed, conducted and reported in accordance with EN ISO 14155 -1 & -2 Documentation to be available: Clinical Investigation Plan (study design, participants, treatment, blinding, dependent variables, follow up, statistical analyses) CIP amendments, rationale and evidence of notification of Regulatory Authority Ethics Committed opinion, informed consent forms, patient information documents Case report forms, monitoring and audit records Regulatory Authority approvals and correspondence Final report signed and dated

46

Harmonised Standards http://ec.europa.eu/enterprise/policies/european-standards/documents/ harmonised-standards-legislation/list-references/medical-devices/index_en.htm

* ISO 14155 (2011)

ISO 14155 2011

More prescriptive requirements for the Clinical Investigation Plan (Annex A) and the Investigator's Brochure (Annex B) Suggestions for Case Report Forms (Annex C) and Clinical Investigation Reports (Annex D). Clear requirements for Adverse Event reporting to meet the requirements of 93/42/EEC and 90/385/EEC (Annex F). The suggested procedure for a literature review have been deleted. This eliminates any confusion between the standard, GHTF N2R8 and MedDev 2.7.1.

48

9. Provided Conclusions and Justification if no Clinical Investigation

Verify justification if no clinical investigation performed. Validate conclusions. NB report should cover proof of expected benefits to patients

50

Conclusions:
Conclusions GHTFN2R8 (2007) Outlineclearlytheconclusions reachedabout thesafetyandperformanceofthedevicefrom theevaluation,withrespecttotheintendeduse ofthedevice.Statewhethertherisksidentified intheriskmanagementdocumentationhave beenaddressedbytheclinicaldata. Foreachproposedclinicalindicationstate whether: theclinicalevidencedemonstratesconformity withrelevantEssentialPrinciples; theperformanceandsafetyofthedeviceas claimedhavebeenestablished; therisks associatedwiththeuseofthedevice areacceptablewhenweighedagainstthe benefits tothepatient. MedDev2.7.1 (2009) Outlineclearlytheconclusions reachedaboutthe safetyandperformanceofthedevicefromthe evaluation,withrespecttotheintendeduseof thedevice.Statewhethertherisksidentifiedin theriskmanagementdocumentationhavebeen addressedbytheclinicaldata. Foreachproposedclinicalindicationstate whether: theclinicalevidencedemonstratesconformity withrelevantEssentialRequirements; theperformanceandsafetyofthedeviceas claimedhavebeenestablished; therisks associatedwiththeuseofthedevice areacceptablewhenweighedagainstthe benefits tothepatientinlightofcurrentstateof theart.

51

Follow MedDev 2.7.1 with Conclusions For each proposed clinical indication state whether:
The clinical evidence demonstrates conformity with relevant Essential Requirements. Device performs as intended by the manufacturer. Device does not pose any safety concerns to either recipient or enduser. Risks associated with the use of the device are acceptable when weighed against the benefits to the patient in light of current state of the art. No need for further clinical investigation or post market clinical follow-up.

10.Justified appropriateness of Post Market Surveillance including PMCF

Verify appropriateness of planned PMCF or justification for not conducting PMCF NB report should cover PMS

53

QMS

PMS NBMed 2.12 Vigilance MedDev 2.12-1 Reactive PMS

Proactive PMS

Post Market Clinical Follow-up MedDev 2.7.1 & 2.12-2

54

Post Market Surveillance / Post Market Clinical Follow-up

Adequate PMS System Plan for PMCF Adequate justification for no PMCF Update Clinical Evaluation with data obtained from PMS/PMCF

Questions & Answers

56

References:
1. Primary total hip replacement surgery: a systematic review of outcomes. 1998 Health Technology Assessment NHS R&D HTA Programme. 2. Evidence Report/Technology Assessment Total Knee Replacement. 2003 Agency for Healthcare Research and Quality. 3. Unicompartmental Knee Arthroplasty for Unicompartmental Osteoarthritis: A Systematic Review. 2005 Australian Safety and Efficacy Register of New Interventional Procedures-Surgical. 4. State of the Science on Implant Dentistry. 2006 The International Journal of Oral & Maxillofacial Implants. 5. Intra-articular visco supplementation, oral glucosamine or chondroitin and arthroscopic debridement treatment of OA. 2007 Agency for Healthcare Research and Quality.

Extra Slides

58

Quality System Requirements (Annex II.3):

1. Procedures ensure suitably qualified persons. 2. Integration clinical evaluation into QMS as continuous process. 3. Procedures for clinical evaluation, clinical investigation & PMS/PMCF: ensure relevance of data to device data includes safety, performance & risk-benefit evaluation use of MedDev 2.7.1 4. Document control of procedures, reports, qualifications & files / dossiers. 5. Identification, evaluation & verification of undesirable side effects, severity and probability of occurrence.

Checklist 59

Where the clinical investigation(s) was performed outside the EU, the manufacturer must demonstrate that the use of the device (including clinical practice and techniques) and patient population are equivalent to those for which the device will be used within the EU.