An Integrated Model For Performance Management Based On ISO 9000 and Business Excellence Models PDF

FindBestStuff.
FindBestStuff.com
Google Search
Web
FindBestStuff
com
Coffee
Cooking Tips
Recipes & Food and Drink
Wine & Spirits
Elder Care
Babies & Toddler
Pregnancy
Acne
Aerobics & Cardio
Alternative Medicine
Beauty Tips
Depression
Diabetes
Exercise & Fitness
Hair Loss
Medicine
Meditation
Muscle Building & Bodybuilding
Nutrition
Nutritional Supplements
Weight Loss
Yoga
Martial Arts
Finding Happiness
Inspirational
Breast Cancer
Mesothelioma & Cancer
Fitness Equipment
Nutritional Supplements
Weight Loss
Credit
Currency Trading
Debt Consolidation
Debt Relief
Loan
Insurance
Investing
Mortgage Refinance
Personal Finance
Real Estate
Taxes
Stocks & Mutual Fund
Structured Settlements
Leases & Leasing
Wealth Building
Home Security
Affiliate Revenue
Blogging, RSS & Feeds
Domain Name
E-Book
E-commerce
Email Marketing
Ezine Marketing
Ezine Publishing
Forums & Boards
Internet Marketing
Online Auction
Search Engine Optimization
Spam Blocking
Streaming Audio & Online
Music
Traffic Building
Video Streaming
Web Design
Web Development
Web Hosting
Web Site Promotion
Broadband Internet
VOIP
Computer Hardware
Data Recovery & Backup
Internet Security
Software
Mobile & Cell Phone

Video Conferencing
Satellite TV
Dating
Relationships
Game
Casino & Gambling
Humor & Entertainment
Music & MP3
Photography
Golf
Attraction
Motorcycle
Fashion & Style
Crafts & Hobbies
Home Improvement
Interior Design & Decorating
Landscaping & Gardening
Pets
Marriage & Wedding
Holiday
Fishing
Aviation & Flying
Cruising & Sailing
Outdoors
Vacation Rental
Copyright 2007
FindBestStuff
Advertising
Branding
Business Management
Business Ethics
Careers, Jobs & Employment
Customer Service
Marketing
Networking
Network Marketing
Pay-Per-Click Advertising
Presentation
Public Relations
Sales
Sales Management
Sales Telemarketing
Sales Training
Small Business
Strategic Planning
Entrepreneur
Negotiation Tips
Team Building
Top Quick Tips
Book Marketing
Leadership
Positive Attitude Tips
Goal Setting
Innovation
Success
Time Management
Public Speaking
Get Organized - Organization
Book Reviews
College & University
Psychology
Science Articles
Religion
Personal Technology
Humanities
Language
Philosophy
Poetry
Book Reviews
Medicine
Coaching
Creativity
Dealing with Grief & Loss
Motivation
Spirituality
Stress Management
Article Writing
Writing
Political
Copywriting
Parenting
Divorce
AN INTEGRATED MODEL FOR PERFORMANCE
MANAGEMENT BASED ON ISO 9000 AND BUSINESS

EXCELLENCE MODELS
Shannon Macey
Submitted
in partial fulfhent of the requirements
for the degree of
MASTER O F APPLIED SCIENCE

Industrial Engineering
at
DALHOUSIE UNIVERSITY
August, 2001
Halifax, Nova Scotia

Copyright by Shannon Macey, 2001
National Library
Bibliothque nationale
du Canada
Acquisitions and
Bibfiographic Services
Acquisitions et
senrices bibliographiques
395 WeUington Street

OltawaON KlAON4
Canada
395. rue Wellington

Ottawa ON K1A W
Cam&
The author has granted a nonexclusive licence allowing the

National Library of Canada to
reproduce, Ioan, distribute or sell
copies of this thesis in microfom,
paper or electronic formats.
L'auteur a accord une licence non

exclusive permettant la
Bibliothque nationale du Canada de
reproduire, prter, distribuer ou
vendre des copies de cetie thse sous
la forme de microfiche/film, de
reproduction sur papier ou sur format
lectronique.
The author retains ownership of the

copyright in this thesis. Neither the
thesis nor substantial extracts fkom it
may be printed or otherwise
reproduced without the author's
permission.
L'auteur conserve la proprit du

droit d'auteur qui protge cette thse.
Ni Ia thse ni des extraits substantiels
de celle-ci ne doivent tre imprims
ou autrement reproduits sans son
autorisation.
Table Of Contents
List of Figures .................................................m....~.........em...........................~.........

x
List of Abbreviations
1.0 Introduction
xi
~
..........................~.......o...~....~....ooo..~~~.e............................1...~e...o......
Performance Management
...................................................................................................... .......1
..
Models for Performance Management ......,..,............. ..................,.......................... ............- 4

Performance Measurement .........................................
.
.
.
. ............................................. 5
.............................,
.
.
....................................... ..........7
Case Study Organisation (CSO)...............................,
.
................................................ . 9
Ornanisation o f the Thesis .............................................. ....................................
.. .. 1 0
Performance Management in R&D
2.1
............................*................~o.....sm..........*.~*...........*12........~...
Introduction .............................................................................................................. . . . 12
2.2
2.0 Literature Review
................................ ............. . . ... .................. .
..............12
2.2.1
Models for Performance Management ................................ ....................................... . 13
2.2.2
Performance Measurement................. .................................. .......................,......................-27
23
...........................................................................................-46
2.3.1
Addressing Issues Unique to R&D .......................................................................................
2.3.2
Benefits of Implernenting Quality Management in an R&D Environment ........................... 49
2.3.3
ISO 9000 in R&D..................................................................................................................
46
51
.......................
.
.
............................................................... 52
Objectives of Proposed Researcb ................................................................................................
53
Motivation for Proposed Research
.................................5
An Integnted Mode1 For Perf~~nance

Muiagemcnt
..............................
.................................................................................................54
Development of the OPIM ......................... ..... .............w........9.9........ ..........................54
Introduction
3.2.1
Framework Development ......................................................................................................
54
3.2.2
Criteria Development ............................................................................................................
57
3.2.3
Development of the Assessment Questions .......................................................................... 66
3.2.4
Establishment of the Scoring Scheme ................................................................................... 66
3.2.5
Scoring Scheme Cornparison ................................................................................................ 73
33
Cbapter Summary
.......................................................................................................................
76
4.0 Seff Assessrnent .............................................................................................. 77
..................................................................................................................................
77
Self-Assessment at the Case Study Organisation .......................................................................
77
Introduction
4.1
4.2
4.2.1
Planning ................................................................................................................................ 78
4.2.2
Assessrnent............................................................................................................................ 83
4.2.3
Final Score ............................................................................................................................ 89
4.2.4
Outcomes .............................................................................................................................. 90
4.2.5
Evaluation o f Assessrnent Process ........................................................................................ 93
5.0 Performance Measurernent ........e....................................................................

96
5.1
Introduction
..................................................................................................................................
96
Cornparison of Assessrnent ResulW to Quriity ~ u d i Results

t
......................
.
.
......................96
Implications for Simultaneous Auditing and Assessrnent ................................................... 102
......................
.
.
...................................................-103
Result-based PerCormance Measurement
Review o f Organisational Performance Measures ............................................................. 103

Review o f the Performance Measures at Case Study Organisation.................................. 104
Self.assessment, Auditing and Result-based Performance Measuremeat
.............................117
Comparing Self-Assessrnent and Result-Based Performance Measurement....................... 117

The Roles of Self.Assessrnent, Quality Audits. and Result B a d Perfomance Measurement
in the Organisational Performance Masurement System ......................................................................
5.5
Chapter Summary
........................................................................
119
.........................................120
6.0 Performance Management In R&D .............................................................. l
........................................................................................ 121
Modifying the OPIM for Use in an R&D Environment ..........................
.
.
.........................122
The R&D Environment at the CS0
6.2
63
6.3.1
Framework ..........................................................................................................................
123
6.3.2
Criteria ................................................................................................................................
125
6.3.3
Scoring ................................................................................................................................ 128
6.3.4
Addressing R&D Issues with the Modified OPIM .............................................................. 130

Chapter Summary ...................................................................................................................... 131
Conclusions .................................................................................................. 132
...................................................................................................132
Scope for Further Research .................................................................. ...............................134
Contributions of the Research
References .................................................................................................... 136
Appendices ................................................................................................... 144
.............................................................................................1 4 5
APPENDIX B .
Modified OPIM for R&D ..................-..... .................................................................160
APPENDIX C .
Triiiium Results .......................,.......
...............................................................169
APPENDIX D - Cornparison of Performance Management Mode1 Criteria ......................................170
APPENDIX E - Approach. Deployment and Results Adapted from the EQA ...................................175
APPENDlX A .
OPIM Questionnaire
List of Tables
Table 2-1:Channeristics of Performance Management Models.......................................-21
Table 2-2:Strengths and Weaknesses of Performuice Management Models ......................22
Table 2-3: A Cornparison of the S c o ~ Schernes
g
from the Quairy A w d .....................24
............................................................ -31
Table 2-5:Evaluation and Scoring Methods of the Performance Management Models .......33
Table 2-6: Description of Evaluation EIements Used in the EQA .....................................34
Table 2-7: Scoring Guidelines from the MBNQA ...........................................................-35
Table 2-8:Sample Score Calculation for MBNQA ..........................................................
36
Table 2-4:Selection of Self-AssessrnentMethod
Table 2-9: Plan-Do-Check-Act Cirde of Audit and Self-Assessment Programs.................. 45

Table 3-1: InmaCaregories of the OPIM ........................................................................57
Table 3-2: Initial fit of Mode1Giteria to OPIM Framework .............................................
58
Table 3-3:Changes to the OPIM Categories ...............................................................
60
Tabie 3-4:Application of Cnteria to Frarnework .............................................................6 1

Table 3-5:Final OPIM Framework with Comespondmg Critexia from Xntegrared Models ..64
Table 3-6:Calculauon of Scores from Preferences............................................................
71
. .
Table 3-7:OPIM Scoring Cntena.................................................................................... 72

Table 3-8: Distribution of Points Arnong Criteria of OPIM, MBNQA, and EQA .............74
Table 4-1:Sample from the Question Mauix ................................................................... 82
Table 4-2: Tirnelines for Self-Assessrnent ..................................................................8
Table 4-3: Final Score Calculauon ................................................................................. 8 9

Table 4-4:Prioritising Areas for Improvement................................................................ 90
Table 5-1: Comparing Outcomes from Self-Assessment and Q d t y Audit ....................... 96
Table 5-2: Recomrnendations, and Noncornpliance Issues and Areas for Improvement ....98
Table 5-3:Comparison of CS0 Scores on Self-Assessmenr to Audit Findings ................101
Table 5-4:Results Categories from MBNQA and EQA ............................................... 103
Table 5-5:Performance Measwes Used at the CSO ....................................................... 105
Table 5-6: Performance Measurement System Interview Questions.................................107
Table 5-7: Applying the BaLnced Scoreard to the Performance Measurtment Syscem of the
Cs0..................................................................................................................... 110
Table 5-8: Performance Mea~uresAccording to QuiLry, TirrP, Cos, and Flexibility ........111
Table 5-9: Performance Arev Addresseci by the OPIM.................................................113
Table 5-10: Suggested Performance Measures to Add Relating to Resources ...................115
Table 5-11: Suggested Pedomiance Meyures to Add Relating to Society........................ 116
Table 5- 12: Suggested Perfonnuice Mevures to Add Relathg to Employees .................. 116
Table 5- 13: Characteristics of Perfoxmance Measurement Methods................................. 117
Table 6-1: Chamteristics of QwLty Management Addressing Issues in R8rD ................. 123
Table 6-2: Strategy Criteria Gom the OPIM .................................................................127
Table 6-3:Moditied Strategy Criteria fiom the OPIM ...................................................128
Table 6-4: The Categories of the Modif1ed OPIM Addressing the R&D Issues ................ 129
Table 6-5: Scoring Sdieme for the Modifieci OPIM ................................................... 130
Table 6 4 Modifieci OPIM Addressing R&D Problems .................................................131
List of Figines
..................................................1
Figure 1-1: P e r f o ~ c Management

e
......................
Figure 1-2: Methods for Performance Improvemmt ..........................................................3

Figure 1-3: The Relationship h e e n Performance Management and Performance
Measurement ..........................................................................................................
Figure 2- 1: Un-g
Framewoxk for the Bddrige National QuaLF/ Award Model ..........18
Figure 2-2: Enablen and Re&
of the EQA .................................................................19
Figure 2-3: The Self-assessrnent Process ..................................... ....................................2 9

Figure 2-4: Sehssessment Process Rrgour Versus Data Qua<ty ......................................32
...
Figure 2-5: OveMew of audit amvines...........................................................................

38
Figure 3-1: OPM Framewotir ...............................................................................

5
Figure 3-2: Measuring Preference in Triilium ...................................................................70

Figure 3-3- A Cornparison of the Point Distributions Among the Models ........................75
Figure 4-1: Steps in the Self-Assessrnent of the CSO .......................................................77
Figure 4-2: Test Operations & Svategic Sourcing.............................................................80
Figure 4-3: MvkRing and Sales................................................................................ 8 1
Figure 4-4: Product Development ................................................................................... 81
Figure 4-5: Excerpt from Find Repon .....................................................................
8 8
Figure 5-1: Cornparison of Scores on Self-Assessmentto Audit Findings ........................ 100
Figure 5-2: Comparing Low Self-Assessrnent Scores to Non-Cornphces ...................... 102
Figure 5-3: Perfomance Measwement Categoxy from OPIM ....................................104

Figure 5-4Measuring Efficiency and Effectiveness in Self-Assessrnent ...........................118
Figure 5-5: Relationship of OPIM and Derforniance Management System....................... 120
l R&D Goals ............................................ 122
Figure 6-1: Deployment of O ~ a t i o n aand
Figure 6-2: The Modified OPIM Framework .................................................................124
List of Abbreviations
CS0 - Case Study Organisation
TQM - Total QuaLty Management

BEM - Business Exceilence M&
MBNQA - Malcolm Baidnge National QuaLty Award
EQA - European QuaLty Award
DP - Deming Plize
NPD - New Product Development
OPIM - Organisational Performance Improvement M d
C E 0 Chef Executive Officer
Go0 Chief Operating Officer
VP - Vice President
QA Qdity Asnirance
PFM - Preference Function M o d e b g
Acknowledgements
Karapevovic, for invithg me to take part
in this research project and for all the guidance and support that he provideci throughout the
process. His knowledge of and experience in rhis field proved Livduable and was a great
1would like to thank m y supervisor, Dr. St&v
source of inspiration.
1wish to give thanks to the case study oqpkation for pamcipating in this project and for a i l
the CO-opemion1received whik performing my research there. Without its fwcial
support, this project would not have been possible. I would specifidy like to thank b o t -
Mark DeKoning and Max Jaffer for their involvement and advice. Their support was of
great benefit to the successful completion of this project.
1would Idce to thank Dr. John Blake for his valuable feedbadc and tirne spent reviewing rhis
work and Dr. Eldon Gunn for his encouragement and support. Thanlu to both for serving
as memben of m y guiding comminee. Thank
ou to Dr. Walter Wdbom for his Ume spent
reviewuig some of the w o i in the thesis and for travelling to Halifax for my defence.
Thanks also to Dr. Car1Sandblom for seming as a member of my guidance cornmittee.
Thanks also to the Department of Industrial Engineering at Dalhousie University for
providing funding as well as the opportunity to serve as a teaching assistant. Thanks to ail of
the Industrial Engineering deparunent staff, especially to Cindi Slaunwhite and Linda
Seamone for their cheer and fnendship.
1would like to express my sincerest gratitude to Greg Delbndge for his inexhawible
encouragement, patience and support throughout my d e s .
In today's global economy, o&tions

mua ourperform &eir cornpetitors in order to
survive. They must set goals and objectives chat will move them into leadership positions in
their indusrries. To ensure that these goals are being me5 cornplnies are developing
performance management systems to help them measure and improve the efficiency and
effectiveness of t h e u activities. S e v d modek have been developed to assist organisations
in these efforts.
Some of these models, such as ISO 9001, provide the minimum
requirements for a quality management system, while orhen, like the National Quality
Awards, provide a broader set of criteria that extend the prinaples of
management to
al1 organisational activities. Companies natudy question which model is best suited for
their performance management needs. It has been suggested that the model should be
selected based on the qulity maturity level of the organisation. The ISO 9001 standard for
quality management provides a necessaxy foundation for the journey towards business
excellenceThe Business Excellence Models can then be used to build onto this
foundation.
In this paper, a model that integrates ISO 9001, the Malcolm Bddnge National Quai;ty
Award, the European QuaLty Award, and the Deming Pxize is developed The
Organisational Performance Improvernent Modei (OPM) is considered to be a combination
of quality assurance standards that provide the minimum recpirernents for sU1i;ty
improvement, and business excellence models that illustrate the opportunities for
improvement.
Using a Canadian hi& technology Company as a case midy, a p h self-assessrnent is

conduaed againn the OPIM.This self-assessrnent pmcess is then compared to the two
other performance meanirement methods used within the organisation to cietennine its
usefulness as a tool for p e r f o m c e irnprovement. Because the case study organisation is
highly focused on its research and development activities, modifications were made to the
OPIM to create a more appropriate model for application and use in an R&D environment.
The OPIM was developed specifically for the case study organisation (CSO) but can be
easdy adapted to be used by other organisations. A comparry c m employ this model to
evaluate its current quality management system against the minimum requirernents in the
model and rhen apply the opportunities for improvement as a guide for continuous
improvement. Self-assessrnent against this model wili be beneficial in enhancing the
organisation's quality auditing program as well as its redt-based perfoxmance measurement
system.
1.0 Introduction
1.1
The purpose of any organisation is to niccessfully achieve in gods and objectives. To

remain cornpetitive, these goals need to be achieved in the most effiaent and effective
manner possible. Effectiveness and effiaency constitute the two fundamend dimensions of
performance (Neely et al, 1995). Effectiveness refers to the exrent to wbich goais are met,
while efficiency is a m m of how economidly the h ' s resouras are udised to redise
these goals. Organtations musr continually improve their processes and the use of th&
resources to realise improved o v e d performance. Perfoxmance management is the overali
process of managing aa organisation's effiaencyuid effectiveness in the adiievernent of its
corporate and funaional strategies and objectives.
Performance management is a dosed loop process. Figure 1-1 illustrates this d e .
Set Goais and Objectives
............................
Audit .....................
Self-Assessrnent
Yes
lui for I.mprwemtnr
and objectives. These cm

As a firsr s e p in this process, the organisation musc set irs
be baseci on benchmarking, standards or other such refemce aiteria.
Once goals and objectives are determineci, performance measurement is used to nadr the
progress towards these goals. Performance measurement is the quantification of the
effiuency and effectiveness of an action (Neeiy et ai, 1995).
To narrow the gap becween the m e n t lwel of perfomunce and rhe levels required to meet
its objectives, an organisation must improve. Performance impmvement is the systematic

process of idennfying and implementing changes to oqpisauonal processes in order to
more efficiendy and effectively achieve set organisational objectives. Performance
improvement can be achieved in two ways. The fvst is to elimioate problems or
deficiencies. The second to initiate actions that Mn aaualky result in improved
characterstics.
There are rwo ways in which companies can elimliate problems. These are corrective and
preventative actions. Corrective actions are those which are taken to eliminate the cause of
an existing non-conformity[the non-fuifilment of a requirement (ISO 9000: 20W)J or other
undesirable situation (ISO 9000: 2000). Preventive actions are those taken to elMinate the
cause of a potential nonconformiry or other potentially undesirable situation (ISO9000:
2000).
Improved characteristics can be achieved through Kaizen or Quantum Leaps. Kaizen means
s m d incremental irnprovernents towards the achievement of objectives. Quantum Leaps or

breakthroughs are larger irnprovements. Both of these reduce the average nurnber of
defects or reduce variation.
Figure 1-2 illusvates the concept of performance irnprovement.
Quaiify = k d o m h m deficiencies
( 0 )
+ imP'0ved characteristics (+)
ELIMINATE
PROBLEMS
GET BETTER
F e ,once a goal or objective is reaiised, new goals should be set to continue the cycle.
Both performance measurement and improvernent are essenria if an organisation wishes to
sirndtaneous~achieve its objectives and maintain its cornpetitive edge.
MANAGEMENT
PLAN
ACT
+
v
DO
IMPROVEMENT
MEASUREMENT
L
The
measurement synem un be seen, according to Bitichi et al. (1997), as the
heart of the performance management qmem.
There are xveral wys in which an organisation can mevure its performance and plan for
improvement. One such method is to compare its puformance t o a given reference model.
There are many such modek used by o ~ t i o ntoday.

s
Some of these models are
focused on specific processes within the organisation such
the ISO 9001: ZOOO standard
for sUai;ty management. Others such as National QuaLty A w d often referred to ar;
business excellence models (BEM) have a broader focus and uy to encompass all aspects of
an organisation. All of these models address the main aspects of performance management:
performance measurernent and performance improvement.
1.1.1.1
ISO 9000: 2000 Standards for Quli;ty Management
T h e ISO 9000 series of @ty
management syste.cn standards is an international set of
nanduds that provide the minimum requkements for an organisation's quality management
system. The core standard of the series, ISO 9001, addresses the creation and
implementation of a quaiity policy, the standardisation of procedures, the identification and
elimination of defects, cor-ve
and preventive action, and the management review of the
quality management systea The standard is process onenteci, anu at customer satisfaction
asniance, and focuses an organisation's attention on the effectiveness of its quaiiy
management sysrem. It was conceived as a set of essential requirements for building a
qualiy management synem that fits the company's ne& in relation to its products, services
and customers.
1.1.1 -2 National Quaiity Awards
Q d y awards are models for business excellence. These models consist of viteria to which
organisations can compare their activities and results. They have been developed based on
the p ~ a p l e of
s Total QwLty Management FQM).TQM is a quality-oried approach
chat consisu of applymg quality management techniques t h r e o u t the organisation. The
e v h t i o n criteria provide puideluies to organisations to measure their progress in these
efforts.
1.1.1.3
Integration of ISO 9001 and the Business Exceknce Models
Bo& the ISO 9000 series and the quality mat& have been adopted by orgaaisations to
measure and improve perfomiuice. Many oqpisations wonder whether one mode1 is
better than another, and whether the choice of model should depend on the type of
organisation. Many authors have mggesteci that both mo&
are important to an
organisation (van der W d e et ai, 2000; Dale, 1999; Karapetrovic and Wdborn, 2lB). The
ISO 9001 standard is seen as providing the minimum reS;rements for a &ty
management
syaem or the basics to stan the joumey towards orgauisationalexceilence. The quality
awardr, provide guidance for the n a sep on this joumq..
Combining these modeis would provide an integrared set of criteria that codd help an
organisation transition smooddy from its quaity management system t o w d the use of the
business exceilence modek for improvernent. There are however, many differences between
these models and as a resuh, they cannot simpiy be cur and pasteci together (Dale, 1999).
1.1.2
Performance Measwernent
To meet a company's organisational goals, performance measurement must be used to

evaluate, control and improve organisational processes (Ghalayini and Noble, 1996). This
process is used to interpret and descnbe quantitativelythe criteria used to mesure the
effectiveness of the organisation.
As noted previously, performance measurement can be defmed in terms of the rwo
fundamental dimensions of performance. Performance measurement is thus the process of
quanufyuig the efficiency and effectiveness of an action (Neely et al, 1995). ISO 9000
(2000) defmes effciency as the relationship bmeen results achieved and resources used and
effecriveness as the extent to which planned aaivities are realised and planned results
acheved
Three main performance mevurcment methods are used byo+tions
today: self-
assessment, auditing and renilr-based performculce rneasurement.
As organisations srrive to implement perfomunce management models, there is a redting
need to meanire progress and improvemenc of the impIementation efforts. Self-assessrnent

involves the regular assessment and monitoring of organisational activities to determine what
is working well, what needs to be improved, and what is missing (Dale and Bunney, 1999).
Instead of compliance with the criteria of the business excelience model, self-assessment
idendes an organisation's strengths and opportunities for improvement in each area of a
predefmed set of criteria. Self-assessrnenu ask "how doesthe organisation meet the
criteria?" and whether this approach is used consistendy (Le. well deployed) throughout the
enterprise.
Self-assessrnent using am-ard models is focused on finding impmvement opportuniries and
new ways in which to drive the organisation down the road of business excellence (van der
Weile et al., 1995).
The performance meanirement merhod used to evaiuate an organisation's qUa;ty

management system is the quaiity audit. The audit is used to evaluate the compliance of the
system widi the criteria of the ISO 9001 standard. The outcome is either compliance or
non-cornpliance. Audits are designed to provide independent, objective and documenteci
evaluation of the evidence that the quality system &S.
Quaity au& are used to measure the effectveness of an organisation's quahqr system
against the selected ISO 9000 model. Audits do not however, meanire the efiaency of the
system and therefore miss one essentd aspect of perfomance.
The audits used with ISO 9000 series registration look for noncornpliance and assess
whether the quJiry mern and its undeliying procedures are king followed. In other words,
+ty
audits ask 'if the organisation meeu the deria", wheeas self-assessrnena ask 'how,
and how well, does the organisation meet the criteria".
1.1.2.3 Result-based performance measurement
Result-based performance measurernent is another method organisations use ro meanire the
effuency and effectiveness of their processes. This involves the collection of information
and data to analyse process performance uid/or input and output characteristics (ISO 9000:
2000). To do this, companies design or adopt performance measures that ~ r o v i &valuable
information about the current performance lwels of organisational activities. Performance
measwes are meuics used to quanafv performance.
The set of performance measures used by an organisation fomis the performance
measurement system. Firms m m carefully select these m e m e s to provide a view of key
performance areas to control and improve performance lwels. It is the seleaion of this set
of meanires that has proved chailenging.
1.2
Research and development (R&D) is an area in which there is much variability and
uncenainty. The ability to innitute systematic decision-making continuous improvement,
experimentation with new ideas, and tearnwork are cntical elements of the TQM philosophy
that have been found to be important for a successful RBrD work environment (Gamin,
1993, Kiella and Golhar, 1997). This area of research however has been found to be weak
as, for the most part, the lirerature deals with the manufamiring and service sectors.
Due to the l w tangibildy and repetiuon found in R m &pmnents, manufamiring-based

quality practices when applied to R&D my prove dislsrrorrr (Kumar and Boyle, 2001).
There are several issues, specific to an R&D organisation, t&atshoulcf be considered.

Individuai achievement,as opposeci to teamwork, has been recognised and awarded in
the R&D envkonments fostering a very inciividualiswc workethic.
The goals and objectives of the R&D funaion are often emphasised over the o v d
company goals, leadhg to a system rhat is not f d y conducive to the achievement of
these organisational goals.
In todn/s competitive marketplace, companies are faced with the prospect of

introducing new products at a fnghtening Pace in order to survive.
In a shon-sighted effort to remain competitive, companies reduce the number of
resources avaiiable. The R&D department is often the fim target. The expectation is
that researchers will continue to perform at the same level with fewer resources.
W1th increased cornpetition, organisations must recognise that R8rD/new
product
development is one of the moa important sources of competitive advanrage for a f m

aayawama and Pearson, 200 1).
The emergence of new technology gready impacts the R8d) environment. R%D mus
view technology and scientific advances as tools to assin the organisation in meeting
company goals (McLaughhn, 1995).
Its responsiveness to and influence on changing customer needs musr drive the R8rD
organisation.
The company used as a case study (case mi+ organisation) for the research in this thesis is
very mu& f a e d on its research and development efforts to maintain its competirive
position in the marketplace. Many of the issues lined above apply to the R&D departments
in this organisation. The n a section provides an overview of the case study organisation.
1.3
Case Study Organisation (CSO)
The companyused for this work, which we wiU caiI the case study organisation (CSO)is a
Cinadian hrgh technology company which desigm, manufactures and markets eleamnic
components, p@
&con integrated circuits (ICs) and thick-filmhybrid circuits, for
specialised applications.
The organisation has rwo faciliaes with total plant space of 70,000square feet, encompassing
all aspects of integrated circuit development, muiufacnviag and marketing. Revenue
exceeds f 107 million per year. The CS0 employs ahost 500 people.
The company bas recendy reorganised into rhree divisions. The divisions are based on
produa families and each indu&s marketin&sales, R&D, and manufacturing functions.
The CSO recognises that in today's fast-changing economy; initiative and ingenuity are
essenal elements of success. In 2000, it renewed its focus on RSrD, ramping up spending
to approximately 17 percent of d e s .
The organisation is dedicated to quality. To ennue hi& quality produas and services, the
organisation's Quality Program is registered to ISO-9001-94 and is constantiy evaluated
through interna and extemal audits. The quai~tysystem is currendy being updated to meet
the ISO 900 1: 2000 standard for quality management.
W
e the quality synem is being rnaitained in accordance with the ISO 9001-94 standard,
the CS0 is aware that these efforts are no longer adequate to maintain their cornpetitive
advantage. The company needs to continue its quality journey and expand its continuous
improvement efforts. Several National Quality A
C S 0 into this nez phase of excellence.
. suggest cnteria that should move the
Organisation of the Thesis
1.4
The remainder of this thesis is organised as follows.

Chapter Two is a nwqr of existing iitennire to develop a background for the problems
addressed in this thesis. These top& indude:

Issues Uniqye to R&D
ISO 9001 in R&D
It dso indudes an o v e ~ e w
of the motivation for and objectives of the research in this
thesis.
Chapter Three describes the development of an integrated model for performance
management based on the ISO 9001: 2000 Standard for QwLty Management and severai
business exceilence models.
Chapter Four subsequendy reports on a pilot self-assessrnent performed in a business
division at the case study organisation. This self-assessment was conducteci against the
proposed model for performance management.
Chapter Five compares diree me&& of perfomance measurement used by the CSO.
These include the self-assessment, the p h t y audithg prognm, and the result-based
performance measurement system.

Chapter Six describes the modification of the OPIM to be more appropriate for use in an
R&D environment.
Chapter Seven condudes the thesis with a nimmary of r e d a and convibutions of the work.
The scope for htrther research is also discussed.
2.0 Literature Review

2.1
Introduction
A nwqr of existing literature has been conducted to review the foliowing aspects of
performance management:
the main elemenu of performance management iaduding mewrement and
improvemenr;
existing frarneworks and modeis for organisationai performance management;
the integdon of the minimum requirements for a quality management synem with die
cntexia frorn the more holinic national @ty
awards;
three organisational performance measurement methob: self-assessment, audiring, and

redt-based performance measurement;
performance management in a research and devdopment environment;
Success of an organisation's business depends on its ability to set svafegy and objectives and
then plan for the resources and activities necessary to achieve them To ensure that these
pluuied activities acnially occur and lead to the fulfhent of the set objectives, the
organisation needs to msnage and improve its performance. Perfomiance management is
"the process by which the Company manages its performance in line wirh irs corponte and
functional strategies and objectivesn (Bititci et al., 1997). Performance management is
essentiai to the successful operation of the enterprise. It has become a key business process.
The performance management process is a dosed loop deployment and feedback ryaem
that sans with the company's vision, broken down into business objectives and strategic
goals. Action plans are created based on the achievement of these goals and objectives. To
monitor the progress towuds the accompiishment of these goals and objectives,
performance measurement is used to collect information, indicating where the nirrent
performance gaps exist. This information is then fed into the performance improvement
process, which narrows the perfomunce gaps and then feeds information back into the
process. In SV,
the main elements of performance management are:
establishment of goals and objectives

pertonnance measurement
performance Unprovernent
These interdependent processes, hctioning hannoniousiy, usix~gvarious resources to
achieve goals related to the effiaency and effeaiveness of organisational processes, make up
the performance management system. (Karapetrovic and Wi11born, 1998A)

There are numerous rnodels for performance management. The next section reviews several
of these.
2.2.1
Models for Performance Management
This section outlines the frameworks and models used by organisations today to design,
implement, maintain, and improve their performance management systems. These range
from models of the basic requirements for a quaRy management system such as the ISO
9000 standard for q d t y management to more holistic models for business excellence. The
latter includes the Malcolm Baldnge National Q d t y Awad, the European Q d r y Award,
and the Deming Prize.
2.2.1.1
ISO 9000:2000 Standard for Quality Management
Customen in today's marketplace require assurance of the quaiity of the products or services
that they are buying. To provide this assurance, organisations look to the ISO 9000 series of
standards for q d c y management to provide guiciance in the planning, implementation and
maintenance of an appropriate quality system.
A @ty
system is a set of interdependent processes tbat h & o n hamoniody, using
various resources, to achieve the objectives relateci to quality (Karapeuovic and Wiibom,
1998A). The quai~tyqmem is a sub-system of the business system of an organisation.
The ISO 9000 senes of standards was developed in 1987 as a set of models that stipdated
the requirements upon which an e x t e d pany could assess an organisation's quality system.
The senes consisted of guidance nandvds and conformance standards.
The guidance
standards ISO 8402, ISO 9000, and ISO 9004, provideci interpretations for using the
conforniance standards. The conformance standards ISO 9001, ISO 9002, and ISO 9003
represented three distinct f o m of funaional capability suitable for extemai qyality
assurance purposes. Companies applied the applicable standard to their organisation to
assure their customers of the @ty
of their products and senices and to make their quality
efforts more uaasparent.

The original ISO 9000 senes was revised in 1994. The ISO 9001 standard in this version of
the model consisted of 20 elements that addressed all aspects of a q d t y srjem. This
sysem induded the organisation, allocation of responsibilities, procedures, processes, and
resources that joindy led to the provision of goods and seMces in accordance with the firrn's
quality policy and objectives (PG,1995).
The standards emphasised the asniance of quaiity of products and services through:
clarification of management poiicies and cornmiunent
identifilcation of roles, responsibility and authority (Elmuti, 1996)
establishment of clear instructions to ali personnel affecting qualiry
development of precise procedures and instructions in al1 areas of operationai activity to
e n w e consistenq and uniformity (kadi et ai., 1996)
This model has been used extensively by organisations 1over the world since its incepuon.
Today,over 400,000 companies world-wide are registered. There have been however,
severai criticisms made of the model:
tao much emphvis is piaced on documentation, creating bureauctacy in convol
rnechanisms and bodenecks when documents need to be c h g e d or updated
(Karapeuovk, 1999; Dale, 1999)

la& of emphasis on impmvement (Chin et al., 1995; Pun, 1998; Najmi and Kehoe, 2000)
la& of a suong focus on nistomer satisfaction @odinson, 1991)

frequendy mistaken as a guarantee for produa quahty (Bum, 1990)
In December 20d0, the ISO 9000 senes was updated for the second time. The 1994
version, which emphasised +ty
assurance, has now been replaced by ISO 9001:2000 that
was developed to assis organisations implement and operate effective quality management
systems. The sUa;ty asnuance standard provided confidence that the requirements for
quaiity will be met (Karapetrovic and Wilibom, 1998A) through quality planning and
conuol. The new series adds the process of qwLty impmvement into a broader model
emphasising quality management. Q d t y management is the combination of quality
planning, q d t y conuol, and quaiity impmvement.
This new emphasis on improvement
renilted in a section of the ISO 9001 standard devoted to measurement, an+
and
improvemenr processes.
The changes manifest themselves in the underiymg p ~ c i p l e of
s the model. Ir is based on
eight q d t y management principles.
1. Customer focused organisation
2. Leadership
3. Involvement of people
4. Process approach
S. System approach to management

6. Continual impmvement
7. F a d approach to decision making
8. M u e beneficial supplier relationships
By connecting the prinaples of qu+
management to organisational processes, the 2000
version of ISO 9 provides a greater orientation tawvds continual improvement and

customer satisfaction, thus widening the scope and the use of the standard
The ISO 9000 senes now consists of a pair of quality management system standards: ISO
9001 and ISO 9004.
ISOSM)I
ISO 9001 s p d e s requiremerits for a qu?lity managrment system that c m be used for infernal application by
oqpnktions, or for c d c a t i o n , or for contracnui purposes. It focuses on the effectiveness of the qu;Jity
management system in meeting customer qukments. (ISO9004:2000)
ISO 9004 gives guidance on a wider range of objectives of a qu;ility management system than does ISO 9001,
particulariy for the continuai impmvement of an organisation's o v d performance and efficiency,as wd as its
effectiveness. ISO 9004 is recommended as a guide for o ~ a t i o n whose
s
top management wishes to move
beyond the requirements of ISO 9001 in punuit of continual impmvement of performance. However, it is not
intendecl for certification or for c o n d purposes. (ISO 9004:2000)
These IWO standards can be used alone or a complementaxy models for an organisation's
quality management system ISO 9001 sets the essenrial requirernents and ISO 9004 shows
the way for future developments. To maintain registration status, an organisation m u s
perform intemal and extemal audits against the ISO 9001 model to show that it has met
these requirements. It can however, enhance the audit process by performing selfassessments againn the ISO 9004 model. This process can help identify strengths and areas
for irnprovement in the quaiity management system in texms of its efficiency as well as its
effectiveness. If the issues identified in the self-assessrnent are addresseci, the results of the
next quality audit should show irnprovement.
While the ISO 9004 standard is a model for improving the efficiency and effectiveness of the
quality management system, other models have been k e l o p e d that address an even broader
set of organisationai activities. These are the National QuaLty Awarb or Business
Excellence Models (BEM) as they are often referred to.
2.2.1.2
National Quality Awardc
QuaLty awards are modeis of business excellence consking of uitexia to whidi

organisations cm compare their activities and resuits. This compvison process, r e f e d to
as self-assessment, involves the regular assessrnent and monitoring of organisational
activiries (Dale and Bunney, 1999). hstead of cornpliance with the criteria of a business
excellence model (BEM), these assessments idenrify an organisation's suengrhs, weaknesses,
and oppormnities for impravement in each area of the criteria.

Three of the most pop& q d t y awards are the Malcolm Baldnge National Qua;ty Award,
the European Qua;ty Award and the Deming Prize. Each of these a w d has its own
focus, famework, mteria and scoring method
This award is named for Malcolm Baldrige, who served as Secretary of Commerce from
198 1 und 1987. His managerial excellence conuibuted to long-term irnprovement in
efficiency and effectiveness of goverment (NIST, 2001)

Organisations that aim at k i n g world-class focus on insrilling core values such as good
leadership, amorner focus, respect for employees, and continuous improvement
(Pannirselvam and Ferguson, 2001). These form the basis of the MBNQAaiteria. The
model embodies these essential values in a framework of seven main categones.
1. Leadership
2. Strategic P l k g
3. Customer and Market Focus

4. Information and Anaiysis
5. Human Resource Focus

6. Process Management
7. Business Results
There are two key assumptions thax devinthe modeL The fim is that the leadership
provided by top management is the primvy driver of the business. Top management m e s
organisational values and goals and direcrs orp.aiutional improvement. The second is that
the basic goal of the slUi;ty process is the delivery of mer-impruvingq d t y and d u e to
customers (Malcolm Baldnge National Quatty Awad, 2000). Figure 2-1 provides the
framework comecting and integrating the categories.
Information and Adysis
Organisations are evduated against a given set of criteria and a maximum total of 1000
points are allocated among seven award criteria Each critenon curies a different number of
points in accordance with its relative value within the a w d
The European Quality Award (EQM)was introduced in 1991 by the European Foundation
for Quality Management (EFQM).
The EFQM Excellence moQl is based on a set of fundamental concepts that formulare the
definition of excellence (European Quality Award, 1999). These are:
results orientation
amorner focus
leadership and consistency of purpose
management by processes and facrs

people dwelopment & involvement
continuous leaming innovation & improvement
partnership development
public responsibility
The mode1 incorporates these Fundamend Concepts in a set of nine criteria; five of which
are 'enablers', four of which are 'redts'. The 'enablers' criteria address what an organisation
does while the 'renilrs' cover what an organiution achieves.
Figure 2-2 ilhwates the model
with the 'enablers' and 'results'.
Enablers
Results
People
Leadership
Processes
u
=
L
J
Key
Performance
Redts
& Resources
Innovation and Leaming
organisations are evaluated agaian a given set of criteria and a maximum total of 1000
points are allocated among nine awd criteria. Each mterion carries ii different number of
points in accordance 4 t h its relative value withiu the a w d
The Deming Prize was established, in honour of Dr.W. Edwards Deming, by the Board of
Directon of the Japanese Union of Saentists and Engineers W E ) in 1951. Dr. Deming,
one of the fathen of quaiity, is best known for his work in Japan, d e r e from 1950 onwad,
he taught top management and engineen' methob for the management of qualiry. The
impact of Dr.Deming's teachings on American manufacturing and semice organisations has
been profound
The orignal objective of this prize was to assess a company's use and application of
statisticd method (Dale and Bunney, 1999).
The prize consins of a cheddisr with ten major categories. These are:
1. Policies
2. The organisation and its operation

3. Assembhg and disserninating information
4. Standardisation
5. Human resources
6. Quaiiry asnirance
7. Mallitenance
8. Improvement
9. Effects
10. Future plans

Each pricnary category has six sub-categories (except for q d t y assurance, which has
twelve). The cheddist expiiady identifies the factors and procedures that constitute the
Company-Wide Q d t y Convol and Totai QwLty Control (CWQCrTQC) process.
Y inherendy presuiptive.
Because it names specific techniques and appm&es, the ch&
There are no pre-designateci points allocarad to the individd sukategories.

The Dmimg PNc ir not b
d on an udaiying fmmework linking concepts, aaiwies, processes and

it does not assume an undcrtying causality. It si.& ~rovidesa lisr of
resuits together. Furthemiore!,
desirable or good gu;ility-orietedmanagement pncticcs. Its focus is on poliaes and plans;

implement;iOon of p h ; information collection, a d + ;mdcontrol; resulrs flowing frorn the
implemenmtion of poliaes and thtir effects; and fuwc improverncnt p h .
(Ghobadian and Woo, 1994)
2.2.1.3
Comparing the Models
Each of these rnodels har its own focus, scope, approach to evaluarion, and improvement
strateW. Table 2-1 illusvates these major differences.
ISO 9000
Conformanu (CO
p r o c h and CO
speafi&ons)
F
Responsibilityfor
Qu&y assurance
spnwlists and top
MBNQA
Iniprovan~~t
(of
rmthods and of
o v e d business
du)
ExeCUfive
management
Idmufication of
conformance and non- nruigths and
conformances, resulting oppominiria for
in a pasdfd verdia
improvemcnt, and
scom i n d i h g the
m;inuity of the
managunmt v a n
Design and production

processes, and dimdyassociateci support
activities
nie entire
management synem,
includlig ludcrstiip,
pluuung and the use
of infornucion and
meuurunuits, the
dorelopmenr and
management of
people, the mcrhods
used for
understanding
clrstomcr and market
rtquirrmenrs,and the
management of kcy
inuiness proceSKs.
EQA
lmprovanenc (of
mcthods a d of
o v d business
Deming Rizc
Statisud Mcrhods
(prevaition of
@ty
~mblm)
rdts)
ExCCutive
Executive
svcngths uid
oppomuiitics for
improvanent, and
scores indiaring the
cnanuity of the
managemcnt syscem
The entire
managanent system,
induding leadership,
people, policy and
nntegv, puuiuships,
mourccs, proceses,
and the management
of key business
processes
existence of
Identification of the
-ch
dement through an
auditing process
Process orientcd
nrong emphasis on
proces
management
Giteria confmed to
the application of
p r o ~ &,
e~
statisticd methoch
and q d t y c i d a
Fnmcwodc
ISO 9000
I Basecion the vroccss
MBNQA
B a 4 on an
undcrjuig fnmework
linlunp conceps,
mkkks, processes
and rrnihs togerher
DrmingPrizt
EQA
Based on an
Does not u n i m e
mdatying
fnmervorklinlung
concepts, &es,
proceses and re&
togtthff
anun-
udty,umpIy
provides a list of
&sirable or good
quality-oricnted
management
pncrices
( G h o b a b and
Woo, 1996)
nonconformanccsand
to pxwent m>ccurrrnce
of pmblans
CoaunlKnlS
ilxpvmmfuung
Quality
lmpmvemcnt
CHECK-ACI' cyde.
conml
PIAN-DO-
thrwghdd
Appendix D provides a detailed cornpuison of the elemems covered by the ISO 9001
standard,the MBNQA, the EQA and the Deming Prize, in each of their cntena Table 2-2
presents an overview of the svengths and weaknesses of each model.
Model
1-
9001
MBNQA
we;ikncsscs
Stmlgths
M i n focus is on the processes wnhia the quality
Weak in the area of humui
management system
Quaiity system model is prescriptive and therefore
the way forward is clear and unambiguous
End point (achieving registration) is weii defineci
and externay valiated (Najmi and Kehoe, 2000)
Constinites an integrai pan of an organisation's
o v e d management systern (Pun et ai, 1999)
Addresses the identification and provision of
resources with a focus on idkanmm, the work
environment and the conml of documents
Forces an organisation to describe its key
processes and make thern more transparent (van
der Wde et al., 2 0 )
Broad model addressing an organisation's o v e d
activities, not oniy those relateci to the quality
management system
Asks how the organisation meeu the criteria
Focused on cunomer management
Provides ody the minimum

requirements for the quaiity
management system
Does not specificafy address
technology or financiai
resources
Primanlyadress iimited areas
of the enabler si& of the BEM's
Little emphasis on areas that
directly impact on business or
organisational effectiveness
(Poner and Tanner, 1996)
Process criteria goes into less
depth than the ISO 9001
stanard
Does not spetlftcalty ddress
the use of resources in its
cnteria
stmlgths
Weaknesscs
B d modei ;iddrcssingan orginisuion's o v d
Process critaia ~ o e into

s less
activitics, not o d y those r c i a d to the qu;ility

management sysrem
Asks h m the o q p i m i o n meets the criteria
Addresses the efficient and effective use of
resourres within the o ~ s i o n
Focused on the relations with the cornmunity, and
customer's and ernployee's satisfaaion
dephrbrnthefs09001
stvldvd
Focused on q d t y control
Focuseci on compuiy wide qu?lay efforts,
conMwus improvement
Provides a List of desirable or good qu;ilisronented management pnctices
Does not &as

how the
orguiis;ition manages customer
rehionships and satisfacrionor
how it contribuus to socieiy
and miaimises environmentai
impact
G t e r i a are confined to the
application of process an&&,

mtisticd methob and q d t y
d e s
Crimia such as Company poiicy
and p h m g d t s , or future
p h are pconcenieci
with qulity wmce activiries
and q d t y resuits
Not b d o n a n u n d e r j i n g
h e w o r k linking concepts,
aaivities, processes and d f s
t ogether
The major points arisiig from this cornparison process are:

The ISO 9000 standard provides oniy the minimum requkmeno for the q d t y
management sysrem but is seen as the foundation of an organisation's total &ty
efforts.
The MBNQA is a broad model addressing an organisation's overaii activities, not o d y
those related to the quality management system. It does not specifiulh/ address the
organisation's use of resoumes.
T h e EQA is also a broad model for business excelience. Ir does not address customer
management in the same detail as the MBNQA.
The Deming PNe is a model focused on quality conaol. The criteria is not as broad as
those in the MBNQA or the EQA but provides some important pieces conceming the
use of quaiity control in a company's continuous impmvement efforts.
Table 2-3 provides a cornparison of the major categories addressed by Bdi of the qudity
award models as weli as th& respective point values.
MBNQA
Ladership
12.5
Deming R u e
(%)
Leadership
10
'
1
Orpiution and iu
Strategic
8.5
Policy and Stntegy
People
Partnenhips and
Planning
Customer and
Market Focus
Information and
Human
8.5
Resource Focus
Assembling and
Resources
Process
Manaeement
j
f
8.5
7
Business Results
Processes
14
information
Qu;ility-=
Miatenuice
Standardisation
10
10
Custorner
Resuits
society
People resuits
Resulu
Key Perfomiance
Results
Irnprovement
Future Plans
Ir is evident that each of the awards places emphasis on different areas of their criteria. Both
the EQA and the MBNQA however, assign the most points to the Results sections of their
model. The Deming Prize does not single this category out as being the most important in
rems of its scoring xheme (each is valwd at 10%). The MBNQA assigns 12.5% of its
points to Leadership and distributes the ren of its points among Strategic Planning,
Cusromer and Market Focus, Infornuon and Analysis, and Human Resource Focus
assigning 8.5% for each. The EQA however, w i p o d y 10% to Leadership d
e givhg
Processes 14%. The People category is worth 9%, o d y slghrh/ higher thao in the MBNQA.
This cornparison process has rwealed that each of the performance management madels
have muiy nrengths. Each one could compensate for aaother's weaknesses. The next
section examines, in further detail, the research that has been done conceming their
2.2.1.4
Integration of ISO 9001 and the Business Exceilence Models
The ISO 9001 standard for quality management is seen by many as a requed foundation for
a company's @ry and performance improvement efforts (van der Weile et al., 1997, Dde,
1999, Karapmvic and Willborn, 20O1B, Russell, 2000, Meegan and Taylor, 1997). For
exampie, van der Weile et al (1997) make the following comment:
Continuous improvment ody m;ikes sense if an organisation h m what is going on in reiation to
the processes, which are undedying the things that need to be improved. The ISO 9000 series forces
an organisation to desaibe the key processes and make them more transparent.
Once the requirements for rhis standard have been met however, companies need to b d d
on these foundations to extend the principles learned to ail organisational activities and not
This means irnproving a

quality synern beyond the level of simply meeting spec&cations to creation of the proper
just to the activities relating to the quaiity management *en
conditions for business excellence.

The synem required for accreditationWUoutline the organisationai structure, responsibilities,
procedures, processes, and resources needed to knplement quaiity management. The m i t i o n zone
[beyond ISO 90001 begins then at the intersection b e e n the operationai and management
infrastnrctures (Meegan, 1997)
The transition from quaiity management to business excellence need not involve leaving ISO
9000 behind to move on to somRhing supenor (Meegan, 1997). To facilitate the movement
from ISO 9001 quaiity management to the o v e d business excelience embodied in the
quality awards, researdien niggest that these models be integrated (Mann and Voss, 2000,
Tonk, 2000, Dale, 1999, van der Wkle et al., 2000, Pun et al., 1999).
Pun et al. (1999), describe a self-assessed qual~tymanagement sysrem using the framework of
the Baldnge M
d forthe integrarion of ISO 9000: 1994, ISO 14000, and the Baldrige
criteria. In a case stuciy, the assessrnent against rhis f r a m e d d l consisted of the regukr
auditkg and performance assessments performed by the organisation.
When there is a co11ection of afksystems or stanfiam different ueu, efficiency is difficult
and misconcepuoas and coafurion often occur. It is &us ben to elimbae possible confusion and
sub-optimisation from the outset by establishing and maintaininga t o d systems approach.
(Puri et 4 1999)
Finai conclusions from Sun (2000), suggea that TQM and ISO 9000 srandards musr be
completely and systematicallyimplemented and i n t e g r a d Researchers must detemine a
method of c o m b i i g ISO 9000 and TQM, and bridging the gap bnween the two.
Mann and Voss (2000) present an innovative approach used by one company IO integrare its
ISO 9000-certified management system with the Baldrige model. The company concemed
developed its ISO 9000 qmem to address all elements of the Baldnge criteria. Of parti&
note is its process improvement approach that prioritises improvemenr projects based on
their expected impact on the company's Baldnge score.

Companies, who are ar present working for an "integrated quality management sysremw
where ISO is a major contributor, go beyond the requirements of the standard ro M e r
irnprove the performance of the organisation in various dimensions. The ISO standard is
thus the comerstone of
TQM and, as such, presents both oppotrunities and challenges.
The challenges involved in integrating ISO 9000 and TQM are well reviewed in the
literantre. We have seen that ISO 9000 has been advocated as the fxst step towards total
quality management. The ISO 9000 series and the excellence models clearly have different
goals and perspectives and define total quai~tymanagement at different levels of maturity
(van der Wiele er ai., 2000). It is obvious then that the uiteria from the ISO 9000 series
cannot simply be incorporateci into a model for business excelience. The criteria represented
in the reference models are different, ISO 9000 representing the minimum requirements and
the quaiity award models k i n g more holisric paie, 1999). If rhis were to o c w , it would
lose much of its potency in the building of the foundaton of an organisation's TQM
activities (Dale, 1999).
The quaiity a
d models and the ISO 9001 standard use self-wessment
and quality
auditing respectively for evaluation of rhe otganisauon against them. To integrate the
models would require the integration of these measurement methods. Karapetrovic and
Wibom (2001B)explore the integration of quality audits and seif-assessment processes.

Another difficulty in integrating the ISO 9000and the @ty
award modeis is the different
'scoring" schernes used by eadi. For example, when Tonk () describes the integration
of the MBNQA and the ISO 9000 system, he suggesu:
The Bddrige scoring system uses percenrages, w h m s the ISO 9000 systern uses, in effect, a pus/fd
(conform;ince/nonconformuice)system Decisions need to be made as to whether to retain the
MBNQA percenage scoring sysrem and if so what scoring levels will be reqwred in each of the
uiterion in the new model.
2.2.2
Performance Measurement
Performance rneaswement is the process of coileccing data pertaining to al1 aspects of the
company's activiues. This data can then be used to help control and correct waning
performance areas and to set new targets for improved performance.
According to Sinclair and Zain (1999, rhis meanvernent is exuemely important to
organisations for several reasons:
Meanires can be analysed to enable appropriate decision making.
Measurement provides feedback and reinforces behaviour.

Measurement is an integral part of continuous impmvement; "if yai m o t meanve an
activity, you cannot irnprove it".
Measwement "helps an organisation direct its scarce resources to the mon aMcnve
impmvement opportunities*.
It has been shown that perfo~llil~lce

improves if individuas are givea targets, and is
.
.
maxunised if targets are seen as cbillenging but achievable.
Measurement is impoxtant in the maintenance of longterm focus.
Measurement aiso plays an important role in quality and producOvYy improvement
(Oadand, 1993). It:

ensures that customer requirrments have been met
provides standards for establishiog cornparisons

provides visibility and a scoreboard for ~eopleto monitor th& own performance levels
highlighrs quality problems and deremines which areas require priority attention
gives an indication of the costs of poor q d t y
justifies the use of resources
provides feedback for driving the improvement effort
Each of the National QluLty Awub provides guidelines for perfonning organisational selfassessments agalin the given set of criteria for business excellence. MBNQA provides a
Guide to Self-assessrnent and Action. The EQA provides detailed guidance to organisations
and suggests a self-assessrnent approach depending on the quaiity rnaturity of the
organisation.
These organisations usually perform self-assessrnent in one of rhree ways (van der Weile et
a., 1999):
taking the a
d guideha and foiiowing these for intemal use in the sarne way as
prescribed in the respective guidance booklets for those organisations who are putting
forward a formal application for the chosen award
adjusting the cnteria and/or the scoring to fit the specifc situation and goals of the
organisation
speQfying the parh and seps to be t&n h m the present situation ro the end goal.
Conti (1993) suggescs that organisations should use already existing data in the assessment
process and as additional information for the goal setting process. Some examples of this
data are:
cunomer survgs, amorner cornplaint statistics, marketing strategies
employee wqs, work safety statistics, illaess rates, absenteeism and fluctuation,
human resources development plans

processes e.g. processing Ume or process liability, ~rojectsand gained improvements
producr e.g. e m r rate. life duration and failure rates

fmancial e.g. trends conceming profits, sales volume, renim on investment (ROI),qwlity
coas &ink and Schmidt, 1998)
2.2.2.1.1
Self-AssessrnentProcess
The self-assessrnentprocess usually follows the Deming PLAN-DO-CHECK-ACTqde.

Figure 2-3 has been adopted from the EFQM (1995). It provides an o v e ~ m
of the selfassessment process.
Wbde the unciersiag pmcess re&

the EQA each has its own m
the same, the ISO 9004 standard, the MBNQA and

a for conducOng the self-assessment.
The assessment method suggested in the ISO 9004 standard involves the evaluation of the
mamrity of the @ty
management synem for each major dause in ISO 9004 on a scale
ranghg from 1 (no formal system) to 5 (best-inclas performance). The standard provides
exvoples of t y p i d questions that cm be asked duriag the self-assessment. It is suggested
that self-assessment can be used in a flexible way according to the needs of the organisation.
Two recommended m d o d s are to:
perfoxm the self-assessrnent on an individual basis for all or part of the quality
management system and then to pursue improvement.
have a cross-function group of people perfonn self-assessment on ail or part of the
quality management system, followed by group review and an*,

building,to detemilie improvement prioriries and action plans.
then consensus
MBNQA
The Baldrige National Qwltty Progra.provides a Gui& to Self-Assesunent and Action.
This document ourlines ten seps ro conducting a self-assessment.
1. Idenufy the boundaries of the organisation to be assessed.
-7
2. Decide on a format for the assessment and action plan.
3. Write the business/organisaon oveMew
4. Select seven champions and teams, one for each criteria categoty
5. Practice self-assessrnent techniques with the seven category

6. Ctegory teams prepare a response for their assigned teams
7. Share responses among te-
and fiiulise the fmdings
8. Assess the organisation's strengths and o p p o d t i e s for improvement

I
9. Develop and implement an action plan for improvement

10. Evaluate and improve the self-assessrnent and action process
-1
JLl
.~m
EQA
According to the EFQM,selecton among assesmient methods should be dependent on the
quality ma-
of the organisation. Table 2-4 presenu self-wessment approaches
correspondmg to the maturity of the oqpnkation and the desired effort level.
Award simulation
Pilot a w d simulation
Facihateci workshop
Pro-formaand wofkshop
Verv detaleci questionnaire
This approach involves the creation of a questionnaire used to gather infomtion about the
perceptions of people within the organisation. Some questionnaires can consisr of simple
y e h o questions. Othen provide a set of possible a m e r s nich as A-Fullyachiwed, BConsiderable progress, C-Some progress, or D-Not started. These questionnaires are
dinnbuted and used alone, or as nippon for more elaborate self-assessment approaches.
In this approach, an achievement ma& is developed For each category of the award,
statements perraliing to possible achievernents by the organisation are set on a scaie from 1
(minimum achievement) to 10 (greatest achievement).
This is used as a basis for
management discussion and sening objectives for improvement.
Pro-forma
Here pro-formas are created for each sub-category. The description of the critexia in the
sub-category is printed at the top of the page with areas to address beneath it. The
assessrnent entails f&g
pro-forma itself.
in suengths, areas for improvement, and evidence collected on the
Woikshor>
The management team is responsible for gathering dam from their unit and presenting the
results to their p e r s in a workshop environment.
Thic appmach is effectivelya simulation of entering for the award. The unit undenaking the
self-assessrnentwntes a full submission document. A team of trained assessors then
evduates this report.
The EFQM dso indudes in the EQA document, a figure that i i l m t e d the relationship
between the rigour of the different approaches and the quaiity of data captured. (See Figure
Qrnhuti
c3
Pro-f o n
Process
Rigour
<workrhop>
4
Low
Based on Opinion
Table 2-5 presents the evduation and scoring methods used by ach modeL
Mode1
ISO 9004
Evaluation and Scoring

Evalu?tes the m;uunty of the qu;rlity managtznenr
Details
1
system for cach major ciause in ISO 9009 on a s d e

rianging from 1 (no formai system) to 5 (best-inlus
performuice).
No systematic approach
cvident, no results, poor
d t s or unpredimbIe
d t s
Problem or systemaric
appfo;idr; minimum data
on improvement resuits
~ a&le
S y s t d c pmess-bued
appmpC4 Fufy =%e of
system~tlcrmprovement ,
&ta avaiiable on
c o n f o m c e to
objectives and existence
of improvement vends
hpmvement process in
use; good resuits and
susuineci improvement
I
I trends
5
Strongiy integrated
impmvement process;
best-in&s results
dernonstrateci
Evaiuates each element in the a d criteria bved on
the appmach, the deplayment of appmch, and resuits See Table 2-6for details
re?lised These are evaiuated on a d e from 0% to
100%.
Evaluates each elexnent in the award criteria based on
the approach, the depIoyment of approach, the resuits
r e d i d , and the assessrnent and review (RADAR) of
See Table 2-7 for details.
these elemenu. These are evaiuated on a s d e fkm
OYo to 100%.
Cfiecklist is evaiuatd Each category is worth the same.
I
Table 2-6 desaibes the evaluation dimensions used by the EQA.
Elernmt is cvaiuatcd based on

Appmpri?teness of the methods to the
rrgUiranmfs
Effectivcness of the use of the methods
Dcsaiptim
Mallod(s) d to
;iddress item
requvcmcnu
Embodies evduation, improvemendleaming

1s b d on diable inforrmtion and &ta
Alignmeat with o ~ t i o needs
d
Eviduice of innovation
Exrem COwhich
a p p r d is applied to
4the
Use of the approach in addressing item
requtements relevant to your organisation

Use of the a p p r d by 1appropriate work
in
each item
r-Results
OutcomesinachieVing
the purposes of the
item
r
Assessrnent & Review
units
The scope of evidence

of measurernent,
leaming and
Unprovernent activities
CuITentperformance
Performance relative to appropriate
compvisons or bencbmarh
breacith, and importance of ~ o u r
performance impmvements
Remilu me;uurement of the effectiveness of
t h b r o a c h and deplayment is carrieci out
Levniag acvities are used to idenufy and
share best practice and improvement
opportuniues
Gtpn from measurement and leamhg is
a
n
w and useci KIidentify, prioritise, plan
and impIernent improvementr
Table 2-7illusuates the wduation dimensions used by theMBNQA
m a c h and Dcployment
no systematic approach aident; a n d o t a l
Rcds
no resuits or poor results in areas reponed
evfy sages of eveloping trends; some
l
I(
1
eariy stages of a transition from re?cting to

problems to generai improvernent
orientation
major gaps exist in deplayment dut wouid
inhibit progress in ?chi+
the prinury
a sound, systematic a p p d , responsive
to the primry purposes of the item
a faa-based improvement process in place
in key areas; more emphasis is piaced on
improvement than on reaction
no major gaps in deployment, though
someunrmayixineariyrvges
igood
improvement trends d
performance levels reporteci for m;ury to
moa ueu of importince to the company's
kqr business fequirements
no p a m of adverse trends d m poor
performance lweis in areas of importance
to the appliant's key business requinmenu
some trends and/or current perfommce
lwels-evaiuated against relevant
cornparisons d m bencbmuks-show
areas of svengrh and/or good to very good
relative performance levels
a sound, systermtic approach, responsive

to overail purposes of the item
a fact-based irnprovernent process is a key
management toil; dear evihence of
refinmient and improved integrauon as a
r e d t of improvement @es and adysk
approach is well-deploye, with no major
gaps; deployrnent may Vary mong some
areas or work units
current performance is good to excellent in

most key ares of importance to the
company's key business requirements
most improvement trends andlor current
performance lwels are sustained
muiy to rnost trends and/or current
performance levels-evaluated a g a ,
relevant cornparisons and/or benchmuksshow areas of leadership and very good
relative performance lwels
current performance is excelient in most
ares of impoxtance to the company's key
business requirements
excellent improvement mds and/or
sustained excelent performance in most
ares
mong evidence of industry and benchmvk
leadership dernonsvated in many areas
a sound, systematic approach, fully
improvements d m eariy good

performance levels in a few areas
results not r e p o d for mvry to most areas
of importance to the cornpuiy's key
business requirrments
responsive to al1 the requirements of the

item
a very nrong, fut-based improvement
p m & s is a key managem&t tool; nrong
refinement and integrauon - backed by
excellent axmlysis
approach is f d y deployed without any
sipificuit weaknesses or gaps in any ueas
In self-assessment, once the evduation of each element is complete, the cmmhtive scores
are used to calculate a fnalscore. In both the EQA and the MBNQk this is a score out of
l. These IWO points have k e n distributed among the caregories of the award by the
developers of these modeis based on the perceived imponan of each.
Table 2.8 uses the MBNQA categones to illusvate how a final score is calculated. First, each
category is evaluated out of 100% according to the process described previously. Nem, each
score is calculated based on the total poim value of the categoty and the evaluation v h .
For the Leadership category, the evaluation wu 60%, the total point value for the category is
125 so the score for Leadership is 75 (125 x 60%).
C;rtegory
Leadership
Srrategic Phnning
Custorner and Market Focus
Information and A d y m
Human Resource Focus
Process Management
Business Resulfs
Totai
Evduation
Aau;il Score
60%
45%
55%
60%
52%
40%
35%
75
38.25
46.75
51
43.35
34
157.5
370.85
Point Vaiues
M h u m Score
125
85
85
85
85
85
450
IO00
In an initial assessment such as this, mon organisations do not score more than 250 out of
1000 points.
World class perfoxmance would achieve a score of around 700 points, while a
score of 500 is exception@ good (Public Service Office, 2001)
A score of 200 is reporte*
typical for an initiai assessment of an organisation at the
beglining of a Baldxige-based quality effort (McWaid, and Gale, 1995).

It is important to emphasise though, the importance of improvement over the a
achieved in any specific assessment.
d score
Qrulity auditing is another method rhP o e t i o n s un use to meastue their performance.
A quality audit is
A systematic, independent, and documenteci proces for ob-
audit M d m c e , and waluati~it objecvely
to determine the extent to which agreed aiteria ue fulfilled
(ISO 9000: 2000)
Severai elernents are essential to the performa~lceof an effective audit Fint, there is the
audit standard. Standards provide guidance on the prinaples of auditing, the management
of audit programs, the conduct of a&
as weil as the competence of auditors (ISQ 19011:
standard for @ty

management system audi*.
auditkg standard however, ISO 19011, is being drafted to integrate quality and
2002). ISO 10011 is the m
A new
environmental management system au&.

There are three ~ / p e of
s audits that c m be p e r f o m d These are first, second and rhird
parry audits. First puty audits, o t h e W e known as intemal audits, are performed by the
organisation on itself. Second p a q audits are initiated by a customer (die dient) to venfy
that the organisation meas their quality requirernents. Third party audits are refemed to as
exvinsic audits. These are performed by an outside f m calied a registrar and are prompted
by the organisation directiy (Goetsch and Davis, 1998).

Before performing the audit, its scope m m be defmed. This should be based on the
objectives o r information desired byrhe client and should be communicated to the auditee
pnor to the audit (ISO 14010,1995). It is prudent ro do so to ensure that the auditee is
prepared wirh necessary documents or objective evidence to demonstrate cornpliance to the
cnteria being auditeci.
Audit criteria are the set of policies, procedures or requirements used as a reference (ISO
19011,2002). Audit evidence are the records, statements of fact or other information, which
evaluation of the coiiected a d t evidence a g d the audit &ria.
Audits should not be stand-done processes, but part of an overali auditing system (Goetsch
and Davis, 1998; Karapetrovic and Wdbom, 2lA). Wdibom and Cheng (1994) suggew
thar a badine audit (a more cornprehensive and inteasive audit) establishes the starting
effectiveness of the sysrem and t h du+ the eariy p e n d of implementation, regulu audits
should be scheduled at reg& intervals to mainuia the inaeases in effectiveness of the
system. As the system matures and the effectiveness stabilises, audits should be scheduied
less frequendy.
The execution of any audit indudes sweral steps. ISO 19011 prodes a dugram to illustrate
the activities involved in the auditing process (See Figure 2-5)
T
I n i t i b g the audit
Appointhg the audit team leader
Defining objectives, scope and criteria
Determinhg the feasibility of the audit
Estabidmg the audit team
Establishing i n i d contan with the auditee
Conduaing document revicw

Review relevant management system documents,
induding record, and determining their adequacy
1
/
Prcparing for in-site audit actRritics

Preparethe audit p h
Arrigning woik to the audit team
Gmducting on-site wdit actinties

ConducMgopeningmccting
Communication during the audit
Roks and responsibilRies of gui& and observen
Cdlectiag aud verifying informaion
Genvllring audit fin+
Prepuing audits conciusions
Conducting closing mecting
Preparing, approving and distributhgthe audit
rtport
Prepariq the audit repon

Approving and distributhg the audit report
Completing the wdit

RRinuigd~cuments
i the audit
I
...-...---...-...- ---....
Conducting audit foilow-up
The objectives of a quality audit, as outlined by the ISO IO0 11standard, are to:
determine the conformity or nonconformity of the quality system elernents with
specified requirements
determine the effeaiveness of the implemented quality v e m in meeting specified
quality objectives
provide the auditee with an opportunityto improve the @ty
systern
meet regulatory requirements and

permit the listing of the audited organisation's quality system in a regiaer
Audits of management sysrems are invaluable in assuring that a system is perfomhg
effenively and meeting its objectives. It is also a necessaxy component in the continuous
improvement process. The achievemem of continu ou^ improvement can o d y occur if
quality audits are used in a dynamic and proactive rnanner (w;ubom, 1990; Banhelezlly and
Zairi, 1994; and Peters, 1998). If organisation use a u d i a only as a tool to verify
cornpliance widi agreed standYb wthout dieduog the aptness of the standards thernselves
or seeking information about the effectiveness of the q d t y system in meeting objectives
(Beeler, 1999), auditiug will not lead to impmvement.
Result-bued perfomunce measurement is a third m n h d for meaniring performance.

Redt-based measurement system is an integnl pan of an organisation's performance
management system.
It is a means of gathering data to support and CO-orciinarethe process
of making decisions and taking action throughout the organisation (Schallcwyk, 1998)
Organisations, and researchers in the performance management field, have developed many
performance mevurement models. These models range from a balanced set of indicators
that periodicdy show an organisation how it is performing in various areas of the business,
to integrated models that provide real t h e information with respect ro how thqr are
performing cmently and forecasts how thq. mry do in the future. As competitiveness
increases, organisations require measurement synems that provide not onty up to date
performance idonnation but also examine how these meanires interrelate so they can make
more informed decisions. Performance measurement systems should be aligned with
organisationai goals and objectives. This helps to ensure that the organisation is measuring
the efficiency and the effectiveness of the activities that have the greaten effect on the
achievement of these goals.
The set of measures selected by a parcicular organisation should provide m e n t information
penaining to the health of the organisation and also link direcrty to suategic objectives.
Traditional perfomiance meaSuTemenrs systems are based m a d y on f i n a n d performance
measures 6.e. retum on invesunent, renun on sdes, productivity and profit per unit
production) that were once sufficient to s u c c e s s express

~
a compmy's perfomunce.
However, organisations need more than a badrwvd giance at how w d rhey have done over
the past p e n d Lgging metrics, apparent on firunad reports,were the r e d t of past
decisions and are not seen as usefd in operational performance asessrnent (Ghalayuii and
Noble, 1996). An organisation neeck to see how w d ir is doing now, in t e m of con, t h e ,
flexibility and quality. Not oniy do they need to k
m their current perfomimce but also
how thqr CUI improve this perfomiance. Neely (1999) suggests that the mevures in today's
Company need to be:
focused on organisational strategy
provide data on @y,
responsiveness and flexibility
encourage managers to se& improvement continually

provide information on what cunomers want and how cornpetitors are performiag
indicate what would happen in the short r e m and the long term
Selecting a set of measures that will fulfil these requirements can be a daunting task. One
method that researchers have used is to i d e n e uiticai performance areas and
corresponding meanires to coiiea the important data This data is then used to control and
improve performance in these key areas.
It is becoming increasinglyimportant for companies to understand how all areas of their
organisation are perforrning. If properly consvucted, performvv~meanires provide
information to management to enable fact-based decision making. They are also important
for effective communication. Their use increases constructive problern solving, increases
influence, monitors progress, and gives feedback and reinforces behaviour (Sinclairand
ZaLi, 1995A). Performance measuremenr helps an organisation allocate scarce resources to

processes with weaker perfomiance because it highghts areas that require these resources
the most.
If an organisation dwelops performance mevurrs for each process, this may result in a very
large lia of measures. The question is, how does one select a few kqr measures that wili
ensure manageable data collection but yet provide a good indication of the naus of the
organisation's health.
M q authon have trieci to develop guidelines or approaches that will help in this selection
process (Keegan et al., 1989, Fitzgerald et ai., 1991, Kaplan and Norton, 1996, Dixon et al.,
1990, Neeiy et al., 1995, Globenon, 1985). To d
e the contributions of these
authors, performance measures should:

be linked to strategy
stimulate action
support long-terni improvement

emphasise whar is important to the organisation currently
cover ali the appropriate elements of perfonnance, not jus the finanaal ones
not be in conflict with each other
enable cornparison with other organisations in the same business
simple and easy to use
be mainly ratio-based Uistead of an absolute nwnber
be under convol of the evaluated organisational unir
be desiped based on discussions with the people involved (cunomers, employees,
managers)
be m a d y objective
Vary berween locations - one meanire is not suitable for all departmenu or sites
change as circumaances do
provide fan feedback
stimulate continuous improvement rather than simply monitor
Nurnerous performance measurernent frameworks have been developed since the late 1980s.
These framewoh provide information about the types of measures an organisation c m use
to track perfoxmance in a "baiancedmway. This ensures the Company is not just lwking at
its financiai situation as an indicator of its perfomunce but that all elements of performance
are being m
d and m o n i t o d Two of such h e w o & are the Balanced Scorecvd
developed by Kaplan and Norton (1996) and the model from Neeiy et al. (1995), which
consists of four elemenrs of perfoimance.
The Balanced Scorecard anernpts to capture an organisation's performance from four
different perspectives. The first is a f i n a n d perspective, Le. how the organisation looks to
th& shareholden. The second is an intemal business perspective, which asks at what the
organisation should excel. The third looks at how the organisation appean to the cwomers
and the founh perspective is one of innovazion and leaming which asks how the Company
cm Lnprove and create value.

This nructure is baseci on the premise t h to achieve finuiciai success, the dt;matr? purpose of a q
organisation must be to satisfy customers. So orginisltons mut optimise their intemai vaiue-creating
processes. To do so,they mua l m and th& employees must grow in their individual capabilities.
(APQ 2 w
Neeb et al (1999) suggest that an organisation's m a n u f a h g performance measures

should f
d into four categones. These are quality, Ume, cost and flexibility. These cm be
extended to cover the performance aspects of other deparmients as wd. T h e performance
of design and development processes for instance can be viewed from these four
perspectives.
2.2.2.4
Cornparison of Performance Measurement Methods
Three methods of performance meanirement have been presented in this lirerature review.
This section wdl review the work done to compare these methods in terms of their
applicability and usefulness to organisations.
2.2.2.4.1
Auditing and Self-Assessrnent
Auditing and self-assessrnent are systematic approaches used to measure performance in an

organisation. The audits used with regard to ISO 9000 series reginnuon are mady looking
for non-compliance and wess whether the system and the undedying procedures are being
followed. O n the other hanci, self-asessrnent against the a w d modek is focused on
to drive the organisation d o m
fmding improvement opportunities and new wys in WU
the road to business excellence (van der Weile et al., 1995).

Both audits and self-assessmentsshouid be dosed loop processes that foiiow Deming's
PLAN-DO-CHECK-ACTloop. Thk mearis planning for the audit or assessment,

perfomiing the a d t or assessment, reviewing rhe auditing or assessment process for
adequacy and effeheness, and making impmvements to the process as identifed in the
review.
Each of these processes is used as an evaluation method to detemine the aatus of the
current syscem A series of audits can be used to meawe the achiwed improvement of an
organisation's quaIity system but do little to indicate how fast it is going in a chosen direction
of improvement (effuency) (Karapetrovic and Wdbom, 2001B). Self-assessmentsare able
to measure this efficiency through the as-
of current approaches to and deployment of
these approaches satisfying a given set of BEM cnteria. The uialysis should indicate
whether the current approaches are sound and integrated and whether there is systematic
and implemented deployment of these approaches. The remit is the ability to predict
whether these are sufficient to lead to improvement of the system. Thus, the objective of an
audit is to venfy compliance with a set of cnteria and self-assessment is used to examine the
driven for improvement (Karapetrovic, and Wdborn, 2001A).
The r e d t of a quality audit is either compliance or non-cornpliance with the criteria. Either
the organisation meets the criteria or it does not In a self-assessrnent process, the objective
is not to cornply with the cntena but to use it as a frarnework for idendjing svengths and
oppominities for improvement in each area of the BEM cxkeria.
When it cornes to perfonning the audit or assessment, the procedure is sirnilar. Both
processes are designeci around DemLig's Plan-Do-Che&-Actd e .
I1
T1
DO
~dentifyo ~ o x d s u u c n i r and
e mq&d
cornpetence for audit program management
Define audit..pr-..
and respoasibdiaes
objectives, wope, mler
Identifywdit teams
Ennve cornpetence and suitab* of auditors
Cummuniute audit pian
Manage the planning and suitability of auitors
Gxnmuniute audit pian
Manage the piamhg and arecution of
P e r f m folow-uPactions and/or audits
I
I
I
1
1
1
1
Select approach and scope
Establish d-assessrnent teams

Execute w e s sment
Identify r e q d actions
Incorponte actions into p h
Implement pians
perfofxxLlnce
A m
Review and irnprove audit prognm h u g h
feedback and recommendations from the

dent. a&
and auditor
Karapevovic and Wdbm (2001A)compare the principles and practices of qualicy audits
and self-assessments for the purpose of examinlig th& compatibihty and providing the
basis for integration.
The authors suggest severai suategies for increasing the capability and potential aiigriment of
audits and self-assessrnenu. These include:
Integration of reference models. This smtegy may involve a g r a d d conversion from
the basic requirements of ISO 9000 to the all-encompassing criteria of excelience

models, suggesred in literafufe. In that sense, self-assessments incorporate quality audits.
Integration of evduation merhodologies. Qwlity audits w o d be used for assessing
'hard controls", while self-usessments would be used for 'softer" issues and idenufying
improvemenu. In pnctice, this strategy is already applied.
Goss-use. Quaiity audits can be applied to enhance self-assessments, and vice-versa.
Perfo~nafliceManagement in RkD
2.3
A large portion of the quliry problems rhat vise on the manufactuxing floor could have
been avoided completely with proper attention to the design process. Quaiity cannot be
tested into a manufactureci p d u a , but musc be designecl imo the produa. Most of the
focus of quality management research however, &als Mththe manufacniring and s e ~ c e
secton and lacksRSrD focus. The quality management pracrices that are successfd in
rnanufacniring environments are wd known and widely publishd The application of these
manufacnuing-based d t y pnctices in an R8rD environment is not as successful due to
the lads of replication and high level of variability in RBd) projeas (Kumar and Boyle,
2001). There are also high levels of uncextainry in terms of the impact of the produa on the
market and the revenue it wiU generare over the followhg years. It is more difficult to apply
practices in such an environment however, the undedying concepts of quality
@ty
management are essenria to decreasing this variab'ilityin the R&D processes. Research is
needed to examine an R&D operation and idennfy pertinent quality management aspects
that are applicable in nich an environment. To adequately address this problem, there needs
to be an understanding of issues that are specific to an R&D organisation.
2.3.1
Addressing Issues Unique to R&D
Individuai Recognition of Achievement and Organisational Goais vs. R&D

Objectives
EspecialIy in North Amencan industry, the manner in which researchers and scientists in
R&D roles have been recognised and awarded has fostered a very individualistic work ethic.
Researchers often prosper by phcipating in a system that rewards individual achievement
and discourages alliance building and CO-operation(McLaughlui, 1995). This individual
recognition crares an amiosphere plagued with individds working toward goals that will
move hem ahead without due consideration for the overall goals of the RgrD funaion itself.
This is counterpfoductive to the achievement of these goais. Individual performance and

mards need to be replaced with systems that reward team accomplishments.
As individuah are rewarded for their personal contributions to R&D activities, the
environment becomes one that encourages the achievemat of persona1 goals

The (R&D) environment is academic and contributions to and support of o%;iniszuonai goals and
objectives are minimai. In fact, mon researchen are unrwve of w h t the o ~ u o goah
d and
objectives are.
(Swyt, 1999)
It is exuemely important that r e s e d organisations ensure they are focused on projects that
address the saaregic needs of the organisation and that rhese needs are being met effuendy.
Fast Pace and Factors that Increase Cycle Tirne

In today's competitive marketplace, companies are faced dthe prospect of introducing
new products at a fnghtening pace. The ability to reduce one's cyde rime in product
development is viewed as the key to innovation nuicess and profitability (Cooper and
Kleinschmidt, 1994). The r e d t is a need for efficient and effective new produa
development (NPD)processes that will ailow the Company to mainuin its competitiveness
within its indunry. What R&D facilities are facing now ody increases the pressure while
acting as a barxier sometimes to the achievement of good processes and competitive outputs.
There is a need for a systematic approach to irnproving design and development processes,
and the management systems surrounding hem, to provide the bea environment for
&taking
cornpeutive advanrage.
Many of the factors that influence NPD cyde Ume are outside the R&D department's
control, making the process that much more difficult. C o p r a t e decision making, customer
needs, and orpan;Sational goals are all factors that impact the t h e to market of many new
products (McLaughLR 1995).
Fewer resources
As companies downsize as a meam of c o s CU&&
the R%D department is ofren the hm
target.
Poor or uncertain economic conditions cui cause havoc with R& funcihg and personnel. These
uncertain conditions have a dmmxic and powerful psychologid effect on projects, innovation, and
creativity.
@kLaughttr,1995)
Executives expect that the R&D function will perform successfullywith reduced resources
and unrealisric objectives (Kumar and Bayle72001). This reduction in resources affects the
spirit of the R8rD teams d o meanwhile must nniggle to keep improving cyde times to
keep up with the market.
W& moa research organisations there is a constant pressure to produce more results with
less budget in less rime. It is imperative therefore t h research organisations ensure their
cnticd resources, the research naff, are focused on proieas that address the strategic needs
of the organisation and that these needs are being met efficiendy (Endres, 1992).
Increased competition
R&D or new ~roductdevelopment is one of the mon important sources of cornpetitive

advantage for a f m (Jayawama and Pearson, 2001). With increased competition in today's
global marketplace, organisations need to not oniy satisfy customers, they need to M e
them widi their ability to foresee and fulfilall their requirements. Gmpanies that do diis the
best are dependent on the^ R&D departments to recognise the needs of their target markets
and translate them into produas and services efficiendy and effectively.
Emergencc of ncw technology

The unergence of new technology gready impacts the RBrD environment. RSrD mua view
tedinology and scient& advames as tools to assis the o~anisationin meeting Company
goais (McLa.gbi.i, 1995). Inevitabiy, tedinology must be manageci within the R%D
organisation to benefit all employees/shuaiolders. The ability to manage effectively can
best be achieved through the implemenmion of qulay management priuciples
(McLaughlin, 1995).
Capturing the customa's changing needs
The R&D organisation m u s be driven by its responsiveness to changing cunomer needs.
This is one of the greatest challenges forRBrD hinctions. At the initial stages of a new
product development project, researchers and scientists musr accurat* and precisely
ascertain customer requirements. These are then translateci into a set of specificationschat
defme the prouct to be designed. A new challenge arises when customers request changes
ro rheir on& requirements in the rnidst of the design stage. This may only necessitate
s m d changes to the design, but could result in the design team starcing again from scratch.
These changes increase cyde rime, costs, penon houn and affect the RSrD schedule, rying
up employees that are require on other projects.
As a r e d t of these factors, the need for teamwork, efficient and effecrive management
sysrems, and appropriate management of technologies has become critical (McLaughLn,

1995).
2.3.2
Benefits of Implementing Quality Management in an R&D Environment
One of the main benefits of irnplementing quality management in an R&D environment that
authors have reponed is the darification of the defdtion of 'quality in RBrD' (Pa&o, 1997,
Kumar and Boyle, 2001, Pearson, et ai., 1998). Several &finitions have been suggeste. A
review of litaamre by K
w and Boyle (2001) provides the following components of
'quaiity in R&D':
an understanding of who the R8rD client is and his or her values and expectations
what the key technologies are and how they can be used to meet R&D clients'
expectations and the needs of the entire organisation
who the R&D cornpetitors are and how t h q will respond to emerging R&D clients
needs
Qing diuigs nght once you know you are working on the nght things?c o n c e n t h g on
continually improving the system.
enabling people by removing barrien, and encoUrap;ig people to make their maximum
connibution
Many authors have written about the need to have systemauc processes for managing quality
to gain and remain in a cornpetitive position (Cooper et d,1990, Colurao, 1998, Auer et al.,
1996). The result of irnplementing such a q d t y approach is a hm that continuody
serves the needs of its customers (both internai and extemai) in an efficient and effective
manner (McLaughhn, 1995).
Q d t y awarb can be w d as a yardstick for measuring 'quality in R&D' progress and
idenufylng gaps for improvement, as well as providuig a comrnon focus o r direction for
the whole Company. (Pearson,et al., 1998)
Systematic decision making, continuous improvement, experirnentation with new ideas
and teamwork are important for a successful research and developnieit environment
( G d ,1993).
Several reasons have been provided in the literature for the limited use of quality
management practices in R8rD. These are:
concem that
on of a qrulity myiagement prognm in RBrD nu&
d e creative
woik and be counterpductive (Brown, et d., 1994)

the dificulty of meamring performance in R%D (Montma, 1992)
Lck of vainiog and acceptame by RBrD personnel of the "sofiwside of @ty,

teamwork and participation management (Montana,1992)
such as
expectations (i.e. ahuays dohg it nght the frrn

the feu that it will raise u~vea~onable
thne) (PaGo, 1997)
concem thar it will d e proper recognition of individual accomplishment (Parino, 1997)

ciifferences in inrerpretation as to what 4
t y realiy is and how it is to be pmued
(Francis, 1992)
PaGo, (1997) suggesfs that some of these barriers can be overcome if there is sufficent
dialogue and darity as to whar ' M t y in RBrD' means.
Several authon have discussed the implications of implementing ISO 9000 in RgrD
environmenu (Cooper et ai, 1990, Davidson and Pniden, 1996, DeWeed-Nederbf et al.,
1995, McLaughlin, 1995).
Process driven quality synem mch as ISO 9001 are potenwy of great vdue to control
the quality of the R&D process (Auer et al., 1996).

ISO 9001: 2000 provides an "ideal" oppominity for R%D managers to achieve the
benefits of total quality management and thus improvement in various kqr performance
dunensions (Jayawamaand Peuson, 2001).
W1thin the RBrD context, ISO 9001 is becomuig more popular as a way of making
people think and plan ahead and as providing a disciplineci way of capniring the R8rD
r e d t s (Cooperet ai., 1990).
The revised version of ISO 9001 look into the dyaamic chamter of organisations m
which issues such as leadership, people involvement, v e r n approach to management,

continuous ;.iprovernent and a fact -basai approach to decision-makingfor ewnple,
receive speciai attention. The need for more dynaxmc approach ro RBrD management
has been emphasised for yeus (De Weed-Nederfiof et al., 1995).
ISO 9001 addresses both the issues of better convol and continuousty raising the R&D
standards (Iayawama and Pearson,l).
ISO 9001 is a good means of providmg a framework for developing a quality syscern in
R&D. It outlines the organisational nnicnup, management responsibilities, procedures
and processes r e q d to set the base for a "hoiistic @ty
management system"
(Davidson and Pruden, 1996).
The Companycan adapt (select) from the specifications in the new standard,enabhg
them to ensure a good fit of the standard to their R&D activities.

The main advaatage of ISO 9000 in RBrD is that it provides a structure and consistency to
an otherwise chaotic process.
'The issues relevant to the application of quaiity management pnctices in an R&D

envitonment have been addressed in this section. The n a section provides an o v e ~ e w
of
the motivation for the proposed researdi.
2.4
Motivation for Roposed Research
In light of the topics reviewed in this litenture search, the following points outline the
motivation for the proposed research.
7he ISO 9001 standard for q d t y management systerns is a stepping aone on the path
to business excellence. Extending these minimum requirements for a quality
management system to encompass ail organisational processes can be a daunting task

Methodologies for evduation of a
d vernis desirable systern statu have been
developed for quality management systerns (quality auditing) and for overd business
performance (self-assessment). Both of these mdodologies have been proven to be
essentiai to an organisation's compet;tiveness. There is a need to analyse the
commonalties and differences between these method01ogies in order to provide a
theoretical baclspufld for th& b e n t and possible integtation (Karapetrovic and
~ b o m2001A).
,
Result-based performance masurement k a method01ogy for measuring an
organisation's performance. Organrwtions are Ehallenged ro select a rneaningful set of
performance measures thar wili provide a broad view of organisational performance.
Whde the abject of q d t y management in a manufacturing environment has been wd
addresseci, the application of these methodologies and theu implications to an RBrD

environment hu not. Further research is needed to explore the applicability of quhty
management frameworks in rhis type of environment.
2.5
Objectives of Proposed Research
This section describes the objectives of the proposed research. These are to:
develop an integrated model for performance management based on the ISO 9001
standard for @ty
management and three national @ty
awards to provide a reference
model for organisations wishing to move from the minimum requirements for their
quaiity management system to overall business excellence
condua a self-assessment against diis new reference model in a case study organisation
compare the self-assessrnent renilrs to the outcornes of a quality audit to furcher
investigate the opportuniries for integration of these masurement methods
review the redt-based performance measurernent system within the case mdy
organisation and compare rhe outputs to the self-assessrnent and the quality audit
moddy the reference model for more appropriate evaluation in an R&D environment
3.0 An Integrated Mode1 For Performance Management

3.1
Introduction
This diapter addresses the development of an integrated model for performance

management combining the concepts and criteria from the ISO 9000: 2000 standards and
three nationai quality a d . The chapter describes the development of the framework for
the rnodel and correspondhg criteria, and the establishment of a scoring scheme.
3.2
Development of the OPIM
Organisations roday must reach bqrond the minimum requkmenrs for their qualiry systws
using a more holisric approach to improve aii elements of their business. The minimum
standard ISO 9001 for an organisation's @ty

system is d considered to be essential to
the foundation of all quality improvernent efforts (Karapeuovic and Wibom, 2001B; Dale,
1999; van der Wiele et al., 2000;Russel, 2000). As these requiremenu are achieved
however, new lwels of improvement need to be reached. Organisations musr continue to
ensure that thv are meeting the requirements of the standard and also are moving forward
with their improvement effoxts. The integration of elements of the ISO 9001 standard and
the criteria from the more holistic quai~qwliryaward models for business excellence provides a
framework that can be used for organisatid improvement.
This section describes the development of the new mode1 for qualiry management named
the Organisational Performance Irnprovement Mode1 (OPIM). The Grrr part illustrates its
framework or underlymg suucture. The next sub-section discusses the application of crireria
from the other models to this framework Fin*,
the development of an appropriate
scoring scherne is described.

3.2.1
Framework Development
The framewok for both the ISO 9001: 2000 standard and the &ty
a w d are based on
severai undertying principles (see section 2.2.1). The fm stage in the Qvelopmem of the
A system is a set of interdependent processes that function haxmoniousiy, using various
resources, to achieve established objectives. Processes transform inputs h o outputs. The

effectiveness and efficiency of the system's operations and use of resources, is quantdieci in
the system's performance results. Applying this definition to any set of processes results Li a
"system's view" (Kuapetrovic and Willbom, 1998A).
A framework based on this 'system's view" wadd result in a comprehensive model

incorponthg all of the essential elernents of the organisation.
The OPIM is a model that describes the C S 0 7 s quality system. Ir focuses on the continuous
meanwmenq monitoring and control of the sysrem's performance and its impmvement
(Kanpetrovic and Macey, 2001). Figure 3-1 represenrs the systems concept for applicability
to the OHM. This concept is based on previous research by Kvapetrovic and Wdborn
(1998A). The dotted lines nirrouning the objectives and resources are meant to emphasise
rhe active role of the processes in the system and to indicate that these elemenrs are used as
inputs to the F e m not to de-emphasise their importance.
RESOURCES
PROCESSES
OBJEC77vES
i MANAGEMENT
.........................
PROCESSES
MANAGEMENT
II
1:
..........................................
RESULTS
PERFORMANCE MEASUREMENT, MONITORING, CONTROL & IMPROVEMENT
Each element of the synem portrayeci in Figure 3-1 is a main category of the OPIM
framework These are:
1. Objectives
2. Resources
3. Processes
4. Results
5. Performance
The fm s e p in the integrauon of other models to fonn the OPIM was to use these
categones and to devise sevenl sub-categones that were thought to encompass the purpose
of each main category. The subsategories were created from a review of the content and
purpose of each of the a w d models un&
evduation and the ISO 9001:2000 nuidud
Table 3-1 shows the i n i d set of caregoris and ~~b-categories

of the OPIM.
3 3 Product =&or ~ t r v i c e
3.2.1 1drnrif;caxion of producc mUor service processes
32.2 Idmrifiminn and c r a c d i
32.3 Produa d / o r m i c e ourput
32.4 Graamapropeny
32.5 V d i d a h of pnxases
3.2.6 Gntrol o f me;;llairing a d monitoring devices
3.3 RvEfuung
3.3.1 Idemifiution of nced
3.32 kifofmaricm
3.3.3 V d a r i o n o f purchascd product and/or services
3.4 Support Proccsscs
3.S. 1 Idear.i6&0n
3.5 Pluinulp
1.0 o b j j v e s
1.1 Intacncd parlia
1.1.1 Cunoma
1.1.2 Ownas/ i n v a t o n
1.1.3 People
1.1.4 Supplias d parTnen
1.1.5 Society
1.2 Mui;ig~ll~llt
syacm
1.2.1 Muugtmmt rtsponsibhy
12.2 Mur;igcment mnew
1.3 Legd
2.0 Rrsouras
2.1 People
2.1.1 Acquisnion
2.1.2 Deploymait
2.1.3 Worksynmis
2.1.4 Wdl-king and saisfaaion
2.1 .S TrJning
p i d d 2a 2? L
2.2.1 lLsponsibilicics
2.22 Policy and nrrtegv
2.3 Infomution
2.4 infnnnicnuc
2.5 TechnoIogy
2.6 Supplim/purnaships
2.7 I 4 k c d s
2.8 N d resourccs
2.9 F i c e s
3.0 P r o c e ~ ~ s
3.1 Design and/or dcvelopmuir
3.1.1 PlaMlIlg
3.1.2 Lnprtts
3.1 -3 Aaiviues
3.1.4 Chpus
3.1.5 Revicw
3.1.6 Verification
3.1.7 Validation
3.1.8 Convol o f changes
3.1.9 Design and developmcnt o f p r ~ ~ e ~ ~ s
3.2.2
4.0
Renrhs
4.1
4.2
4.3
4.4
4.5
Cunoma
d and uurket
People
Supplia and m e r
Soery
4.6 G p n k i o d effectivena
5.0 lmprovanent
5.1
monitoring and control
5.1.1 S y n m i performance
5.1.2 Cuscomasatisfaction
5.1.3 i n t d audit
5.1.4 Processes
5.1.5 Prodm a d o r service
5.1.6 GICIM~
of nonconformhy
5.2 An;ityPs o f clam
5.3 6 d v e anion
5.4 Prrvcntive acon
Criteria Development
The OPIM shodd be considered as a combinarion of q d t y asnvance standards that

provide the minimum requirernents for q d t y improvement and business excellence
rnodels, illustating the opportuniues for improvement. The criteria in each categov of the
mode1 are split into two parts. The fmt presents the minimum requkments for the quality
management system. The second provides opportunities for improvement over and above
these minimum requirements, descnbing the continuous improvement of the category. This
split wiU enable ui organisation to maintain ifs quaivmanagement system as a foundvion

for its conkuouc impmvement efforts. niis nsn section describes the dwebpment of the
criteria for the OPIM.
Once the categories of the OPIM were created, each awad model was reviewed to provide
an i n i d fit of rheir categories and sub-categoriesinto the OPIM framework Table 3-2
shows these matches.
OPIM
ISO9i: (
IS09004-.
1 MBNQA I
EQA
DP
ISO 9001:
OPIM
rOOO
32 Rohua d o r service
32.1 Idadkaionof ~roduct
75.1
73.9
, 75
1
d o r u n i i c c ~
3 1 2 khtificPioa
d traceabd'y
33.3 Produa d o r savice

32.4
32.5
propaty
VIliduion of prousses
3.2.6 Cbtmf of masuxing and
7-53
7.3.8
MBNQA
ISO 9001:
2060
7.53
5d
6.2
DP
EQA
6.7
75.3
7.55
7.6
7.6
6.7
conml
5.1.1 S ~ c m p a f o ~ c e
5.1.2 G ~ ~ a m
satisfaction
n
5.1.3 h a m i audi
5.1.4 Proces~s
5.1.5 l'mduci urd/or savicc
5.1.6 GmmI of noaconfonnity
5.2 AiillyPs of d m
5.3 Cormxivc action
5.4 Prcvmuvc action
4.1
1 82
82.4
8.2.13
6.6
6.5. 6.4
6.7
822
82.3
83
8.4
83
8.4
8.52
8.53
4.2
6.1.6263
6.1.6.2, 6 3
3.3.8.3
Sb,%
Sb, Se
6.5.62
IO
After ensuring that moa of the categories from the models were mapped to the OPIM, the
contents of the awards and ISO 9001 were reviewed to ensure that the categorymatches
were appropriate. This process resulted in the rearrangement and renarning of some of the
caregones in the OPIM to provide a more streamllied set of cnteria.
Category 5.0, labelied Irnprovement, is provided as an example. Irnprovement cannot corne
from corrective (5.3) and preventive (5.4) actions done because they just remove the causes
(existing or otherwise potential) of problems. They do not lead to Breakthroughs, which
break new ground in improvement. Breakchroughs cm be graduai (Kaizen) or innovation /
Anorher review of the category 5.0 rwealed that it &O indudeci analyus of data, intemal
auditing, and measurement and monitoring These elements are all a pan of an
organisation's performance management system. The category was renamed Perfozmance
under which three sub-categories were creatd Management, Monitoring and Conmiling,
and Improvement. Matching elexnenu from the other models were reshuffled to fit into the
new structure of the OPIM.
Table 3-3 illustrates the process as describeci above. The first parc of the table shows the
initial sub-categones in category 5.0 and correspondmg criteria The second pan shows the
modXed mb-categoies and their comespondlig criteria.
MBNQA
5.0 hprovemcnt
5.1 Mcinirrmair, monitoring and convoi

5.1.1 S y n a n pafomunce
5.12 Luorner satisfaction
5.1.3
5.1.4
5.1.5
5.1.6
I n t e r d audit
Pmrs~s
Product and/or semice
Conml of nonconformiry
5.1 An3ysir of du
5.3 Conmive: action
5.4 Prcvmtive action
5.0 Perfonnancc
5.1 Management
5.1.1 S p e m
5.1.2 Key Pcrformuicc Outcomcs
5.1.3 K y Perfonnancc Indiuton
5.2 Monitoring and Controlluig
5.2.1 Audits
5.2.2 Self-Assasrnaits
5.2.3 Puformancc M a s u m e n t
5.3 Improvanent
5.3.1 C o k v e and Prnteative Actions
5.3.2 Kaizcn d Quuinun L a p s
8.2.1.3
82.1.5
8.1,8.2.1. 8.4
8.5.1
8.52,8.5.3
8.5.4, Anna
EQA
illustrates the caregones and comsponding de~cn~tiom

for categoxy 5.0 Performance.
5.0 Performance
F i OPIM
5.1 Mui;igcxnent
5.11
Systan
Describe the +an applied to masure org;iniutionai performance. Illuruve the

inurrcltioriships mong the internai audit, d-assusment and r c s u i t - b d syncms for
performancernasurrment.
5.1.2
Describe haw rht i.urrcl;uionshipsof i n d audits, d-wessmcats and rrsuh

m~surrmcnts
are managcd and imprwd
Key Performance Outcornes (EQA, 91)
Rczpciramnr
Desaibe the +cm
appiieci to nmmre the ky paforrmnce outcornes idurtified in section
4.6.3 of OPM
5. t .3
Describe how the m c m m m n t v a n for ky performance outcornes is continuaily i m p m t d

Key Perfoxmance Indicltors (EQA, Sb)
Describe the system applied to mcasurc the key performance outcornes idendicd in Kctions
4.6.1 and 4.62 of O P M
Describe how the meuwpient spem for key performance indiators is contllidy
impmved
5.2 Monitoring and Conmlling
5.2.1
Audits
5.2.2
Repmaens (ISO 1,8.2.2)

Dexnbe the syscun applied t o perfonn internai and extemai audits against the rquircmcnts of
the ~lectedmanagement nandards, induding ISO 9001: 2C and/or I S O 14001:1996.
Qpmozm ( I D , 8.2.1.3)
Describe h m the effectiventss, effiaency and consistency d the audit sysrun are continuay
;mpmvcd.
Self-Assessments (IS04,8.2.1.5)
Describe the system applied to perform rcguiar self-assessrnenuagainst the The CS0
Performance improvcment M d
qyJmmm3
Describe h m the effmiveness, efficiuicy and consisturcy of the self-wcssmcnt synern are
concinuaiiy improveci
3-4- I
5.2.3
Performylce~wumncnt~SOI,8.1,821,8.4)
(MBNQA, 4.1al and 42)
Describe the performance mevurrmait systan d to collea, moaitor, adysc, synthcsise and
act upon puvpios (cnan;J)and performanceindiators (iuitamal) &ring to:
C U S t O ~
shareholdus
people
suppkandputnas
-ccy
(MBNQA, 4.1d)
Descrik h m the effecUvcness, efficimcy aud connsVncy of the performuice masurement
systcm arc contindy impmved
5.3 Improvemcut(ISO1,8.5.1; IS04,8.5.1;DP,9)

Acons
5.3.1
Corrrcuve and Pmr&
i?qmmm( m l , 8.52 and 8-53)
Describe the v a n applied to m o ~ o ridenufy,
,
+, and rcmzve the cxisting ;and potenriai auses
of nonconformiriesin inpmccws, outprru, objectives, rcsources and d u .
(IS04,8.5.2 and 8-53)
Desaibc how the conrctive and prevcntive actions ur used in the business piau and how theu
effcctivaress is continulty improvcd.
5.3.2
Kaizen and Quannun b p s (IS04,8.5.4 and Anncx B)
Describe the syncm applied to c o n ~ u c d impme

y
the feuurcs in inputs, processcs, outputs,
objectives, NSOWCS and rrnilu.
Describe how the effecWenus of the continuai impmamnt +an
is i
To better integrate the requirements of the ISO 9001 standard and the non-prexiipnve
criteria in the qualiry a w d , each category of the OPIM was split into rwo aspects, the
'MinimumRequirements' and 'Opportunities for Improvement'.

The OPIM should be considered as a combination of quaiity assurance srandards that provide the
minimum requinmenu for quality improvement and business excellence modek, iliustnting the
oppomuiities for improvernent.
(Karapetrovic and Macey, 2001)
Table 3-5 outlines the fuial OPIM and the comesponding cnteria of the MBNQA,EQA,
Deming Pnze, and ISO 9001, ISO 9004.
Please note that there is no entry in element 5.2.2 from the MBNQA or EQA. The reason
for this is that these a w d do not specifically address self-assessrnentin their uiteria but
provide further guidance in sepante documents on this process. Please refer to the
Approach to Evduation t h supplement the EQA and
Guidelines for self-assessment
that accompanies the MBNQA for comsponding elemems.
Once the framework and ait& for the OPIM were cornplme, a document cded 'OPIM
Fundamentas" was created to describe the undertying frarnewods of the OPIM, its content,
terminoiogy and application (Karapev~Mcand Macey, 2001).
This document is meant to accompany the OHM Criteria and contains general information
about the OPIM induding its relationship with the CSO's performance management system,
the framework of the OPIM,the content of its criteria, a description of terminology used,
and notes pertaining to its application.
3.2.3
Development of the Assessment Questions
Once the model w u complete, it was ready to be wd as a reference for conducting selfassessments. To couen appropriate information for a self-assessment, the criteria were
translated into a set of questions to be used in assessment interviews. The questions were
based on the cnteria of the OPIM and were specific enougb to capture suffiuent
information with which to evaluate each element in the OPIM.
The questions were designeci to be open ended to obtain as much information as possible.
Appendk A contains the set of questions for the assessment. Please note that category four
is the r e d t s section that is used to evaiuate the results obtained in all other sections of the
model. For this reason, there are no questions for this category in the questionnaire but
evidence of the results in each pertinent category are collecteci from interviewees when
questions pertaining to these categories are asked. These questions ask the inte~eweeto
provide perception measures or performance indicaton that indicate performance relative to
the respective category critena.
3.2.4
Establishment of the Scoring Scheme
This model was developed as a set of criteria against &ch a self-assessrnent could be
conducted in a praaicai sertlig. Although, not the main puipose of self-assessment, it is
ofren beneficial to calculate a score from the assessment that can be compared from one
assessrnent to the next IO monitor i r n p r o v ~ ~ia~al quantitative

t
muiner. As seen in
Chapter 2, each of the refemce models has its own scoring scheme and evduation methods.
An appropriate scoring xheme w u required for the OPM. The rwo a w d considered as
models for the scoring xheme for the OPIM were the MBNQA and the EQA. These have
the most complete and comprehensive methods of evaluation. The evduation scales are
detailed, enabling the wesson to be more p r k in their evaluation The evduation of

each element is based on the three or four criteria outlined (approach, deployment, r e d s ,
assessrnent and review).
To create a scoring scheme for the OPIM, it was fim necessvy to assign point values to
each of the categories of the OPIM. The OPIM was developed to help an organisation
improve its business performance. It wu therefore important to incorporate the perceiveci
contribution of each categoq to the organisation's perfomiuice into the final disuibution of
points. A multi-criteria decision making methodology, was used to determine this
distribution of point values. This approach enables each organisation ung the OPIM to
incorporate the impacts of each category to their specifc business.
3.2.4.1
Preference Function Modeliing
Mdti-cnteria decision making involves the selection of one alternative from a given set of
altemarives based on a set of criteria. Preference Function Modelling
is a decision
analysis method010gy for mulUsriteria decision makiog. It is used to create a mathematicai

mode1 of the decision-maker's preferences for alternatives based on a set of critenk more
detaii see Banilai (1998, 1997).
A typical multi-cnteria decision involves the cornparison of several alternatives based on a
set of criteria. Take for example the purchse of a car. A decision-makerwould have a set
of cars to choose from. To make this choice, several elements rnight be considerd colour,
pnce, fuel economy, wa~anty,etc. These are the cri&
for the decision. The outcome of
the decision would the selection of one car out of the set of cars.
The Trillium software (Badai, 2001) MS developed to encapsdate preference function

modeliing rhmiy into a foxmat for practical use. To use Trilliu111,rwo inpurs are requked.
These are a set of alternative outcornes for a decision, and a set of aiteria upon which the
deusion wiu be made. Once these are fed imo the model, decision-makersrank the
alternatives on each uitenon by compaxing tnplets. The result of diis process is a
positioning of these aitematives on an interval sale (a preference function). N a , the usen
indicate uade-offs between the criteria This allows waghu to be detennined for each
critenon. The set of weights is then used to resolve e d preference function (or rankings
on each interval scale) into one overail preference function.
R e f e h g again to the car example, once the altemacives are listed in the software, the
decision-maker is then asked to describe the set of criteria that wili form the basis for the
decision. Next each of the altemative are compared on one criterion. For instance, if we
have a blue Ford, a red Chiysler, and a green Toyota, the decision-makeris asked for his or
her preference for a blue car when compared to the altematives for green and r d This is
done for each criterion. Once this is done, the decision-maka provides mde-offs between
the criteria. For example, how many units of price would one trade for how many units of
fuel economy?
This methodology is used to select one out of a set of alternatives. However, in this
application, the software was used for a slighdy different purpose. The hope was to develop
a function of values to be applied to the OPIM categories. The software was used to rank
the caregones of the mode1 based on their relative contributions to business performance.
These ranks would then be trandated into a set of vaiues for each of the categories. Thur
instead of selecting one car, we are creating a set of weights that would dow us to evaluate
any given car on a single, uni-dimensional scde that weights the car's characteristics
The foilowing describes the development of this function of values for the OPIM.
TOdetennine the dimibution of scores among the categones of the OPIM,the Trillium
software was iwd In this o v e d function, the htghest ranked element f~anslatesinto the
category that is worch the mon points.
The relative contribution of each category to business performance w u captured using the
opinions of top management and those who had experience with BEM and self-assessrnent
process. A review of literature rwealed several instances where authors that have vied to
determine the relationship betweai quality management practices and business performance.
In 1999, Teniovski md Samson used several performance ourcomes as representations of

business perfomiance. These agreed with the connnicrs used by other researchers. The
performance outcomes were produaivity, competitive advantage, emplayee monle, coa of

quaiity, customer relationships, and product &ty.
These outcomes were adopted to help
defme business perfomance in the decision modelling process.
To dwelop the peference function for OPIM,the foUowiq process was followed. The
alternatives are the categones in the OPIM and the criteria are the six components of
business
identifid Sweral people, independent of one another, ranked the set
of alternatives in sets of five on each decision cnteria Figure 3-2 shows the scale used in the
Trillium software with severai elements from the OPIM positioned on it. After these
rankings were complete it was necessary to determine the uade-offs between the cntena to
indicare to the software which cnteria was the moa important to the organisation at the
present Ume. The Chief Executive Officer, Chief Operathg Officer, Vice President of
Finance, and the Vice President and General Manager of Division A were asked to indicate
the importance of each of these to the CSO in the Trillium software.
Appendix C contains the raw data taken from the Trilliurn software.
The end renilr of this anal*
was a rmkiqof all the OPIM categories on an intemai scale
from 50 to 150.
It was decided that 1000 points would also be the t o d point value to facilitate cornparison
with the other award assessments.
The following process was used to change the ranks of the alternatives on the preference
function into scores out of 1000 for the OPIM.
1. The preferences on the scale were summed.
2. The sum was divided by 1000 to give a f-~rctor.

3. Each individual preference value was divided by this factor to uuisform them into
scores. The sum of al1 the resulting values wu 1000.Table 3-6 shows the results of this
process.
4. The resulting scores were rouded to give the result in Table 3-7.
I
People
Pmfamcw
48.49
1%
* m t
customcr Rdalionships
148.77
48.09
144.44
46.69
Performance Managamt
132.05
42.69
Performmce h p m e m c ~ l t
131.94
42.65
Strvegy ikvelopment
131.75
42.59
131.12
42.3 8
Peopk Resulu
129.62
41.90
De*
128.73
41.61
lnfonnuion Resources
12526
40.49
h
*
122.54
39.61
120
38.79
and Deploymtnt
Performance Monitoring and Cent*
Scotc out of 1000
Proccsxs
Resourccs
F~cialResources
St;ikehol&r Objectives
118.99
38.46
Raiisation Ptoccrxs
118.98
38.46
Cwomer Wts
110.3
35.65
103.n
33.54
Shareholdcr Results
99.48
32.16
System R
94.95
30.69
Supplier and P m a M
dts
89.38
28.89
Support Processes
86.8
28.06
l'urchasing Processes
82.8
26.77
50
16.16
Society Rmilts
l
Total
3093.56
Faaor
3.09356
ZOO0
4.4 SU&&~S
4.5 S O U m
4.6 Svstem
and Pumas
cz
33.50
16.00
30.50 1
' 5.2Monitoring and G n u o l h g

5.3 Improvement
T-.-I
42.00
43.00
I
r-
nnn
The process of g a t h e ~ information

g
or preferences from the decision-makeisin the CS0
was difficult. niey quenioned the impact of their decisions and were initially concemed that
they did not f d y understand the Triilium software. Decision-makersunderstood the
concept of favouMg one cntenon over another but were a little unsure of the merhod used
in the software to capture these uade-offs. Once this was clarified, thqr were more
comfortab1e wRh entering their preferences into the mode1 for an+.
element to comprehend wu the idea of uade-offs.
The most dZficult
the MBNQA criteria in the OPIM Processes utegory are p e r than the poinrs in the
EQA The uiteria from each of these a
d that fdinto the Results catego'y of the OPIM
make up between 43 and 45% of the respective award points. The aiteria from each of these
awads that f u l inro the Performance category of the OPIM make up between 7.5 and 8.5%
of the respective awad points.
Objecllves
Tod
Resources
8.5%
4%
4%
4%
OrganhonalTp
Public Responsibilityand Citizenship
Development
Sv?te&~
Smtegy Deployment
Custorner and Market Knowledge
12.9%
Work Systerns
Employee Education, Tninllig and
Development
Employee Well Being and
32.8%
Rwults
Tod
19Yo
Measurement of Organisationai
Performance
Analysis of Organisationl
Performance
Perfol.;mance
Tod
Processes
Total
borner Focused Results

Ficiai and Mvket Results
Humas Resource R d u
Supplier and Partner Resuits
~ t- i o n aEffenivmess
i
Reda
12.8%
Customer Satisfaction and
Relationships
Cusromer md Market Knowledge
P d u a and Service Processes
Supplier and Pamer Processes
Suppon Processes
22.5%
Leadership
10%
8%
Processes (14%) s 1/5
7.8%
4%
24.5%
3.5%
People
9%
2.5%
P m e r s h i p s and
Resources (9%)s - I / S
7.2%)
20.8%
2.5%
8.5%
11.5%
1 1.5%
Satisfaction
Tod
Policy and Smtegy
8%
2.5%
11.5%
45%
Customer Results
People Results
SocietyResula
Kcy Pcrfommcc
Results ( l job) x L 12
18%
20%
9%
6%
1 7.541, 1
1
42.5%
Kcy pcrtormancc
4%
4.5%
8.5%
4.5%
4%
5.5%
1.5%
1.5%
17%
Resulrs ( 15%)
>)s
1/ 2
7.5%
7.5%
Pracesses ( 1 -4%) s 4 / . V 1 3 ,
Pannerships and
1.8%,
Rttsuurces (9%) s 1/ 5
13%
Cornparison of Category Point Values
i O OPIM
1 HMBNQA
1 O EQA
Objectives
Resource
Processes
Results
Performance
OPIM Categories
There are siight differences between the models in ternis of the preference given to each
category. The EQA and the MBNQA are both very focuzed on the results categones. A
large percentage of the total points have been assigne to this category in these cwo models.
The s c o ~ scheme
g
developed for the OPIM has points that are more evenly disvibuted
among the categories. n i e point value of the resources categoiy in the OPIM is aiso verdifferent from those of the other models.
The Resources category was deemed to be of greatest importance to the organisation's
perfoxmance. This is very 1ogica.I. It is often forgotten that no systern exists widiour
resources. A V e m is not jus a set of processes thar interact to produce an output.
Resources are ememely important as input into these processes. The q d t y of these
resources needs to be high to obtain quality outputs. One cannot cast inarmpetent
employees, unreliable equipment, or poor materials into a sysrern and expea satisfacto'y
results. It is interesting to note that resources are worth the lean in the scoring scherne of
the MBNQA.
41
1
This redt is a h v e y important to companies moving beyond ISO 9000 towub business
excellence. The main focus of IsO 9001 is on pmesses. TOmove beyond this standard,
companies musr begin to concentrate on &ek resources as a next sep.
The distribution of these points in the OPIM is based on the opinions of people in the CSO.
This distribution of points my have a great effect on the final score in a seif-assessment.
Chapter 4 describes the performance of a self-assessrnent against the OPIM. A sensitiviry
analysis is done to answer this question.
3.3
Chapter S u m n w y
This chapter described the development of an integrated model for performance

management based on ISO 9000 and business excelience models. This involved the creation
of the model framework and corresponding criteria. A scoring scheme was also set up
capnuing the specific relationships h e e n the elernenu of the modeL The next chapter
describes how this proposed model was used to conduct a self-assessrnent 4 t h a division
at the CSO.
4.0 Self Assessrnent

4.1
Introduction
The development of the OPIMwas describeci in Chapter k.

This chapter illustrates
how a self-assessrnent against this proposed mode1 was used as a meanire of organisational
performance.
4.2
Self-Assessrnentat the Case Study Organisation
The CSO's quality system has been registered to the ISO 9001 standard since 1995. Since
thar tirne, quality audits have been conducted to ensure the continwus cornpliance of this
system. The organisation had not however, conducted a self-assessment against a mode1 for
quality management or business excellence. Sweral seps were involved in conductiag the
fwst self-assessment a g h the OPIM (See Figure 4- 1). This induded planning, garhering
idonnation, evduating the r e d s and the assessment process, and incorporating the r d t s
into the business planning process. This secrion describes the seps involved in the pilot
assessment of one of the (30's business divisions.
Planning
4.2.1
The planning phase of the self-assessrnent induded definmg the scope, self-messment
method, resources, and tim$ines. The scope refen to the areas within the organisation rhat
This can ange from one depanment to the whole organisation. The selfassessrnent method concerns how the infonnation will be gathered to complete the
assessment. (Chapten 2 and 3 ourline several nich methods). It is important to defme the
wili be assessed
resources required to conduct a self-assessrnent before it begtas. These consist of

personhoun for coUecring and analysmgthe assessment info~nation.The tirneline for the
assessrnent must also be detennined in advance. This incldes times for information
coileaion, a d y s ~ and
~ , presentation of the resuits to senior management.
4.2.1.1
Scope
It was decided that the self-assessrnent wouid be c o n d u d in one division (Division A) of

the cornpany's business. This decision was based on several factors. Fim of all, this selfassessment effort was the fm in the organisation's hisrory. Seleclhg a d e r ~ b s eof
t the
organisation wu deemed to be a more realistic underilkingthan a t t e m p ~ to
g assess all
areas of the business. Seconcy, the rime and resources available limited the scope of the
assessment. Fin*,
this particular division was selected because it was the fastes growing
business area of the Company.

4.2.1.2
Self-assessrnent Method
It was necessary to select a self-assessrnent method that would be appropriate for the
organisation as wd as meet the requirements and expectations of the CSO.
There are sevenl choices for appropriate self-assessment method, as seen in Chapter Two.
The seleaion of a self-assessrnent mode1 involves wsessing the maturity level of the
Company.
The CS0 had progressed well in the implementation ofits quaiity system. This suggests that
it has srarted its joumey towars excellence, and w u 'on its wy". As a r e d t , a more
rigorous assessment approach was deerned to be appmpxiate.
The main objective of this
assessment exerQse was to collect information from the organisation to identify the
suengrhs and areas for improvement in each area. The initiators of this process were also
interesteci in gaining as much information as possible from the assessment. Because of the
amount of infoxmation required and the quality manuin/ level of the organisation, a variation
on the pro-forma approach (see section 2-22) was selected.
The pro-forma approach involves the creation of a set of pages where each page
corresponds to one sub-critenon. The description of the sub-critaion is printed at the top
of the page with areas ro address beneath it. The information requested on the page
indudes suengths, areas for improvement and evidence. I n t e ~ e w are
s conducceci to collect
evidence for rhis process. Once these are finisbed, the assesson then complete the proformas.
Ln this self-assessment, a set of questions relating to the sub-criterion was asked of the
inte~eweesro uy to capture its essence. The assessor then identifieci the strengths and
areas for improvement based on the responses of the interviewees. Thus, the pro-forma
was completed by the assessor through the collection of information and evidence from the
interviewee.
Ir was necessary to select a subset of emplcyees with which the interviewswould be

conduaed. The set of 159 questions was disuibuted among a number of interviewees,
seleaed by the managers of the division This redted in a set of 25 interviewees from the
following areas and with the following memben of the executive:
Marketing
* Research and Dwelopment
Test Operations & Strategic SourSng
- Materials Planning
Vice President of Finance

Chief Executke Officer (CEO)
Chief Operating Officer (COO)
Vice President of Division A
The following organisational charcs indicate which representatives fiom eadi are? were
intervieweci.
A mauix contaning the question set and the names of the inten?ewees w u created. This
was used to ensure that dl questions in the set were asked at least three times to three
different individuals. Questions were distributeci to employees through a nntified randorn

assignment (Mendenhall and Sinach, 1993). The employees were srratifid into
management and non-muugement to ennve that the questions would be asked to the
appropriate parties, appropriate for their function. Table 4-3 is a piece of rhis mauix.
Interviewees are listed at the top and the questions run down the length of the ma&.
'Yes" in a cell indicates that a question wrr asked of a corresponding inte~ewee.The 1st
row in the marriv monitors the number of questions asked each interviewee in total to
ennue an equal distribution The second last colwnn indicates the number of times a
particular question had been asked of an inrehewee to ensure at least rhree. The lasr
column indicates if ail the intewiewees asked the question were able to provide a usable
answer to the question.
If this number wu not at leas three, the question was posed to
more inteMewees to get the three answers required.
Question
How do leaders communiate the importance
of meeting cunomer as weii as reguiatory and
legal requirements?
Whar is the process for iden*
and
targeting cusromers, customer groups and
market segments?
How does the CSO address and improve
workplace heaith, safety and ergonornic
factors?
How is design validation incorporated into the
design and deveiopment processes?
How do you detennine key suppon process
requirements? What is the systern applied to
measure your key performance indicators
relating to organisationai effectiveness
(meeting key objectives)?
How do you && what key compaxtive data
and information to use?
How do you continuaiiy improve the
effectiveners, effiency rad consistency of the
performance mevurement systern?
Number of questions for each interviewee
Ys
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes Yes
Yes
Yes
Ys
3
Yes
3
Yes
3
Asked Aaswer
3
Yes Yes
Yes
3
3
4.2.1.3
Timelines
A Mieliae was developed to indude al stages of the assessment process. These were
developed based on previous experience in other organisations. Table 4-2 ilhinrates the
suggested and a
d rimes for the assessment.
St;ise
Discussion with wessment ini&ors re:

scope, wessment team, assesscnent
process, tirnelines, evaiuilfion methods
Presentation to managers in the division
p*
1 ~ e v e l o ~ m mofr
manix
M d up session with (QA manager and
the &miew
Conducc and analyse interviews
u
Idenyfy OFh and r a s(3 OFI'r and 3 +'s
Der careeonr)
12.00
l&
1M
ldy
20.00
6.00
2d;iys
8.00
25
75.00
80.W
100.00
15
30.00
Wrrkr
1 Week
1 I weekl
week
wIFk
1 da^
we$rr
1h
Total Person Hours

-
4.2.2
Hours
10
2-3W&
ld?y
2-3 W&
Presentation to Management
IdV
Write the final reporc
Report
1%
ldy
000)
1 Compare interviews and d
Time
Persons
Involved
Person
Tii
Esum;ited
377.25
--
Assessrnent
After the ~luiningwas complete, the next phase of the assessment involveci conducring
interviews, reviewing and e v a l b g responses, reviewing resuits, and crearion of the final
report.
InteMews were scheduled through intemal e - m S Each interviewe was sent a List of
questions to feview before his or her scheduled interview M i e . The interviews were
conducteci by the aurhor and were recorded dire+
ont0 a laprop computer. Aher each
meeting, the notes taken were rwiewed and if any dacificarion was necessary, a phone c d or
e-mail was made to the interviewee.
Objective evidence wu also coiiected from the interviewees. This evidence included the
following:
customer satisfaction nwqr re&
employee satisfaction nwqr results
monthly performance data from the result-based performance memement system
This evidence was used to evaluate the Redts section of the OPM The data was reviewed
ro see if trends from the pas few yean have shown continuous impmvement.
4.2.2.2
Review of Responses
At the end of the interview process, the ansvers from each interview were reviewed. It was
here that the i n i d analysis of the responses took place. The three answers to each question
were midied to determine if thqr described the same approach. If al1 three answers were
sirnilar, this would indicate that there was good deployment of the approach across and
down through the division. If an inte~ieweein a management position described a strong
approach but other respondents did not concur, diis indicated that the deployment of the
approach was weak. If, after getting these dvee answers, they wue ail different, the question
was posed to yet another h t e ~ e w e e .If
al1 four of these responses were different, this
niggested that there was no consistent approach to the element in question throughout the
division.
After ensuring that each question had been arked a SuffiCient number of thes, the author
and a member of the quaity assurance team ma to discuss and evaluate the amvers. The
three or four responses to each question were Nmmanred before the evduation Each
s u m m a h d question was evaluated first accordllig to the approach describeci. Appendix E,
adapted from the EQA, darifies the scores given for approach, deployment and results.
Each evaluator scored each question independen* as per these definitions If the scores feu
within 20% of each other then the average score was taken for that question. This range was
felt to be acceptabie to the assessrnent team since, as we can see in Appendix E, the range
foreach score level is 20%. If however there w u a gap greater than 20%, then discussion
ensued This was a vduable part of the evaluarion process. A large gap occurred onlytwice.
The discussion that followed led to a re-scoring and a s d e r gap.
The following is provided as an examp1e of the evaluation process. It describes the
evduation of element 5.1, Performance Management.
These questions were asked in order to mess this aspect of the organisation's business.
What are the systems used to mevure organisationai performance?
H o w are these synems interrelated?
How are these relationships manageci and improved?
The responses to these questions provided enough information to evaluate the approach and
deployment cf the methods to satisfy the given element.
Approach:
There wu some evidence of a sound approach to internai auditing and remit-based
performance meanirement.
There was no self-assessrnent process in place at that t h e .
There w u some evidence
the performance rneasurememt systern suppoxted policy
and strategy and that the interna1 auditing process w u well defined
Each department tracked its own perfonavrce measures but had its own merhod of
daetmiaing which performance areas needed monitoring and how this information w u
colected.
Ther- was no evidence that the relationship berween the intemal auditing systern and the
performance measurement system is managed or improved at d
The average score given was 20%. Refeming to Appeadk E, we can see that the CSO had
oniy some evidence of systemauc approaches to satisijmg the criteria in the given category.
Deplqment:
The audit resulu were used u a performance measure in the q d t y u s u r u i c e

department.
There was some evidence that the approach to internai auditing and remit-based
performance measurement was implemaued and deployed in a systematic way.
There was no evidence that any type of self-assessment process was implementedor
deplayed
There was no evidence of a weil-deployed and implemented method of remit-based
performance meanirement.
The average score given was 13%
Again, the assessrnent reveaied that the CS0 had little to
no evidence of systematic deployment of approaches satisfyuig the criteria in the given

category.
4.2.2.4
Results
M e r the xoring w u complete for each sub-category, a lia of svengths and areas for
hprovement w u compiied. These were based on a bendimuking ana+
to derennine the
arez Some of &e m g t h s and areas for improvemenr were

evident from the cornparison of the organisation's a p p r d e s to each part of the critek If
there were apparent gaps in approaches or the deployilent of these approaches through the
division, these were mentioned as part of this list as w d
ben p&ces
within
A final presentation of the results of the assessrnent was made to the managers and directors
of the division The results were also communicated in a Gnal reporr that w u left wirh the
CSO. Each page of the body of the report conrained the sub-category title, the fmal score
for that section, the criteria in that category, the list of strengths and areas for irnprovement,
and some examples of what excellent companies do. Figure 4-5 is an excerpt from this
repon.
5.1 PRfo1111;~1ce
Management
5-13 Key Performanceindiaton
Describe the qaan applied to masure

Dcsaik the s)rrtan applied
thc krypafomlalce OutcOmCs
tomtvunthc~
perf0~OutCOmcs
&nt&d in section 4.63 of
identifid in sections 4.6.1 and 4.6.2 of

O P M Describe h m the mtuutemtni
OPIM. Describe hoprthe
for ky performance indicvon

is contindy improvcd.
syffem
muuraeait systan forkcy
mwuranmtsare cxlaqcd and
irnproved
wbat Excellent Compuiics Do

Each depumient is mponsiblc for u?dring iadiriclual
iodicaton of effe&eness
Resuits-bwd performance mevures ur mricwed
mon*
in the FORM
Thcre is a system of ESD audits and i n d ISO qu?lny
auditz
The R&D process masures cover proccss h,

m,
efficicncy, and planning quai~ty
RSrD mevurrs are avaiiable on-iine (the inmet)
Areas for lmprovement

There is no formal recognition of the i n t d t i o n s h i p s
between the interd audit, self-assessrnent and =suitbased synems
There are sepante systuns in cadi department for u?cking

r e d t s - b d mevures
It is undear which performance measures convibute the
mosr to which keycompany goais
There is no evi&nce of a sysccmatic approach to
impmving the measunment synem for kcy performance
outcornes, kcy process, and mource performance
indicaton
The= is a Lck of rncmms dating to the effective use of

RU)UTCeS
Finand mevia covcr the past,

prcxnt and fucurr pcrformuice
The o q p i d o n lmows the uuc cons

of its processes and producrs/senrices
Cycie Umc for ail ky processes is
me;isurcd
Key processes have b e n identifid in

each unit,funaion, and depvuncnt
of the organisation, and pmcess
masuns have b e n & f i for a c h
bpProcess mesures uc codatecf
with product./servicc
charameristics or performance factors
tht uc of prime imponuice to
cwomen
dire+
*ll)LC organisation bu dcveloptd and

o v e d safety in& thu is u;idrtd u
l u s t once a month, and consists of
several output masures Wre timc-lon
accidents as wd as the numbu of
pmrcnt?tive or behaviod musure
Fimd Score
4.2.3
Once earh categoxywas evduated, the scores were uuisfured h o the final scoring template
to caiculate a f;nal score for the self-assessment.
4-3:
-
F
I
1
OPIM Citcpory
Objdva
2000 Score
50
1
1
Pemat Score
39%
Maximum Score
Raccsses
60
27%
Proces Managanent
10
49
220
Ginorna rationships
12
23%
25%
Design ~
13
30%
13
3096
23
Shucholder Rcsulrs
309b
30
People Rcnilts
10%
44
Supplier ancl Pvrncr Rcnrlu
5%
33
Society RcsulKs
5%
10
~gProccsscs
Pcrfonnna Monitoringand Gnuolling
12
27%
Perforxuncc knprovanmt
12
28%
43
28%
IO00
Ova;ill OPIM S c a c
281
43
A score of 284 on the first assessrnent indicates that the CSO and the division that
undenvent the d-assessmuir have embarked on the joumey towarb excellence. This
score is not to be taken as an absolute accurate 'proportionw of business excellence (Le. the
Company being about 28% excdlenr), but as an indicator of which areas within the
organisation are further dong than others (Le. some categories received a score of 50h
whereas others scored 30%, meaning the fint category is not as developed as the second
one). It is not the absolute score in each category that is the most important eitha but the
translation of scores into priorities for impmvement that is the most value a d d d
4.2.4
Outcornes
Once the fmai scores of these categones were caldateci, it was important to prioritise the
results. This would assist the in the process of selecting the area on which to focus
improvement efforts.
This prioritisation was based on the score achiwed in a speciiic catego'y and the value of the
category. Table 4-4 illustrates the prioritisation process. The categones are ordered by score
achieved resulting in a score index. The categones are then ordered by total point value
giving a value index. These index values are then multiplied resulting in the total index.
for
IGtcgory
Score
Achiwed
1 Score Index (
4
10
23
15
17
13
11
People Results
Process Management
Information Resources
1Techno10gy Resources
&orner relationships
11
12
Perfomiuice Management
Supplier and P m e r Rcsults
Resoufce Management
Perfommce Monitoring and Conuolimg
R;ilis?tion Prucesses
Desien Processes
11
2
14
12
10
13
Vaiue of
Gtegory
44
44
1 Vaiue Index 1 T o d I
19
20
13
41
45
49
24
44
33
47
43
37
43
18
8
23
16
10
17
456
300
289
286
264
22
I
12
25
7
10
14
8
2 16
200
161
160
140
136
n d
Performance Impmvement
Infr;isuucnueResources
Work Environment
NaturalResources
Shareholder Renilrs
System R e d t s
Strategy Development & Deployment
Support Processes
People fiesources
&orner Results
Pudashg Processes
Financiai Resources
SocietyResults
Management
Score
Achieved
12
8
14
4
9
4
17
8
25
16
7
18
1
27
Scorekidcx
9
19
6
24
16
22
4
18
2
5
20
3
26
1
Vaiue of ' Value Index Totai Index

Gccgory
43
30
45
29
30
28
42
24
53
35
23
40
10
51
135
133
126
120
15
7
21
5
6
96
88
56
54
14
3
26
9
2
12
1
25
1
1
52
45
40
36
26
25
The self-assessrnentprocess providecl the CSO with severai important outcornes. Firn, the
self-assessrnentprocess itself was i m p o m t . The pilot assessrnent will provide valuable
information to the organisation regardmg how best to conduct their self-assessrnent process
in the future. A h , a review of the strengths and areas for improvement found in the
assessrnent will be beneficial as an input to the m u a i business planning process. The final
score on the assessrnent wili be a usefd benchmarkuip tool as weli as a good quantitative
indicator of the statu of the quaiity management system and its level of business excellence.
The observations and recommendations resulting from this wessment were the following
Objectives
Suppliers and partners are not involved in the strategic planning pmcess.
There is no syscematic process to follow-up on action items to determine if they have
been completed and to evaluate the gaps.
Processes
Lack of syste~llitticincorporation of leamhg and customer feedback into design
processes.
No systernatic process for incorporation of current and projected infernal Nsromer
requirements and priorities into the design of support processes.
No review of customer related processes to ensure they are dmeeting custorner's
requkemenu.
Collecf and use data from day to day contact with customers to enhance customer
satisfaction levels.
Resources
Provide managers with a report of only those budget items that thqr have control over.
0
Develop meanires for detennining the effectiwness of employee development and

training.
Develop a formal process for providing timely feedback to suppliers and parmers on
their performance.
Develop a synem for waiuariag the uade-off berween maintaining 'old" techdogy
and investing in new technology.
Results
Review the reporting of perfomance measures ro ensure that vends for the pan three
years are evident and that rargets are explicitly xared.
Performance
Irnplement a review of performance measures used to track rhe performance in each
deparunent to enwe thqr are reflecting the key drives of your business.
Focus on closing the loop. Irnplement processes to e n m that outputs from co~~ective
and preventive actions become inputs to design and process control activities.
4.2.5
Evaluation of Assessrnent Rocess
As this self-assessrnent was the k t for the CSO, the organisation and assessors reviewed
the process to evaiuate its effectiveness and to suggest impmvements. This should become
a reg&
pan of the assessment proceu. Severai questions should be asked within this
process.
Was this process perceived to be successful by the employees, managers,

assessors, and otha interested parties?
This process wu perceived to be successfd by the managers, and directon. The M

presentation was proceeded by many questions and exceiieat feedback This indicated that the
suggestions arising from the self-assessrnent would be reviewed and used as pur of the business
planning process. There was a lack of inte-
on the part of some of the interviewees. This
wouid suggest that more information and support coming from management to encourage active
participation in the process by employees should orin before the n m self-assessrnent is
conduaed.
Was the tirneframe appropriate for pmper feedback into the business planning
process and 0 t h processes that would require input fiom this process?
This self-assessrnent did not oc= at the proper t h e for the results to be fed into the business
p l h g process. These self-assessments should be perfomied doser to the beginning of the
new planning q d e for the year.
Did the question set provide sufficientinformation to enable good collection of

information?
The questionnaire developed for use in this pilot assessment enabied good coiieccion of data and
information. The questions were clev to the interviewees as evidenced from the responses.
Did the selection of inte~eweesresult in a good representation of a cross-section

of the organisation or unit king assessed?
This proces nee& to be reviewed and imPmved in the next assessment. The inte~eweeswere
selected by department managers and directom. A better cross-section of employecs including,
for instance, more people involved in the re;ijisatiion processes, could be selected.
Was the scope of the assessment appropriate?

For this pilot wessment, the xope was entirely appropriate. For the n w assessrnent however,
rhis should be thought out before it is conmicred and reviewed again once it is complete.
Was the number of assessors appropriate and sufficient to conduct a good
assessment?
It is benefiaal ro have a number of assessors from al1 areas of the business. This would result in
a more efficient and effective coiiection of information, since thete would be a better
understanding of the s p d c proceses Mthinthe Qputmenfs. This would eluninate Ume spent
on understanding the organisational processes.
Was the management cornmitment to the process sufficient?

Management commiunent w u fady weak No xnanagen above the lwel of k o r were
present at the finai presentation of the assessment redts. Self-asessrnent redts also showed a
la& of management commiunent.
This self-assessmentprocess involved many people wirhli the CSO. The tirne involved was
lengthy. As this was the first assessment of ths kind performed within this organisation,it is
useful to evduate its value to the Company. One wayto do rhis is to compare the process
and r e d t s to those of a q d t y audit performed in the same rime LamP. This anparison
indudes:
the resources required to perforrn the self-assessrnent and the quality audir
the processes themselves

the type of r e d s obtained from each process
the value of idonnation obrained from each to the o v e d performance of the business
The ne= chapter deais Mth such a comprison proces. It also lodrs at remit-based
perfommnce measufement in &on
to self-assessrnent and quality a u d i ~ g .
5.0 Performance Measurement

5.1
Introduction
The CSO uses several methods for measuring organisational perfonnance. The selfassessrnent desuibed in the previous chapter is now compareci to wo other types of
perfonnance measurement used by the organisation: a quality audit conducted in the same
time frame, and the red-based performance mevureman used to continuously monitor
the processes 4 t h the organisation.
5.2
Cornparison of Assessrnent Results to Quality Audit Results
At the same time as the self-assessrnentwas conducted, an internd audit of the C W s

quality system was performd In the year 2000, the self-assessrnentw u conducted h e e n
October 23d and December 15* (8 weeks) and the q d t y audit occurred during the week of
October ZPto November 2" (1week). The internal audit team within the Quality
Assurance deparunent perfonned this audit with the assistance of an extemai consultant
acting as lead auditor.
Both the audit and self-assessrnent resulted in a set of recornmendauons for improvement.
The audit's recommendations resulted from the non-confonnities and opportunities for
improvement identified by the lead auditor. The recommendations from the self-assessrnent
arose from the areas for improvement found throughout the process. Table 5-1 shows the
main areas requiring attention that were identified in each of these processes as a remit of
the prioririsation process oudines in Chapter Four.
Self-assessrnent (Oct 23-DeclS. MOO)

Highest ladar
People Resuits
Process Miusagement
Information Rtsourca
TcchnologyRsourccs
rnomcr relatioiuhips
Stak&ol&r Objecrives
I n t d Quaty Audit (Od27 Nov 2,2000)

Eiemenrs Requinag Major Attcnuon
4.4 Design Conml
4.6 Rirch;rsing
4.9 Process G n t m l
4.1 1 Inspection, Masuring And Test Equipment
4.14 CoRMiVe And Prcvcntawe Action
4.17 internai Qu?liry Audits
At first glance,the elements identified in the self-assessrnent seem to be ver-different from

those identified in the qualiry audit This is not surprising. The elements identifiecl in the
self-assessment are minly categones that are not covaed by the ISO 9001-94standard The
CSO's quality system is not concenuated in these areas and as a result, these may need
M e r development.
The Process Management category h m the OPIM however was found to be the weakest
element. This corresponds well to the audit findings since Process Management
incorporates elements 4.4,4.6,4.9,and 4.1 1 of the ISO 91 standard.
The category identifid in the self-assessment that does not seem to have been identifieci in
the quality audit is Customer Reiationships. Elemmts 4.14 and 4.17 were identied in the
audit but no related elements stood out in the self-assessment. Customer Relationshps in
the OPIM indudes element 4.3 of the ISO 9001 standard and a srnail part of element 4.4
deaiing with identification of cusromer requirements. Element 4.3 was not identifieci as
requiring major attention but f d e r an+
indiates an agreement h e e n the weaknesses
within Customer Relationships and those found within element 4.4 Design Conml.
Table 5-2 shows the main recormnendations, and non-cornplance issues and ares for
improvement idendied in the self-assessrnent and audit respecciveiy. The table lins the
main caregories in the OPIM with corresponding recommendations.
T h e non-compliuice
issues and opportunities for improvement redting from the audit have been listed
accorciingly. The issues underlined within the table have been identified as si&
issues
arising from the audit and self-assessment. As was illustrated in Table 5-1, many of the
recommendations from the self-assessrnent do not correspond to any in the audit as the
audit covers onty those items in the ISO 9001: 1994 standard. Those that were identified as
major issues in the audit may not have been major issues in the self-assessment. This is
because other issues arising in the self-assessrnent were ranked at a higher impomnce level.
M Qurlity Audit (Oct 27 NOV2,2000)

Objectives
Focus G devcloping a process ro follow-up on
action items to dccumine if thcy hne been
completcd m d to ev;ihiue the gaps
Review the timing of rhe digment m e y to ensure
t h R is opirml to pmvide input into the ;inrnul
Provide mui;igen with a report of ody thosc budget

items that they hne control over
Devclop mevurs for detuminig the effecllveness
of employee devtlopmc~itand uaining
Wew ROI pioccss to msurc consistent use ;uross
the division
Cdlm and mriew dadinfomrztion coaccrning the
effiuency and effmiveness of informuion systems
and availabilityof data

Deveiop a synem for e v a i w the uac-off
betwecn mintaining 'oldw technology and investing
in new technology
Qriltrysysta
Use ISO 90012m direction for dcploying stratcgic
ptnn;np prinC;Plcs t o &me bmuiinkage berweai
svPegic objectives & the systcm dtslgned to achieve
tbmi
N o compiiancc issuCs
No compiiancc issucs
N o compluace issues
No c o m p ~ c isnics
e
N o compLUnce issues
Design Control
Processes
Develop a process to incorporace cumnt and

projected i n t e d cusromer requkmcnts id
priorities into the design of nippon proces~s
Implement a rcview of amorner rctted processes to
ensure thy ue aiil mceting customa's rcquircments
Com and use data from day to day contact with
custorners to enhance customer satisfaction levek
for
to
s-
..
on k
IKtfofminrC
Revicw the rcponing of performance rnclsurrs to

ensure ttut uuids for the put thne ycvs am cvidcnt
Consider using quahy funaion depiuyment to

idaiufy the uue c u n o m a speufications & not what
tbey think thy wuit
Consider ;idopUng suicca DFMEA & PFMEA
during the e V t y p k s
Use tfeasibiity Jgn-off a p p n d i
Move t
d tnic supplier qu;ilif~cationin pkcc of
product quaMcations
Consider a formal sys~cmfor achicving the above
i n t a d Qu?lity Audit (Oa27 NW2,2000)
Conrtrive and Prevenuve Action

lmplemcnt a rcview of perfomunce mesures used
to crack the performance in #ch dcpvunent to
ennuc thcy arc rrflccting the key &ers
of your
business
t cause ui;3yUs is i n a d a p t e
Corrrcllvc actions not responded to in a timeiy
muuicr, and somc n a rcspondcd to at d
l
Mcthod for esublishing effecrivcnessof solution is
Intenul Audits
Prioritise audit ;ictivillcs b w d on due-added and
effective w of reu,urces
Numaous comcWe isnies ourstuiding. Consider
an &on
Clause
No objective evidence that mut?gcmuit personnel
rwiewcd audit fndings
Indepuidence of auditors not evident in di cases
Statistid Techniques
Several issues can be ewacred from this comparison. A mjor problem identifid in the
q d t y audit (element 4.17) was that there was no objective evidence that management
personnel reviewed the audit fmdings. This was also emphasised by the fact that there were
numerous corrective issues found to be outstanding. This is one of the main differences
berween the self-assessrnent and auditing processes. It is evident that the outcornes or
recommendatons from the quality audits are not reviewed and acted upon to an acceptable
level. The results are not fed back into the business planning process. The self-assessrnent
fmdings on the other hand were presented to senior management and were considered
usefil to the planning process.
As discussed previously, the renilrs of an audit are simple pass or f
d judgements. The
assessrnent performed against the OPIM was given a score based on the approach and
deployment of the approaches that the organisation has in place to mea the given criteria of
the model. It is difficult, therefore, to compare the f i score or outcome of these two
processes. If there were a methoci for cornpuison, the findings would show that the score
on an assessment againa rhis model Qcreases with fhe number of non-conformances and
elements requiring major attention in a qualicy audit performed in the same rime frune.
The OPIM has attempted to incorporate the basic reqykments for the organisation's q d r y
system. If there were any non-conformances arising from the audit, it should follow that the
corresponding element in the OPIM would be judged as inadequate. If the minimum
reguirements have been met, the score for the eiemem in the self-assessment should be
hrgher. It does not indicate however, that ail the critexia in the corresponding element of the
seIf-assessrnent have been met. Figure 5-1 iliumtes this concept.
The ana of the triangle
represents the relative percentage of elemenu with:

no non-conformanceswith a low score on the assessment
no nontonfonnances with a high score on the assessment
non-conformances with a low score on the assessment

nonsonformances with a hi& score on the assessment
Self-Assessrnent
Low
I
Wouid h?\e nonconformanceswhilc
0bt;ilring a hi& score
on the self-asxssment.
Audit
The r e s h of the pilot self-assessrnent and the @ty

audit with respect to the concept in
Figure 5-1 are illustrateci in Table 5-3. A l m score on the seif-assessrnent is a score in the
bottom @e
of the xores on the self-assessment. A hi& score is one in the top quanile
of the scores.
QualityAudit
SeIf-assessrnent Score
Low
Conforrmnce
Nontonformance
This corresponds well with the hypothesis presented in Figure 5-1. The highest percentage
of elemenrs is in the quarter for low scores and non-conformances. The lowest percentage
of elements is in the quarter for high scores and non-conforniances. The other rwo quarters
are almost equal.
Figure 5-2 compares the scores on the OPIM to the elemenu of the quality audit that weie
found to have non-conformities. On average, the self-assessrnent xores were lower when
corresponding ISO 9001 elements were found to have non-conforniinces. The results do
Vary. Keep in rnind that the elements of the OPIM in moa cases cover more than
corresponding elements of ISO 9001. It is possible to have a low score on the OPIM men
if there were no non-conformances found in the audit. There should not be however, mun,
elements in the OPIM that achieve a high score if rhere are non-conforniances found in the
corresponding element from the ISO 9001 standard
5.2.1
Implications for SirnultaneousAuditing and Assessrnent
The self-assessmentperfoxmed againa the OPIM provided much information regarding the
effectiveness and the efficiency of al of the organisation's processes. In particular, suengths
and areas for improvement were identifieci in processes related to quality management. A
comparison of the resuits of the self-assessrnent and the quality audit rwealed that similar
issues were idenufied in each process. The implications of this are far reaching.
It is evident from this exercise that performing self-assessrnent on the elements related
to q d t y management c m provide valuable information regardhg rhe effciency of
these processes. This c m cl+
of such processes.
the quality audit, which aims to judge the effectiveness
The men& and weaknesses identified in the self-assessrnent process are incorporated
into the business pknning pmcess. This ensures that the areas for improvement, related
to the quality management system, are iopns h o rhis planning process as w d . This
compensates for the fact that the results of quality audits are not incaporated into the
business planning process.
management, especdy if perforrned
Self-assessrnent of the processes relating to +ty
a opposing times of the year from the interna1 quality audits, would lead to further
improvements to the efficiency and effectiveness of these processes.
5.3
Result-based Performance Measurernent
In addition to quality audiring and self-assessment, the organisation uses resulr-based

performance meanires to u n d e m d the curent perfomiyice lwel of its processes. This
section fm describes a review of the measures cumntly tracked by the CSO and a
comparison of rhis set of measures with measures suggested by researchers and in the
national quality award models. Fin*
a cornparison of these three performance
meanirement me&& is described in the final part of the chapter.

5.3.1
Review of Organisationai Performance Measures
Self-assessmenr against the MBNQA and the EQA involves the evduation of organkationd
results. These reniks are assessed based on the history and trends of m m e s that have
tracked perfomiance in each of several areas. Each a w d suggests areas of performance

that should be monitored by an organisation. These are found in the Results caregory of the
EQA and the Business R e d t s categoxy of the MBNQA (See Table 5-4).
MBNQA (Business Rendu)

Gstomer
Finuicial and Market
Human Resource
Supplier and P m e r
Omnisauond Effectiveness
EQA (Result4
Customer
People
SRY
Performance
It is the a
d results (trends, levels, ben*
in each of these areas rhat are evaluated
in a seIf-assessment. The self-assessrnent does not appraise whether the organisation

amdlymeasures perfonnuice in each of the areas. If there is no measure for a speciGc
element of performance, the element cannot be wessed.
Gregory 5.2.3 of the OPIM, Performance Meanirement, uks the organisation to describe
its perfomiance measurement V e m (see Figure 5-3). It is at this point that the
organisation's resuit-based perfomiuice measurement system is wessed It is also here that
the organisation has the opportunity to i d e n e any gaps in ternis of areas it needs to be
meamring.
5.2.3
Performance Mcuuremait (IS04,8.1,82 1, 8.4)

hpmmus (MBNQA, 4.1a1 and 42)
Describe the performance measurement systcm uscd to collect, monitor. analyse,

synthesise and act upon perceptions (extcrnal) and performance indicators (intemal)
relating to:
~0~
people
supplias and partnen
Society
qptm-
WNQA4, . 1 4
Describe how the effectiveness. emciency and consistency o f the performance

measurement system are continually improved.
OPIM
5.3.2
Review of the Performance Measures at Case Study Organisation
The CSO has IWO main divisions. Each of these has its R&D, marketing, and manufacninng
components but are assisted by the same support departments. These are cornputer
services, quality conuol, quality asniance, hurnan resources, and finance. Division A
focuses mady on new produa development. This means the design of new integrated
circuits @Cs) to meet the needs of its customen. Most of the new products developed are
manufactureci out of house. Division B has both new product development and on-site
manufacturing. The divisions are quite different h m one another but both must conuibute
to the ovedl adiwement of profitable gr&
Most departments have dwdoped their perfoxmance measures from the objectives of the
depanment, which in tum are developed h m the CSO's overall objectives. To improve
the C D ' s performance measurement system, it is important to identify gaps where aspects
of performance are not being measured.
The CSO's performance measurement system is made up of several perfomiance meanires

uacked in each depanment of the organisation. The following measures are reportecl on
monddy (See Table 5-5).
Explmation
The pcrcmtage incmasc in d e s over the pmiious ycu (5 veu compound)
(Prr-tuincomc-invesmicntinromc-tucs)/~oril~-cuh+ ST-
Ilbiliries)
Qualicy
Assurance
(to QA) in pans & - d o n (AIQ

@pm))
Producr Group 1
P d u s t Group 2
Prodm Group 3
(Incomc/ average equity)

(incorne beforc tax + d a r i a and bcnefits)/(numba of crnployces)
Percaitigc inarve in shvt pice over the pmrious y a r (5 y a r compound)
This is the numba of defects o k c d on di batches submitted to QA for
-pli%
This is the nurnber of defms observai in thc acceptable batca a the QA

(AOQ ( P P ~ ) )
CAOQ (cuxoma average
outgoing quaiity)
Qwlny Assurance Inspection
cvdc
umc
This is che number of defective renirns observd in the total quuitiry s d c i
This is rime from buch urivai ro arrivai u stores

g produas want to knaw the 'rooc cause" of fdurc
Clrnomcn d
..
A d r m e d life testing QuLfying
Coafirmcd dcfecve from cunoma by QA
This is the p e r c a i w of corponte revenue genented from p d u c t s
NP1 New
Roduct
Introductions
Sales
revenue
invcuaiin'~~~
Pmrntage
effort for top ten
. .
This is the pcruntagr of anplqrcc t h e spenr on the cumnt top tai priority
Producrion rate vs. MPS (%)
-&on
Wdcr Fab. yield (%)

BasiiiccyderLN
Product mU vs. MPS
'
Cyde
1
1
1
1
Amui pfodunion vasrrr the w produccion schcdulc

The d e r yield before elcarialunLig (physicd ody)
T i from the stzrr of firn operrtioa in production to end prochicr into
invmrory (m
produns)
The actuai mix of produas compvcd to the sch&
mix of produas in
nwerproducrionschcdulc -
1
Asscmbly Opcrations/Test/Extemal
Master Production Schedulc

(MPS) Rue (%)
Yicld (%)
Asscmbly
(w*)
Accuai prafuaion vernir the miner produaion schcdule
Number of finished pro du^ v a w numba of produns stvtcd

T i from the ninof firn operaion in production to end prodricr into
uiventory
The amai m ofproduns ompued to the d d c d mix of pro& in
Testing
(Extemal
and Intemal)
The actual mix of produar testcd cornpucd to the scheduicd mir of turing in
~produaionschcdulc
.
AIQ (ppm)
Prototype cyde timc inhowc/offshorc (w-bys)
Several interviews were conmicted to gain information about the current measures being
used. These measures were reviewed with the manager of QA, the VP of Research and
Developmen~the VP of Manufaauring, and the VP of Finance. Each one was asked a
number of questions pe~ainingto these measures.
Quesuons
Which
performance
indicarors are most
important to you in
determinhg the
performance of
your department?
VP Finance
meaSures are
imponant with the
possible exception
of the Quai~ty
Assurance
Inspection Cyde
Time, which is
done for customer
satisfaction
reasons. These
meilSufes are a
good indication of
the performance of
the QA
deriment.
measures thu you
feel are not quite as
useful as they
could be?
measures are
manufacnuing
COR
on-rime
delivery
qdty(%
defects)
production of
appropriate
volumes
Other
healthand
safety
although these
need to be
explained
thoroughiy to the
attendees at the
montMy
performance
meetings so that di
measures used are
undersrd
Quesuons
Do you fed tht

yourperformuice
indiuton are in
line with the
go;ils/objectives of
Yes
How ue these
How does the

performance of
p u r deparunent
relate to the o v e d
perfommce of
The organisation?
tracked and
reported on?
How is the
information from
these measured
used?
depaKmenf
determines the
outgoing qu;ikty of
products and the

reliability of new
desim. This has a
g r 2 effm on The
Measures are
reporteci on
monthiy
Measures are used
to fmd areas w here
d t y
performance is not
up to targets
How are targeu for The targets for aU
these measures set? these measures are
1s bencharking
25% improvement
done? If so how is over previous year.
t h done?
Target was 50h
but after improving
the 'low huiging
fruitn,it was more
difficult to achiwe
this 50%
improvement.
Yes
There is a database
for tmcking these
measures (on-line
NP1 &tabue)
Mon of these
n w s w e s are
tracked in the QIS.
Finaud mesures
are uacked in
aflother
inforniwon
sysrem.
The performance
of the
~llil~lufacfuring
dep;i~mentaffects
cusfomer
satisfaction and
~lll~lufacniiing
The performance
of the R&D
department
determines how
fast the Company
grows
It is the primvy
cost driver of the
business and
deterrnines the
quality of produm.
The performance
of the QA
organisation's
How often are

these measures
Yes
Yes
M m e s are
CalcUlated by
accounting d e r k
from information
in another
information
syst=.
The finuicial
measures are the
m?in indiaors of
The orgmktion's
performance
COSU
Measures are
coiiected and
reponed on
mon*
This information is
used to focus
improvement
efforts
Me;isures are
collected weekly
and reporteci on
mon*
Measures are used
CO frnd problems in
the production
pmcess
Measures indicate
how the
organisation is
performing o v e d
Set realisticaiiy
based on historiai
Targecs are set
Targets are set
baseci o n
forecasteci dernad,
some
based on an
increase of 20 in
saies and revenues
each year
benchmarking such
as on-time delivery.
Benchmuking is
done m d y from
a process
perspective not just
to determine
tuners.
coiiected monthly
Based on this set of interviews, the foflowinglin of strengrhs and areas for improvement
was created.
Strengths
The measures used in the Quaiity Assurance, R8rD and Manufactwing are felt to be
adequate to capture perfoxmance in each depanment.
The meanires are felt to be in-line with the organisation's strategy.
The CS0uses its performance measurement infommion in monddy meetings.
A r e a s for impravemcnt
Most measures are monitored monrhly. Some may need to be mcked more o h .
There is no systematic review of these measures to ennue that they are d l meeting the
needs of the organisation.

Littie bendimuking is done to set competitive targets for improvement.
Targets are set according to historiai performance in R&D.
5.3.2.1
Identifying Performance Measurement Gaps
Several perfoxmance measurement models were reviewed in diaprer 2. It is useful to
compare the set of measures used by the CSO to these models to idendy a .areas whem
the organisation does not meanire performance.
Table 5-10 places the CSO's perfoxmance measures wirhin the four penpecrives of the
Balanced Scorecard (BSC).
Fimacial Perspective
Sales growth
RRurn on Average Net Assets (ROANA)

Rcturn on Equity (ROE)
I n d Business Perspective
Design cycle time
De*
4 w
Devclopmait effort
Vaiue addecl per pcnon per year

EPS growth
Milestones on time
Manufaawingcyde Urnes
Design-ins
Yield
Supplier on-time delivery
Qu?lity of supplier products
Billingacmmq
Customa Perspective
Customer Satisfaction Survey
Fdure Assessrnent (FA) success nte
Customer renvnstechnid
CAOQ (CunomerAverage Outgohg Qdq)
Prdua reliabhy Ievels
On-Ume shipments
Backorden
Cil response
Ability to accept orders
Innovation and Lcaming

hvestment in NP1
New produa inittiom
New producc introductions
Tnining hours per employee
Training costs as a % of salary
Ir is evident that the C D ' s measures fit well inro the BSC famework The weakest category
seems to be Innovation and Learning. The CSO should find several more measures
penaining to its performance in this area Sorne suggestions follow in the next section.
Neeiy et ai (1999) niggest that an organisation's manufacturing perfomiance measures

should f
d into four caregories. These are qyahty, time, con and flexibility. These can be
extended to cover the performance aspects of other deparunents as wd. The performance
of design and development processes, for instance, can be viewed from these four
perspectives. Table 5-8 shows how the CSO's measures can be assigned to Nedy's elements
of performance.
Mmuf?aiPing
AIQ ( P P ~ )
AOQ ( P P ~
CAOQ (customer average outgohg quality)
Product reliabiikty levels
Customer rcnunstechnical
Production m e vs. MPS (%)
P
&
Yield
Prohuct mix vs. MPS
Sdety
FA
m e
R&D
Mask d o
% First tirne right
Humui Rcsourccs
ProductiY;ty/E mployee
TrainingH o m per employee
Hit me (%)
Qulityof hire
Rtention rate
Testing and Stntcgic Sourcing
Quality of WO accepted
Muiufacturlllg
Qu;ility Assurance Inspection
Muiufacnuing Lead
Tie
Rare of production
ind&on
Deliver lead time
Due-Date performance
Frequenq of delivery
Production Cycie times
O n T e Dehiy
R&D
'
Manufaauring cost
Vdue added
S e h g pice
Running con
Service con
Cost of Qu;iLty
Design con
Distribution con
Cost dative to
cornpetitors
Materiah
Inventory
Design Cyde Uaie

Development effort
Milestones on time
Average v k c e
Human Rcsourccs
Ovenime - # of employees > 100 hourdyear
Tcning and Strategic Sourcing

On time shipments from Suppliers
Flembiliry MPaGlquliry
Manufrcturing
B;ickordas
Rtw
Ncw p&
;n;t;lt;ons
(MG)
New prohua imroducaom (MR)
Humm Resourccs
0urPutQu;itty
N-vproduct
Modify~h
D&mbility
Volume
Mix
Resourcemix
Ov&
Those from the C S O ' s

set of pafonxunce
Other
- # of cmployecs > 100 hours/year
Tcsting and Stmegic SourQng

RespoWveness of supplies (1 low - 5 hi&)
Sales growth
R O M
me?surrs thu do not
ROE
fit in these ategories
Value addcd per person per year
EPS growth
% Revenue h m NPI projects iaunchcd within last 3 years
Investment in NPI
NP1 investment as % of forcas menue
Percentage effort for top ten priority projects
Design-ins
Sales
This set is comprehensive. The con category however, is quice weak. This uea should be
reviewed to ensure that the organisation uuly understands its costs. O n e important aspect is
the identification of the c o s of quaiity. This is not addressed at all by the CSO's
performance measurement system. This indudes looking at preventative, appraisal and
fadure costs to show management the costs of poor quaity, to idenufy opporninities for
improvement, and to idenafy hiden costs.
A third set of measurement categories is found in the Resuhs sections of the OPIM. The
OPIM incorpontes the meanires suggested by the MBNQA and the EQA. Table 5-9
compares these measures to those that are mcked bythe CSO.
i OPlM
Reput;ition
Ourpum
Case Study Orguiisation

-orner
Satisfaction Survey
Fdure fina+ Success Rate

-0rnerRcnims
CAOQ
On-the shipments
Interfaces
Backordcrs
GIl Response Rare (Applications)
Financial
Sdes Growth
ROANA
ROE
value Added
EPS Growth
Non-Finuiciai
Design-hs
New Prociuct Inm>micuons

Tirne to rnarket (N'PIcyde Urnes)
People
Interfaces
Motivation
Satisfaction
Hours of Tnining
Employee Alignment Survey
RRenaon
Absen&
Achiwemeats
Supplien and
Parmen
Society
Services
Lnputs
Interfaces
CRizenship
Cornmunity
Environment
Hdth and S a f q
-of
Hire
Enuy level/expenence %
Empluyee Productivity
Accession (% hedcount)
AiQ for Offshore paru
Offshore cyde tirnes
On-tirne deliveries
Quai~tyof WO accepted
Responsiveness
Health and Safety

Accidents
Incidents
Days lost
Interfaces
Design and Development
IC cyde Ume
Development
W
t
y
T o d Development
Effort
AIQ for Offshore
P-
Offshore cyde h e s
On-Ume deliveries
Qu;ility of WO
accepted
Responsiveness
Production rate vs.
MPS
Produa mix vs.
MPS
Production cycle
times
Yield
AIQ
Quality-ce
Ins~ectionT i e
Management
Finances
RONA
ROE
[nformauon
[nfnsuucturr
People
Produ&+ per
employee
Design Effon
-
Nature
Work EnWonment
EPS
Net Profit
On-tirne shipments
Schedule Accuracy
FA Success Rate
Mes
Design-ins
Accordhg to the utegories of perfonnuice in the OPM, the CS0 is missing perfomiance
measures that reflect the effiaent and effective use of the organisation's r e m c e s . Table 510 provides suggesred meanires that shodd be incorporateci into the CSO's m
t set of
masures to monitor the use of resomes. The suggested meanires have been taken from
several sources induding the E Q A
- Suggested Pcrfamancc Measuns to Add to Existing
Pcrformancc Ar#
Source
Sct
Exterd Resources
C
Supplier psice
Numkr and vaiue added of p d p s
Inventor-leveh and timing of deliveries
Number and vliut added of innovative product solutions
1 Recoenition of m e r s ' conmbuons

c.
Financial Resources
EQA
t ratings
Defea rates of mu&

Resources
Technology Resources
I
Information Resources
inventory tumover
Utility consumption
Lhihation
~ & i i iof~toolr
Value of i n t d d property
Patents
Royalties
Accessibility
kitegrity
Relwance
Timeliness
Sharine and u s m knolk'ledee
Vdue of intellecd capitd
Y
Shaw, 1999
Csb flowitems
Balance sheet items
Depreciation
Maintenuice corn
M
EQA
EQA
EOA
EQA
EQA
EQA
EQA
EQA
EOA
EQA
EQA
-.
EOA
- -
Ba*,
1994
EQA
EOA
-
- -
EQA
EQA
EQA
EQA
EQA
EOA
EQA
In addition to the resource indicators, the CSO does not have indicators that show what the
organisation is adUeving in relation to s o c i q . Some of the measures that are s-ested
in
literanue are k e d in Table 5-1 1. The CSO should inco'pome seved of these mevures
into its m e n t set of performance measures.
Pcrform;uice Arca
Performance as a responsible a&
in volve men^ in the communitieswhere it

operates
Reporting on activities to wist in the

presemation and SUSt3iZI;Zbility of resources
S u-g- g c d Pcrfmance Masures to Add [ Source
toExistingSet
Disdosures of information relevant to the
cornmunRy
E q d oppominities practices
Impact on I
d and nationai economies
Involvement in education and &&g
' Volumvy work a d phiianthropy
Support for sport and leisure
Choice of avupon
Ecologid impm
Reduction and eiimktion of waste and
EOA
-x--
EQA
EQA
EQA
EQA
EQA
EQA
EQA
The CSO d s o does not report on services for employees. Table 5-12 indudes a lia of
suggested performance measurrs covering rhis perfomiance area.
Services for Employees
Benefit lwels
Provision of facilities and services
EQA
EOA
In general these tables address the three areas where the CSO's perfomiance meanirement
system is weak Thus, performance measures should be designed or adopted to reflect
performance in the following areas:
innovation and learning
use oI resources
interaction with society

evaluation of empluyee services
The 0should a h rmiew the suengdis and areas for improvement related to its remitbased performance measurement system identifid in this chapter.
5.4
Self-rssessment, Auditing and Resuh-based Performance Measurement
It is evident that self-assessment, audiring and result-based performance meanirement are ali
benefiaal to an organisation's performance improvemem efforts. Each one requires a
different cype of input and each r e d t s in outpins that become inputs into the performance
management system of the organisation. Table 5-13 recaps the characteristics of these
performance mesurement methods.
Self-assessrnent
Inputs
' Effort
Outcornes
Measure
Opinions
Obiective evi&nce
Possible hi& effon l
wei
Qulttative
Subjeaive (Based on
Result-baseci pe dormance
measunment
Numeric &ta
Audrnng
Objective evi&nce
1
Medium effon ievel
opinion)
Pfocess-based
Dependent on system
Quantitative
Ma+ Objective
Specific
Holistic
Effiaency anci Effecveness
Effectiveness
Efficiency and Effectiveness
Qualitative
Objective
These methods each have its specific role to play but are obviousiy quite interdependent.
This next section describes the relationship arnong these meanirement methods and how
they can be used together to improve organisational perfoxmance.
5.4.1
Cornparhg Self-Assessrnent and Result-Based Performance Measuremcnt
The OPIM, like the EQA and the MBNQA, contains cntena that address the Enablers and
Results of the performance management system. In self-assessment, the efficiency of the
Enablers is judged. Their effectiveness is mevurd by the Redts. Figure 5-4 illusmates this
relationship.
SELF-ASSESSMENT
Efficiency of
Effectiveness of
Ir is the R e d t s category, in fact, that represents the organisation's result-based performance

meanirement system. Performance measures are used to detennine the nirrent effecWeness
of organisational activities (Enablen). In other worb, to what exrent these activities are
contributhg to the overall performance (Results) of the organisation.
The CS0 has a well-developed result-based perfomance meanirement synem. The
meanires in this system provide quantitative and qualitative information concerning the
efficiency and effectiveness of activities. For instance the CSO uses Quahy Asnirance
Inspection Cyde Ume as a measure of the how efficiendy the QA department performs its
inspections. It also racks customer satisfaction, which is an indiwor of how effectively it is
meeting cunomer requirements.
O n e wodd ask then, why wouid an organisation need self-assessment. The anmer lies in
the limited abiiity of these meanires to evaiuate the approaches and deployment of
approaches used to perform organisational activities. It is hem that self-assessment becomes
important. This process imrolves a more in-depth a d p of~why the activity is at the
current performance lweL Self-wessment, for instance, can provide information about
how the QA department performs its inspection processes thus shedding light on why it's
effiaency is where it is.
Resuit-based performance measurement provides a snapshot of performance levels. Selfassessrnent evaiuates the causes of these performance levels. It is &dent that these nvo
performance meanurnent methods are complementary. Used together they can provide
valuable information for improvernent.
5.4.2
The Roles of Self-Assessment, Quality Audits, and Result Based

Performance Measurement in the Organisational Performance
Measurement System
This chapter has illustrated the importance of each type of performance measurement to an
organisation. Figure 5-5 illusvates how these fit into an organisation perfonnance
measurement system. The result-based perfonnance measurement system consins of
perforrmnce meanues. The OPIM exisrs as a reference mode1 for self-assessrnent and
encompasses the elemenu that are audited against ISO 9001. The results of self-assessments
against OPIM will provide an indication of the CSO's overall performance, and therefore
will be added to the set of existing performance m e m e s .
PERFORMANCE
MEASUREMENT
w
Audit
Assessrnent
i
5.5
Chapter Suimnary
Li this chaprer, the processes of self-wessment and @ty
audit*
were cornpared. niis
reveded the several implications for the simuitaneous use of audits and self-assessments
within an organisation. The chapter dso described the review of the CSO's perfonnance
meanvernent system and rhen compared the use of redt-based perfonnance meanirement
to the use of audiring and self-assessment.
6.0 Pdormance Management In RdrD

6.1
Introduction
This diaprer outlines the modification of the proposed model, the OPIM, for use in an
R&D environment. These modifications were based on issues raised in the literature review
and practical experience gained t h u g h the self-assessrnent conducteci at the CSO.
6.2
The R&D Environment at the CS0
The CSO, as an organisation, is very much focwd on its R&D efforts to compete in the
fast paced marketplace. There are several R&D reams woiking in each of irs three business
areas. After interviewhg several of the managers in each of these areas, it was dixoverd
that mon of the issues mentioned impact the efficiency and effectiveness of the R&D
function.
The R&D management is very much focused on reducing q d e Urnes and ensuring that
products reach the marketplace as dose to fun as possible. It is currently woriung on
increasing the awareness of the vaiue of time amongst R&D ream members. Through
several benchmarkhg mdies reporteci on in the CSO's strategic pianning meeting, it was
discovered that Ume delays con much more in the long r e m than does exceeding the project
budget to get the work done on cime.
One author found, mggesteci that in a market with 20% annual growth and 12O/0 pnce drop
per year chat a delay of six months while on budget wdl earn 33% less profit over life but
introducing the same produa on rime with a 50% budget overrun cuts profits by only 4%.
(From the 0 ' s strategic planning meeting)
The CSO is very aware of the need to malitain its RgrD efforts. About 20% of net revenue
is invested back into R&D. The CS0 is constantly exploring other markets and expanding
its opporruniaes.
Division A is involved in the design of integrated Circuits for th& application to a particular
segment of indu-.
Most of these products, once designed, are manufacnued off-site.
7'he mlin activities of the division are therefore R&D relatecl.

6.3
Modifying the OPIM for Use in an R&D Environment
The OMM is based on standards and models that have not s p e a f i d y been taiiored for use
in an R&D environment. To be effective as a mode1 for R%D, several modifications needed
to be made.
R&D goals and objectives shodd be defined in a waythat shows de+ how each
researcher's activities contribute to the company's goals. These should be aligned from the
a m with the goals of the organisation. If organisational objectives are undear, then the
nom is generally for individwls to outperfom othen through a system that r e w d vaiues
conduave to organisational goals (McLaughLn, 1995).
diat are not
Overaii Organisational Goais
R&D Goals
Organisationai goals should be translated into goais for the R&D effort and subsequently
individual goals for each researcher.

The literature review outlined a number of issues that F&D groups are facing in today's
marketplace. It is essential to consider each of these when designing a qwLty management
system for the R&D environment. Table 6-1 iflusvateshow the characteristics of a wddesigneci quality management system could vldress each of the issues.
Issues Affccting the RdtD Enviranmcnt

Individual recognition of achicvcmcnt
ChPrctaStics
Emphask on developmcnt of team baseci
Oqpisationll goais vs. R&D objectives
recognition
R e w -d s based
- on- the dievernent of
o ~ o n agoals
l
Fast pace
An environment that is focused on flexibiky and

adptabiiity and empowered rcsarchcrs
Incrclsing cycle time
A systan t h e n c o q e s the systemtic
I
I
Fewcr rcsourccs
Inaeased cornpetition
GQfLving the customa's changing nccds
Emcrgence of ncw technology
6.3.1
I d e rimes
I
A system that leas to improvernent in the
effectiveness and efficiency of the R&D proceses
which enables improved performance with fewer
1 resources
'
A syaan that piaces the o r p k i o n in a position
to unckmnd ascorner's current and future needs
to continuously iinprove its ab*
to fuifil these
nees and remin competive
A sysrem that enables the effective management of

technology
Framework
The R&D hinction can be viewed as a mbsystem of the organisational management sysrem.
In this way, R&D should develop its own quaiity management system chat is aiigned and
easdy integrated with the o v e d organisational management system. This will ensure that
the R&D mbsystem is operating to achieve goals that are w d aligneci with the organisation's
overall goais. The system is comected to the overall system thmugh balances and checks to
ensure continuous alignrnent of goals and objectives. Figure 6-2 illustrates the framework
OPIM to which an RBrD subsystem has been ad& The mows in the
figure depict the interrelationships between this subsystem and the o v e d OPIM to ensure
for the modified
continuhg aignment. For example, compare Figures 3-1 and 6-2. h e OPIM can be
applied to the qmem as a whole but the items can atm be tailoreci to be used for the R&D
system.
PROCESSES
RESOURCES
PROCESSES
i MANAGEMENT i
MANAGEMENT
R&D
PROCESSES
The research and development funaion can be viewed as a system of processes whose most
important resources are technology, people and understanding of the customers'
requirements and expectations.
Once the f~ameworkfor the modified OPIM was created, the next s e p was to modify the
cnteria to reflect the specific requirement of a quaiity management system in R&D.
The categones of the OPIM should address the follo*
Obieaives
The Objectives section of the modifieci OPIM should focus on management, R&D nategy
fomulation and alignment with organisational suategy, and the main nakeholders of the
R&D function. These stakeholders include interna and external astomers, senior
management, people, suppiiers and Pamiers, and society.
Resources
This section should cover all resources indudeci in the origiuai OPIM but should focus on
how the RSrD function plans for, acquires, deplay$ and controls these resources Li order to
develop produas and new technology effiaendy and effectively.
Processes
The Processes section should adress process management and the planning and conuol of
customer related processes, design and development processes, purchashg processes, and
support processes. Realisation processes originally addressed in the OPIM are not required
in this version.
This section should determine how customer requirements are completely defmed. These
requirements 'drive" the design convoi process so customers are aswed that the
organisation will design the appropriate processes to ensure cornpliance. The R8rD
professes should be planned and designed frorn the customer's point of view. This indudes
everyrhuig normally associateci with design, development, test, prototyping, and
evaluation/assessment.
The R&D organisation cian re~uestrhu supplien demonstrate evidence that pudaseci
product will m a all quality reS;rements. This permits the R&D organisation to ensure
that purchased items and seMces meet the needs of the depanment.
The key support pnxesses r e q d for effective and efficient R&D should be planned and
conuolled.
Results
In the R8rD environment, the rime lag becween action and outcome is prolonged This
makes difficult the development of a comprehdve set of masures that accurateiy reflect
the outcornes of the R&D activities.
Sevenl authon have addrrssed this problem and have corne up with suggestions for
meanires that can be used in R&D organisations.
The funaion of the R&D organisation is not only to develop but &O to transfer technology to o p t i o n s
and the marketplace, to enable the compuiy to meet its business goais and therefore maximise stieholder
value. R&D's conuibution, therefore, is n a to be m
d simpiy by i n t d success mevurements but
by its performance against interoll ;and e x t d customer satisfaction short and long-term pals. (Patiiio,
1995)
In general though, these meanires should reflect the achievement of set objectives relating to
customers, people, suppliers and pannes, society, and the synem itself. The meanires
should be reported in terms of the a
d level of performance, targeted levels of
performance, trends over the past three to five years, uid performance relative to
cornpetiton.
This section should address the systems for perfomiance management, performance
monitoring and conmlling,and performance improvernent related spediuUy to the R&D
depumients. It wu not necessvy to modify this section h m the O&
OPIM since the
departmental performance management, monitork and controulng and improvement
should be si&
to that of the CSO.
Modifications were made to the OPIM to d o w it to be more applicable to an R&D

depanment within an organisation that develops, mukers, and manufactures new produm.
AppendM B conuins the m&ed
version of the model. The 'organisation" refers to the
whole organisation induding marketing and manufacturing departrnents. The "R&D

funcrion* refers to the RBrD department or mit.
The following is provideci as an sumpte of the modification made to the criteria of the
OPIM.
Section 1.2 of the onguial OPIM perrains to the establishment of quaLty policies and
objectives and the o v e d suategic ph.ningprocess of the organisation. Table 6-2 illusuates
the critexia from this category of the OPIM.
1.2 Strategy
1.2.1
Development
i?q&mm&(ISOI,5.3 and 5.4.1)
Describe the sysiem for establishing quaty poliaes and objectives.
(IS04,5.3;MBNQA,2.1; EQA, EQA, la and 2c; DP,1:l-1:3)

Describe rhe strategic planning process appLed to improve organisational performuice
and set appropriate strategies and pliaes.
1.2.2
Deployment
h p m m n k (iS01,5.4.2,5.5 and 5.6)
Describe the processes of management planning and review, as weii as the wociated
respoasibilities and authorities.
qnpimomies @Sa, 5.4-5.6; MBNQA, 1. Ib and 2.2;EQA, 2d and t e ; CP,1:4 and 15)
Descnbe h m suacgic pians are deployed, reviewed and improved upon.
The developrnent and deployment of strategy in R&D ne&
to be linked to the overail
business planning process. This requires uuisktion of the organisatiod goals and svategic
objectives into R8rD department goals and objectives. This transformation enables the
R%D funaion to better understand the corporate stratefies and to ensure the aignment of
its suategic goals and purpose.
Table 6 3 illusvates the modifications that have been made to this categoiy of the OPM.
OPM
Rqukwa @SOI, 5.3 and 5.4.1)

Describe the syscem for rransiaMg organisational qulity policies and objectives into
p h and objectives for the R&D function.
Qpamm (IS04,5.3;MBNQA, 2.1; EQA,EQA, la and 2c; DP, 1:1-1:3)
Describe the stmtegic pluroig process applied to improve R&D performance and set
appropriate strategies and policies.
1.2.2
Deplayment
ibptmm @Sol,5.4.2,5.5 and 5.6)

Describe the processes of R&D management pluining and review, as wd as the associated
responsibilities and authorities.
(IS04,5.4-5.6;MBNQA, l.lb and 2.2; EQA, Zd and 2e; DP,1:4 and 1:s)
Describe how strategic plans are deplayed to the R&D function. Describe how the
strategic plans, translated to the R&D function, are reviewed and irnproved upon to
ennire continuing alignment wirh organisationai p h
The scoring scheme for the OPIM wu developed from the opinions of people w i t h the
organisation. These people undemuid the importance of each part of the criteria to the
performance of the business. This sarne process could be foilowed to modify the scoring
scheme for R&D application. Expen opinion from those within the R8rD depments
could be c a p d using the PFM software and a new distribution of points could be done.
This wu not possible in this case because of tirne collsvaints, so a change in the disuibution
of points was made to address the issues identifieci in the litennue nwqr.
Table 6-4 shows the categoxies of the modified OPIM that mon &cdy
Issues Affectiag the R&D Envimmcnt

Individual recognition of achievement
0rpUi;sauonal goais vs. R&D objectives

Fast pace
effect these issues.
1.1 MaMganait
1.3 Stakeholders
2.5.1 Hunan Resource Sysvm
2.5.3 Satisfaction, Involvernent and Empowerment
4.3 Peonle Rcsults
1.1 Managrnient
1.2 S m q y
4.6 System Results
2.5.1 Human Resoutre System
3.3 Design and devdopment processes
4.6 System Results
2.5.1 Human Rcsource Systern
3.3 Design
- and devdopment processes
4.6 System Rcs&
Fewer resources
Increased cornpetition
Capturing the customer's changing neeb
Emergence of n m technology
2.5.1 Hunan Resource System

3.3 Desigu and development procases
4.6 System Results
3 2 Customer duionships
3.3 Design and development processes
4.6 System R d t s
3.2 Customer reiationships
4.1 Customer Fkdts
2.6 Technology
4.6 System Results
These categories should be vaiued the highea within the modified OPM model. Table 6-5
presents the s c o ~ scheme
g
for the modified OPIM.The items in boldface rype are those
caregories that have become the moa important in the rarised s c o ~ scheme.
g
Criteria
Ctcgoy
Po*
1.0 Objectives
1.1 Management
1.2 Stntcgy
OPIM
Item Points
129.00
48.00
43.00
38.00
129
13 Stakeholdm
Modif~cdOPIM
Item Points
Points
m~g0ry
146.00
50.00
48.00
48-00
146
2.3 Information
2.4 Lnfnrrnicnire
2.5 People
40.50
2.6 Tcchnoloev
2.7 Nature
2.8 Work Environment
48.50
44.00
29.00
43.00
--
' 3 3 Design and adcveloprnent

- -
- -
---
3.4 Purchasing
3.5 Relis;ition
3.6 Support
4.0 Resulu
4.1 Customa
4.2 Shareholders
42.00
26.00
38.00
4.4 Suppliers and Partners

4.5 Sociew
28.00
190.00
36.00
32.00
42.00
33.50
16.00
4.6 System
303
190
4.3 People
-
5.0Performance
128
128.00
220
- - - -
110
47.00
26.00
0.00
28-00
220.00
50.00
30.00
48.00
3 1.00
16.00
45.00
110.00
-
5.2 Monitoring and ConuoUing

5.3 hnprovement
Totai
6.3.4
39.00
39.00
50.00
46.00
28.00
40.00
40.00
42.00
43.00
Io00
- - -
40.00
41.00
Io00
Addressing R&D Issues with the Modifed OPIM
The moci&ed mode1 for perfomiance management addresses many of the issues we have
seen that RgrD funcrions face today. Table 6-6 outhes how the modifications respond ro
these problems.
Mocfied Elcmcnts
Design of work and job functions
Addfcssing R&D h b l e m s
Improvement of this design udi support co~
opecation, collaboration and innovation
The system used to reward and recognise R&D
Irnprovement of this system to encourage
- teampem&el
b d r e w d and recignition
The development and deploymor of RdrD
Improvement to ensure the alignxnent of R&D
stmegic objectives
objectives with organisational goals
The use of technology
to
achieve
R&D
Improvement
to ensure efficient and effective
objectives
&of technology in R&D projms to meet
organisationai goais
The identification of nistomer reqllemenfs and Impmvement to the pnxesses of idendj&g
the Licorporuion of these recpkments into the customer qukements to ensure thar changing
& s i p process
customer needs are being met
n i e management and improvement of design
Impmvement to increase the efficiency and
and development processes
effectiveness of RBrD processes
to shortai cycle
I tmes and maintain comOefiuVeness
The rneasuremenr of results reiated to people,
Improvernent of the mednument
to
customers and the system as a whole
ensure the effectiveness of the R&D
I
1 performance management system
-
6.4
Chapts Summary
This chapter illusuateci the modification of the OPM for use in an R8rD environment. The
modifications were made to address the issues outlined in the literature review. The next
section closes this thesis with a summiuy of the conclusions arising from the studyas well as
some oppomuiities for future research.
7.0 Conclusions
This chapter discusses the main conuibutions of the worir presentd in the thesis, followed
by recommendations for future research.
7.1
Contributions of the Research
In Chapter Three, a proposed m d for performance management that inteptes the new
ISO 900 1standard for +ty
management wirh three modek for business excelience w u
developed. The resulting model is based on a sysrems view of organisational performance.

It incorporates the requirements for a quality manageaient systern from ISO 9001 with the
identifid strengths of each model for business excellence. The redting set of aiteria
represents a path towvds business exceiience. The organisation can evduate its c u r w t
&y management system against the minimum requirements in the model and then use
the opportunities for improvement as a guide for conMuws improvement.
A scoring scheme was developed for the model using a decision an+
model that captured
the opinions of senior management regarding the relative importance of each element in the
model for business performance. The resource element was considered to be the mon
important, representing the transition from the process-based ISO 9000 to a more system-
based business excellence.

Chapter Four describes how the proposed model was used to perform a seif-assessrnent of
one division in the case study organisation. A set of questions was developed to obtain
information through interviews with a cross-section of employees from the CSO. The self-
assessrnent was successful in obtaining mudi idonnation penainiag to the organisation's

acrivities. It provided the CS0 with a comprehensive lia of areas in which to focus
improvement efforts.
Chapter Five compared the three perfomunce measurement rnethods used by the case study
organisation. This induded a cornphon of the seif-assessrnent outcornes with the results
of a quality audit perfonned at the urne rime as a m-
of investigating the compatib*
of
the IWO measurement methods. The chapter ais0 compareci the use of result-based
performance measurement to the ourcomes of the self-assessment and audit.
This process
reinforcd the importance and interdependence of each method in an organisation's

performance measurement efforts: result-bd performance measurement is used to capture
the CUrrent performance levels withli the organisation; @y
a u d i ~ is
g used to detemiine
managenent system. Selfif an organisation is meeting the requkements for its

assessrnent is used to enhance both of these measurement processes, as it requires that a
Company evduate the effiaency of its activities. This reveals why a process is perfonniflg at
its current lwel and suggests why a qualRy au& has found non-cornpliance.
The cornparison of the auditing and assessrnent showed the foIlowing
y
can provide
Perfomiing self-wessment on eiements related to q e d d tmanagement
valuable infomiation regarding the efficiency of these processes. 'Iliis enhanas the
@ty
audit, which aims to judge the effectiveness of mch processes.
The svengths and weaknesses identified in the self-assessrnent can be incorporated into
the business planning process. This ensures that the veas for improvement, rekted to
the quality management system, are considered, compensahg for the absence of quality
audits results, in the business planning process.
Self-assessrnent of the processes relating to quality management, especiaily if p e r f o d
at opposing times of the year from the internai quality audits, would lead to f d e r
Mprovements in the efficiency and effectiveness of these processes.
Chapter Six ouined how modifications were made to the OPIM to create a more
appropriate model for use in an R%D environment. The modiGed model addresses many of
the issues facing the CSO's R&D departments in today's marketpIace. This is a renilr of the
new focus on the most important aspects of the OPIM to W. These areas are:
Suategy dwelopment
Stakeholders' objectives
Human resounces
Techn01ogy resources
Oisromer related processes
Design and development processes
People renilu
Syaemresults
The scoling scheme was +Lo changed ro reflect the new focus of the OPM for R8rD. The
vaiue of the above listeci catepies wu increased Using rhis proposed model, the
organisation can assess its R8LD activities in more detaiL
In summaxy, the main contributions of the wotk in this thesis are the:
development of an imegrated mode1to move beyond the minimum requirernents of ISO
9001 to create the proper conditions for business excellence
funher investigation into the compatibiityof self-assessment and audithg for future
alignment or integration of these performance measunment methods
examination of the roles of self-assessment, audiMg, and redt-based performance
measurement in the o v e d performance mevurement system
a modifieci version of the OPIM which provides a good framework for future
development and application of this model in an R&D environment
7.2
Scope for Further Research
The following are recommended issues for further research:

the use of the modified OPIM to conduct a self-assessment in an R&D environment
investigation into the use of self-assessment to enhance the @ty
audithg process
smdies on the integration of self-assessment and audit te. using an integareci evaluation
methd for measurernent against the OPM.

funher smdy of the relationships between redt-based performance measurement and
self-assessrnent and their impact on business performance
funher use of the OPIM for organisationid self-assessrnent and investigation into the
effectiveness of the m d for improvernent
8.0 References
Auer, A., Karj*e!n,

J., Seppanen, V. (1996), 'Improving R%D processes by an ISO 900 1based quality management system", jar mal^^ Ahaime, Vol. 42.
B d e l e m y , J. L., Zairi, M., (1994), 'Makmg ISO 9W Work: The Role of Auditingw,TQM
Mag&
VOL6, No. 3, pp. 44-47.
B a d & J., 'Preference Function Modeliagn,~://~~~.Scien~c&asiom.com/,
(6
Augus, 2001).
Bardai, J., (1998),"Mea~urement

Foundations for Preference Funaion Modelhg,"
l h ~ d n g ~ o f t h l o l E EIn&nduzdciqkbxz<n
E
S+ttms, hiha, a n d C j b m ~pp.4038-4044.
~~,
Bandai, J., (1997), "ANew Methodology for Deaiing with Contradicting Engineering
Design Criteria," l + t x d ptfi4.e 18thA d Mehg fPbe Ammkan h e t y f i EnM'pp. 73-79.
Beeler, D. L., (1999),"Intemal auditiag the big liesw,QuakyAug>ea, Vol. 32, No. 5,pp- 7378.
Biutu, US., Cimie, A.S., McDevtt, L., (1997), 'Intepteci performance meanirement
systems: a development guide ",-1
J d t f O p e r i & ISulrrIio2 Manugemt, Vol.
17, NO. 5, pp.522-534.
Bodinson, G. W., (1991), 'Waming: ignoring ISO standards may be h
company's fumen,.i&
Manrqgeney Vol 33, Pt. 2, pp. 11-12.
d co your
Bohoris, G. A, (1995), 'A comparative assessment of some major qulliy awardsn,

r ~ / d o f Q & y r D d R $ t r t t n l i ~ ~V d
, 12, No. 9, pp. 30-43.
Brown, M. G., Hitchcock, D. E., and W h 4 M. L., (1994), Why QM Fa& and Whit mdo
A b It, Irwin Professional Publishg Bum Ridge, IL.
Brown, M. G., Svenson, R. A., (1988), 'Meanuing R&D Pruductiviry",R e s e d T
Mm~q3etiem,Vol. 31, No. 4., pp. 11-15.
Burr, J. T., (1990), 'The future necessisr, Quriby h p s , June, pp. 19-23.
Chin, K. S., Yam,C. M., and Rui, K. F., (1995), 'Continuou Qwi;ty Improvement
beyond ISO 9O0on, Total Qaky RaiRu,VOL5, No. 1, pp. 53-59.
Glurcio, M., (1998), 'ISO 9000 quaiity v e m and product innovation: new evidence from
shoe manufaauring factories", PnmzhfirmPSP 4 E U S M InaornatidlPlMConti, T., (1999), "Vision 2000: positionhg the new ISO 9000 standards with respect to total
Vol. 10, Nos. 4/5, pp. 454-464.
management modek", T d Q d t y
+ty
Conti, T., (1993),
Taal @&y: A
A&M d a q p m ~Ch;anui
,
& Hall, London
Cooper, R G., Kleinxhmidt, E. J., (1996), 'Wiming Businesses in P d u a Development:

d-T
M ~ ~ E Jdy-AuP z ,
pp. 18-29.
The Gitical Success Factors", R
Cooper, R G., Kleinschmidt, E. J., (1994), 'Deterniinants of Tiieliness in Produn
-cf
~~UZ&PZ ,
vd 11, pp.38 1-396.
Dweiopment",Jd
Cooper, R D., Dormuth, K. W., Gug,1. D., Sherman, GR.,Truss, K. J., and Wlborn, W.,
(1990), "App& @ty
assurauce to R&D projectsw,Q z u k y h p s , J u k
Dale, B. G., Bunney, H., (1994), T d @&y UmqgRlait, Blackwel Publishers Ltd,Mass.
Dale, B. G., (19991, M e - ,
3d Ed, Blackwell Publishers Ltd,Oxford.
Davidson, J., Pruden, A., (1996), "Qualitydeployment in R&D organisationsw,ResearrS

T-&=W=4
Jan-Deming Application Prize (1996), DenilgPme Culcilojbr(ksnnr Co>lpilier, Japanese Union
of Sciensts and Engineen.
De Weerd-Nederhof,P.C.,Fisher, O. A. M., and Kerssensvan-Drongelen, 1. C., (1995),
'Managing quality in R%D:social-dynamicaspects and the importance of generating new
organisational configurations", P d g s , 3" EIASM I
.E9oduct Amkpmt
Mmqpmm Gpyfbmz In Jayawarna, D., Pearson, A. W., (2001), 'The Rde of ISO 9001 in
ManagLig the Qu?Lty ofR&D Activities" , n e 7QM M a g e Vol. 13, No. 2, pp. 120-128.
Dixon, J. R,Nanni, A. J., Vollrnann, T. E., (IWO), The Neru I+gbmmC h h g - Mes*
qmmim~&Wd-GlrrsCcnpmk, DOWJones-Irwin, Hommvood, Il.
Eknuti, (1996), 'World-class standards for global competitiveness: An overview of ISO

. M h p m a t , Sept/Oct, pp. 5-9.
9000" I
Endres, A. C., (1992), 'Resdts and Conclusions from Applyiig TQM to R e s e h " , 464
A d @&y Ca?g>ar, May, pp. 426-430.
European Foundation for QuLty Management (1999). Asssmqql6r Exc$lerre A P r .

S$Asses911~~,E~uropeanFouadation for QuaLty Management, Bnissels.
Eutopean @
t
y
Management
A w d (1999), EFQM E x d b m M d ,Empean Foundation for Quality
~~d
a h s , 1996,
European Foundaaon for Q d t y Management, (1995), S e ~ A s s e s s n r G
Bnissels.
Fktgedd, Le,
Johmon, R,Bngnail, S., Silvestre, R,Voss, C., (1991),
in Sm& bu si ne^^, CM& London.
Meaamrrnt
Franck, P. H., (1992), 'P&g Qd~v

h o the R&D Process", R e s d-T
Man<sanem, Jdy-Aug~n,pp. 16-23.
Garvin, D. A., (1993), 'Building a learning organisationw,U n d Burins

August, pp. 78-91.
Garvin, D., (1988), M-ug
July-
Q&y &Strmirgicd GmplzeEt&, Free Press, New York,
NY.
Ghalayini k M., Noble, J.S., (1996), " n i e changiog basis of performauce measurementn,
r
w
j
d4 - h
6 hxhufim IlroLgenar, Vd. 16, No. 8, pp.63-80.
Ghobadian, A, Woo, H. S., (1994), "Ch~aceristics,benefits and shortcomings of four
major q d t y awarbn,hnemiiairdJ i x m d ~ Q u h t y a d R e M z & t yVol.
~ ,13, No. 2,
pp. 10-44.
Globenon, S., (1985), 'Issues in developing a performance crite& system for an
organisation", I - I d o f
B u h m &sud, Vd. 23, No., 4, pp. 639-646.
Goetsch, D L , Davis, S.B.,(1998), U-md
Prentice-Hd, Inc., New Jersey, 1998.
ISO 9000 and I S O M ,
Grandzol, J. R, Gershon, M,(1998), 'A Survey Innnunent for Srandardising TQM

Modehg Research", Intemamal Jd
c-f
S c i k ~VOL
,
3 No. 1,pp.80-105.
Ho, S. K. M., (1999), "Change for the better via ISO 9000 and TQM", Manu--,
Vol. 37, No. 4, pp. 381-385.
Hutdiins, D., (1990), In P
d tfQuhy, Plmian Pubiishing, London.
ISO 10011, (1990) G d & n e s j b Q & y Azuhq Q d t y Systm~,I n t e d o n a i Organisation for

Standardkation, Geneva, Switzerland
ISO MO 10, (1996), Gd&t15h

E + v m d A h g Gemd Phncqde cfEmSiIIPtlSttrtl
Al<drtmg,Intemational Organisation for Standardisation, Geneva, Switzerland
ISO DIS 19011, (2W1), G ~ m q ~ a r r d / a r m ~ ~ ~ y n s > u m d m n g ,

Intemational Oqpkation for Standardisation,Genwa, Switzerland
ISO FDIS 9000 (2000), QulLty i k k q p m t .ymmC - FunbmMk a d V
International Organisation for Standardisation,Geneva, Swkzrland
ISO FDIS 900 1 (2000),Qudrty Ahqpmzt Spam - R e q k m t s , hternauonai Organisation

for Standardisation, Geneva, Switzehd.
ISO FDIS 9004 (2000), Q d t y
IFysrem - GU*^ l b j b h p~ m ~a m~~ ,
International Orguiisation for Standardisation, Genwa, Switzerland
Izadi, M., Kashef, A. E., dr Stadt, R W. (1996). 'Quaiity in higher educauon: Lessons
Deming prize, and ISO 9000 regktration"./ou+
leamed from the Baidnge
I m k d&
T
VOL 33,No. 2, pp. 60-76.
~~
Jayawama, D., Peuson, A. W., (2001), 'TheRole of ISO 900 1in Muiaging the Quality of
VOL 13, No. 2, pp. 120-128.
R&D Activities", k 7QM
Kaplan, R S., Norton, D. P., (1999, T
*
S t r -~ Z Z Action:
O
ihe Balmnd S 4 ,
Harvard Business School Press, Boston, Mass.
Karapetrovic, S., (1999), 'ISO 9000: the system emerging h m the viaous cirde of
compLanceW,?he 7QM M
m VOL 11, No. 2,1999, pp. 111-120.
Karapetrovic, S., Wdbom, W. (200lA), "Audit system: Concepts and Pmctices", TotalQclLry
M-,
Vol. 12, No. 61, pp. 13-28.
Karapetrovic, S., Wdborn, W. (200l), "Audit and Self-Assessment in Qualiry Management:
Cornparison and Compatibility",Managend AudithgJournal,Vol. 16,No. 6, pp. 366-377
Karapeuovic, S., Macey, S., (2001). ' A Model for Perfoxmance Management and
Assessment", Working paper, Department of Mechanid Engineering, University of Alberta.
Karapetrovic, S., Wdbom, W. (2000), "Genenc Audit of Management Systems:
W A&mg/<lmill, Vol. 15, No. 6, pp. 279-294
Fundamentals",M
Karapeuovic, S., Wdbom, W., (1998A), "The Systems View for the Clarification of Quality
J d q f Q d z t y & R$tabiltry-,
Vol. 15,No. 1, pp. 99Vocabulary",
120.
Karapetrovic, S., Wdbom, W., (1998B), "Integated audit of management synemsn,
r i j d d w v 1 o d R $ i a b i l i q~m<qpmm,v d 15 NO.7,pp. 694-711.
Kielia, M. L,Golhar, D. Y., (1997), 'Tod quaiity muiagemait in an R&D environment",

ClllZIAndrrIaar UPulp"int, V d 17, NO.2, pp. 184-198.
IntematCQnalJd c f O p e r h
Keegan, D. P., Eler, R G., Jones, C. R, (1989), 'Are your performance mevures
obsoke?", Mamapmz AcJune. pp. 45-50.
Kumar, V, Boyle, T., (2001), 'A Qualiry Management Implementation Framework for
Manufacnving-basedRBrD EnWonments" ,I n d J a r m r J o/Q&y dnd R*e
M
m Vol. 18, No. 3, pp. 336-359.
L a w , C.W., Noble, J.S., (1998), "A Framewotk for Business System and QuaLty
Management Integrationw
,I u J a m i r d dQdzty& R$latrlny Mbqpm, Vol. 15, No.
6, pp.566-581.
Laszlo, G. P., (1997), 'U.S.and Canadian national quality awards: increased emphasis on
business resulu", 7Z.w 7QM Ahqpzzw?VOL9, NO.5, pp. 381-383.
Lazlo, G.P., (1996), 'Quahy a w d

No. 5, pp. 14-18.
- recognition or model", The QMIti;zp&,
Vol. 8,
Loomba, A. P. S., Johannessen, T.B., (1997), 'Makoh B&ge N a t i o d Q d t y Awad

Gitical Issues and Inherent Valuesn,-@&y
Mmgmnt & Tiirhmlogv,Vol. 4,
No. 1, 1997, pp. 59-77.
Malcolm Baldrige National Q d r y Award (2000), Cri8tzjbrAqhmzmExdm, NIT,
Gaithersbug, VA.
Mann, R, Voss, M., (2000), 'An innovarive process irnprovement approach rhat integrates
ISO 9000 with the Baldnge frameworkW
,Berhdmg An Irrtematio?al Jd,
Vol. 7, No. 2,
pp. 128-145.
McLaughlui, G. C., (1995), T d @&y
Q d t y Senes, Florida.
bz Reemrhrmd-,
The St. Lucie Press, Total
McWaid, T., and Gale, R, (1995) ''Summyr of the quality self-assessrnent of the precision
enginee~gdivision", Planning Workshop of the NIST Pr~isionE n g i n e e ~ gDivision,
Defence Systems Management CoIIege, Fort Belvoir, VA
Meegan, S. T., (1997), 'A model for muiaging the transition from ISO 9000 to TQM",
T r i rQi*rlP/, Vol. 5, No. 1, pp. 35-39.
Meegan, S. T.,and Taylor, W. A., (1997), 'Factors iduencing a successfd transition h m
ISO 9000 to TQM the influence of understanding and motivation", Intencmd J&qf
@&y & RdddqiMnqpm~,
Vol. 14, No. 2, pp. 100-117.
Montana, A. J., (1992), 'If it isn't Perfect, Make it Better", R e s d TeoSndogy-M

Jdy-Augun pp. 38-41.
Najrni, M., Kehoe, D. F., (2000), 'An integrated fkmework for post-ISO 9000 +ty
developmentn,Iizasn<IuotrJ]OUtTtd@M 6 R$rabJiry Adkgmnt, Vol 17,No. 3, pp.226228.
Neely, A., (1999), "The Perfomiance Measurement Revolutiorx Why and What Next?"
-1
Jd
of OpPr&
ard&u.htim ,
V d 19, No. 2, pp. 205-228.
Neely, A., Gregory, M., Plans, K., (1995), "Performance Measurement Synan Design; A
Literature Review and Research Agendan,-nI
Jd
o*f&
rOd h z h m
,
M
Vol. 15, No. 4, pp. 80-116.
Oakland, J. S., Total Qdtyltouigrnien- 7heRwte t>hpvwgA+mun, 2nd e,

Bunerworrh-Heinmann, Oxford, 1993. In Sindair, D., Zain, M., (1995), "Performance
Measurement as an O b d e to TQM",%QM M@,
V d 7, No. 2, pp.42-45.
Pannirselvam,G.P.,Ferguson, L.A., (2001), 'A audy of the relationships bmeen the
Bldrige categoriesn,ItttemdtMrtal Jd
&&&y
dltdR$ia6Jiru hkmpmz, Vol 18, No. 1,
pp. 14-34.
Patiao,H.,
(1997), 'A p p b g Total Quahty to R&D at Coors Brewing Company", Resead
TkhinkgyMnqpm, September-Oaober, pp. 32-36.
Pearson, A. W., Vaughan, N., Butler, J. (1998), 'The implemenration of TQM in R&Dn,
r
i J d o f T-VOL 16, NO.4/5/6, pp. 405-432.
Peters, J., (1998), 'Some Thoughts on Auditing", TQMMagazzne, Vol.10, No. 1, pp. 4-5.
Porter, L. J., Tanner, S. J., (1996), Assess*
Oxford
b i n e s Exo$letrp, Bunewod-Heinemmq
Puay,S.H.,Tan,K.C., Xie,M.,Goh, T. N., (1998), 'A comparative study of nine national

quality awars" , 7fa 7QM ~~, Vol. 10, No. 1, pp. 30-39.
Public SeMce Office, 'Next Steps Reportw,hnp://www.offiddocumatsco.uk
/document/an38/3889/chap-2b.htm (25 July, 2001).
Pun,K.P.,Chm, K.S., Lau,H., (1999) 'A self-assessed quality management system based
on inregration of MBNQA/IW 9000 and ISO 14m,I n t ~ ~ d . f Q d t y r O d
R&h@ Ultrgenea,Vol. 16, No. 6, pp. 606-629
Pun, K.F.,(1998),"Implementhg an integrad quality m e m e n t system", ProLmeduigs

of the 52 Annual Quality Coagress, PPhiLdphia, PA,
pp. 356-362.
m,
h
i
,
S.C.,(1995),ISO 9000 e>tgEumc.rd T d @&y
Standards-Qualiy
Management Group, Ottawa-Nepean, Ontario, Canada, pp. 181 183.
Russell, S., (2),'ISO 9000: 2000 and the EFQM Excellence Model: cornpetition or cooperation?", T d @&y
VOL l l,Nos. 4/5 & 6,pp. 657-665.
Schalkwyl5J. S., (1998),'Td Quahty Management and the Perfomiance Barrierw, The
I p M Magazk, Vol. 10,No. 2, pp. 124-131.
Shaw,A, (1999),A Gude toR+mam? Meaammeitad N m - F i d lidEurans, Mariison
Public Relations, The Foundation for Performance Measurement, S u q
Sinclair, D.,Zaki, M.,(1995), " Effective pprocess management through performance
meastuernent - Part 1 ",BusmeJs Anses Rem& M m q p m m ] d , Vol 1, No. 1,
pp.75-88.
Sinclair, D.,Zai~$M., (1995). " Effective process management through performance
measurement - Part II", Burinss RUBs R e - e n m & M m q p m t J d , VOL 1, No. 2,
pp.58-72.
Sinclair,D.,Zain, M.,(1995)," Effective process management rhrough performance

h Arrss Re-engkmkg & M
J d , Vol. 1, No. 3,
measurement - Part III ",B
pp.50-65.
Sun, H., (2000), "Total quaiity management, ISO 9000 certification and performance
Swyt,D.A.,(1999),"Experiences in initiating Baldrige-based quality in a NIST technid

divisionw,7be 7 Q M MaguzZne, Vol. 1 1, No. 3, pp. 142-156.
Texziovski, M, Samson,D., (1999),'The Mc b e e n total q d t y management pnctice
and organisationai performancen,I
d Jd
6Qafity& Rd&&
kbqpmt, Vol.
16,N0.3,pp. 226-237
Tonk, H S.,(2000),'Integrating ISO 9001:2WO and the Bairige Criteria:A pmpod for a
2 in centuy @ty
paradgmw, QrdIy Aqpa,August, pp. 5 1-55.
Tummala, V. M.R Tang, C L,(1996). 'Stmegic qwlity management, Malcolm Bddnge
and European quaiity awards and ISO 9000certification: Core concepts and c o m p d v e
a.nalyssn, Intanmuxlal J d o f Q d k y a d R $ t a b r l r t v ~Vol
, 13,No. 4, pp. 8-38.
van der Wiele, T., Bmmm, A, MiIlen, R,Wh& D., (ZOOO), 'hprwement in
Orgamsational Performance and Self-Asse-nt
Practices by Selected Amencan Firmsw,
AbuJ
t VOL 7, NO.4, pp. 146- 157
van der Wiele, T., Brown, A, (1999), 'Seif-assessrnent practices in Europe and da",
1
,Jdqf@&yrIdR$iabilityV d 16, No. 3, pp. 238-251.
van der W~ele,T., Dale, B. G.,and W k , A. R T., (1997), "ISO 9000 series registration
to t o d quaiity management the transformation journey", I n t d Jd
dQ&y ScEslao,
Vol. 2, No. 4, pp. 235-252.
van der Wiele, A., Williams, A. R T., Dde, B. G., Gner,Ga,Kolb, F., Luzon,D. M., A.
Schmidt, A., Wallace, M., (1996), 'Self-assessment: A midy of progress in Ewope's leading
organisations in @ty
management practices", I n W Jd
of Qdty dR$iabilrty
Mlm<qge>le~,
Vol. 13, No. 1, pp. 84-104.
van der Wiele, A., Williams, A. R T., Dale, B. G.,Grtu,G., Kolb, F., Luzon,D. M., A.
Schmidt, A., Wallace, M., (1995), 'State-of-the-an snidy on self-assessment", 7he TQM
M a g k , VoL7, No. 4, pp. 13-17.
WiIIborn, W., Cheng,T.C.E.,

(1994, G i a M<maameit qfQuhy A s w i Systena, M c G m HiIl, Inc., New York. Li Karapetrovic, S., Wdbom,W. (2W1), "Auditand Self-Assessrnent
in QuaLty Management: Cornparison and Compatibilky",Managerial Auditing J o u d Vol.
16, NO.6, pp. 366-377
Wdbom,W., (1990), "Dynamic Auditing of Quahty Assurance: Concept and Method",

-1
J d o f Q d t y m r d R $ t a b r l s ~M~ i q p r m ~VOL
, 7, No. 3, pp. 35-4 1.
Zink, K. J., Schmidt, A., (1998), "Practice and Implementation of self-assessment",

Zn&m&ud J o d of C&dity Sarna: Vol. 3, No. 2, pp. 14%170.
Zink, K. J., Schmidt, A., VoB,W., (1997), " T o d quality "down under": the Aunnlan
Quality Award - an innovative mode1 for Business Excellencew,The 7QM Mug&
Vol. 9,
NO. 3, pp. 217-220.
9.0 Appendices
APPENDIX A - OPIM Qucstior~~zirc

APPENDIX B - Modified OPIM for R&D
APPENDrX C - Trillium Results
APPENDXX D - Cornparison of Pafomunce Management M d Criteria
APPENDIX E - A p p a c h , Deployment and Results adapted from the EQA
ORGANISATIONAL PERFORMANCE IMPROVEMENT MODEL

QUESTIONN.
1. Objectives
1.2 Management
1.2.1
system
1.
How do you idennfu the processes needed foryour +ty

systern? Whu are these processes?
2.
How do you determine the sequence and interaction of these processes?
3.
How do you continually impmve the processes and the management of

the relationships h e e n processes of the +ty
management system?
management
1.2.2 Leadership
4.
H m do leaders cornmunicate the importance of meeting customer as

wd as regdatov and legd r e q k e n t s ? (IS04,S.l)
5.
How do leaders ensure the availability of resources to meet these

requirements?(IS04,S. 1)
What is the management review process and at what frequency is it

conducted? (IS04,5.1)
How are leaders involved with customen, parmers and representatives of
6.
7.
society?
How are these processes continually improved?
8.
1.2 Suategy
1.2.1
Development
9.
What is The organisation's suategic planning process?
10.
How are strategic plans reviewed and impmved upon to ensure

alignrnenr with the nuTent and funire business environment?
How is the oqgmktion's m t e g y t r m s w into actionable items and
meaninble goals?
H m are!these communicated and deployed to each management level to
assure individual contribution to achievement?
How are responsibilties for these items eaablished and communicated
to the people in the organisation?
H m is progress a g a k t these plans rneasured?
How are the p r o c e s for deplayment of strategy conUnwiiy impmved?
1.2 Stakeholders
Customers
1.2.1
16.
17.
H m do you determine and review your customers' key requirements?

How is rhis process continually improved to keep up with current and
future business needs?
18.
How are these needs and expeaations deployed and communicated to

people throughout the organisation?
19.
Please provide any perception measures andior performance indicaton

that indicate performance relative to the needs and expectations of
customen. State their current levels and trends. List the targeted levels of
these performance indicators.
1.2.2
Shareholders
How does the organisation idenufy and review shareholders' needs and
expeaations?
How is this identification process continualiyimproved?

How are these needs and expectations deployed and communicated
throughout the organisation?
Please provide any perception measures a d o r performance indicaton
that indicate perfomiuice relative to the needs and expeaations of
shareholders. Include both financial and non-financial iadicators. State
th& current levek and uenb.List the targeted levels of these

performance indicators.
1.2.3
People
How does the organisation identify and review people's needs for
recognition and w o k satisfaction? (ISO), 5.2)

How does the organisation idenrify and review people's needs for
cornpetence and knowledge developmem? (IS04,5.2)
How are these processes continuaUy improvdi

How are these neeck and expeaations deplayed and communicated
h u g h o u t the organisation?
Please provide any perception measures &or
performance indicaton
that indicate perfonnatlce relative to the needs and expectations of
people in the organisation. State their current levels and d.

Lin the
q e t e d levels of these performance indicators.
How does the organisation dwelop partnerships that iink to its arategic
objectives? (CABE 6.1)
How does the organisation idenufy the needs of these pamerships?
How are these pannerships reviewed to ensure rhey are still meeting
objectives of the relauonships?
How are these processes continually improved?

How are these nec& and expectations deployed and communicated
Please provide any perception measures and/or perfomunce indicaton
that indicate performance relative to the neeb and expectations of
piumerships. State their current lwek and trends. List the tvgered levels
of these performance indicators.
1.2.5 Society
How do you i d e n e and manage the impacts of your products, seMces
and operations on society?
How do you identify other needs and expectations of society in the

process of sening your objectives?
How are these needs and expectations deployed and communicated

How are these needs and expectations waluateci?
How are these processes continuallyimproved.?

Please provide any perception masures d o r performance indicaton
that indicate performance dative to the needs and expectations of
society. State their current levels and trends. List the targeted levels of
Resources
2.
2.1
Finances
4 1.
How does your department manage its budget?
42.
How do you monitor your accual spending relative to your budget?
43.
How do you continually irnprove this process?

Please provide any performance indicaton relating to fmanal resources.
Stare their current levels and trends. List the t q e t e d lwels of these
indicators. (OHM 4.6.2)
44.
45.
How does your department manage idonnation to achieve set

Objectves?
46.
47.
48.
49.
How do you monitor and convol the use of information?

How do you contindly improve these processes?
How do you ensure avdabilty of information?
How do you detexmine the effectiveness of the information system.'
50.
Pl-
provide any performance indicators r&ting to information
resources. State di& current lwels md mnds. ~ i sthe

t targeted levels of
these performance indicators. (OPIM 4.6.2)
2.3
Infrastructure
5 1.
H m does your deparunent manage infasmicntre resources?
52.
How do you monitor the use of infrastructure resources?
53.
How do you continualiy impmve these processes?

Please provide any performance indicaton relating to infrasrncture
resources. State rheir curent lwels and trends. List the targeted levels of
these performance indicators. (OHM 4.6.2)
54.
2.4
People
2.4.1
Human Resource System

How do you idenufy characteristics and skills needed by potential
employees that match the organisation's values and needs?
How are work and job funcrions designed to support CO-operation,

collaboration, individuai initiative and innovation?
What is the process for hiring and reuuiting new employees?
How is this process continually irnproved to meet future business needs?

How do you link reuwd and recognition of individuai's performance to
your suategic direction? (CABE 4.1)
How do you link reward and recognition of team performance to yuur

strategic direction? (CABE4.1)
Please provide uiy performance indicators relating to the human
resource system. State their w e n t lwels and trends. Lin the targeted
lwels of these performance indicaton. (OPIM 4.6.2)
2.4.2
Competenues, Training, and Development
H m do YOU as~ess/evaluatedepartmentai skill levels to idenufy u e a ~for

vainiog and education?
How do you wess/evduate individual skills and areas for individual

naining/ development and career progression?
H m do you seek and use input from employees and th&
s u p e N i s o r s / ~ e r son education and uining needs, expectations, and
design? V N Q A 5.2)
How do you design or idenrify educational and vainiog prognms that

support individual and organisation needs?
How do you derexmine if an educationd or training has been effective or
successful? What type of follow-up is thml
H o -does nianagement reinforce and encourage employees to use the
skills, tools and techniques that thqr have leam&
How do you idennfy ski11 requimnents for future business needs?
Please provide any performance indicaton relating to cornpetencies,
tminhg, and development. State their current levels and trends. List the
targeted levels of these perfomiance indiators. (OPM 4.6.2)
2.4.3 Satisfaction, Involvement and Empowerment
How are people's suggestions and ideas captured, encouraged and

implernented?(CABE 4.2) How many suggestions have you received to
date? How have these suggestions improved the division's performance?
How are people encouraged to innovate and take Rsks in order to

achwe goals? (CABE 4.2)
How does The organisation detexmine the key factors that affect
employee well-being, satisfaction and motivation?
Mease provide uiy performance iadicators relating to employee
satisfaction, wd-being, and motivation. State their m e n t lwels and
trends. Lin the targeted levels of these performance iadicators. (OPIM
4.3.1, OPM 4.3.2)
2.5
Technology
75.
How does your deparmient manage technology to achieve set objectives?

How do you monitor and control the use of tedinology?
76.
How do you contindy "prove these processes?
77.
Please provide any performance indicaton relating to tedinology
74.
resources. State their current levels and trends. List the targeted levels of
these performance indicators. (OPIM 4.6.2)
H m does The organisation address and improve workplace health,

s a f q and ergonomie factors?
How do emplcryees take PUS in idenafying these factors and in
ixnproviog wodspiace safery!
Please provide any performance indicators relating to health and s a f q
results. State their current levels and trends. List the targeted levels of
How does the organisation determine when to upgrade/improve the
work area, decor, and equipment in your location?
Please provide any performance indicaton relating to the work
environment. State their m e n t levels and trends. List the targeted levels
of these performance indicators. (OPIM 4.6.2)
2.7
Nature
83.
What is the process for idenufyuig naturai resources that can influence
the organisation's performance? (IS04,6.7) What are your plans for rheir
continued availability or contingency plans to prevent or minimise
negative effects on the organisation's performance? (IS04,6.7)
84.
How do you optimise your connimption of utilities? (EQA, 4c)
85.
What do you do to d u c e and recycle waste? (EQA, 4c)
Processes
3.
3.1
Management
87.
How do you p h and develop the key pmcesses within your depamnent
and how they intedate? (IS04,7.1)
How do you manage these pmcesses and their iaterrelationships?
88.
How do you improve the efficiency and effecriveness of these processes?
89.
Please provide any performance indicators relating ro the key processes
86.
within your d e p m e n t . State their current lwels and trends. List the
targeted levels of these performance indicators. (OHM 4.6.1)
inomer relationships
90.
How do you design processes that affect the organisation's rekrionships

with customers?
9 1.
Hm are these processes implemented and maintaineci?
92.
How do you continuaily improve these processes?

Please provide any performance indicators relating t o cunomer
relationships. State their cunent levels and uends. List the targeted levels
of these performance indicators. (OPIM 4.1.4)
93.
Knowledge
3.2.1
94.
What is the process for idenufying and targeting customers, customer

groups and market segments?
95.
96.
3.2.2
How do you consider customers of cornpetitors in this process?

How do you continuaDy hprove the process of idennfying customen?
Satisfaction
97.
What processes are used for determinhg Nnomer satisfaction?
98.
How do you follow up with customen on produas to receive prompt

and actionable feedback?
How do you obtain customer satisfaction information relative to
cornpetitors a d o r benchmarks?
99.
3-3
Design and/or development

3.31 Planning
101. How do you plan design and development processes to ensure that they
respond to the neeb and expectations of the organisation's cusromers
and other interested parties? (IS04,7.3.1)

102. How do you implanent and maintain these processes?
103. How do you contindy improve these processes?

3.3.2
Inputs
104. H m do you ensure that inputs to design and development are accurate
and complete? (IS04,7.3.2)

105. How is rhis process continua& impmvd
3.3.3
Activities
106. How do you irnplement and maintain design and development

processes?
107. How are these processes conrinuah/ improved?

3.3.4
Outputs
108. How do you validate output againn input requirements and acceptance
criteria to achieve interested p a q satisfaction? (iS04,7.3.2)
109. HOWis this process continually improved?
1IO. How is esign &ew
incoqmrated into the design and development
processes?
i 1. How is the design review process continualh/ improvdi

3.3.6
Vedication
112. H m is design verification incorponteci into the desigri and development
processes?
113. H m is the design veficaton process contin.aUy impfoved?
3.3.7
Validation
114. H m is design validation incorporateci into the design and development

processes?
11S.
3.3.8
H m is the design validation process conUnually impmved?
Changes
116.
H m do you incorporate changing customer/market requirements into

your designs and development processes?
117. How is the process of making design changes continualiy improved)

3.4
Purchasing
3.4.1
Planning
118. How do you i&nufy the resources needed, which cannot be obtained
interna& in order to defme a purchasing strategy? (IS04,7.4.1)

1 19. How do you idenufy and select capable suppliers so as to ensure
consistent quaiity and dependable nipply? (IS04,7.4.1)
120. How are these processes continualiy hproved?
for purchased
121. How do you define processes to ensure that ~~ecification~
product meet the requirements of the orpi&on and interested parties?
(IS04,7.4.1)
How do you implement and maintain these processes?

123. How do you continually improve these processes?
122.
3.5
Realisation
Pianshg
3.5.1
124.
How do you plan and design realisation processes to ensure that they
respond to the neeb and expeaations of the organisation's customers
and other interened parties? (IS04,7.3.1)
125.
How do you contiauaUy improve the design of these processes?
Convol
126. How do you implement and maintain realisation processes?
3.5.2
127.
How do you improve the effciency and effectiveness of these realisation
processes?
3.6
Support
3.6.1
Planning
128.
How do you determine key support process requirements?
129. How do you plan and design suppon pnxesses to meet ail key
requirements?
130. H o w do you conth&

3.6.2
improve the design of rhese support processes?
Conuol
131. How do you impiement and maintain key support processes ensure
meeting key perfomiance mquirements?
132. How do you improve the effiuency and effectiveness of these processes?
4.1
4.2
4.3
4.4
4.5
4.6
Customer
4.1.1 Reputation (OPM 32.2)
4.1.2 L~~&~(OPhil3.2.2)
4.1.3 Outputs(OPIM1.3.1)(OPhil3.2.2)
4.1.4 Interfaces (OPIM 3.2)
Shareholders
4.2.1
Financial(OPM1.3.2)
4.2.2
Non-Finanaal (OPIM 1.32)
4.2.3
Interfaces (OPIM1.3.2)
People
4.3.1
Motivation (OPIM 2.4.3)
4.3.2
Satisfaction (OPIM 1.3.3)
4.3.3
Achievemenu (OPIM 2.4.2)
4.3.4
SeMces (OPIM 2.4.3)
Suppliers and Panners

4.4.1
Inputs ( o p h l 3.4.2)
4.4.2
Interfaces (OPIM 1.3.4) (OPIM 3.4.2)
Society
4.5.1
Citizenship(OPIM1.3.5)
4.5.2
Gmmunity(0PIM 1.3.5)
4.5.3
EnWonment (OPIM 2.7) (OPIM 1.3.5)
4.5.4
Heaith and Safety (OPIM 2.6)
4.5.5
Interfaces (OPIM 1.3.5)
System
4.6.1 Processes (OPIM 3.1)
4.6.2
Remurces (OPIM2)
4.6.3
Objectives (OPIM1.1.1)
Performance
5.
5.1
Management
5.1.1
System
133. What are the systems used to measure organisauond perfoxmance?
134.
How are these gcitems interrelafed?
135. How are tbese relationships manageci and improved?
Key Performance Outcornes
5.1.2
136. What is the system applied to mevure your key perfoxmance indicators
reiating to organisational effectiveness (meeting key objectives)?
137.
How do you continwUy improve this measurernent system?
Key Perfomiuice Indicators
5.1.3
138. What is the syscem you use to measure your key resource performance
indicators?
139. How do you continuah/ improve rhis measurernent system?
140.
What is the system you use ro meanire your key process performance
indicators?
141. How do you continuaUy "pmve this measurnent system?

5.2
Monito~g
and Controllhg
5.2.1
Audits
142. What is the process for the performance of interna1 audits againn the
requirements of the seleaed management standards indudiag ISO

9001:2W0 OS0 9001:1994) and/or ISO 14001:1996?
143. How do you continuaMy improve the effectiveness, efficiency and
consistency of this process?
144. What is the process for the perfo~nanceof exremal audits against the
reS;rements of the selected management standarb induding ISO
9001:Z O S 0 900 1:19%) uid/or I S O 14001:l996?
145. How do you c o n t i n d y @mve
consistency of this process?
the effectiveness, efficiency and
5-2-2 Self-Assessments
146.
What is the process for perfomiing regular self-assessments against the

organisation Perfomunce impmvement Model?
147.
How do you c o n t i n d y b p o v e the effecriveness, efiiency and

consistency of rhis process?
5.2-3 Performance Measurement

How do you design or select the indicaton used to masure and analyse
the performance in your department?
How do you idenufy and manage the interrelationships bmeen rhese
measures?
How do you decide what key comparative data and information ro use?
How do you support the organisational performance review and planning
with analysis of data and information?
How do you link the organisationai level results to R&D depamnend
and project tearn operations to enable effective support for decisionrnakllig?

How do you c o n r i n d y irnprove the effectiveness, efficiency and
consistency of the performance measurement system?
5.3
Improvement
5.3.1
Coftective and Prwentive Actions
154.
How do you monitor, iden*,
&,
and remove the exisUng causes of
nonconformities in inputs,processes, outputs, objectives, resources uid

results?
155. How do you indude plans for and renilu of corrective and preventative
action in the business plan?
156.
How do you evaiuate the effectiveness of these actions for rheir

continuw improvement?
5.3.2
K a h n and Quantum Leaps
What the process for initiating and c
e out quautun leap projects
that lead to the revision and irnprovement of existing pmcesses or the
irnplementation of new processes?

What is the process for initiating and ovrying out Kaizen (small
incremend impmvement efforts) for the continual improvement of

features in inputs, processes, ourpufs, objectives, resources, and results?
How do you wduate and improve the effectiveness of these
improvement processes?
APPENDIX B Mded
OPTM for R&D
ORGAMSAnONAL PERFORMANCE IMPROVEMENT MODEL

CRITEIUA FOR1. Objectives
1.2 Management
1.21
System
Describe haw the R&D funaion establishes, "plements, and maintains its quality
procedures and plans to meet the qwiity policies and objectives set by the
organisation.
OS@,
4.land 6.1; DP,2:l-25)

Describe how the R&D function continuously improves its &ty
pians and the
effectiveness and effiaency of its qu?liryprocedum. Desuibe how qu&y is defineci
d Boyle, 2001)
for the R&D fuaction (Kumar a
D&be how the cornmitment of R&D management to achieve policies and

objectives set by the organisation is ensureci.
(MBNQA, 1.1%EQA, 1b-d; DP, 1:6 and 10)

Describe how leaders of the R&D function conmbute to the development,
implementation and continuous improvement of the organisational management
system, as well as their Uivolvement with di sdteholders.
1.2 Stategy
1.2.1
Dwelopment
hpmmts (ISOI,5.3
and 5.4.1)
Describe the syscern for translating organisational quahty policies and objectives into
plans and objectives for the R&D funaion.
(IS04,5.3;MBNQA, 2.1; EQA, EQA, la and 2c; DP, 1:1-1:3)

Describe the nntegic planning process applied to irnprove R&D performance and
set appropriate suaregies and poliaes.
Rqmmm (iS01,5.4.2,5.5 and 5.6)
Describe the processes of R&D management p h h g and review, as well as the

associateci responsibilities and atithorities,
qDportrmrtJs (IS04,5.4-5.6; MBNQA, 1.1b and 2.2; EQA,2d and 2e; DP, 1:4 and 15)
Describe how m e g k p h uc deplaycd to the R&D function. Describe how the
nntegic pians, m d a e d to the R&D function, ue revicwed and improved upon to
ensure continuing ;ilignment wi3i organisational pians.
Describe how st;ikeholders are iden~ed,and how th& needs and requiremeau
[iudiUiinp legal, regultory, MWP-tal,
producr and protes standuds, as well as
openriod) are identied, deploye, communicated and evaiuated.
@xmmtm ( i S 0 4 , 5 2 ; MBNQ& 1 2 uid 3.1; EQA,21 and 2b; DP, 2:6 and 4:l- 4:s)
Descrik h o w the process of identification, deplayment, communication and

evaiuation of stakeholders and th& a& and requirements is continualiy irnproved
1.2.1
Customers
1.2.2
People
1.2.3
O r p w a t i o d Leaders
1.2.4
Suppbers and Puniers
2.1 Management
h p t w m (IS01,6.1)
~ ~
D e s d e how the resources listed below in d o n s 2.2 to 2.8 and 3.4 of OPIM are
managd Describe the intdtionships among these resources and how they are
used to support processes listed in section 3 of OPIM.
(IS04,6.1)
Describe how the acquisition and deployment of resources is optimiseci to ensure
effectiveness, efficienv and consistency of system operation and achievement of set
objectives. JJiustrate how the use of the resources is continuously improved Describe
how the commitment of long-tenn resources to R&D effom is ensureci. ('da
&
G o h , 1997)
2.2 Finances OS04,6.8; EQA, 4b)
Desuibe how finulciai resources are planned, acquired, deployed and convolled to
achieve R&D objecrives.
QF==
Describe how the efficiency and effectiveness of the use of fmancial resources is
impmved.
2.3 Information
Describe how idonnation resources are pianneci, acquired, deployd and conmiled
to adiiwe RStD objectives.
(IS04,4.2and 6.5; EQA, 4e; DP,3)

Describe how the efficiency and effecfiveness of the use of information resources is
;mprovCd
2.4 Infrvvucturc
OS01,6.3)
Dexribe h m
to d e v e R&D objectives.
resources are pianned, acq;ed, deployed and contrdled
(IS04,6.3; EQA, 4 4
Describe how the efficiency and effecciveness of the use of
'
'
buuctureresources
is
2.5 People
2.5.1
Describe how characteristics and skiils needed by potend employees are identifieci
to ensure h t the current and future ne&
of the R&D function will be met.
Describe h m work and job functions designcd to suppon CO-opedon,
collabontion, i.adividu?lhithive and innovation.
Q p f w z t k (MBNQA,5.1; EQA 3%3 4 3e)
Desuilx how the human resourre system is conUnuoUS, improved.
2.5.2
Campetenaes, Training and Development

~ ( I S O 1 , 6 2 )
Describe the systems used to ensure continuing competency of R&D perso~eL
C&nmumm (IS04,6.2.2; MBNQA,5.t; EQA, 3b)
Desuibe how k q cornpetencies are developed and sustained to achieve set levels of
R&D perfomiuice. Describe how the efficiency and effectiveness of these systems
are continuously improved
2.5.3
Satisfaction, Involvement and Empowerment

&panmm (ISOI, 6.2)
Describe the system used to reward and recognise R&D personnel.
9lpnrrrniher @ S a ,6.2.1;MBNQA, 5.3; EQA, 3c)

Iliunrate how a work environment that conmbutes to employee satisfaction,
involvement and empowerment in O& to improve effecfiveness and effiaency is
developed, maintaineci and enhancd
Describe how technology is used to achieve R&D objectives. Describe how business
and technology strategies are integrated, communiated, and understood by aU
depuunents (Kumar and Bayle, 2001)
9eipomotaier
Describe how the efficiency and effectivcnss of the use of technology is improved
Describe how av;Utbilityof ntunl resours in cnnved in order to prevent adverse

impacts on the a w i m n f n ~and
t negatbe effecu on the R&D perfonnuice.
q.pBd?5
Describe h m the use of n a 4 resources is optaised to prevent negative

environmentai impacts and improve organisationaiperformance.
Describe how work environment is
t o achiwe R&D objectives and provide
a healthy and d e workpiace.
(IS04,6.4; EQA, 4c)

Desaibe how a positive influence of the work environment on human resources is
maintaineci and impmved.
3.
Processes
Rqamm#
@SOL,7; EQA, Sa and 5d)

Describe how R8rD pmcesses listeci Mow confom to the criteri? stated in ISO
9001: 2000 standard,section 7.
(IS04,7;EQA, Sb; DP,6 and 7)

Desuibe how the effectiveness, efficiency and consistency of processes are
continuoudy improved, referxing to the guidelines mted in ISO 9004:2000, section
7.
3.1 Management (ISO1,7.1; IS04,7.1)
Describe how R&D processes are managed

3.2 Customer relationships (EQA, Sc and Se; MBNQA,3.2% ISO1,7.2.3; IS04,7.2)
Describe how processes rhat affect the R8rD function's relauonship 6 t h customers
are desiped, implementd, xmintained, and impmved
3.2.1
Knowledge (iSO1,7.2.l and 7.2.2; MBNQA, 3. la)
3.2.2
Describe how prucesses for idendymg internai and extemai customers and their
requirements are designed, implemented, maintained, and improved.
Satisfaction (MBNQA,3.2b)
Describe how processes for determining internai and externai customer satisfaction
are designcd, implemented, maintained, and irnpmved
3.3 Design ancilor development

3.3.1
Pl?nning (IS01,7.3.1; I n , 7.3.1)
Describe how design and developrnent processes are p h e d , implernented,
maintaineci and improved to ensure t&at thev resmnd to the neeb and exuectations
Inputs (iS01,7.32; ISW7.3.2)

Describe how the R&D function ensurcs t h t inputs to design and deveiopment are
accurate and compled. Describe h m this process is conrinuaily improvd
Activities (MBNQA,6.11)
Describe how design and development processes are implemented, m;iintained and
impmved
Outputs (ISOI, 7.3.3; IS04,7.3.2)

Desaibe how ourput requinmenu are validateci against input requirements and
acceptana criteria to achieve interesteci pvty satisfaction. Describe how this process
is conMii?Uy impfoved
Review (Isol,7.3.4; I n , 7.3.3)
Desaibe how the design review is incorporateci h o the design and deveiopment
processes. Describe how this process is continuousfv improved.
Verification (ISOL7.3.5; IS04,7.3.3)

Desaibe how the design verification is u i c o r p o d into the design and
Qvelopment processes. Describe h m this process is continuody improved
Validation (IS01,7.3.6; IS04,7.3.3)
Describe how the design vaii&tion is i n c o r p o d into the design and dweiopment
processes. Describe how this process is c o n ~ u o u d improved.
y
Changes (IS01,7.3.7; IS04,7.3.3)
Describe how changing customer/market requirements are incorponteci into R&D
design and developrnent processes
3.4 Purchasing (EQA, 4a)
3.4.1
Planning(IS01,7.4.1;IS04,7.4.1;MBNQA;6.3aland6.3a2)
Descnbe how resources, which m o t be o b d intemaiiy are idenUfied in order
to define a purchasing strategy. Describe how capable suppiiers are identified and
seiected so as to ensure consistent quaiity and dependable nippky. Describe how are
these processes continua& improved
Describe how processes ro ensure that specifications for purchad producc meet the
requirements of the organisation and infefened pmies are defined, implemented,
maintaineci and improved
3.5 Support
3.5.2
Describe how requirements for key support processes are determined. Desuibe how
these support processes are pknned, &signe4 and hproved
Conml @Sol,
7.5.3-7.6; IS04,7.5.2-7.6; MBNQA,6.214-5)
Describe how key support processes are implemented, maintained and impfoved to
ensure meeting kqr performance reguirrments.
4.
Results
List the perception measUres and performance indica~orsrelating to the following

four componcnts of customer r d t s . S m c thci,current levels and uends.
QP-4.1.1
List the targetcd leveis of perception mevurcs and performance indicators.

Reputaion (EQA, 611 and 6b1; MBNQA,7.Id)
Luydty (EQA, 6a4 and 6b4; MBNQA,7.1a2)
4.1.2
4.1.3
Outpufs (ISO 1, 8.2.4 and 8.3; IS04,8.2.3 and 8.3; EQA, 612 and 6b2; MBNQA,7.1a3)
4.1.4
Interfaces (EQA, 613 and 6b3; MBNQA,7.lal)
4.2 People (MBNQA, 7.3;IS04, 8.2.41)
List the perception mevures and performance indiators &ring to the following
four componenu of people results. State their current levels and m d s .
List the targeted leveis of perception measures and perfonnuice indiaton.
4.2.l
Motivation (EQA, 711 and 7b2)
4.2.2
Satisfaction (EQA, 7 d and 7b3)
4.2.3
4.2.4
Achievements (EQA, 7bl)
Services (EQA,7b4)
4.3 Suppliers and Panners (IS04,82.4c)

List the perception measures and performance indicaton reiating to the foiiowing
cwo components of suppliers and panners relationships. State their current levels and
trends.
9.lpnaprras
4.3.1
4.3.2
Lin the targeted leveis of perception mesures and p e r f o m c e indiators.

Inputs(IS01,8.2.4;IS04,8.2.3)
Interfaces (MNBQA, 7.4)
4.4 Society ( I D , 8.2.4d)
List the perception measures a d p e r f o m c e indiaton reiiating to the foiiowing

five components of the rektionship to Society. State th& current leveIs and trends.
List the t v g ~ e d
levels of perception mesures and performance indiCuos.
4.4.1
Citizenship (MBNQA, 7.512; EQA,811)
4.4.2
Community(EQA.812)
4.4.3
Enknment (EQ& 814)
4.4.4
M t h and S a f q (EQA, 81))
4.4.5
htcrfaces(EQ&8b)
4.5 System
4.5.1
Processes (EQA, 9bl; MENQA, 7.511, IS04, 8.2.2)
Qv-leJ
List the tvgeted lweis of key process perfommnce indicators.
-9biantrorias
4.5.3
5.
5.4
List the key perfomance indiators rekang ro processes stated in section 3 of

OPM Sute their tleveis and vends.
List the key performance indiators relating to r e s o m s stated in section 2 of OPIM.

State their current levels and trends.
List the targeted lm& of key resource performance indiutors.
Obj&es(EQA,
91)
List the key performance indiutors reiating to R&D effectiveness. State their current
lwels and trends.
List the targeted lwels of key R&D performance indicators.
Performance
Management
Describe the system applied to measure RSrD performance. Iiiustrate the

interrelationships among the intemal audit, self-assessrnent and result-based V e m s
for performance measurrment.
Describe how the intedxionships of i n t e r d audits, self-assessments and result

rnevurements are manageci and improved.
Key Performance Outcornes (EQA, 91)

Describe the system applied to mcasure the key performance outcornes identifieci in
section 4.6.3 of OPM.
Describe how the measmmmt synmi for key performance outcornes is conUnually
impmved.
Describe the system applied to masure the key perfonnuice outcornes identifid in
sections 4.6.1 and 4.62 of OPIU
Describe how the me;isuremcnt sysrem for key pcrfomance indiators is c o n ~ w i l y

improved.
Monitoting and Controiiing
5.5
5.5.1
Audits
Describe the V e m applied to perforrn internai and extemai auclits against the
requirements of the selected management stui-,
including ISO 9001: 2000
and/or ISO 14001:1996.
Qpzwntm (IS04,8.2.1.3)
Describe how the effectiveness, effiaency and consistency of the audit system are
c o n t i n d y improved.
5.5.2
Self-Assessments( I D , 8.2.1.5)
Describe the systern applied to perfonn regular self-assessments against the
Oqpmauonal Performance Impmvement Model.
Descnbe how the effectiveness, efficiency and consistency of the self-assessment

system are c o n t i n d y impmved
5.5.3
Performance Measutment (iS04,8.1,8.2.1,8.4)
(MBNQA, 4. la 1 and 4.2)

Describe the performance measurement synem used to cokct, monitor, an*,
synthesise an act upon perceptions (extemal) and performance indicators 'nternai)
relating to:
- customers
suppliers and pvtners
soaety
Describe how the effecciveness, efficicncy and consistency of the performance

measunment system are continuay improved.
hpmvcmait(ISO1,8.5.1;IS04,8.5.1;DP,9)
5.6.1
Corrective and Prevcntive Actions
5.6
R t q m m m (Bol,8.52 and 8.5.3)
Describe the systmi applied to monitor, i d e n e , analyse, and remove the exisung and
potend causes of nonconformiries in inputs, pfocesses, outputs, objectives, resources
and d
f s .
(iS04,8.52 and 8.5.3)
Desuibe how the c o d e and preventive actions are used in the business pian and how
5.6.2
th& effectiveness is continuaiiy imprwed.

Kaizen and Quantum Ir?ps (IS04,8.5.4 and Annex B)
Describe the systern applied to continuousiy improve the features in inputs, processes,
outputs, objectives, resources and resuits.
Desuibe how the effecUveness of the conUnd impmement systern is incre;ised.
REFERENCE ClUTERK
Demig Application Prize (1996), Dsnirg F%E G d
DP
Orsnoiv C k p z i s , Japanese
Union of Saentists and Engineers
European Qu;ility A d (1999), EFQM ExadlmP M d , European Foundation for
EQA
ISO1
1s04
MBNQA
Quatty Myiagement
1!50 FDIS 9001 (2), Q&y

h h p a m S y a m - Rqbmmzs, Intemational
Organkation for Standudiz;ition, Geneva
rso mrs 9004 (20001,
G-~W
,International O ~ u o forn Standardization, Genwa
Maicolm Baidrige Natiod Quaiity Award (2W0), Crieiiafi A+nmx? E x d h ~ M
, n,
Gaithersburg
Kieiia, M. L,G o k , D. Y., (1997),
'Td Quaiity Management in an R8cD Environment", inrenraakll
J d o f O p e r m i c m & R d d m Mmqpmt, Vol 17, No. 2, pp. 184-198.

Kumar, V., Boyle, T., (2001), 'A quality management implementation framework for manufactuing-based
R&D envkonrnents", I
359.
M JuanafofQuhy & R$raEilrtyMhqpmt, Vol. 18, No. 3,2001, pp. 336-
APPENDIX C Trillium Resuits
Trilliwn Resdts
Scoring for OPIM
98.77
Management
81.75
Su;itegy Development
75.46
61.72
87.58
100
100
81.47
90.55
72.92
68.74
67.77
100
38.39
56.41
76.17
97.87
86.09
50.5
90.02
71.84
86.72
68.86
58.26
10.55
71.03
97.7
87.43
31-01
89.75
83.8
85.75
81.41
59.37
53.86
87.37
91.46
74.14
19.13
94.47
77.82
89.93
91.7
66.39
80.06
92.97
71.42
55.63
48.32
60.13
48.72
77.21
72.19
49.43
79.06
79.56
79.33
99.5
89.2
88.49
87.1
72.36
37.98
30.47
100
50.06
89.84
61.66
51.83
30.76
68.29
58.45
69.6 1
71.4
37.9
51.85
50.51
62.94
49.16
48.41
51.41
95-36
59.66
75.78
55.09
62.8
28.53
72.49
72.98
O
67.21
72.86
59.52
and Deplaymcllt
68.99
87.49
l
85.19
39.38
84.38
84.83
94.44
78.73
32.8
68.98
36.8
60.3
49.48
79.62
53.77
O
44.95
82.05
81.12
StakeholderObjectives
Management of
Rwources
Rsources
People
Technology
Nature
WorkEnWonment
Management of
Processes
customeir
Relationships
Design Processes
Purchashg Processes
&uon
Processes
Suppon Processes
Customer R e d u
'Sheholder R e s u l c
People Results
Supplier and Panner
Renilts
SociqResults
System Results
Performance
Management
Perforrmnce
Monitoring and
-
Conuoltng
48.67
78.02
63.42
100
---
52.52
46.26
48.21
51.11
54.63
61.79
28.82
100
80.56
89.39
12.47
70.03
74.99
93.49
69.65
42.06
88.67
37.65
63.03
68.46
35.34
88.91
37.17
43.72
53.31
47.53
46.5 1
92.56
69.44
89.37
91.1
54.56
89.05
APPENDLX D - Cornparison of Perfknance Muiagemmt Model G i t m a

Catcgory
ISO9001:2OOO
h k i n g for
evidence of
management
coIIMifment to the
gwlity
management
system and that
top managanent
MBNQA
senior leaders
gui& the
or&;inis;ition and
revicw
or&;inistional
perfolmance
eaSureS
The needs and

expectwons
of interesteci
pmies are met
The quaiity
poiicy is
EQA
Examines how
leaders develop
and facihate the
achievement of
mission and vision,
develop values,
and ensure t
h the
o
~
o
d
management
systern is
developed and
maintained
Dcming Prize
Addresses
undersraading,
knowledge and
enthuskm of
leaders, hmeffectively leaders
impiement TQM,
and senior
management's role
in the management
v-
dehd
objectives and
the qiulity
management
system (qms)
is pianned
Responsibility
and authority
is defmed
The review of
the qms
1 Examines how the 1 Examine how the 1
Generai
Looking for
evidence of a
p o h and
the planning of
Generai
qdty
objectives
the quaiity
management
mem
company addresses
its responsibilities
to the public and
how it practices
g d cihnship
Suaregy and policy
Examines the
company's strategy
deveiopment and
deplayment
processes
compaxxy's leaders
are involved with
representatives of
society
Examines how the
compvry
unplements its
mission and vision
Examines how the

company fulGIs its
s d
mponsibilities
concerned with
quaiity assurance
miVities and
q u ; i l i ~ ~ ~
Exunuies h m the
company pians and
manages its
intan?l resources
in order to suppon
its policy and
strategy and the
effective operation
of its processes
&den& that the

~qpktion
determines and
provides the
resourccs nceded
to implement and
&tain
the
qu;ility
II1;Ul?gement
system and to
enhance customer
satisfaction
Not speaficaliy
induded
Examines how the

compv~yui;iEyses
i n f o r d o n on
qu;ility, customen,
opemionai and
finuicial
Examines how the

compauy's
information sysrem
is m a q e d to
ensuredata
validity, integrity
performance to
and security
support decision
Exvnines how the

company pians and
Lx>okmg for
evidence t h t the
compuiy
&termines,
provides and
maintains the
Examines how
exted
information is
collecteci and
=bed to
support decision
making
-
.
-
manages its
buildings,
equipment and
Not induded
illhsmcture
needed to achieve
Examines how
effectively the
Company manages
its miued
Not specifically
induded
Not included
Not specificaiiy
included
Na specificaiiy
included
Not included
resources and
supplies so that
material
inventories are
m e is mkiimised
Examines how the
Eompany evduates
and exploits
techno1og)r
Examines how the
~ompany
's
manages its
Il~lilllcialrrSOurceS
Not induded
Not induded
Catcgory
ISO9601:ZOOO
MBNQA
uaLlreSOurtem;inge
Examines how the
EQA
Examines how the
compaq-es,
develops and
reieves the full
potend of its
Looking for
evidence that the
company ensures
the necessary
cornpetence for
Eduution and
Training
paformuice
Examines how the
Company secks
input h m
ernployecs and
their supervisors
on.educaon
and
.
trvnrngnceds
Examines h m the
Company support
business objectives
people
Examines how the

company promotes
the involvement
and empowerment
of its people
Exvnincs how
people's
kncswiedge and
cornpetencies are
idenaed,
deveioped and
personnel
performing woiir
Dcming f i e
ent
susuineci
;iffectingproduct
qu;iliv
Company appiies
Qu;ilityconcepts
into its human
resource docation
and management
Not induded
Examines h m the
companyf
eduatiod
poliaes and plans
consider QC
management and
how it reviews the
effectiveness of the
educationai
Prognms, and
whu eciucatiod
compuiies
Examines how the
Employee wellbeing and

satisfaction
Not induded
organisation
maintains a work
enWoument that
contributes to
employee wdking and
satisfaction
Examines how
people are
m;uded,
recognised and
cared for and how
the company
promotes the
dialogue between
i d and the
wo~le
Not induded
Cjtegory
ISO900t:2OOO
MBNQA
Processclualiw
Examines how the
ComPvIy m=%=
its key product and
-l
evidence that the

company pians and
develops the
processes a&
for p d u c t
service design,
delivery, and
suPpo*
EQA
DcmingPrizc
Examines how the

company designs,
-a,
=d
improves its
processes to
support its policy
and strategy
Great emphasis on
the quaiity
assurance of
products and
services
Examines what the

company is
achieving in
&on to:
Exterd
customers
People
Society
Key
Performance
R4ts
Prim+
concerned with
quality assurance
activities and
Resuits
Examines the
compq's
performance and
Unprovernent in
the folluwing key
ue?s
Gmomers
Finuiduid
Not inciuded
market
cwomer
requirements are
deterrnined and
fulfied with the
aim of enhancing
customer
satisfaction
homer
management
Humui
resoufces
organisational
effecriveness
r management and s;i
Examines how the
compvry
determines
requiremenw
expectation, and
preferences of
cunomers and
Customer
satisfaction
perception of
fuifiient of
Not included
Not inciuded
Examines what the

zomp;uiy is
ichieving in
relation to its
exterd customers
Not inducfed
markets
b
evidenc h t the
company monitors
inforniluon
WtyfeSults
Examines the
compuiy's
perfonnuice and
improvement in
customcr
sarisfaction
Cpcgory
Genenl
ISO9001:2OOO 1
MBNQA
1
EQA
SuppIiers/pumtn mamgamc a d performvlce
Lcdnng for
Ervnincr how the Examines how the
&ce
rhr the
manages
cornp h and
corncompany cv;ilu?us its supplia and
manages its
and selecu
me ring
extad
nippliers based on processes
pvtnerships to
th& ab&y to
support.itspolicy
and stnregy and
W P P&P~
in
accorcfance with
the effective
the company's
operation of its
requiremcnfs
proccsses
DcmingPrizc
Focused on
excension of
qu?lity
management CO
suppliers of the
fum
APPENDIX E - Apprt ich, Deploymi it and Results Ada~tedlrom the EQA

Score
Attributes
Sound:
Approach h u a clear rationaie
There are wcii efined and eveloped
processes
Approach focuses on nakeholder needs
Intcgnttd:
Approach suppons policy and nrategy
Approach is linked IO othcr approaches as
appmpriate
Approach is implemented
0%
75%
No evidence or
anecdotal
Some evidence
No evience or
anecdotai
Some evidencc
No evidence or
anecdod
Somc evidencc
No evidence or
anecdotal
Some cvidence
No redts or
information
Positive trenb
and/or satisfactory
pcrfomiiuicc on
somc muhs
Positive trends
and/or sustainc
food pcrformzncc
onmanyds
over at leas ihrcc
Y='
Strongly positive
trends md/or
sustaincd excellent
pcrfomunce on
mon rcsutts over at
lm 3 y m
Strongiy positive
vmds and/or
sustaincd excellent
performance in a
vcls over at leas
5 Y-
No r e d s or
anecdotai
information
Favourable and
appropriate in some
FavowbIe and
appropriate in
Favounble and
appropriate in mon
Excellent and
appmpriate in
mon
Gnnprehcnsive
cvidencc
dcar evidence
Syncrnatic:
Approach is deplyed in a nniauredway
T d
Trends:
T m & are positive and/or there is
sunained g d p c r f o m c e
Tugcts:
Targcts arc achievcd
Targets ue appropriate
a
mecdow]
areas
mvry-
arcas

An Integrated Model For Performance Management Based On ISO 9000 and Business Excellence Models PDF

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

An Integrated Model For Performance Management Based On ISO 9000 and Business Excellence Models PDF

Uploaded by

Copyright:

Available Formats

FindBestStuff.

Mobile & Cell Phone

AN INTEGRATED MODEL FOR PERFORMANCE

MANAGEMENT BASED ON ISO 9000 AND BUSINESS

MASTER O F APPLIED SCIENCE

Halifax, Nova Scotia

395 WeUington Street

395. rue Wellington

The author has granted a nonexclusive licence allowing the

L'auteur a accord une licence non

The author retains ownership of the

L'auteur conserve la proprit du

List of Figures .................................................m....~.........em...........................~.........

Models for Performance Management ......,..,............. ..................,.......................... ............- 4

2.0 Literature Review

................................ ............. . . ... .................. .

Models for Performance Management ................................ ....................................... . 13

Performance Measurement................. .................................. .......................,......................-27

Performance Management in R&D

Addressing Issues Unique to R&D .......................................................................................

Benefits of Implernenting Quality Management in an R&D Environment ........................... 49

ISO 9000 in R&D..................................................................................................................

An Integnted Mode1 For Perf~~nance

Framework Development ......................................................................................................

Criteria Development ............................................................................................................

Development of the Assessment Questions .......................................................................... 66

Establishment of the Scoring Scheme ................................................................................... 66

Scoring Scheme Cornparison ................................................................................................ 73

4.0 Seff Assessrnent .............................................................................................. 77

Final Score ............................................................................................................................ 89

Evaluation o f Assessrnent Process ........................................................................................ 93

5.0 Performance Measurernent ........e....................................................................

Cornparison of Assessrnent ResulW to Quriity ~ u d i Results

Implications for Simultaneous Auditing and Assessrnent ................................................... 102

Result-based PerCormance Measurement

Review o f Organisational Performance Measures ............................................................. 103

Self.assessment, Auditing and Result-based Performance Measuremeat

Comparing Self-Assessrnent and Result-Based Performance Measurement....................... 117

6.0 Performance Management In R&D .............................................................. l

The R&D Environment at the CS0

Scoring ................................................................................................................................ 128

Addressing R&D Issues with the Modified OPIM .............................................................. 130

Conclusions .................................................................................................. 132

References .................................................................................................... 136

Appendices ................................................................................................... 144

Table 2-9: Plan-Do-Check-Act Cirde of Audit and Self-Assessment Programs.................. 45

Tabie 3-4:Application of Cnteria to Frarnework .............................................................6 1

Table 3-7:OPIM Scoring Cntena.................................................................................... 72

Table 4-3: Final Score Calculauon ................................................................................. 8 9

Figure 1-1: P e r f o ~ c Management

Figure 1-2: Methods for Performance Improvemmt ..........................................................3

Framewoxk for the Bddrige National QuaLF/ Award Model ..........18

Figure 2-2: Enablen and Re&

of the EQA .................................................................19

Figure 2-3: The Self-assessrnent Process ..................................... ....................................2 9

Figure 2-5: OveMew of audit amvines...........................................................................

Figure 3-1: OPM Framewotir ...............................................................................

Figure 3-2: Measuring Preference in Triilium ...................................................................70

Figure 5-3: Perfomance Measwement Categoxy from OPIM ....................................104

Figure 6-2: The Modified OPIM Framework .................................................................124

TQM - Total QuaLty Management

ou to Dr. Walter Wdbom for his Ume spent