FRONTIERS OF E-BUSINESS RESEARCH 2004

Development of a Performance Measurement System in a Knowledge-based Public Organization

Hannu Rantanen1; Tuija Oikarinen2 Professor, Lappeenranta University of Technology, Lahti Unit, hannu.rantanen@lut.fi 2 Researcher, Lappeenranta University of Technology, Lahti Unit, tuija.oikarinen@lut.fi
1

Abstract The study presents some principles and problems which arise when a knowledge-based public organization develops a performance measurement (PM) system. The study is based on empirical evidence from two case organizations located in Finland. The first one is a university and the other one a state agency subject to the Ministry of Trade and Industry. The development and implementation process differs remarkably from the way how it is realized in the private sector. The procedures and recommendations for the development and implementation of a performance measurement system have been usually developed from the perspective of enterprises and mainly large industrial companies (see eg. Neely 1998). In knowledge-based public organizations the problems from the managements point of view in these organizations are diversity of objectives and goals, unclear command chain and defective flow of information. There are plenty of different customers in these organizations. The development of a PM system can be seen as a learning process in an organization. Knowledge in an organization manifests itself primarily in its competencies and capabilities. An organization needs to know what it knows, it needs to have a clear understanding of its knowledge assets competencies before it can fully exploit, manage or develop them. Keywords performance measurement system, knowledge, public organization

Introduction
Integrated performance measurement systems are nowadays very common and extensively utilized in private companies, and they have become popular also in public organizations. The procedures and recommendations for the development and implementation of a performance measurement system have been developed from the perspective of enterprises and mainly large industrial companies. The strategies and objectives are clearly defined in the companies and the ultimate goal of business is making profit. In knowledge-based public organizations the visions and strategies are also fairly well documented today. The problems from the managements point of view in these organizations are the diversity of objectives and goals, unclear command chain and defective flow of information. There are plenty of different customers in these organizations. Who is the real 340

FRONTIERS OF E-BUSINESS RESEARCH 2004

customer? Is it the state that gives the money, the citizens who get the benefits finally, the industry which utilizes the services, or someone else? Defining the most important aspects or measures for these specialists is very difficult. This is an especially difficult problem when the organization structure is multifaceted, and a clear chain of command is absent. The management organization structure and management system are highly problematic e.g. in universities and other public specialist organizations. In companies the number of separate measures in the integrated performance measurement system should normally be kept down (e.g. max 25 items depending on the size of the company and level of measurement) to ensure the usability of the designed PM system. In the knowledge-based public organizations there are plenty of measures on offer, and many highly educated specialists to whom information itself is important. In a knowledge-based organization, knowledge is the most important factor of production and thus the most important factor to manage and measure. Knowledge in an organization manifests itself primarily in competencies and capabilities of the organization. It often becomes embedded not only in an explicit form in documents and databases but even more so in organizational routines, processes, and norms, and in competences of individuals. (see e.g. Argyris & Schn 1978; Davenport & Prusak 1998; Maula 2000) An organization needs to know what it knows, it needs to have a clear understanding of its knowledge assets before it can fully exploit, manage, develop or measure them. If there is uncertainty and a lack of consensus with respect to what those competencies and capabilities actually are, the organization cannot actively manage or measure them. The PM development projects make the management and the employees of the organization build the PM system themselves and thereby clarify their visions, core competencies, strategies and objectives. The development of a PM system can be seen as a learning process in an organization.

Development of a PM system
The development and implementation process in the private sector is rather clear. The literature presents a lot of different kinds of process models for the implementation of a performance measurement system (see eg. Kaplan & Norton 1996; Neely 1998; Bourne 2000 or Simons 2000). These process models have mainly been developed from the perspective of large companies. In these companies the strategies and objectives are usually clearly defined. Moreover, the ultimate goal of business is unambiguous it is profitability. In modern companies the stakeholder, and especially the owners, shareholders, are important. Customers are also important and usually clearly defined and recognized. The measurement is often based on adding the value for the customer and increasing the wealth of the owners. There is an abundance of legislation and other rules to guide the operations of companies. In the public sector the development and implementation process is somewhat different. Also public organizations define their strategies and objectives in these days, but they are not always as clear as in private companies. Often the ultimate goal of operations may also be unclear. The customers of the public organizations are often multifaceted. It is not quite clear 341

FRONTIERS OF E-BUSINESS RESEARCH 2004

who the customer is. For example in the universities the customer can be the state who gives the money, but it can also be the student, or the companies which hire the students. Especially in knowledge-based public organizations the management culture can be rather loose. The difference between workers and management is not always well-defined. For example in the Finnish universities the management is elected every third year. In addition, there might be some kind of political steering at state or regional level for the public organizations.

The development of a PM system as a learning process

The development of a PM system requires the organization to learn. For the organization, its members act as learning agents: insight and innovative ideas occur to individuals (see e.g. Argyris 1999; Nonaka & Takeuchi 1995). Of course new technologies, new organizational forms or new working methods may well emerge in the process of organizational development, but to fully exploit them, learning of individuals is a prerequisite (Drejer 2000). The development of a PM system first demands the individuals of the organization to create the system. They have to agree on measures, ways of data collection, responsibilities and schedules. This can be seen as an innovation process, which includes many kinds of interaction and co-operation of participants. The participants articulate their own conceptions of the companys objectives, goals and customers. These ideas are to be shared and a common meaning to be developed in the innovation process of the PM system development. Secondly, the created PM system has to be implemented. Besides invention of new forms, the learning of an organization requires their diffusion and adoption to the organization (see e.g. Crossan et al. 1999; Schein 1996; Sthle & Grnroos 2000). The created PM system has to be embedded in the organizational memory, i.e. it becomes a part of the organizations systems, rules, routines and infrastructure. After being formalised in the memory function, the PM system can begin to guide the actions of individuals. This can be seen as an institutionalizing process, which aims at ensuring that the planned actions occcur and become routinized. During this process the outcomes of innovative processes are transferred to those members of the organization who have not participated in the innovation. In knowledge-based organizations the development of a PM system is challenging. Knowledge in an organization appears in its competencies and capabilities and is embedded e.g. in shared interpretations, governing values, various administrative mechanisms, strategies, rules, routines, organizational structure, or in the expertise of its individuals. Often individual expertise is crucial. Measuring individual competencies is problematic however: should the focus be on an individual-related or task-related approach, and are the outcomes of competence or the development of competence in focus? Thus, the development of a PM system offers the organization a platform to reach an understanding of its knowledge assets. Rationalistic managerial principles emphasize that a consensus of organizations knowledge assets is a prerequisite for exploiting, managing and developing them (Birchall & Tovstiga 1999). Developing a PM system may significantly contribute to the organizations abilities of knowledge management as a whole. 342

FRONTIERS OF E-BUSINESS RESEARCH 2004

Case studies
In this paper two examples of the implementation process in a public knowledge based organization are presented. Both case organizations are located in Finland. The first one is a university and the other one a state agency subject to the Ministry of Trade and Industry. The implementation processes started in winter 2003/2004, and are still ongoing processes. The study is composed of the information gained during the performance measure system development processes in these organizations, and some preliminary observations are presented here. The university The objectives for the integrated performance measurement system in the university were rather wide. The target was to utilize the system e.g. in strategic steering, decision making and management at the university. There was a need for applications e.g. in performance measurement at all levels, quality management in all sectors, and co-operation between the university and the Ministry of Education. The rector founded four subgroups for the development of the measurement system and one coordinating group for controlling the process. The subgroups were founded to develop measures for teaching, research, service to the community and for administration and support organizations. There were three levels where the measurement system was to be utilized; the level of the whole university, departmental level and the level of teams and individuals. The development process started in spring 2003. The planning, commissioning and constitution of subgroups were done in March. During April the training was arranged. The information was given and an example from another Finnish university was presented. The subgroups had meetings in the summer and autumn. In the winter 2003/2004 the departments gave their statements concerning the proposals of the subgroups. The autumn 2004 and winter 2004/2005 are the time for decision-making and implementation. Some preliminary remarks of this process of developing an integrated performance measurement system at the university can be made. For example there seems to be too many interest groups, who affect or want to affect on system. Both the management of the university and the employees want to put their mark on the PMS. Also the departments, regional units, research units and even the ministry of education have different needs. As a result of the development of the PMS there were plenty of measures and targets of measurement. For example there were 53 measures or targets of measurement at departmental level and 15 targets of research measurement at university level. The problem seems to be that the main purpose or aim of measurement is not clear. They could not make the decision of whether it was to be a management tool, a data bank or data basis, or whether it was made for reporting to stakeholders.

343

FRONTIERS OF E-BUSINESS RESEARCH 2004

Another problem was that the cost effectiveness of the system might be poor. It requires a lot of work to collect and handle the data. There are many fuzzy measures which were difficult to measure and there are no good enough tools available (databases, programs, systems) for handling that data. The result of the development and implementation process seems to be databases which serve every one and no one. The state agency The preliminary target in the State agency was to develop a productivity measurement system. They wanted to know how effective the organization is and how to satisfy the stakeholders needs of information. They also wanted to measure the productivity of the organization. They knew the inputs but the output measurement was out of focus. After the first session the agency determined that the objective was to develop a performance measurement system for the organization. They wanted to measure the performance in different sectors and improve it. The development process started in the end of the year 2003. At the beginning of 2003 the start up, collection of background information and entry of external support were done. During the year 2004 four development meetings were held and the process will reach the decision making and implementation phase in the spring 2005. Some preliminary remarks of this process of developing the integrated performance measurement system in the state agency can be made. For example the main purpose or aim of the measurement was not clear. They did not know whether they were measuring the units or actions or sectors or level to manage. Also the nature of the customer was somewhat unclear. Who is the customer? Who pays for the "products"? The flow of information did not work properly in this organization. There seemed to be some problems between the units and persons in the organization, and there were also problems between the organization and external support. New critical information was brought out in every meeting. For example the focus of action made by the board directors came out in the third meeting. There were many overlapping projects in the organization. They defined the measures or indicators in various levels and units. There were differences in the identification and definition of processes. The relationship between these projects was undefined and unclear. The employees are specialists sensitive to status and self-respect. They are also autonomous, independent and sensitive to colleagues. These elements make the specialists difficult to manage. Also in this organization the management of these specialists seemed to be problematic and the management seemed to be too loose or soft.

344

FRONTIERS OF E-BUSINESS RESEARCH 2004

Conclusions
The development and implementation process in the knowledge-based public sector differs remarkably from the way they are realized in industrial private sector companies. The aims and targets for the development and utilization of performance measurement systems seem to be unclear in the case organizations. These organizations do not know exactly what they want to do with these kinds of methods or tools. The primary reason for the system is not determined. The management of specialists is difficult because they are the best experts in their area and they know it, but they do not understand or accept the fact that they perhaps are not specialists in the field of management or measurement. This kind of situation demands additional competence in the leadership from the managers. The divergence between service and payment leads to an abundance of customers. If the organization does not recognize its customers or the customers are multifaceted, the result is an abundance of measures. The abundance of measures for its part leads to ineffectiveness of operations. However, in a knowledge-based organization it may be inadequate to only concentrate on measuring the outcomes and efficiency i.e. the exploitation of current knowledge. Some measures, which illustrate the development of knowledge, may also be needed to promote future competitiveness and self-renewal in organizations. The abundance of measures and uncertainty about the primary reason for the measurement system can lead to a performance measurement system that will not serve anyone.

345

FRONTIERS OF E-BUSINESS RESEARCH 2004

References
Argyris, C. (1999) On Organizational Learning. Cambridge, Blackwell Publishers. 2.ed. Argyris, C. & Schn, D.A. (1978) Organizational Learning: a Theory of Action Perspective, AddisonWesley, Reading, Massachusetts Birchall, D.W. & Tovstiga, G. (1999) The strategic potential of a firms knowledge portfolio. Journal of General Management, vol. 25, no. 1 Autumn 1999, pp.1-16 Bourne, M. 2000. Designing, implementing and updating performance measurement systems. International Journal of Operations & Production Management, Vol. 20, No 7, s. 754-771. Crossan, M.M., Lane, H.W. & White, R.E. (1999) An organizational learning framework: from intuition to institution. Academy of Management Review, Vol. 24, No. 3, pp. 522-537 Davenport, T. & Prusak, L. (2000) Working Knowledge How Organizations Manage What They Know. Boston, Harvard Business School Press Drejer, A. (2000) Organisational learning and competence development. The Learning Organization, Vol. 7, lss.4; pg. 206 Kaplan, R.S. & Norton, D.P. 1996. The Balanced Scorecard: Translating Strategy into Action. Boston, Harvard Business School Press Maula, M. (2000) The senses and memory of a firm implications of autopoiesis theory for knowledge management. Journal of Knowledge Management, Vol. 4, Iss. 2, pp. 157 161 Neely, A. 1998. Measuring Business Performance. London, Profile Books Ltd., The Economist Books. Nonaka, I. & Takeuchi, H. (1995) The Knowledge Creating Company. Oxford, University Press Schein, E.H. (1996) Three cultures of management: the key to organizational learning. Sloan Management Review, Cambfidge: Fall 1996. Vol. 38, lss. 1; pp.9-21 Simons, R. (2000) Performance Measurement & Control Systems for Implementing Strategy. New Jersey, Prentice Hall. Sthle, P. & Grnroos, M. (2000) Dynamic Intellectual Capital Knowledge Management in Theory and Practice. WSOY, Helsinki

346