INTRODUCTION A lot seems to have been written about evaluating training.

This may be in part because there seems to be a fair amount of confusion around, as well as for some a sense of something missing, of not being quite right. In this series of short articles well try to clarify things by asking some key, but basic killer questions, and not straying too far from them! The end result is some clarity as to what training evaluation is, where it fits in to the different worlds of the trainer, the customer/manager, the learner and the shareholder/owner, what methods and models exist and which ones to use in various different situations. There is also the realization that training and evaluation isnt just a matter for the trainer, and that the training perspective is not the only one to consider, if at all. In essence, a Dummys Guide to Evaluation. In this first part we will set the scene by asking those questions, and by defining a few useful things against the backdrop of briefly describing some of the work of probably the most well known contributor to this field, Donald Kirkpatrick. In the second part we will identify some of the different approaches, models and frameworks that exist, with some comparative pros and cons. In part 3 we take a brief look at which to use and when before wrapping it all up in part 4 with a look at some of the pitfalls that need to be overcome to address the sense in some quarters of things not being right, and how to tackle them. KILLER QUESTIONS So, lets start with some key questions. Ive taken the liberty of borrowing some from Fred Nichols as he asked these questions with impact some time ago, back in the early 1990s: Evaluate? Evaluate what? Training? What do we mean by training? Whats to be evaluated? A particular course? The trainees? The trainers? The training department? A certain set of training materials? Training in general?

More to the point, why evaluate it? Do we wish to gauge its effectiveness, that is, to see if it works? If so, what is it supposed to do? Change behaviour? Shape attitudes? Improve job performance? Reduce defects? Increase sales? Enhance quality?

What about efficiency? How much time does the training consume? Can it be shortened? Can we make do with on-the-job training or can we completely eliminate training by substituting job aids instead?

What does it cost? Whatever it costs, is it worth it? Who says? On what basis? What are we trying to find out? For whom?

Source: http://home.att.net/~nickols/evaluating_training.htm

These are all killer questions. In these articles well answer some of these questions, and others, including: when to evaluate, and when not to; who is responsible for which bits; when quick & dirty is OK, and when it isnt; which approach to use and when, and what is the bottom line or ROI contribution.

SOME WORKING DEFINITIONS Lets start with some definitions. There seem to be two different focuses for evaluation: focusing on the actual process of training or performance improvement (whats known as formative evaluation), and focusing on the final product or outcome of the process (whats known as summative evaluation). Evaluation seems to mean different things to different people, hence some of the confusion. Its suggested by some of the killer questions that the answers and meanings depend upon perspective who is asking the questions and why. Lets identify and define these different audiences. For shareholders and managers of an organization the need to evaluate is to help answer the related questions of will training fix my problem and/or help achieve our goals? and will it be worth it or should I invest my resources elsewhere? The final question is now more obvious: was it worth it? For the trainer perhaps evaluation is driven by the need to answer different questions, such as was the training effective? and did it achieve its objectives? And for the employee the evaluation questions are likely to be will it help me do my job better/easier/faster?, will it help my career development?, what am I doing here? and

whats in it for me? Given that most of the thinking on evaluation seems to have been done by those in the training world, is it any wonder that there is some tension between each of these 3 groups when evaluation comes up for discussion? TRAINING EVALUATION The process of examining a training program is called training evaluation. Training evaluation checks whether training has had the desired effect. Training evaluation ensures that whether candidates are able to implement their learning in their respective workplaces, or to the regular work routines.

PURPOSES OF TRAINING EVALUATION The five main purposes of training evaluation are: Feedback: It helps in giving feedback to the candidates by defining the objectives and linking it to learning outcomes. Research: It helps in ascertaining the relationship between acquired knowledge, transfer of knowledge at the work place, and training.

Control: It helps in controlling the training program because if the training is not effective, then it can be dealt with accordingly. Power games: At times, the top management (higher authoritative employee) uses the evaluative data to manipulate it for their own benefits. Intervention: It helps in determining that whether the actual outcomes are aligned with the expected outcomes.

PROCESS OF TRAINING EVALUATION Before Training: The learner's skills and knowledge are assessed before the training program. During the start of training, candidates generally perceive it as a waste of resources because at most of the times candidates are unaware of the objectives and learning outcomes of the program. Once aware, they are asked to give their opinions on the methods used and whether those methods confirm to the candidates preferences and learning style.

During Training: It is the phase at which instruction is started. This phase usually consist of short tests at regular intervals After Training: It is the phase when learners skills and knowledge are assessed again to measure the effectiveness of the training. This phase is designed to determine whether training has had the desired effect at individual department and organizational levels. There are various evaluation techniques for this phase. TECHNIQUES OF EVALUATION The various methods of training evaluation are:

Observation Questionnaire Interview Self diaries Self recording of specific incidents

THEORETICAL BACKGROUND OF TRAINING EVALUATION Training Evaluation - Meaning Evaluation involves the assessment of the effectiveness of the training programs. This assessment is done by collecting data on whether the participants were satisfied with the deliverables of the training program, whether they learned something from the training and are able to apply those skills at their workplace. There are different tools for assessment of a training program depending upon the kind of training conducted. Since organizations spend a large amount of money, it is therefore important for them to understand the usefulness of the same. For example, if a certain technical training was conducted, the organization would be interested in knowing whether the new skills are being put to use at the workplace or in other words whether the effectiveness of the worker is enhanced. Similarly in case of behavioral training, the same would be evaluated on whether there is change in the behavior, attitude and learning ability of the participants. Benefits of Training Evaluation Evaluation acts as a check to ensure that the training is able to fill the competency gaps within the organization in a cost effective way. This is especially very important in wake of the fact the organizations are trying to cut costs and increase globally. Some of the benefits of the training evaluation are as under:

Evaluation ensures accountability - Training evaluation ensures that training programs comply with the competency gaps and that the deliverables are not compromised upon. Check the Cost - Evaluation ensures that the training programs are effective in improving the work quality, employee behavior, attitude and development of new skills within the employee within a certain budget. Since globally companies are trying to cut their costs without compromising upon the quality, evaluation just aims at achieving the same with training. Feedback to the Trainer / Training - Evaluation also acts as a feedback to the trainer or the facilitator and the entire training process. Since evaluation accesses individuals at the

level of their work, it gets easier to understand the loopholes of the training and the changes required in the training methodology. Not many organizations believe in the process of evaluation or at least do not have an evaluation system in place. Many organizations conduct training programs year after year only as a matter of faith and not many have a firm evaluation mechanism in place. Organizations like IBM, Motorola only, it was found out, have a firm evaluation mechanism in place. The Way Forward There are many methods and tools available for evaluating the effectiveness of training programs. Their usability depends on the kind of training program that is under evaluation. Generally most of the organizations use the Kirk Patrick model for training evaluations which evaluates training at four levels - reactions, learning, behavior and results. After it was found out that training costs organizations a lot of money and no evaluation measures the return on investment for training, the fifth level for training evaluation was added to the training evaluation model by Kirk Patrick which is called as the ROI. Most of the evaluations contain themselves to the reaction data, only few collected the learning data, still lesser measured and analyzed the change in behaviour and very few took it to the level of increase in business results. The evaluation tools including the Kirk Patrick model will be discussed in detail in other articles.

Training Needs Analysis Training is an expensive process not only in terms of the money spent on it but also the time and the other resources spent on the same. The most important question therefore is determining whether or not a need for training actually exists and whether the intervention will contribute to the achievement of organizational goal directly or indirectly? The answer to the above mentioned question lies in training needs analysis which is the first step in the entire process of training and development.

Training needs analysis is a systematic process of understanding training requirements. It is conducted at three stages - at the level of organization, individual and the job, each of which is called as the organizational, individual and job analysis. Once these analyses are over, the results are collated to arrive upon the objectives of the training program. Another view of the training need is that, it is the discrepancy between what is and what should be. Taking cues from this the world bank conducted a needs analysis and arrived upon the conclusion that many of its units in eastern regions of Europe required transformation from state owned business to self sustaining organizations. A number of universities were then contacted to develop the necessary modules and conduct the training upon the same. Although each step in the entire training process is unique in its own, needs analysis is special in that it lays the foundation for the kind of training required. The assessment gives insight into what kind of intervention is required, knowledge or skill or both. In certain cases where both of these are present and the performance is still missing then the problem may be motivational in nature. It thus highlights the need and the appropriate intervention which is essential to make the training effective. As mentioned earlier, the needs analysis / assessment is carried out at three levels organizational, Individual and Job. We now take up each one of them in detail. Organizational Analysis The organizational analysis is aimed at short listing the focus areas for training within the organization and the factors that may affect the same. Organizational mission, vision, goals, people inventories, processes, performance data are all studied. The study gives cues about the kind of learning environment required for the training. Motorola and IBM for example, conduct surveys every year keeping in view the short term and long term goals of the organization.

Job Analysis The job analysis of the needs assessment survey aims at understanding the what of the training development stage. The kind of intervention needed is what is decided upon in the job analysis. It is an objective assessment of the job wherein both the worker oriented - approach as well as the task - oriented approach is taken into consideration. The worker approach identifies key behaviors and ASK for a certain job and the task - oriented approach identifies the activities to be performed in a certain job. The former is useful in deciding the intervention and the latter in content development and program evaluation. Individual Analysis As evident from the name itself, the individual analysis is concerned with who in the organization needs the training and in which particular area. Here performance is taken out from the performance appraisal data and the same is compared with the expected level or standard of performance. The individual analysis is also conducted through questionnaires, 360 feedback, personal interviews etc. Likewise, many organization use competency ratings to rate their managers; these ratings may come from their subordinates, customers, peers, bosses etc. Apart from the above mentioned organizations also make use of attitude surveys, critical Incidents and Assessment surveys to understand training needs which will be discussed in detail in other articles.

Training and Development - A Key HR Function Training and development is one of the key HR functions. Most organizations look at training and development as an integral part of the human resource development activity. The turn of the century has seen increased focus on the same in organisations globally. Many organisations have mandated training hours per year for employees keeping in consideration the fact that technology is deskilling the employees at a very fast rate. So what is training and development then? Is it really that important to organisational survival or they can survive without the former? Are training and development one and the same thing or are they different? Training may be described as an endeavour aimed to improve or

develop additional competency or skills in an employee on the job one currently holds in order to increase the performance or productivity. Technically training involves change in attitude, skills or knowledge of a person with the resultant improvement in the behaviour. For training to be effective it has to be a planned activity conducted after a thorough need analysis and target at certain competencies, most important it is to be conducted in a learning atmosphere While designing the training program it has to be kept in mind that both the individual goals and organisational goals are kept in mind. Although it may not be entirely possible to ensure a sync, but competencies are chosen in a way that a win-win is created for the employee and the organization. Typically organizations prepare their training calendars at the beginning of the financial year where training needs are identified for the employees. This need identification called as training need analysis is a part of the performance appraisal process. After need analysis the number of training hours, along with the training intervention are decided and the same is spread strategically over the next year. Development Lots of time training is confused with development, both are different in certain respects yet components of the same system. Development implies opportunities created to help employees grow. It is more of long term or futuristic in nature as opposed to training, which focus on the current job. It also is not limited to the job avenues in the current organization but may focus on other development aspects also. At Goodyear, for example, employees are expected to mandatorily attend training program on presentation skills however they are also free to choose a course on perspectives in leadership through literature. Whereas the presentation skills program helps them on job, the literature based program may or may not help them directly.

Similarly many organizations choose certain employees preferentially for programs to develop them for future positions. This is done on the basis of existing attitude, skills and abilities, knowledge and performance of the employee. Most of the leadership programs tend to be of this nature with a vision of creating and nurturing leaders for tomorrow. The major difference between training and development therefore is that while training focuses often on the current employee needs or competency gaps, development concerns itself with preparing people for future assignments and responsibilities. With technology creating more deskilled workers and with industrial workers being replaced by knowledge workers, training and development is at the forefront of HRD. The onus is now on the human development department to take a proactive leadership role in responding to training and business needs.

Training and Development - Meaning, its Need and Importance Training and development is vital part of the human resource development. It is assuming ever important role in wake of the advancement of technology which has resulted in ever increasing competition, rise in customers expectation of quality and service and a subsequent need to lower costs. It is also become more important globally in order to prepare workers for new jobs. In the current write up, we will focus more on the emerging need of training and development, its implications upon individuals and the employers. Noted management author Peter Drucker said that the fastest growing industry would be training and development as a result of replacement of industrial workers with knowledge workers. In United States, for example, according to one estimate technology is de - skilling 75 % of the population. This is true for the developing nations and for those who are on the threshold of development. In Japan for example, with increasing number of women joining traditionally male jobs, training is required not only to impart necessary job skills but also for preparing them for the physically demanding jobs.. They are trained in everything from sexual harassment policies to the necessary job skills.

The need for Training and Development Before we say that technology is responsible for increased need of training inputs to employees, it is important to understand that there are other factors too that contribute to the latter. Training is also necessary for the individual development and progress of the employee, which motivates him to work for a certain organization apart from just money. We also require training update employees of the market trends, the change in the employment policies and other things. The following are the two biggest factors that contribute to the increased need to training and development in organisations:
1. Change: The word change encapsulates almost everything. It is one of the biggest factors

that contribute to the need of training and development. There is in fact a direct relationship between the two. Change leads to the need for training and development and training and development leads to individual and organisational change, and the cycle goes on and on. More specifically it is the technology that is driving the need; changing the way how businesses function, compete and deliver.
2. Development: It is again one the strong reasons for training and development becoming

all the more important. Money is not the sole motivator at work and this is especially very true for the 21st century. People who work with organisations seek more than just employment out of their work; they look at holistic development of self. Spirituality and self-awareness for example are gaining momentum world over. People seek happiness at jobs which may not be possible unless an individual is aware of the self. At ford, for example, an individual can enroll himself / herself in a course on self awareness, which apparently seems inconsequential to ones performance at work but contributes to the spiritual well being of an individual which is all the more important.

Systems View of Training - Stages in a Training Program The success of a training program is evaluated in terms of the end result or the increase in the work ability, skill or competency in the trainee. For any training program to be successful it is very essential to follow a certain process. The basic process as illustrated in the figure below consists of four stages which are assessment, development, delivery and evaluation.

The process of training begins with the needs assessment stage. The aim of the assessment stage is to understand whether or not training is required. If the answer is yes; the next step is determining competency or skills gaps and the appropriate training intervention required. The training intervention is essentially decided in terms of attitude, knowledge and skill (ASK), the combination of which is called as competency. The assessment also called as the training needs analysis is undertaken at three levels, the job, the individual and organisational analysis.

Once the training needs analysis is complete, the next stage is that of Development. This stage involves the development of content and the training material. Right from designing the appropriate environment to deciding the various tools, everything is taken care of in the development stage. Games, A/Vs, Case Studies, Class room intervention are various means that may be decided upon apart from the content delivered. For example, in behavioural training emotional intelligence, teamwork, listening are examples of competencies that are required to perform superior work. The same may be transferred into the trainees through any of the above means depending upon various factors like demographics, job nature etc which are taken care of in the first stage. The most important stage and perhaps the least talked upon from the training process is the delivery. Once the development stage is over it is time to conduct the training. Factors like time and venue of delivery are already decided in the earlier stages. There are various factors that determine the process of delivery like the participant demographics, the training intervention, the individual style of the trainer etc. This brings in a lot of diversity to the training programs. Evaluation is the last stage in the training process and more important from the perspective of evaluation of the effectiveness of training. Needless to say, it is aimed at analysing whether or not the training has been effective in achieving the objective (bridging the competency gap, changing the attitude, developing new skills etc). There are various ways in which the effectiveness of training programs can be evaluated but not many are able to answer in terms of ROI. The most effective tool for evaluation of training is the Kirk Patrick Model of Evaluation.

In order for the evaluation to be effective the both the criteria and design for training program is decided so that there is no discrepancy and the participants are able to evaluate the benefits effectively for themselves. The evaluation is made on the basis of participant reaction to the training, their learning and the change in behaviour. This feedback is then reused in the first step training needs analysis for making future training more effective.

Kirkpatricks Model of Training Evaluation Organizations spend a huge amount of money for training their employees at various levels and on various competencies, behavioral and technical. Every year new tools are designed to try and cater to individual learning styles and make the training more effective. After all an organization is concerned about its spending and the return on the same! Donald Kirkpatrick, professor emeritus, university of Wisconsin began working on evaluating the effectiveness of training very early in his life. His early work on the same was published in the year 1959 in a journal of American Society of Training Directors. He laid out four levels for evaluation of any training. This model is arguably the most widespread for evaluation in use. It is simple, very flexible and complete. The four levels as described by Kirkpatrick are as follows:
1. Reaction of the Trainee - thoughts and feelings of the participants about the training 2. Learning - the increase in knowledge or understanding as a result of the training 3. Behavior - extent of change in behavior, attitude or capability

Results - the effect on the bottom line of the company as a result of the training. The fifth level which is the ROI has been recently added which is not but a part of the original model. The graphic description is as follows:

Level

Results

Was it worth it ? Level Job ? 3-Behaviour KSAs being used on the

Level

Knowledge

Did they learn anything ? Level 1 Was the suitable ? The beauty of the model is that each level can only be predicted when the lower level prediction is complete. Thus evaluation at the level of behaviour may not be useful unless evaluation at the knowledge has been completed. Reaction Reaction implies how favorably the participants have responded to the training. This evaluation is primarily quantitative in nature and is a feedback to the training and the trainer. The most common collection tool is the questionnaire that analyses the content, methodology, facilities and the course content. Learning At the level of learning the evaluation is done on the basis of change in the ASK (Attitudes, skills and knowledge) of the trainees. The evaluation involves observation and analysis of the voice, behaviour, text. Other tools used apart from the observation are interviews, surveys, pre and post tests etc. Behaviour Behaviour evaluation analyses the transfer of learning from the training session to the work place. Here the primary tool for evaluation is predominantly the observation. Apart from the observation, a combination of questionnaires and 360 feedbacks are also used. Reaction environment

Results The results stage makes evaluations towards the bottom line of the organization. Here the definition of the results depends upon the goal of the training program. The evaluation is done by using a control group allowing certain time for the results to be achieved. There are many other models that are unique in their own ways, nut as mentioned earlier Kirkpatricks Model is the one that is accepted and used widely across all industries and with wider applications.

OBJECTIVES OF THE STUDY

To evaluate the training given to the employees of I-FACT TECHNOLOGIES PVT. LTD. To study the satisfaction level of employees towards training at I-FACT

TECHNOLOGIES PVT. LTD. To understand the factors influencing the training given to the employees. To identify ways to improve the existing training evaluation system. To analyze the relationship between demographic factors and satisfaction towards training process. SCOPE OF THE STUDY The study can be used to make the present evaluation system more effective and satisfied among employees. And it could be used to identify training and development needs for the individual and the organization as a whole. The study helps to understand the employees perception to recommending organization as a preferred place to work. The management can identify the employees feedback of the existing training evaluation system.

LIMITATION OF THE STUDY Unwillingness and inability of the respondent to provide true information.

Limited Resource and Time constraints.

RESEARCH METHODOLOGY Research Design A research design is the arrangement of conditions for the collection and analysis of data in a manner that aims to combine relevance to the research purpose with economy in procedure. In fact, the research design is the conceptual structure within which research is conducted; it constitutes the blueprint for the collection, measurement and analysis of data. The present study attempts to assess the Training Evaluation in I-FACT TECHNOLOGIESPVT.LTD. It tries to assess the image of an organization as a great place to work in the minds of its current employees and key stakeholders. Hence this study is a descriptive study. Sampling Design Population 255 employees working in I-FACT TECHNOLOGIESPVT.LTD, Corporate office, Bangalore forms the universe. Sample The survey was conducted with 70 employees of I-FACT TECHNOLOGIES and the questionnaire was distributed to them. Sampling Design A sampling design is a definite plan for obtaining a sample from a given population. Sampling method Simple Random Sampling

Simple Random Sampling In a simple random sample ('SRS') of a given size, all such subsets of the frame are given an equal probability. Each element of the frame thus has an equal probability of selection: The frame is not subdivided or partitioned. Furthermore, any given pair of elements has the same chance of selection as any other such pair (and similarly for triples, and so on). This minimizes bias and simplifies analysis of results. In particular, the variance between individual results within the sample is a good indicator of variance in the overall population, which makes it relatively easy to estimate the accuracy of results. Simple Random sampling from a finite population refers to that method of sample selection which gives each possible sample combination an equal probability of being picked up and each item in the entire population to have an equal chance of being included in the sample. 255 is a finite population and 70 is a sample that we choose at random. Here the researcher used the Simple Random sampling method to collect the data. Method of Data Collection Data collection Data can be classified as primary data and secondary data. Data collected for first time under the direct control and supervision of the investigator is called primary data and those derived from other sources and not collected by the investigator for first time is secondary data. Data collection is classified into two namely 1. Primary data collection 2. Secondary data collection

1. Primary data Data observed or collected directly from first-hand experience of the researcher. In this research the primary data was collected by using questionnaire method

Questionnairemethod: It is a research instrument consisting of a series of questions for the purpose of gathering possible true information from respondents. 2. Secondary data Secondary data means data that are already available i.e., they refer to the data which have already been collected and analyzed by someone else. When the researcher utilizes secondary data, then he/she has to look into various sources from where he/she can obtain them. Statistical Tools Used in the Study The collected information has been processed both manually and with the help of computers and analyzed suitably. The following statistical tools are used in this study, 1. Simple percentage analysis. 2. Chi-square 3. Regression

1.Simple Percentage Analysis Simple percentage analysis refers to a ratio. With the help of absolute figures it will be difficult to interpret any meaning from the collected data, but when percentages are found out then it becomes easy to find the relative difference between two or more attributes. No. of respondents Percentage = Total no. of respondents 2.Chi-Square Test Chi-square is a statistical test commonly used to compare observed data with data we would expect to obtain according to a specific hypothesis. 100

3.Regression The regression is a statistical measure that attempts to determine the strength of the relationship between one dependent variable (usually denoted by Y) and a series of other changing variables (known as independent variables).

Hypotheses

In Symbols In Words HO 1=2= The presence of others does not influence helping. HA Not Ho
3

The presence of others does influence helping.

Given the null hypothesis (in this case HO: 1=2=3), the two variance estimates should be equal. That is, since the null assumes no treatment effect, both variance estimates reflect error and their ratio will equal 1. To the extent that this ratio is larger than 1, it suggests a treatment effect (i.e., differences between the groups). It turns out that the ratio of these two variance estimates is distributed as F when the null hypothesis is true.