Kirkpatrick: Evaluating Training Programs

What is quality training?

How do you measure it? How do you improve it?

The Ten Factors of Developing a Training Program 1. 2. 3. 4. 5. Determine needs Set objectives Determine subject content Select qualified applicants Determine the best schedule

Evaluation Questions:
Increased production? Improved quality? Decreased costs? Improved safety numbers? Increased sales? Reduced turnover? Higher profits?

Guidelines for evaluating results:

Use a control group. Allow time for results to be achieved. Measure before and after the program. Repeat the measurements, as needed. Consider cost versus benefits. Be satisfied with evidence if proof is not possible.

Types of Assessments Used at Each Level

Level 4 - Results
Was it worth it?

KSA being used on the job?

Level 3 - Behavior

Level 2 - Knowledge
Did they learn anything

Level 1 - Reaction
Was the environment suitable for learning?

Level One - Questionnaire. Level Two - Demonstrate competency, create action plans through group simulations. Level Three - Follow-up to determine if action plans were met (specific steps to implement concepts of what was learned). Level Four - Ongoing process of tracking business indicators.
6

Reaction - What Is It?

How favorably participants react to the training (Customer satisfaction)
Collects reactions to instructor, course, and learning environment Communicates to trainees that their feedback is valued Can provide quantitative information

Reaction - What It Looks Like

Questionnaire - Most common collection tool
Content: I enjoyed the content. Methods: The seminar approach helped me learn. Media: The AVs were helpful to me. Trainer style: I liked the instructor. Facilities: The room was useful for my learning. Course materials: The materials provided make my learning better.
SD: 1 2 3 4 5 6 :SA

Learning - What Is It?

Knowledge Skills Attitudes

Learning - What It Looks Like

Media used to measure learning:
Text: I felt the textbook help me learn. Voice: The audio materials increased my knowledge. Demonstration: I learned well from the demonstration.

Methods used to measure learning:

Interviews Surveys Tests (pre-/post-) Observations Combinations
10

Learning - How to Perform

Use a control group, if feasible Evaluate knowledge, skills, and/or attitudes before and after Get 100% participation or use statistical sample Follow-up as appropriate

11

Behavior - What Is It?

Transfer of knowledge, skills, and/or attitude to the real world
Measure achievement of performance objectives

12

Behavior - What It Looks Like

Observe performer, first-hand Survey key people who observe performer Use checklists, questionnaires, interviews, or combinations
I believe coming in early is helpful. I work better on the new production system.

13

Behavior - How to Perform

Evaluate before and after training Allow ample time before observing Survey key people Consider cost vs. benefits
100% participation or a sampling Repeated evaluations at appropriate intervals Use of a control group
14

Results - What Is It?

Assesses bottom line, final results Definition of results dependent upon the goal of the training program

15

Results - What It Looks Like

Depends upon objectives of training program
Quantify: Bottom line, productivity, improvement

Sample Questions:
I believe the new system has improved productivity. My training has allowed me to be more productive. My boss is a better leader after my training.

16

Summary & Assignment

Level I: Level II: Level III: Level IV: Evaluate Reaction Evaluate Learning Evaluate Behavior Evaluate Results

Write five questions for each level using 4 different training scenarios.
Use I questions. Insure they apply to the content Tricks: Thesaursus, book TOC, ask each other

17

Post-test Questions
(1) Name three ways evaluation results can be measured. (2) Do all 4 Levels have to be used? (3) Do they have to be used in 1,2,3,4 order? (4) Is Kirkpatricks method of evaluation summative or formative? (5) Which developmental view does Kirkpatrick use? (discrepancy, analytical, diagnostic)

IF YOU THINK TRAINING IS EXPENSIVE, TRY IGNORANCE.

and, remember, the definition of ignorance is repeating the same behavior, over and over, and expecting different results!