A Synopsis of Corporate IT Training

A Document prepared by Dr Sameh S El-Atawy and published by Kemetex.com,

Alexandria, EGYPT.

Attended Readership

This document should be read by business owners, executive officers, HR managers

and/or IT managers involved in corporate IT training.

This document addresses the needs of a business or corporation embarking on a

project including Information Technology training for its personnel. Examples
however, have been drawn from different areas to exemplify points and show the
universal applicability of most of the mentioned guidelines. Indeed, any pre-
placement or in-house training project may use most of this document.

Nota Bene

This document is by no means comprehensive and does not pretend to cover the
subject at hand. It is merely a collection of guidelines, suggestions, and an insider’s
view based on the Author’s experience in this area.

This document was based in no small part on the contributions of others, all of whom
the Author wishes to thank heartily.

References on the subject are vast and textbooks, conferences, and papers have been
dedicated to individual topics mentioned here. Please consult the literature for details.
The author has attempted to use established nomenclature whenever possible to
facilitate further readership.

About The Author

Dr Sameh S El-Atawy has been heavily involved in training staff on various locations
and at various levels of experience. Being a veteran software and internet developer
himself, he trained several other members of staff at mess@ge, VeBIT, and is
currently working at Kemetex.com.
Besides that, his educational experience includes lecturing IT to staff at the Faculty of
Medicine of Alexandria University, in planning and conducting pre-placement
training for a major private Hospital in Alexandria, and Programming, Databases, and
SQL privately. He was among the first to teach ICDL courses in Egypt with a
UNESCO fund during 2001.
Dr El-Atawy practices Medicine as head of the ER at a corporate private Hospital and
runs his own small clinic where he welcomes trainees as well as patients.
 Corporate IT Training www.kemetex.com

CORPORATE IT TRAINING DEFINITIONS ............................................................................................ 4

STAGES OF CORPORATE IT TRAINING ................................................................................................ 5
PLANNING ......................................................................................................................... 5
Requirement Analysis......................................................................................................................... 5
Objectives ................................................................................................................................... 5
Targets ....................................................................................................................................... 5
Skills .......................................................................................................................................... 6
Topics......................................................................................................................................... 6
Constraint Definition ......................................................................................................................... 7
Trainer Constraints ..................................................................................................................... 7
Trainee Constraints ..................................................................................................................... 7
Tool Constraints ......................................................................................................................... 8
Time Constraints......................................................................................................................... 8
Imposed Constraints.................................................................................................................... 9
Resource Definition ........................................................................................................................... 9
Course Material .......................................................................................................................... 9
Human Resources ....................................................................................................................... 9
Educational Tools ..................................................................................................................... 10
Venue ....................................................................................................................................... 10
Time ......................................................................................................................................... 10
Course Planning .............................................................................................................................. 10
Course-Topic Relations............................................................................................................. 10
Course-Session Relations .......................................................................................................... 10
Course-Skill Relations .............................................................................................................. 11
Session Interdependencies ......................................................................................................... 11
Session Planning ............................................................................................................................. 11
Resource Conflicts .................................................................................................................... 11
Topic/Skill Conflicts ................................................................................................................. 11
Adequate Time Distribution ...................................................................................................... 12
Adequate Time Allocation ........................................................................................................ 12
Assessment Timing ................................................................................................................... 12

Page 2 of 24
 Corporate IT Training www.kemetex.com

IMPLEMENTATION .......................................................................................................... 13
Preparation ..................................................................................................................................... 13
Availability: .............................................................................................................................. 13
Functionality:............................................................................................................................ 13
Reliability: ................................................................................................................................ 13
Stability: ................................................................................................................................... 13
Suitability: ................................................................................................................................ 14
Accuracy: ................................................................................................................................. 14
Delivery........................................................................................................................................... 14
Instructor-Trainee Interaction: ................................................................................................... 14
Trainee Feedback: ..................................................................................................................... 14
Peer Interaction: ........................................................................................................................ 15
Pop Quiz Methods: ................................................................................................................... 15
Lab Exercises ........................................................................................................................... 16
Real-Life Situations .................................................................................................................. 16
Unattended Practice .................................................................................................................. 16
Assessment ...................................................................................................................................... 17
Type of Questions ..................................................................................................................... 17
Scope of Questions ................................................................................................................... 17
Difficulty of Questions .............................................................................................................. 18
Weighing of Questions.............................................................................................................. 18
Assessment By Assignments ..................................................................................................... 18
Ambiguity of Questions ............................................................................................................ 19
Ambiguity of Responses ........................................................................................................... 19
Creative Responses ................................................................................................................... 19
Gaussian (Normal) Distribution of Assessment Results.............................................................. 19
Standardization Questions ......................................................................................................... 19
Elimination of Sources of Potential Errors ................................................................................. 20
EVALUATION .................................................................................................................. 21
Data Collection ............................................................................................................................... 21
Instructor Questionnaires .......................................................................................................... 21
Trainee Questionnaires.............................................................................................................. 21
External Reviewers ................................................................................................................... 23
Skill Achievement..................................................................................................................... 22
Statistical Significance of Assessment Shifts ............................................................................. 23
Target/Objective Achievement .................................................................................................. 23
Decision Support ............................................................................................................................. 24
Scenario 1:................................................................................................................................ 24
Scenario 2:................................................................................................................................ 24
Scenario 3:................................................................................................................................ 24

Page 3 of 24
Corporate IT Training www.kemetex.com

Corporate IT Training Definitions

 A body is the corporation or business entity requiring a training
operation.
 A body can be tiny, small, medium, large, or huge in a different sense
here than in comparable business parlance. When categorizing a body
for purposes of training, it is prudent to use potential trainee count as
a judge of operation size.
 Of course, more professional categorization of training operations
depend on trainee hours to be covered and variance in training
requirements. This only needs to be applied for large and huge
bodies.

Tiny Small Medium Large Huge

Potential Trainees Up to 5 5 - 50 50 - 500 500 - 5000 Over 5000

Simple Plan Must Must - - -

Complex Plan - - Must Maybe -

Multiple Plans - - - Maybe Must

Own Material No Maybe Maybe Must Must

Hire Instructors Maybe Maybe Maybe - -

Third Party Maybe Maybe Maybe - -

Own Department - - - Must Must

Page 4 of 24
Corporate IT Training www.kemetex.com

Stages of Corporate IT Training

Planning

Requirement Analysis

Objectives
 The body must derive a list of specific objectives from its Mission
Statement or Stated Aim(s) in accordance with its Policy
Statement(s).
 The list of objectives can be neither very broad nor very narrow in
interpretation. Simple ‘offering the best customer support’ is not
adequate but rather: ‘Be able to provide support for the vast majority
of clients and issues in a timely fashion noticeably ahead of the
competition without incurring support costs over the projected
estimate used in calculating product pricing and without undermining
the quality of support provided’.
 An objective must be able to speak for itself in terms of which
department(s) or task force(s) it refers to, as well as what level of
achievement is deemed satisfactory. In the above example, it is clear
that the support department is involved and that satisfaction is
declared if three requirements are met: speed of delivery, staying
within budget, and maintenance of quality.
 An objective must be reasonable to expect. For example, the above
may not specify: ‘resolution of the most difficult support issue at all
costs to preserve corporate image’ and at the same time expect to stay
within budget without flexibility. Priorities have to be stated.

Targets
 From the objectives defined previously, more specific targets are
defined for each. These include measurable quantities or indicators of
objective satisfaction.
 Targets can have variables included. For example, the target for sales
could be a certain quota or market share this year, but a different one
the following year.
 Targets are formulated centrally but must be made open to scrutiny
by department (or task force) staff according to their own insight.
This helps keep expectations realistic.
 Targets may have to be published and made known to involved body
members as a motivational and accomplishment self-metering tool.
 For example, the above statement could define targets as:
1) Satisfaction of at least 80% of support issues over the phone within 2
phone calls of under 20 minutes overall.
2) Satisfaction of at least 50% of issues requiring trained personnel
through one visit to the client of under two hours by one support
member.

Page 5 of 24
Corporate IT Training www.kemetex.com

3) Customer satisfaction of at least 90% measured by printed and online

customer questionnaires. These are to be later reviewed anonymously
for fair judgment.
4) Comparison of support costs with fiscal planning estimates on a per-
product basis. Sub-reports on certain geographical areas and certain
staff members will be furnished quarterly.
5) Comparison of statistics with at least two major competitors to be
found significant if a 10% difference is found in average and
worst/best scenario support times.

Skills
 To achieve the targets defined, a set of skills will be needed. These
are enumerated then divided over different job descriptions as the
need arises.
 Skills need to be highly specific and tailored to specific hardware,
software, tools, or knowledge topics in use.
 For example, the above targets can include the following skills (For
Phone Support Personnel):
1) Ability to talk on the phone eligibly and politely.
2) Ability to take notes while on the phone without the customer having
to talk slowly or repeat him/herself.
3) Ability to obtain feedback from a customer and document it.
4) Ability to absorb customer anger or discontent.
5) Ability to use the installed redirection software to route the call to
another member or schedule a call-back.
6) Ability to use the installed knowledge-base software to obtain
answers to questions quickly, etc.

Topics
 Once all the skills are enumerated, skills are then grouped together
into logical topics.
 Grouping is based on these factors:
A. Ability to grasp and master grouped skills simultaneously.
B. Better ability to grasp and master a skill if compared to another.
C. Usually, skills that involve the same hardware/software/tool and
likely to be used by one job description are grouped together.
D. Skills that require specific educational resources are grouped
together.
E. Some skills are simple enough to be included within the instruction
of another skill.
 For example, to train staff on the above Phone Support skills, one
might define a topic called: ‘Phone etiquette’ including the first four
skills. These require a support lab with phones and PCs and recording
hardware to be instructed by senior support staff. The last two skills
mentioned are grouped into another topic defined as ‘Phone Software
skills’ which can be taught in a PC lab without recording hardware
and in larger trainee groups to be instructed by software engineers.

Page 6 of 24
Corporate IT Training www.kemetex.com

Constraint Definition

Trainer Constraints
 Number limitations: The number of available experienced
instructor-level staff may be limited. This must be noted on a per
topic basis.
 Competency: Each instructor needs to have a check of his/her
qualifications, personal merits, interpersonal skills, leadership skills,
and specific skill levels checked. This is noted on a skill basis per
instructor.
 Motivation: Available instructors may not have the ability or
inclination to teach and may wish to delegate this task to another.
Being an instructor is not something to be forced upon an unwilling
person. Motivation may need to be created in form of incentives or
repositioning.

Trainee Constraints
 Competency: Some trainees may not need to attend a specific topic.
Forcing them to do so would be a waste of time and cost in addition
to giving them a feeling of worthlessness of the whole training
experience. This will in turn reflect negatively on the instructors
themselves and on their peers.
 A more serious problem is trainees that do not perceive a need to
attend a topic whilst in fact they do. Convincing somebody to be
retrained on something they’ve been doing for a while may seem as a
declaration of mistrust or belittlement.
 One solution to the above problem may be to ask them to attend in
order to submit reports to higher management. These reports should
include an evaluation of the instructor and a list of new skills they
feel they had acquired from the attendance. This way, they feel more
in charge and valued for their contribution, thus they pay more
attention.
 Aptitude: In most instances, trainees will fail to learn a foreign
language if above a certain age. Training English-only speakers on
computer software in French is likely to also result in failure.
Teaching mathematics to surgeons who have long forsaken this
exercise may as well be skipped. These are examples of lack of
aptitude. For a trainee to grasp or master a skill, certain prerequisites
or a certain background may be necessary.
 Motivation: This was discussed regarding competency, but another
aspect needs to be addressed. Do the trainees feel they will be
rewarded (or de-merited) in any fashion if they attend (or fail to
attend) training.
 Attempting to motivate trainees by purely positive rewards may be
successful, but threatening punishment only is deemed to failure. At
best, trainees will attend without paying real attention. A combination
of positive and negative scoring based on both attendance and post-
training assessment is likely to bring forth the best results.

Page 7 of 24
Corporate IT Training www.kemetex.com

Tool Constraints
 Certain material may need to be delivered in certain ways. Skills may
also need certain tools to be mastered. These are, first of all,
enumerated.
 By definition, tools are anything that enable the delivery of material
to trainees; or the practice or assessment of certain skills before or
during implementation of the training operation including
assessment. This includes hardware, software, classroom facilities,
office equipment, etc.
 Each topic and skill is given a set of tools to be provided. Each tool in
turn, will later be allocated to any session where the topic or skill to
be covered requires it.
 Tools obtained from outside the body, that incur high costs, or that
may not be available are marked accordingly. Either such imposed
constraints are resolved or changes may be needed.
 Such changes may include course material, venue, skills, or the entire
objectives-targets hierarchy. If at all possible, imposed tool
constraints should be resolved by the body and not made to reflect on
the training operation.

Time Constraints
 Overall duration: Certain courses composed of a number of
sessions, topics, and related skills, may need a minimum of time to be
adequately mastered. The classic example is a foreign language. Each
language of course has a different minimum overall duration. Crash
courses simple cannot remedy such a constraint.
 Session duration: Sessions covering large, difficult topics or where
classroom assignments can be long will have to be given a minimum
duration. An example being Architectural Drawing sessions.
 Sessions covering highly focused, intensive material may need to
have a maximum duration set, as for instance when dealing with
Modern Physics or Biochemistry. Expecting a trainee to stay alert and
absorb new data for three hours of lecturing is impossible. Therefore,
it is best that sessions be allotted topics according to the expected
minimum and maximum durations of each.
 Session repetition: Needless to say, skills developed over time in the
classroom may have to be spread over several sittings; giving trainees
enough time to gain insight, speed, and discover potential pitfalls
before undergoing assessment.
 Instructors may also need to deliver sessions more than once to
divide a large number of attendants into groups. This may be set
according to the personal need to be delivered to each attendant, or
practically, to be accommodated at the chosen venue comfortably.

Page 8 of 24
Corporate IT Training www.kemetex.com

 Inter-session interval: Sessions may need to be spread out with two

considerations:
1) Feasibility: One cannot expect the main activities of the body to be
halted whilst training goes on. Moreover, one cannot expect trainees
to attend twelve hours of sessions a day. Sessions need to be spread
out with working hours, breaks, holidays, and other activities in
mind.
2) Availability: The venue, tools, instructor, and trainees must be all
available without interfering with other activities, including other
training times.
 Pre-assessment interval: Trainees to be assessed at the end of a
course may need some time for assignments, dissertations, practice,
etc. This needs to be noted on a course basis.

Imposed Constraints
 The body imposes certain constraints on training plans that do not
arise from the training process or normal activities.
 Examples include deadlines for completion, cost of training, pass
rate, etc.
 As a general rule, these constraints tend to be perceived higher than
inherent constraints. This is the worst management decision that
could be taken. Best practice dictates that if training is to be
implemented, it should be effective. Imposed constraints may need to
be negotiated with higher management and overcome if they prevent
satisfaction of inherent constraints. Failure to do so results in a bad
operation and a waste of valuable resources to no avail.

Resource Definition

Course Material
 This will be given out to instructors to use as a checklist for each
topic in a course.
 This will also cover what trainees are to be given to review what they
have learned or prepare for assessment.
 This also includes presentations, computer software, multimedia
clips, reference list, etc.

Human Resources
 A list of instructors and trainees are to be provided prior to
scheduling.
 A task list for each instructor and trainee as to what he/she will need
to prepare, what sessions are to be attended, and what constitutes
satisfactory completion of instruction duties.
 After scheduling, this will translate into a timetable.

Page 9 of 24
Corporate IT Training www.kemetex.com

Educational Tools
 Hardware, software, classroom facilities, office equipment, etc. By
definition, anything that enables the delivery of material to trainees
before or during implementation of the training operation.
 After planning, these resources are to be allocated to training as
needed. This will then translate into a timetable. It would be
convenient in most situations if sub-lists were also made out per
venue.

Venue
 Where training will take place. These are to be outlined in full.
 A list of sessions to be held on each venue. After scheduling, this will
translate into a timetable.

Time
 When training will take place. The outline of working hours and
working days is preset as well as deadlines if imposed.
 Scheduling commences with all the above documents. After
scheduling, timetables are issued and distributed for each instructor,
trainee, venue, and to tool personnel. Preferably, course timetables
and a master timetable could be filed with higher management but
such documentation would not always be feasible.

Course Planning

Course-Topic Relations
 Each course has a list of ordered topics to be covered.
 This is the academic map used to compile course material from a
common pool or reference library.
 It may also be cross-referenced to the skills covered by each topic. As
such, it is also used in assessment to suggest the knowledge base to
be queried in exams, quizzes, etc.

Course-Session Relations
 Each course has a list of ordered sessions to be attended. A session
may include more than one topic or a topic be repeated in more than
one session.
 This is the practical map used by scheduling. It must therefore
mention the time constraints discussed above.

Page 10 of 24
Corporate IT Training www.kemetex.com

Course-Skill Relations
 Each course has a list of unordered skills to be grasped or mastered
by trainees. These may be grouped into essential (the trainee does not
pass if skill is not attained) or complimentary (the trainee is to be
merited, rewarded, or placed according to these skills but not said to
have failed).
 This is the assessment map used to compile assessment material such
as exams, quizzes, competitions, etc. It is also used for placement as
each set of skills may be valuable in a certain position in the body.

Session Interdependencies
 When planning, a certain session may need to be placed after another
one (or more). This type of dependency should be respected across
courses. It is thus vital to be documented separately.
 This is a crucial aid to proper scheduling in an average-complexity
operation. Very simple and very complex operations may dictate that
this information be either included with other documentation (as
would be the case when only two courses are being scheduled) or
divided into several such documents (as would be the case with a
large University).

Session Planning

Resource Conflicts
 No resource (instructor, trainee, tool, venue, etc) should be allocated
with any overlap between instances.
 For example: If two sessions need a Lab equipped with telesales
hardware and software, and only one such lab exists; they should be
planned to NOT coincide and to leave adequate time in between for
check-out and check-in as well as Lab maintenance if necessary.

Topic/Skill Conflicts
 A topic may cover more than one skill. The same skill may be
covered in more than one topic. To cover all needed skills within one
course, a suitable set of topics is needed. Some overlap may thus be
allowed. The same skill may be covered twice in a course since it is
included in more than one topic.
 On the other hand, topics should not be included that contain only a
duplicate of already-covered skills.
 For example: If topics A, B, and C are defined as follows:
Topic A: covers skills 1 and 2; Topic B: covers skills 3 and 4;
Topic C: covers skills 1 and 3
 So to cover skills 1 through 4, only topics A and B need to be
planned in this course context.

Page 11 of 24
Corporate IT Training www.kemetex.com

Adequate Time Distribution

 If the available time frame for course completion is two weeks, and a
total of four sessions are needed for training, these can be distributed
as: 2 on the first week and 2 on the second, or 3 on 1 week with the
remaining session on the second. It would not seem prudent to cram
all 4 sessions in the first week.
 Another consideration is the daily hours’ quota: trainees should not
be expected to attend six hours on two days and then nothing for the
following 3 days. It would be more profitable to space trainees in
rotation over the sessions of the day; this way, every trainee gets to
work (on each day) the same number of hours as everyone else.
 For example: Lets say trainee groups A and B need to cover each six
hours over two days.
o We will let group A start on day 1 for 3 hours, followed by 3
hours for group B.
o The following day, we repeat the rotation. Thus, we fulfill all
defined time constraints and satisfy total time quotas.
 This of course, is governed by time constraints mentioned earlier.

Adequate Time Allocation

 This was covered earlier. At the end of the planning phase, all time
constraints and potential conflicts need to be checked. Finally, the
time allocated to each course, session, topic, skill, instructor, and
trainee need to be checked against the required quotas.

Assessment Timing
 Assessment should be given the defined lead time for each trainee or
group of trainees.
 It should also allow sufficient time for implementation, review,
publication, and decision-making.
 Following that, a contingency plan may need to be laid in case
satisfaction criteria are not met. This usually entails a whole new plan
dedicated to those who did not obtain ‘pass’ status on final
assessment.
 Another caveat is that the whole training operation may prove
unsatisfactory on evaluation. In this case, management may have to
decide upon a new course of action:
1) Re-implement the whole training operation with a different set of
targets/skills/instructors/trainees, OR
2) Accept less-than-satisfactory results to define a new ‘pass’ status for
trainees. This entails deferring re-implementation to another work
cycle in the future. In this case, which is usually what happens after
large operations, full documentation of all phases are kept for use in
the future.

Page 12 of 24
Corporate IT Training www.kemetex.com

Implementation

Preparation
 All defined constraints are used as a checklist on the following levels
(per course, per instructor, per venue, per tool, and per trainee). Each
of the latter is an element in the operation.
 Once an element in the operation is found to fulfill the checklist, that
element is handed his/her/its timetable and informed of expected
duties.
 Instructors and trainees are expected to prepare by reading course
material, doing any necessary research or study, answering a pretest
assessment quiz, presenting a report, or compiling a specific set of
data.
 Material, venues, and tools (non-human resources) are examined by a
party other than those responsible. They must satisfy criteria of
usability, reliability, and acceptable quality. This is one of the hardest
tasks known to man. An abundance of research is available on this
problem.
 In view of the difficulty of assessing material and tools for usability
and quality in education, we will only discuss applicable criteria in
corporate IT training:

Availability:
 These elements must be procured, unpacked, and ready to use.

Functionality:
 The compliment of functions built into the element must fulfill the
requirements of instructing trainees on all related skills.

Reliability:
 Reproducible results are necessary. A tool must produce foreseen
results on repeated use.

Stability:
 This is defined as persistence of adequate functionality in the event of
misuse.

Page 13 of 24
Corporate IT Training www.kemetex.com

Suitability:
 This applies to materials. They must cover all necessary topics and
skills to the extent required by an instructor and not include an
unreasonable context not applicable to corporate environment. The
latter needs an example: Teaching basic pharmacology to medical
representatives cannot be done from Pharmacopeias containing too
much irrelevant material. More suitable textbooks or monographs
need to be found.

Accuracy:
 In fast-changing fields, material must be as up-to-date and valid as
possible. For example: A course on basic computer use cannot be
expected to train your staff efficiently if based on Windows 3.11
textbooks while your staff will be using Windows XP. While the
same concepts do apply, and your staff will not be required to use the
latest features of Windows, the difference in terminology will make
your trainees lose grasp of most skills when put to work.

Delivery
 In this document we will only mention some guidelines for delivery
of Corporate IT Training. The research on teaching methodology
itself is extensive.

Instructor-Trainee Interaction:
 An instructor must be allowed to direct and accept questions to and
from trainees. This keeps trainees alert and paying attention as well
as sharing potential problems or misconceptions with the rest of the
class.
 Sometimes it would be prudent to limit questions to a certain portion
of the time allotted to the session but only in large groups.

Trainee Feedback:
 An instructor must have some means of recording/documenting
feedback from trainees either on the spot or immediately after a
session. This allows:
1) an understanding to develop between an instructor and his/her group
of trainees,
2) an assessment of the trainee and a self-evaluation of the instructor,
3) an indicator that some skill needs more explaining or discussion,
4) a suggestion to be made on how to improve corporate work or avoid
potential problems.

Page 14 of 24
Corporate IT Training www.kemetex.com

Peer Interaction:
 Allowing a trainee to assist another has a good and bad side to it. In
corporate IT training it is almost always unwise to allow this:
1) It makes some trainees feel inadequate when they need assistance
from another classmate.
2) It can make some trainees feel overconfident.
 A nice way around this is to have a trainee show everyone else what
he/she did to perform a certain task or assignment. This shows a skill
in a way different from that of the instructor and does not indicate to
other trainees that they are inferior.
 Another exercise would be to assign two trainees to one job. Trainee
A does the first part which is then handed over to trainee B to review
or improve. Then the situation is switched and results compared. This
method is particularly useful when instructing on semi-creative skills
such as word processing or desktop publishing.

Pop Quiz Methods:

 Trainees are always intimidated by exams. However, fear is not
alleviated by knowing that a certain quiz will not score towards the
final result.
 In corporate IT training, a preferable method would be to use
anonymous quiz papers a few times. The instructor calls a pop quiz,
distributes quiz papers or a question/task/assignment to all trainees. A
trainee them answers and keeps the result to him/herself. When the
answer or assignment result is announced, each trainee sees how well
he did on the quiz. Papers are anonymous. The instructor then
collects answers for assessment of the entire group without being able
to score individual trainees.
 Once this is done, fear is slightly overcome. A formal quiz may then
be attempted midway during the course. Scores are given to each
answer/task/assignment and the instructor knows how well each
trainee performed. These should not be made public but can be
discussed with certain trainees at a convenient time.

Page 15 of 24
Corporate IT Training www.kemetex.com

Lab Exercises
 For trainees to get a grasp of skills, preplanned exercises need to
staged and documented for their first encounter with an IT tool or
method.
 For example: A trainee may be required to set an NTFS quota on his
hard drive and then attempt to outstrip his/her quota. Failure will
result. Such a simple example must have the following criteria to be
an adequate lab exercise:
1) It must be available to all trainees to conduct.
2) It must give the same result to all trainees (without major variation).
3) It must be fully reversible and leave no permanent damage even if
badly conducted.
 This leaves trainees with a sense of achievement, a feel of what its
like to perform the task, and a preparedness for results.
 It also shows trainees what to expect if the system works normally,
and thus they are able to realize abnormal behavior in real-life
situations.

Real-Life Situations
 For trainees to feel motivated and get a sense of the importance of
any theoretical training delivered beforehand, practical situations are
necessary.
 We define Real-Life Situations as those which are likely to recur in
the workplace and those that are not specifically planned for simple
completion. In the latter sense, they are different from Lab Exercises.
 The importance of these is threefold:
1) Motivation and involvement of trainees,
2) Obtaining feedback from trainees on any problems they might have,
3) Testing the hardware, software, or real-life tools or methodology for
pitfalls that might need to be remedied. It can easily be considered a
final field-test for new corporate software for example, before it can
be deemed workplace-worthy.

Unattended Practice
 Leaving trainees unattended is a good step towards final skill
mastering. Absence of a guiding hand puts strain on each trainee to
perform the tasks assigned to him/her without supervision. It also
alleviates the fear of being left alone in the workplace.
 This should ONLY be performed once the instructor feels that a
trainee or a group of trainees has attained sufficient grasp of the skill
at hand. Leaving trainees unattended before this is an exercise in
futility.

Page 16 of 24
Corporate IT Training www.kemetex.com

Assessment
 Assessment is by asking questions and checking for responses.
 In the context of an entire operation, a question pool is built out of all
possible components. Questions are linked to certain skills.
 On building an examination method for a certain course, the topics
covered are listed, the skills to be tested are derived, and questions
are selected from the pool to satisfy the spectrum (or most of the
spectrum) of skills covered.

Type of Questions
 In the current setting, corporate IT training, the following question
types have been tested and perform well:
 TFDK : True / False / Don’t Know Responses to Statements.
 MCQ : One answer from a set is allowed as a Response (A, B, etc.)
 EMQ : Extended Multiple Choice Questions.
o Popular in medical and vocational examinations, The EMQ
format consists of a theme with a number of related questions.
As the theme develops into a more complete picture, a
different kind of response may be needed. This shows how a
trainee will respond in the face of a problem or situation with
limited information and what use will be made of information
as it becomes available.
 In both MCQ and EMQ formats, the context of the question may be
more important than the alternatives to choose from. It is
understanding a situation, more than remembering actions to take,
that is being assessed.
 CEQ : Cause-Effect Questions: A statement is made along with
another statement that seems to be related.
o Response is ‘A’ if both statements are true and one causes the
other.
o Response is ‘B’ if both statements are true but are not related.
o Response is ‘C’ if the first is true and the other false.
o Response is ‘D’ if the first is false and the other is true.
o Response is ‘E’ if both statements are false.

Scope of Questions
 Questions test the trainees grasp or mastering of a skill. As such a
question can apply to any course containing or requiring that skill.
Questions assigned to named skills are said to be Skill-oriented.
 Questions may test ability to understand a concept or remember
stated rules. This could be listed in a topic but not a skill. If some
topics cover data that are not skill-based, these can have questions
assigned to them as well. Such questions are then Topic-oriented.

Page 17 of 24
Corporate IT Training www.kemetex.com

Difficulty of Questions
 Linguistic difficulty should NEVER be an issue. All questions must
be phrased easily and legibly. Difficulty is based on whether the
underlying skill is a basic one or an advanced one. This is not used
for scoring but may be used for ordering trainees later on.
 If 9 trainees out of 10 pass the course successfully. Difficulty of
questions they missed could be used as a guide to their relative level
of competency.

Weighing of Questions
 Questions are NOT weighted by difficulty but rather by the relevance
of the skill to the assessment at hand. A very basic skill weighs more
than a complex skill that is seldom used even though the latter is
more complex to master.
 Weighing is designed to filter those who pass from those who fail.
When two trainees obtain the same mark, it is because they have at
least most basic skills covered. Higher skills may differ in difficulty
but not in weight. Difficulty is then used to order out the top
candidates for future training or promotion, etc.

Assessment By Assignments
 When testing IT skills, nothing beats a real-life simulation
assignment.
 Examples include:
1) Handling 3 incoming calls from angry customers simultaneously.
2) Quickly writing and publishing a report from the database on
demand.
3) Troubleshooting a standard system with a missing device driver.
 Whatever assignment may be required it must be similar to, or
closely related to, a real-life situation set forth during training.
 Scoring for assessment is based on:
A. Completion of the task.
B. If completed, time of completion.
C. If within acceptable time, accordance with acceptable standards and
protocols.
D. If in accordance with all requirements, the quality of the results.
 The fact that assessment follows a structured method and no score is
awarded for a criterion unless the previous criteria are met, gives this
method its name: OSE.

Page 18 of 24
Corporate IT Training www.kemetex.com

Ambiguity of Questions
 There should be NO ambiguity in question statements or available
responses. Ambiguity arising from a question’s language or syntax
will cause it to be disqualified from the final assessment, weakening
the scoring system.

Ambiguity of Responses
 This should only arise in Viva interviews or free notes questions.
These are not encouraged and will not be discussed further.

Creative Responses
 This should only arise in Viva interviews or free notes questions.
These are not encouraged and will not be discussed further.
 Creative responses to assignments are dealt with on a peer
comparison basis. Is the creative response superior to, or inadequate,
compared to that of peers taking the same assignment. Failing to find
comparison, does creativity distract from the time given to the
assignment or hinder the full completion of the latter. If not, human
judgment is left to decide on the score to give to that response.

Gaussian (Normal) Distribution of Assessment Results

 Results of both Pre-Training and Post-Training Assessments should
fall into a near-Gaussian distribution. This tends to be notable in
larger sample sizes but should be identifiable in samples as small as
fifty candidates tested adequately.
 If the results do not match a near-Gaussian distribution, probably the
defect lies in the difficulty of questions; whether too difficult or far
too easy. Either way, a refinement of scoring based on difficulty or a
retake of the examination may be indicated.

Standardization Questions
 These are questions with a known correct answer rate derived form a
larger, pre-tested, and acclaimed pool. If your trainees answer such a
question with the same correct answer level, their results need no
correction. If the correct answer percentage is much higher than
expected, results are shifted down to set this to average. This will
geometrically correct the Gaussian distribution of results.
 If your trainees fail to answer such a question with the defined correct
answer level, compensate for this by shifting up their scores.
 Such questions are compiled only for large operations and cannot be
obtained for every conceivable operation. Some large corporations or
government agencies reuse their questions over periods and thus keep
their question bank highly secret. If your operation is large enough,
setting up your own standardization questions would be a good idea.

Page 19 of 24
Corporate IT Training www.kemetex.com

Elimination of Sources of Potential Errors

 Potential errors may arise from giving undue weight or scores to
skills not attributable to training or giving less than due (including
zero) weight to some skills amenable to, and covered by, the training
operation.
 If a skill has not tested, another examination may be indicated.
However, if some questions are totally answered correctly or totally
answered falsely (or left blank/DK) these can be omitted safely.
 In practice, this is usually hard to find. So the following method may
be useful:
 Eliminate the weight of the question with the highest correct answer
rate and that with the lowest correct answer rate.

Page 20 of 24
Corporate IT Training www.kemetex.com

Evaluation

Data Collection

Instructor Questionnaires
 A questionnaire is prepared with simple yes/no or multiple choice
questions aiming at evaluating the training experience. Blank spaces
are left to allow the instructor to make suggestions, complaints, etc.
 Instructors are required to fill in the questionnaire answering to their
satisfaction of course material, adequacy of venue and tools,
cooperation of trainees, adequacy of time constraints, etc.
 These are filed anonymously and should be kept for later review. If
instructor names are to be known, the instructor should be made
aware of this.
 However, under no circumstance should an instructor be
reprimanded, rewarded, or otherwise held responsible for any
statements or replies made herein. This violates the idea of feedback
and may render future evaluation attempts futile.
 Examples for evaluation questions are:
1) Did you have enough preparation material to read? (yes – no)
2) If so, was it delivered on time? [select one] (on time – too late)
3) Were there any mistakes in the material? (many – few – none)
If so, please enumerate some mistakes in the text box below.
4) Was the material you were given [select one]: (easy – difficult – easy
but length – easy but incomplete)

Trainee Questionnaires
 A questionnaire is prepared with simple yes/no or multiple choice
questions aiming at evaluating the training experience. Blank spaces
are left to allow the trainee to make suggestions, complaints, etc.
 Trainees are required to fill in the questionnaire answering to their
satisfaction of course material, adequacy of venue and tools,
cooperation and competency of instructors, adequacy of time
constraints, etc.
 These are filed anonymously and should be kept for later review. If
trainee names are to be known, the trainee should be made aware of
this.
 However, under no circumstance should a trainee be reprimanded,
rewarded, or otherwise held responsible for any statements or replies
made herein. This violates the idea of feedback and may render future
evaluation attempts futile.
 Examples for evaluation questions are:
1) Did you have enough preparation material to read? (yes – no)
2) Was the material you were given [select one]: (easy – difficult – easy
but length – easy but incomplete)

Page 21 of 24
Corporate IT Training www.kemetex.com

3) Did the software you were using [select any number]: (frequently
hang – give unpredictable responses – take too long to run)
 For the following list of instructors [table below], rate each on a scale
of 1 to 10 in terms of [place numbers in columns A to C for each
instructor]:
A. Being able to answer my questions
B. Being able to teach me
C. Being considerate to my problems
 As you can see, some questions apply to both instructors and trainees.
Others are specific. Some questions rely on others. Some require
tables to be filled. Some require text boxes to encourage free writing.
 The nature of such questionnaires and the later analysis of these
implies a computer system. In our experience, a web-based system is
ideal. Of course, the only problem here is to guarantee some privacy
and security and make sure that every instructor/trainee get only one
vote in each question, even if he/she answers the questionnaire more
than once.

Skill Achievement
 This is done through the comparison of pre-training and post-training
assessments.
 Trainees are assessed before embarking on training. There are two
ways of keeping track of this.
1) Each trainee is given a score and this is filed under his/her name but
he/she is not held responsible for it. This scenario is good enough if
trainees are to be enrolled in courses only if they do not pass
preliminary assessment.
2) If all trainees are to be included in a course, regardless of their score,
an anonymous system is used. Each trainee is given a secret code or
number on pre-testing. The same number is to be used on post-
testing. All such codes/numbers are to be kept secret at all times,
known to nobody except the trainee.
 If Pop Quiz methods are used as described above, these are NOT to
be considered pre-test assessments.
 Final (Post-training) assessment is done as standard and are either
matched with pre-training assessment for each trainee (named
scenario) or their overall score (best, average, and worst) with
percentiles are compared to those obtained via pre-training
assessment (anonymous scenario).
 Trainees can finally be asked to divulge their secret codes/numbers so
that each can obtain his/her final mark. Now that evaluators know
which anonymous pre-training paper is linked to which post-training
assessment paper, a further analysis is due:
 The relative change of each trainee’s skill levels can be calculated
and another chart be drawn (best, average, worst) with percentiles.
The latter does not assess trainees but rather evaluates the training
process.

Page 22 of 24
Corporate IT Training www.kemetex.com

Statistical Significance of Assessment Shifts

 Using T test of significance (or similar), the difference between
average scores obtained Post-Training should be significantly better
than those obtained for Pre-Training Assessment of the same trainees
for the same course.

Target/Objective Achievement
 Higher management may add a final step to data collection by setting
relative scores to skills and targets, defining the relative importance
of a skill in reaching a target and the relative importance of a target in
the achievement of a final objective.
 These scores are specific to a skill-target and a target-objective pair.
The sum of all scores given to skills in the same target is summed,
equated with 100% and the percentage of target satisfaction
calculated arithmetically from the achievement of individual skills
weighted by their scores.
 The sum of all scores given to targets in the same objective is
summed, equated with 100% and the percentage of objective
satisfaction calculated arithmetically from the achievement of
individual targets weighted by their scores.
 A further step is performing a parallel evaluation of corporate work
after a period of post-training activity. If an improvement in objective
achievement is felt and given a percentage (according to business
rules), this can then be compared with projected numbers calculated
above.
 This last step is done to evaluate the evaluation process itself or
discover hidden targets that may have been overlooked. It requires a
feasible quantitative evaluation of business performance. Since this is
a function of other departments, a margin of error (deviation from
projected numbers) can safely be accepted in most environments.

External Reviewers
 An external party may be called upon to conduct trainee assessment
or post-training evaluation. These parties report to higher
management in statistics.
 The inner workings of such parties is their own concern. This
however, does not mean that you need not handle your own
evaluation as well.

Page 23 of 24
Corporate IT Training www.kemetex.com

Decision Support
 Management must decide on a plan of action pursuant to evaluation
of a corporate IT training operation.

Scenario 1:
 In case targets and/or objectives were not met in spite of a
statistically significant difference in assessment results.
 This is probably due to miscalculation of targets/objectives as related
to skills. The usual course of action will be to carry on with business
workflow and judge the adequacy of training after a while.
 This is the most common scenario since Management tends to be
overzealous in expectations of training. It is not unusual to find
managers requiring 90% reduction in the cost of outgoing phone calls
made in response to support issues. Clearly mastering skills of fast
telephone communication and etiquette will not reach that level of
accomplishment in real-life.

Scenario 2:
 In case no significant difference is noted after Training, action should
be:
1) to discover drawbacks in current operation and stage a repeat training
to correct any defects.
2) to put the staff to a real-life test and then judge on adequacy of
training. This is the course of action most commonly taken. It
frequently saves resources, avoids disappointment, and can discover
flaws in assessment or evaluation.
 The idea is: maybe the lack of improvement was due to faulty
assessment or misinterpretation of assessment results, not actually
faulty training.

Scenario 3:
 In case all targets and objectives were met, all statistics were
significantly positive, and everything seemed to be perfect.
 This is the worst scenario. No training operation is complete and
error-free. Lack of recognizable defects will need ongoing scrutiny. A
decision is made to accept the results and delegate trainees to
business workflow. Then periodical or random measurements are
taken, indexing performed, or other data collected on the per-skill and
per-target performance of trainees.
 If real-life work experience proves a significant improvement on the
pre-training era, the results are finally accepted as true and the
operation is deemed a success.

Page 24 of 24