Training Course For Instructor

IMO MODEL COURSE 6.09

Evaluating Students
By : Arizal Hendriawan, M.Sc

Objective
After completion this course the
participants will be able to:
Analyse the uses of evaluation
Identify measures of performance
correctly
Select appropriate evaluation
methods

Outlines
Evaluation, Assessment &
Measurement
Feedback to students and
Instructor
Methods of Measurement
Assessing & Grading the
students
Reliability, Validity, Practicability

Introduction :

Course Design Model

Evaluation, Assessment & Measurement

Measurement : involves collecting information or
data from students.
Ex: giving a written test

Assessment: analysis of the data, interpreting the

measurement data in some way
Ex: giving 1 student 50, other 75
giving 1 student fail, other pass

Evaluation: involves in making judgement

Ex: considering the assessment result and deciding how
well a student has performed

Evaluation, Assessment & Measurement

Main purposes of
assessment & evaluation:
Motivation
assisting in making
administrative judgements
Providing feed back to the
students and instructors

Must be clear about the

objectives of such assessment

Main Purpose
Motivation
Can be an incentive for the
students to apply themselves
more diligently and more
intensively to their studies
Student motivated to try
harder: Risk being excluded
from the course or having
repeat a year

Main Purpose
Grading
Provide rewards in many
diffferent way
Promotion
Better pay
Admission to higher form of
study
Lead to pass a course unit

Main Purpose
Feed Back to students and instructors
Preparative evaluation
Formative evalution
Summative evaluation

Preparative Evaluation
Getting data of students prior
knowledge
Provide appropriate starting points for
instruction
Guide for selection learning activities
Enable to allocate students to the
most appropriate groups in order to
enhance their learning

Formative Evaluation
Takes place during the course of
instruction
Uses to assist the teaching learning
process by diagnosing any difficulties
that students may be having
Actions can be taken to remedy the
problem

Summative Evaluation
Occurs at the conclusion of the
course
Provide information students
achievement to the standard set
Provide information about
effectiveness the course and
instructional techniques employed
Often used for certification

Methods of Measurement
Need to select the method most
useful to the situation
Selection the methods depend on
the aims objectives, and contents of
elements
Consider criteria : reliability, validity, &
practicality
The methods may be objective or
subjective

Methods of Measurement
Objective Type
The answer are known in advance
Types:
Select the answer type
True false
Multiple choice
Matching
Short answer supply type

Major advantages:
Easy or marking
Consistency scoring by different markers

 True False Question

Statements must be selected that are strictly
true or false without qualification
Have limited usefullness
Ex:
The displacement of a ship may be defined as the mass of
water displaced by the immersed hull section ( T / F )

 Multiple Choice
Student is required to select the correct answer from a
number of possibilities that are provided
Have advantage can be quickly and easily marked
Can be very effective but good questions are time
consuming in their preparation
Very nature makes them subject to guessing
THE DISTRACTORS MUST BE WRONG ANSWER, BUT THEY
SHOULD APPEAR TO BE POSSIBLE ANSWER TO THOSE WHO
DO NOT KNOW THEIR SUBJECT
Ex:
A life boat must carry fuel sufficient for :
a. 6 hours steaming at 4 knots
b. 4 hours steaming at 6 knots
c. 24 hours steaming at 6 knots
d. 36 hours steaming at 4 knots

 Matching questions
Required students to match items in one
list against items in a second list
When setting such question, put extra
items in one list to prevent answers being
arrived at by a process of elimination

 Short answer supply type

Requires students to recall or create the correct answer
It helpful to indicate the length of answer required by
use dotted lines or spaces in the layout of the test
Have advantage of being easy and quick to mark
Ex:
Describe the shape, colour,light and top mark of a
safe water mark
......................................................................
......................................................................
......................................................................
......................................................................

 Subjective type
Essay type
Involve interpretation and
judgment by the marker

 Essay or extended answer

Useful to evaluate
a students ability to deal with a complex topic
or problem
To demonstrate a depth of understanding

Take a little time to construct, but require

more time to mark and check
Number of question limited by time
constrain
Tend to be more subjective, more difficult
to score on a consistent or reliable basis
It is necessary to have an agreed marking scheme
among two markers
Use more than one marker, averaging the marks

 Open Book Examination

Mostly to test ability to think, analyse, or
criticise rather than memory
Students are allowed to take their text or
similar material into the examination
Students are asked their knowledge in some
way
Aprropriateness depends upon the nature
of the topic
Has disadvage may take to much time
searching through their book rather than
answering the question

 Oral Examination
Students are asked questions orally and
give their response in like manner
Be used to assess
a students fluency and comprehension
Personal characteristics and attitudes
Ability to think and solve the problem

Very flexible and powerful tools to probe

deeply into students capabilities
Problems:
Students need to be relaxed but must be
presented with challenging questions
Outcomes can be very subjective
Examiners must be able to separate the
substance from the manner

 Other forms of measurement

Keeping of workbook
Diaries
Task
Project work
Exhibition and report

Choice of measurement method:

Objectives of the course
Purpose of learning process
What is considered to be most suitable for
measuring students outcomes or
performance

Weaknesses & Constrains

Halo Effect
Contrast Effect
Focus in progress, instead on achievement
Unclear assesment scheme & standard
Discrimination / Unfair

Assessing & Grading

Criterion referenced assessment
Need to be decided a norm or criterion reference
method to be used when making a decision on a
students assessment
Standard of performance required and to be
achieved is decided upon prior the teaching
Types of grading :
Pass, if students meeting the stated criteria
Fail, if students failing to meet the criteria
Often used for competency based assessment

 Norm Referenced Assesment

Students are rated against one another
rather than against an external criterion
Determines whether a student has learned
an element of work better than his/her
peers
Often used for marking extended answer or
essay question
Grading systems type:
Grading by letters of the alphabet
Grading by percentages

Reliability, Validity and Practicability

Reliability
Reliability is the consistency with which the
test measures whatever it does measure
Validity
Test is valid if it measures what is supposed to
measure
Closely relating with objectives and content

Practicability
Suitable and pratically possible to do so
Consider to time constraints

Competency Based Training &

Assessment
Seafarer required to hold a certificate or license
authorizing them to serve in particular capacity
onboard ship
Authority satisfied itself that the holder was competent
to serve safely and effectively
Human error was major cause maritime casualties
The STCW 1978 convention: set minimum global
standard, have weaknesses:
Did not include uniform standard of competence
Did not provide for effective international over sight to verify
compliance
Outdated technical reference that did not address the modern
shipboard system

 STCW 1978 as amended 1995 (

adopted 1 February 1997), coverage,
include:
Uniform standard of training and
certification of mariners
Mandatory rest period
Requirement english proficiency
Basic safety training
Creation of STCW Codes, contains almost
all technical detail.

 Competency
Considered as what knowledge and skills
an employee need to have in order to
apply that knowledge and skill to the
workplace situation
It takes in all aspect of work
performance and includes:
ability to perform individual task
Respond to certain situations
Manage different task

Meeting the STCW competency

standards
STCW Part A.I/6
Each Party shall ensure that:
.1 the training and assessment of seafarers, as required under the
Convention, are administered, supervised and monitored in accordance
with the provisions of section A-I/6 of the STCW Code; and
.2 those responsible for the training and assessment of competence of
seafarers, as required under the Convention, are appropriately qualified
in accordance with the provisions of section A-I/6 of the STCW Code for
the type and level of training of assessment involved

Should be clear learning objectives and training outcomes

(IMO Model course 7.01 7.04 as guidance)

Mandatory standards at STCW Code A tables

 Since evaluation criterion written in

qualitative terms

Assesor need to developed particular

circumstance of the skills being
assessment

Open LPN 1.1.1

Membiji hasil penilaian

Tes Uraian (Subyektif)
Tandai pekerjaan siswa secara anonim
Gunakan metode memeriksa keseluruhan, soal
demi nomor soal
Sebelum mulai memeriksa, kunci jawaban harus
dibuat sejelas mungkin.
Perlu dibuat bobot soal berdasarkan tingkat
kesukarannya.
Contoh rumus biji: jumlah (Biji x Bobot)
Jumlah Bobot

Membiji hasil penilaian Tes Obyektif

Tes Benar salah
Untuk menghindari kemungkinanmenebak, dpt
digunakan rumus pembijian, sbb:
b=BS
b = T 2S 0
b = B + 0/2

Keterangan :
b : biji
T : jumlah pertanyaan dlm tes
B : Jumlah jawaban betul
S : jumlah jawaban salah
0 : jumlah jawaban kosong (tdk dijawab)

Membiji hasil penilaian Tes Obyektif

Tes Multiple Choice
Untuk menghindari kemungkinanmenebak, dpt
digunakan rumus pembijian, sbb:
b=T(P/P1)S-0
b = B S/ P - 1
b = B + 0/P

Keterangan :
b : biji/nilai
T : jumlah pertanyaan dlm tes
B : Jumlah jawaban betul
S : jumlah jawaban salah
0 : jumlah jawaban kosong (tdk dijawab)
P : jumlah kemungkinan jawaban yg disajikan

Menganalisis hasil penilaian Tes

Obyektif
Analisis secara Obyektif
Daya pembeda (discriminating power)
Tingkat Kesukaran (difficulty level)
Efektivitas distractor

Penilaian Acuan Norma (PAN)

Tes untuk menghasilkan tata tingkat
hasil prestasi belajar murid dari tingkat
tertinggi sampai terendah.
Bisa untuk membentuk kelompok
kurang, sedang, baik.
Lebih tepat untuk pembinaan.
Standar relatif, mengacu pada nilai
rata-rata kelompok.

Penilaian Acuan Patokan

(PAP)
Mengacu pada konsep belajar tuntas.
Lebih untuk keputusan batas lulus.
Standar mutlak, mengacu pada
pencapaian TIK.

Thank You

Any Questions
or Comments?