ASSESSING STUDENT MARCH 21, 2017

LEARNING
ONE HEALTH TEACHING, RESEARCH AND
COMMUNITY OUTREACH WORKSHOP

ONE HEALTH TEACHING CLASS

dr. Diantha Soemantri, M.Med.Ed, Ph.D

SCOPE
• Definition of assessment
• Basic principles of student assessment
• Key aspects of assessment (in general and
also related to healthcare context)
• Development of assessment blueprinting
LEARNING OBJECTIVES

• Participants will be able to explain the

definition of assessment and the basic
principles of student assessment
• Participants will be able to explain the key
aspects related to student assessment
especially in healthcare context
• Participants will be able to develop an
assessment blueprint
DEFINITION AND BASIC PRINCIPLES OF ASSESSMENT
 WHAT IS ASSESSMENT?
According to Case & Swanson (2002)

The purpose of any assessment is to permit

inferences to be drawn concerning the skills of
examinees: inferences that extend beyond the
particular problems (or, equivalently, cases or
test questions) included in the exam to the
larger domain from which the cases (or
questions) are sampled

Assessment = Systematic sampling

Case S & Swanson D (2002). Constructing written test questions for the basic and clinical sciences, 3 rd ed. Philadelphia: National Board of Medical Examiners
WHAT IS ASSESSMENT?
A systematic basis for making inferences
about the learning and development of
students

The process of defining, selecting,

collecting, analyzing, interpreting, and
using information to increase students’
learning and development

Section 6: Assessment in Dent JA & Harden RM (eds), A Practical Guide for Medical Teachers, 3nd ed (2009). Elsevier, Edinburgh, pp.
303-357
Shumway JM & Harden RM (2003). The assessment of learning outcomes for the competent and reflective physician. Med Teach
25(6):569-584
st
PRINCIPLES OF There are 2 groups of students, each consists
ASSESSMENT of 60 students.

They are given some readings in the course

they attend

Group A Group B
Read 55 25
Not read 5 35
 ASSESSMENT

“
DRIVES
LEARNING......
changing the examination system without changing the curriculum had
a much profound impact upon the nature of learning, than changing the
curriculum without altering the examination system

G.E. MILLER
PRINCIPLES OF Work-based assessment,
ASSESSMENT eg. Portfolio, logbook, MSF,
Mini-CEX

Clinical and practical

assessment, eg. OSCE, long
case

Written
assessment, eg.
MCQ, EMI, essay

Written
assessment
PRINCIPLES OF
ASSESSMENT
Competence (Sadler, 2013): the capability to
orchestrate knowledge and skill independently, in a
range of contexts, on demand and to a high level of
proficiency
Sadler, D. R. (2013). Making competent judgments of competence. In Blömeke, S., Zlatkin-Troitschanskaia, O., Kuhn, C. & Fege, J. (Ed
Modeling and measuring competencies in higher education: Tasks and challenges. Rotterdam: Sense Publishers. (Ch. 1, 13-27).

Experts

Proficient Novice  Expert

professionals (Dreyfus, 2004)
Competent
professionals

Learning professionals

Unskilled or novice
PRINCIPLES OF
ASSESSMENT • Constructive alignment: the relevance
between assessment and learning
objectives

• No one method can assess all

• Adequate sampling  ‘case specificity’

• Multiple and systematic sampling

(across content, cases, skills, domains,
examiners, context)

• Holistic assessment
“
No single assessment method can provide ALL the data required for
judgment of ANYTHING so complex as the delivery of professional services
by a successful physician

G.E. MILLER
KEY ASPECTS OF ASSESSMENT
 • cognitive
WHAT NEEDS TO BE CONSIDERED WHEN Learning
• affective
domains
DEVELOPING AN ASSESSMENT? • psychomotor

Purpose
of • summative
assessmen • formative
t

• written examination
Assessmen • practical
t (performance-based)
instrument examination
s • workplace based
assessment

Standard • norm-referenced
setting • criterion-referenced
COGNITIVE DOMAIN

https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/
AFFECTIVE DOMAIN
(BLOOM’S)

Acts consistently due to an internal belief, can

articulate a philosophy or world-view, can break
Characterizing down complex situations and respond accordingly
based on values, develops and lives by a code of
personal behavior
Values become systematic, can compare and
Organizing contrast values and choices, begins to order and
prioritize values, chooses to commit to certain
values and behaviors
Valuing Motivated to invest. Chooses to behave in a certain
way frequently. Begins to identify with a behavior
and commit to it

Responding Willingly participating, obedient, volunteers, finds

satisfaction in participating, ready to respond

Receiving/attending Willing to be aware of the setting/situation, gives

attention by choice, open to the experience
PSYCHOMOTOR
DOMAIN (HARROW’S)

Use effective body language, such as gestures and

Non-discursive communication
facial expressions

Skilled movements Advanced learner movements as one would find in

sports or acting

Physical abilities (fitness) Stamina that must be developed for further

development such as strength and agility

Perceptual abilities Response to stimuli such as visual and auditory

Fundamental movements Basic movements such as walking or grasping

Reflex movements Reactions that are not learned, such as an

involuntary reaction
FORMATIVE VS
SUMMATIVE
ASSESSMENT Formative assessment is usually
contrasted with summative
assessment in the following
way: when the cook tastes the
soup, that’s formative
evaluation; when the guests
taste the soup, that’s summative
evaluation

Bob Stake, 1991

STANDARD SETTING
Student performance is assessed based
on the performance of other students
(after the examination has been
conducted)

Classifying candidates
Choose candidates based on the
predetermined quota
Knowing how far someone has
learned, compared to the others
STANDARD SETTING
Student performance is assessed against
(a) predetermined performance
criterion/a (before the examination
takes place)

Evaluating what has been learnt

Diagnosing student’s strengths and
weaknesses
Certification of competence
Mastery testing
DEVELOPING AN
ASSESSMENT BLUEPRINT
WHAT IS ASSESSMENT
BLUEPRINTING?
It is the process of mapping the
examination content against the
learning objectives/outcomes

Constructive
alignment

Amin et al (2006). Key concepts in assessment in Practical guide for medical

student assessment, 1st ed. World Scientific Publishing, Singapore
ASSESSMENT BLUEPRINT
ASSESSMENT BLUEPRINT

BENTUK INSTRUMEN FREKUENSI BOBOT (%)

Ujian tulis I Esai 1 30
Ujian tulis II Esai 1 35
Praktek Lembar penilaian 1 10
Makalah Lembar penilaian 1 25
Total 100

BENTUK INSTRUMEN NBL (setelah skor dikonversi

ke nilai)
Ujian tulis I Esai 70
Ujian tulis II Esai 70
Praktek Lembar penilaian 75
Makalah Lembar penilaian 70
ASSESSMENT BLUEPRINT
Kompetensi/subkompetensi Jenis tes Jumlah Ranah Bobot
soal
Mendesain instrumen evaluasi tertulis (written Esai 3 K5 20%
assessment) berupa MCQ, EMI dan essay, sesuai
sasaran pemelajaran yang ditetapkan dan kaidah
penyusunan masing-masing instrumen

Mendesain instrumen evaluasi untuk menilai Esai 2 K5 20%

keterampilan mahasiswa (performance-based
assessment) berupa OSCE dan Mini-CEX, sesuai sasaran
pemelajaran yang ditetapkan dan kaidah penyusunan
masing-masing instrumen

Mendesain instrumen evaluasi untuk menilai afektif Esai 1 K5 10%

mahasiswa, sesuai sasaran pemelajaran yang
ditetapkan dan kaidah penyusunan instrumen evaluasi
afektif

Mendesain instrumen evaluasi untuk penilaian Esai 3 K5 15%

kemampuan peserta didik di tempat kerja (work-based
assessment) berupa logbook dan portfolio, sesuai
sasaran pemelajaran yang ditetapkan dan kaidah
penyusunan masing-masing instrumen

Mendemonstrasikan cara pemberian umpan balik Praktek 1 P4, A5 10%

sesuai prinsip yang tepat (lembar
penilaian)
Memilih metode standard setting yang paling tepat Makalah 1 K6 25%
sesuai instrumen evaluasi yang akan digunakan (lembar
penilaian)
 Performance assessment >>
observation >> checklist/rubric
ASSESSMENT
INSTRUMENTS Workplace-based assessment

1. Logbook
2. Portfolio
3. Multisource feedback (360
degree assessment)
>> rubric
Rubric: descriptive and holistic rubric
Written examination

1. Multiple choice questions How to make a rubric?

2. Essay
3. Extended matching questions 1. Find models of rubric
4. Short answer questions 2. Determine the dimensions
3. Determine the scale
Evaluation of 4. Formulate the description for each
report/paper/scientific work >> level of achievement (cut off
form/checklist/rubric point/criteria) (in
Buku descriptive
panduan rubric)
KBK Dikti, 2008
EXAMPLES OF RUBRIC

Buku panduan KBK Dikti, 2008

THANK YOU