Paper presented at the International Journal of Arts and

Science Conference, Malta, 2014.

Assessment Literacy and School Accountability in the

Context of Education Reform in Malaysia

Othman bin Lebar, Siti Eshah Mokhsein, Ahmad Fuaad Dahalan,

Azali Rahmat, Omar Hisham Baharin
Sultan Idris Education University
Malaysia

Abdul Wahab Muda

Aminuddin Baki Institute
Ministry of Education Malaysia

Abstract

This paper examines the notion of assessment literacy and some of its possible
components and the increased pressure of school to maintain accountability.
Assessingstudentperformance isoneofthemostcritical aspects ofthejobofa
classroomteacher;however,manyteachersdonotfeeladequatelypreparedtoassess
theirstudentsperformance.Inordertomeasureteachersassessmentliteracy,a
survey using the Classroom Assessment Literacy Inventory (CALI), which was
designed to parallel the Standards for Teacher Competence in the Educational
AssessmentofStudents(1990).Specifically,thepurposesofthestudieswere:(1)to
measureanddescribetherelativelevelsofassessmentliteracyforteachers,and(2)to
statistically compare the relative levels of assessment literacy between teachers
accordingtosubjecttaught,teachingexperienceandacademicqualification.Othman
Lebaretal.studywasfocusingonassessmentliteracylevelamongschoolteachers
who teach Physical and Health Education, Music Education and Arts Education,
while Abdul Wahab Muda was focusing on Mathematics teachers in the primary
schools.AsurveyusinginstrumentsadaptedfromMertlersClassroomAssessment
LiteracyInventorywasconductedtoteachersintheStateofPerakandDistrictof
KualaTerengganu,Terengganu.Thefindingsofbothstudiesindicatedthatthelevel
ofassessmentliteracyamongteachersintheprimaryschoolsislow. Thisposeda
questionastowhatextentteachersarecompetenttocarryoutassessmentsthatare
valid and reliable? Pressure for increased school accountability is a distinctive
hallmarkofthepresentperiodofeducationalreforminMalaysia.To develop genuine
accountability for students accountability that enhances their educational
opportunities we need to develop policies that use assessments to guide
educational improvements, rather than to further reduce the amount and quality of
education students receive.
Background

Pressureforincreasedschoolaccountabilityisadistinctivehallmarkofthepresent
period of educational reform in Malaysia. Accountability, as presently defined in
Malaysianeducationalpolicy,includesfourmajorideas:theschoolisthebasicunit
for the delivery of education and hence the primary place where teachers and
administrators are held to account; schools are primarily accountable for student
performance,generallydefinedasmeasuredachievementontestsinbasicacademic
subjects;schoolsitestudentperformanceisevaluatedagainstexternallysetstandards
that define acceptable levels of student achievement; and evaluation of school
performance is typically accompanied by a system of rewards, penalties, and
interventionstrategiestargetedatrewardingsuccessfulschools.

One of the reforms initiated in Malaysian education system is the assessment system.
The Ministry of Education Malaysia has introduced school-based assessment
framework to be implemented in primary and secondary schools beginning 2011. The
assessment framework had shifted its focus from centralized exam to school-based
assessment. The introduction of school-based assessment framework had raised
several conceptual and operational issues that need special attention by the Ministry in
order to ensure the quality of assessment of student learning. School readiness is one
of the issues that need to be addressed in the implementation of the school-based
assessment.

Darling-Hammond (1996) asserted that there is evidence about the critical aspects of
education that matter most for student learning. Key factors include well-prepared
teachers, well-designed and coherent curriculum, skillful instruction that is adapted to
students needs, and personalized learning environments in which students are well-
known by their teachers. Providing these key features of a sound education is a major
foundation of an accountability system. Well-qualified teachers are at the center, as
they are the vehicles for developing a sound curriculum, implementing successful
pedagogies, and designing more personalized schools. Research has found that
effective teachers have strong content knowledge and pedagogical training in the field
in which they teach, as well as an understanding of students and how they learn (for a
review, see Darling-Hammond, 2000).

Shepard (2001) mentioned about classroom assessment should support instruction and
enhance students' learning as advocated by many studies. However according to
McMillan (2001) studies also show that teachers have consistently used a variety of
factors in their assessment practices and consequently make erroneous decisions.
Even more disturbing is that most teachers lack effective assessment knowledge and
skills; that is, when evaluating student academic achievement, teachers exhibited
misconceptions about assessment practices. In short, while many seem to understand
assessment, more seem to misunderstand it instead.

This paper examines the notion of assessment literacy and some of its possible
components and the increased pressure of schools in Malaysia to maintain
accountability.
AssessmentLiteracy

Assessment literacy has been defined by Paterno (2001) as the possession of

knowledge about the basic principles of sound assessment practice, including
terminology,thedevelopmentanduseofassessmentmethodologiesandtechniques,
familiaritywithstandardsofqualityinassessment...andfamiliaritywithalternativeto
traditionalmeasurementsoflearning. Anothersimilardescriptionisprovidedby
Stiggins (1995) when he asserted that assessment literates know the difference
betweensoundandunsoundassessment.Assessmentliterateeducators(regardlessof
whether they are teachers, administrators, or superintendents) enter the realm of
assessmentknowingwhattheyareassessing,whytheyaredoingit,howbesttoassess
the skill/knowledge of interest, how to generate good examples of student
performance,whatcanpotentiallygowrongwiththeassessment,andhowtoprevent
thatfromhappening.Theyarealsoawareofthepotentialnegativeconsequencesof
poor,inaccurateassessment.Inshort,assessmentliteracyisrelatedtothereadinessof
aneducatortodesign,implement,anddiscussassessmentstrategies.

Whyisassessmentliteracyimportantforteachers? Newfields (2006) asserted that

therearethreecompellingreasons.First,assessmentisawidespread(ifnotintrinsic)
featureofmosteducationalsystems.Teachersareestimatedtospendfrom10%50%
oftheirworktimeonassessmentrelatedactivities.Inmanyschools,agoodportionof
thebudgetalsogoesintoformaltesting.Withsomuchtimeandmoneydevotedto
assessment,it'sworthcriticallyunderstandinghowassessmentdecisionsaremade.A
secondreasonassessmentliteracyisessential becauseit'snecessarytounderstand
muchoftheeducationalliterature.Afamiliaritywithbasicstatisticaltermsisneeded
not only to critically read specialized journals, but even many general articles in
academicpublications.Afinalreasonassessmentliteracyisneededisthatitallows
teacherstocommunicatetheirownclassroomresultswithothers.

The American Federation of Teachers, the National Council on Measurement in

Education, and the National Education Association developed seven Standards for
Teacher Competence in Educational Assessment of Students to address concerns
about classroom assessment and delineate teacher assessment literacy. (AFT, NCME,
& NEA, 1990). These standards were intended to guide the preparation of pre-service
and in-service teachers as effective and skilled educators. [1] The standards were
skills and knowledge in: (1) choosing assessment methods appropriate for
instructional decisions; (2) developing assessment methods for such decisions; (3)
administering, scoring, and interpreting results of externally-produced and teacher-
produced assessment methods; (4) using assessment results in making decisions about
individual students, instruction, curriculum development, and school improvement;
(5) developing valid grading procedures using pupil assessments; (6) communicating
assessment results to students, parents, lay audiences, and educators; and (7)
recognizing unethical, illegal, and inappropriate assessment methods and uses of
assessment information.
AssessmentandAccountability

Accountability occurs when policies and practices work to provide good education
and to correct problems as they occur. Accountable systems increase the probability of
high-quality practice, leading to positive outcomes. They reduce the probability of
malpractice or educational harm, and they call attention to problems and needs.
Furthermore, accountability must be two-way: If students are accountable for learning
to certain standards, schools, districts, and states must be accountable for providing
them with the necessary resources for learning.

School systems that are accountable to children will ensure that they have well-
qualified teachers in adequately resourced schools that are designed to support
teachers in providing good instruction. Assessments of learning and other indicators
of school conditions can help evaluate the extent to which educational goals are
accomplished. But the focus of accountability must be kept on what is needed actually
to improve achievement as well as on how progress is to be measured.

Newmann,etal1997identifyacompleteschoolaccountabilitysystemasincluding:
(1)Informationabouttheorganizationsperformance(e.g.,testscores);(2)Standards
forjudgingthequalityordegreeofsuccessoforganizationalperformance(e.g.,a
meanachievementscorehigherthanotherschoolswithcomparabledemographic
characteristics);(3)Significantconsequencestotheorganization(i.e.,rewardsand
sanctions such as bonuses to teachers in the school) for its success or failure in
meeting specified standards; and (4) An agent or constituency that receives
informationonorganizationalperformance,judgestheextenttowhichstandardshave
been met, and distributes rewards and sanctions (e.g., the state department of
instruction).

StudiesonAssessmentLiteracyinMalaysia

IndiscussingteacherassessmentliteracyinMalaysia,tworecentstudiesonthistopic
willbespecificallydiscussed. ThestudiesareconductedbyOthmanLebaretal.,
(2013) and Abdul Wahab Muda (2012). It was the intent of these studies to
investigate the concept of assessment literacy and attempt to measure it as
delineatedbyTheStandardsforTeacherCompetenceintheEducationalAssessment
ofStudents(1990).Specifically,thepurposesofthesestudieswere:(1)tomeasure
and describe the relative levels of assessment literacy for teachers, and (2) to
statistically compare the relative levels of assessment literacy between teachers
accordingtosubjecttaught,teachingexperienceandacademicqualification.

OthmanLebaretal.,studywasfocusingonassessmentliteracylevelamongteachers
whoteachPhysicalandHealthEducation,MusicEducationandArtsEducation(n=
322)whileAbdulWahabMudawasfocusingonMathematicsteachersintheprimary
schools(n=365). AsurveyusinginstrumentsadaptedfromMertlersClassroom
AssessmentLiteracyInventory(2002)wasconductedtoteachersintheStateofPerak
andDistrictofKualaTerengganu,Terengganu.

Thefindingsofbothstudiesgenerallyindicatedthatthelevelofassessmentliteracy
amongteachersislow.InOthmanLebaretalstudy,asshowninTable1andTable
2,majorityoftherespondents(86.6%)achieveatthelowlevel,10.9%achieveatthe
moderatelevelandonly2.5%achieveatthegoodlevel.ANOVAanalysisindicates
nosignificantdifferentinmeanscorebetweenthethreegroupofteachers.

Table 1

Score and Assessment Level According to Subject Taught

____________________________________________________________________
Subject Assessment Literacy Level Total Sig. F
____________________________
2.00 3.00 4.00

Music Ed 1 15 90 106
Phy & Health Ed 2 11 97 110
0.430
Arts Ed 5 9 92 106
Total 8 35 279 322
Note: 1.00 Excellent (Score 17 and above); 2.00 Good (Score 13 - 16);
3.00 Moderate (Score 11 - 15); 4.00 - Weak (Score 9 and below)

Table 2
Mean Score According to Subject Taught

Subject Mean Std Dev Sig. F

Music Ed 40.16 10.03
Phy & Health Ed 38.27 10.50
0.763
Arts Ed 38.86 10.10
Total 39.09 10.90
_____________________________________________________________________
________________________
p>0.05

Table 3 shows the level of assessment literacy according to the sub-

skills which indicated that quite a number of respondents are able to
chooseassessmentmethodsappropriateforinstructionaldecisions (mean=66.39)
and developing assessment methods appropriate for instructional decisions
(mean=43.63). However, skills in administering,scoringandinterpreting
the results; using assessment results when making decisions about individual
students,planningteaching,developingcurriculum,andschoolimprovement; and
developingvalidpupilgradingproceduresthatusepupilassessments is well below
the required competent needed to carry out effective assessment.

Table 3
Respondents Performance According to Sub-skill
Sub-Skill N Mean Std Dev

Choosingassessmentmethodsappropriatefor
Instructionaldecisions. 341 66.39 26.17
Developingassessmentmethodsappropriatefor
instructionaldecisions. 343 43.63
27.29
Administering,scoringandinterpretingtheresults 334 33.43 14.68
Usingassessmentresultswhenmakingdecisions
aboutindividualstudents,planningteaching,
developingcurriculum,andschoolimprovement. 346 19.26 20.17
Developingvalidpupilgradingproceduresthat
usepupilassessments. 347 39.05 21.56

Valid N 322
_____________________________________________________________________
________________
A two-way ANOVA as shown in Table 4 indicated that there is no
significant difference in level of assessment literacy for each sub-
skills among teachers who teach Art Education, Music Education and
Physical and Health Education.

Table 4

Literacy Level According to Sub-skills and Subjects

Taught
_____________________________________________________________________
________________________
Sub-skills Art Ed Phy&Health Ed
Music Ed Sig. F
Choosingassessmentmethods
appropriateforInstructionaldecisions. 64.32 64.78 68.00
0.563
Developingassessmentmethods
Appropriateforinstructionaldecisions. 40.17 44.93 45.94
0.214
Administering,scoringandinterpreting
theresults 31.48 34.27 34.61
0.448
Usingassessmentresultswhenmaking
decisionsaboutindividualstudents,
planningteaching,developing
curriculum,andschoolimprovement. 17.54 19.33 20.94
0.769
Developingvalidpupilgradingprocedures
thatusepupilassessments. 38.03 39.03 40.09
0.231

Valid N 322
p>0.05

Abdul Wahab Muda (2012) has conducted a test to determine whether there is a
significant different in mean score between graduate and non-graduate teachers and
mean score between teachers according to teaching experience. A t-test result in
Table 5 indicated that there is no significant different in mean score between graduate
and non-graduate mathematics teachers. In other words, graduate and non-graduate
teachers level of assessment literacy is of the same level.

Table 5
Level of assessment literacy between graduate and non-graduate teachers

Teacher n mean d.f. t Sig.

status

Graduate 110 7.50 363 0.527 0.299

Non- 255 7.37

graduate
p > 0.05

A one-way ANOVA test result as shown in Table 6 indicated that there is no

significant different in mean score between teachers according to teaching experience,
(F (6, 358) = 0.0802), p > 0.05). It means that teachers level of assessment literacy
across teaching experience is of the same level.

Table 6
Level of Assessment Literacy According to Teaching Experience

Source Sum Squared Mean d.f. F Sig.

Squared
 Between 22.984 3.831 6 0.802 0.285
Group

Within 1709.191 4.774 358

Group

Total 1732.175 8.605 364

p > 0.05

Discussion

ResultsofstudyconductedbyOthmanetal.,andWahabMudaclearlyindicatedthat
generally teachers level of assessment literacy is low. This result support what
Stiggins(2001)statementswhenhementionedthatweareseeingunacceptablylow
levels of assessment literacy among practicing teachers and administrators in our
schools. He continues by stating that this assessment illiteracy has resulted in
inaccurateassessmentofstudents,causingthemtofailtoreachtheirfullpotential.In
Malaysiancontext,thisproblemisquiteunderstandablesinceMalaysiaispracticinga
centralised education system and standardization is given much emphasis in the
implementation of the curriculum and assessment. Teachers basically act as an
implementerofthenationalcurriculum. Assessmentinschoolsisemphasizingon
standardizedachievementtestandthetestsareconstructedcentrallybytheselected
teachers.Teachersbasicallyutilizedteststhatarepracticallybeingpreparedforthem.
Accordingly,thevalidityandreliabilityofthetestisalsoquestionable.

Anotherquestions thatneedtobeaddressedistowhatextent teacherpreparation

programmatchedwiththeteachersneedinassessingstudentseffectively?Research
hasshownthattraditionalteacherpreparationcoursesinclassroomassessmentarenot
wellmatchedwithwhatteachersneedtoknowforclassroompractice(Schafer,1993).
The traditional focus has historically been on largescale (standardized) testing
(Schafer, 1993), although this trend is changing. One course in assessment and
measurementmaytrulybeinsufficienttocovereverythingteachersneedtoknow.

As a comparison, Mertler (2005) indicated that assessment literacy means meeting the
seven competence standards delineated by AFT, NCME, and NEA. He compared both
in-service and pre-service teachers' assessment competence and the effect of
classroom/teaching experiences on assessment literacy. The assessment literacy of the
two groups was found to differ statistically on Standards 1, 2, 3, 4, and 7; in-service
teachers did better on these specific standards than pre-service teachers. However,
Mertler did not clarify whether the in-service teachers had taken assessment courses
during their teacher preparation; thus, they may have scored higher for this reason
than the pre-service teachers who were taking assessment at the time of testing.

Withrespecttotheconceptofassessmentliteracy,Popham(2003)hascalledforan
increasedeffortamongthemeasurementcommunityatlargetopromoteassessment
literacyonthepartofpolicymakers,practitioners(includingteachers,administrators,
andcounselors),public,andparents.Amoreassessmentliteratecitizenryislesslikely
totoleratemisuseofassessmentand,specifically,assessmentresults.Stiggins(1995)
offers several guiding principles for educators to follow in order to promote
assessmentliteracy.Theseguidingprinciplessuggestthateducatorsshould:
startwithaclearpurposeforassessment,
focusonachievementtargets,
selectappropriateassessmentmethods,
adequatelysamplestudentachievement,and
avoidbiasanddistortion.

Anotherquestionworthyofconsiderationandfurtherresearchiswhetherornota
majorityofassessmenttrainingisanonthejobtypeoftraining;inotherwords,are
assessment skills bestlearned through classroomexperience as ateacher, perhaps
onceteacherscanplacethenotionofassessmentinaspecificcontext,asopposedto
learning them as an undergraduate? Does undergraduate training provide the
necessaryfoundationforthisonthejobtraining?.Chen (2005) study could be used
to guide the further development and modification of assessment courses in teacher
preparation programs and to motivate teacher candidates to become assessment-
literate in accountability-driven environments. Ultimately, providing rigorous
assessment courses to teacher candidates can help their future students strengthen
academic learning. As Stiggins (2001) indicates, classroom assessment practices need
to be reformed so that assessment processes can become integrated into instruction to
promote student learning, support instructional decision-making, and provide
feedback for teachers on their instructional effectiveness.

To develop genuine accountability for students accountability that enhances their

educational opportunities we need to develop policies that use assessments to
guide educational improvements, rather than to further reduce the amount and quality
of education students receive. In Malaysia, we have developed an assessment policy
that promote school-based assessment and schools are given more autonomy to carry
out assessment accordingly. However, much needed to be done to improve the
practice of assessment in the classroom in order to provide the best and wisest form of
education for all children and eventually be able to create genuine accountability. A
well-qualified and competent teachers are the necessary ingredient to support genuine
accountability.

References

Abd. Wahab Muda, (2013) Mengukur Tahap Literasi Pentaksiran Guru-Guru Sekolah
Rendah Daerah Kuala Terengganu, Tesis Kedoktoran, Universiti Pendidikan Sultan
Idris

Chen, Peggy P. (2005). Teacher candidates' literacy in assessment, Academic

Exchange Quarterly. Accessed on Nov 2013 from
http://www.thefreelibrary.com/Teacher+candidates%27+literacy+in+assessment.
DarlingHammond,L.(2006).Standards,Assessments,andEducationalPolicy:In
PursuitofGenuineAccountability,EducationalTestingService,Princeton,New
Jersey.

Mertler,C.A.,(2003).PreserviceVersusInServiceTeachersAssessmentLiteracy:
DoesClassroomExperienceMakeaDifference?.Paperpresentedattheannual
meetingoftheMidWesternEducationalResearchAssociation,Columbus,OH(Oct.
1518,2003)

Mertler,C.A.(2003).Classroomassessment:Apracticalguideforeducators.Los
Angeles,CA:Pyrczak.

Mertler, C. (2002). Classroom assessment literacy inventory. Accessed on April, 2011

from http://pareonline.net/htm/v8n22/cali.htm.

Newfields, T. (2006). Teacher development and assessment literacy, Authentic

Communication: Proceedings of the 5th Annual JALT Pan-SIG Conference.
May 13-14, 2006. Shizuoka, Japan: Tokai University College of Marine Science. (p.
48 - 73).

Othman Lebar, Siti Eshah Mokhsein, Ahmad Fuaad Dahalan, Azali

Rahmat, Omar Hisham Baharin,. (2013). Pembinaan Standard Pentaksiran
Berasaskan Sekolah Bagi Sekolah Rendah di Malaysia, Laporan Penyelidikan FRGS,
Universiti Pendidikan Sultan Idris.

Paterno,J.(2001).Measuringsuccess:Aglossaryofassessmentterms.In Building
cathedrals: Compassion for the 21st century. Retrieved July 24, 2003, from
http://www.angelfire.com/wa2/buildingcathedrals/measuringsuccess.html

Plake, B. S., Impara, J. C., & Fager, J. J. (1993). Assessment competencies of

teachers:Anationalsurvey. EducationalMeasurement:IssuesandPractice, 12(4),
1012,39.

Popham,W.J.(2003).Seekingredemptionforourpsychometricsins. Educational
Measurement:IssuesandPractice,22(1),4548.

Popham,W.J.(2005).Assessment Literacy for Teachers: Faddish or Fundamental?.

Theory Into Practice, 48:411, 2009

Schafer,W.D.(1993).Assessmentliteracyforteachers.TheoryIntoPractice,32(2),
118126.

Stiggins,R.J.(1995).Assessmentliteracyforthe21stcentury. PhiDeltaKappan,
77(3),238245.

Stiggins,R.J.(2001).Theunfulfilledpromiseofclassroomassessment.Educational
Measurement:IssuesandPractice,20(3),515.