ABSTRACT Challenges in Language Assessment : Are We Ready? by *Assoc. Prof. Dr Ratnawati Mohd.

Asraf and Jaya Pushani Ponnudurai Paper presented at the International Conference on Developments in the Pedagogy of International Languages: A Gateway for Practioners sponsored byInstitute Perguruan Bahasa-Bahasa Antarabangsa (IPBA) Kuala Lumpur June 4 at Hotel De Palma, Kuala Lumpur

Educational accountability is increasingly demanded at all levels of society considering the important decisions made about test-takers and the power it wields over ones future (Shohamy, E. 2001). Reforms in assessment and examinations announced in the 9th Malaysian Plan (The Star December 25, 2005) where new subjects to meet the demands of a changing globalised world will be introduced inevitably raises concerns about teachers preparedness to cope with the change. The shift away from examinations implies active involvement of teachers at both school and state level. Teachers who until now have been merely administering tests prepared at the state level or using exercises from commercially prepared workbooks will now find themselves in the position of having to develop tools and procedures for monitoring, recording, and assessing learners' progress and achievement in the classroom on a more systematic and formal basis (Brindley, 1997). It is therefore crucial that all involved in test design have assessment literacy(Fullan,1999). Frequent interactions with teachers in the process of conducting courses on language assessment and evaluation appear to underline this concern. This paper will outline what constitutes teachers assessment literacy (Fullan, 1999) and make suggestions on how teachers can be trained to become more confident as assessors.

*Assoc. Prof. Dr Ratnawati Mohd. Asraf is a lecturer with INSTED, International Islamic University Malaysia.

I TRODUCTIO

Student assessment is an integral part of a teachers role and Good teaching and good testing go together. (Cunningham,G.K.,1998:3) Teaching and assessment are interwoven and cannot be considered as separate entities because assessment is an ongoing process. If examinations were traditionally used to report students achievement and to sort them into categories, today there is a change today the focus is to assess for the purpose of educating and to improve learning rather than to only report grades. However, if this change is to succeed, we need to look at the change agents who are critical to the success of this proposed change in the educational assessment of Malaysia. These change agents are our teachers who need all the support they can get because any change requires careful planning and ongoing management (Fullan and Stiegelbauer, 1991 as cited in Brindley, 1997) and it is reported that teachers are finding themselves in the position of having to develop tools and procedures for monitoring, recording, and assessing learners' progress and achievement in the classroom on a more systematic and formal basis Examinations in Malaysia enjoy a status of confidentiality. Therefore one tends to rely on public sources of information such as the local newspaper to report related policies and issues before it becomes an official document. So the headlines Education Shift Towards Quality: Ministry working out blueprint to set new standards and criteria for students(New Straits Times, September 16 2006) followed by a Dewan Rakyat reporting (New Straits Times, March 28 2007) that an Evaluation minus exam was soon to be introduced as part of the National Blueprint albeit it would first be implemented in the 300 schools in the cluster of excellence; nevertheless raised concerns about teachers state of preparedness in addressing this innovation in assessment. THE MALAYSIA SCE ARIO - CURRE T PRACTICE In recent years Malaysia appears to have moved away from the traditional one-off examination to include assessment procedures that are on-going and more formative in nature. Changes can be seen at both system and classroom level. The School-Based Oral Assessment is a case in point. In Malaysia, English language is examined in a Standardised test at the end of 6 years, 9 years and 11 years. These tests are part of the Public Examinations namely UPSR (Ujian Pencapaian Sekolah Rendah) at the end of 6 years of schooling, PMR (Penilaian Menengah Rendah) at the end of 9 years of schooling and SPM (Sijil Pelajaran Malaysia) at the end of 11 years of schooling. These three major public exams at key points of schooling are set by the central examining body - the Malaysian Examinations Syndicate. Besides these, there are known to be monthly tests and trial examinations before major examinations. However, these tests come with a variety of names Ujian Diagnostic (Diagnostic Test), Ujian Pengesanan (Diagnostic / Progress test?), Pre-Test and Post-test, Ujian Bulanan (Monthly

Test), Peperiksaan Pertengahan Tahun, Tahun Lima (Year 5 Mid-Year Examination). So whats in a name; one might ask. This is because the name of a test holds several answers to questions on the realibility and validity of a test which directly affects the decisions teachers and stakeholders make about the test-taker. These questions are: . CHA GES A D PROPOSED CHA GES Developments in assessment have created new perspectives regarding teacher assessment. For example, when portfolio was introduced in the Childrens Contemporary Literature classroom in Year 4 primary, teachers must become familiar with new methods of assessment and testing. This is because the Portfolio calls for keen observation and for a comparison between the perceptions teacher and student of a students achievement (Brindley, 1994) Language teachers need to be equipped with new sets of competencies which contain new features; one of them being diagnostic competence i.e. the ability to interpret students growth in language learning and to skillfully deal with assessment material (Edelenbos, P. and Kubanek-German, A., 2004). The School-Based Oral Assessment would have posed another challenge to the ever-increasing challenges teachers face. Teachers would have had to administer new procedures, use new instruments and interpret new criteria. The Malaysian Smart School has a broad curriculum that considers the different capabilities and needs of all students and an on-going assessment that supports good instruction. The assessment should also take into consideration differing abilities, styles, and paces of learning. (Ministry of Education Malaysia, 2006). While teachers were given training on pedagogical concepts, there is little evidence of equipping teachers with the know-how of on-going assessment design that caters to Smart Schools The Ministry of Education, Malaysia announced that it was coming up with a blueprint to set new standards with new assessment procedures and criteria to reflect a seismic shift in the next five years (New Straits Times, September 16, 2006). The new systems is supposed to focus on skills and general ability and character building(ibid). We see the positive trend in the direction that assessment is going; that is, striking a balance between assessment that promotes learning as well as a test of the academia that measures knowledge. He also reminded that this did not signal the end of all public examinations and probably shares Spolskys suggestion to introduce a Why assess? How are students assessed? What is assessed? When are students assessed? Who assesses them? Who designs the test? How are marks used? (Falchikov, N., 2005)

multilevel system that combines testing and assessment (as cited in Hancock, C. 1994). Again there are echoes of concerns as there continues to be relatively little emphasis on assessment in the preparation of, or professional development of, teachers and administrators (Stiggins as cited in McMillan, 2000). At a presentation by an officer from the Malaysian Examinations Syndicate in March 2008, it was stated that the Syndicate was responding to a demand for change in assessment practices because there appears to be a mismatch between curriculum content and assessment practices. One of the demands is to empower the school and teachers to assess their own students. It has also been observed that assessment of learning (summative assessment) carried out by and at the school level has not been taken into account in the determination of grades, that school assessment has not been monitored and received little professional support and guidance from the Malaysian Examinations Syndicate and that there are big disparities of assessment practices between and among schools. Considering that most teachers would have only received knowledge of assessment in their basic first degree or professional qualification which preceded their teaching experience, would it be fair to assume that they are equipped and ready in their roles and responsibilities as assessors over time? From being raters of pencil and paper tests to using varieties of measurement tools to gather more data about their own students so that they may understand them better their innate abilities, their strengths, weaknesses, talents, attitudes, interest and personalities is a big leap into unknown territory.

FROM EXAMI ATIO S TO ASSESSME T Up to the present, teachers have been more familiar with tests which in simple terms is a method of measuring a persons ability knowledge or performance in a given domain. In the language domain, tests can take the form of a set of techniques, procedures, or items that requires performance on the part of the test-taker (Brown, D.H., 2004: 3). It is explicit and it is structured. Assessment on the other hand is an on-going process that encompasses a much wider domain (Ibid. 4). Students contribution during group-work or how they challenge themselves to answer a difficult question maybe subconsciously be assessed by their teacher. Various stages of a piece of written work may be assessed by the self, teacher and even other students. Assessment includes a balance of formal normative tests, formal assessments, informal classroom work samples, performances, and observations and student selfassessment. Many different purposes and audiences are listed certification, reporting to school boards, states, evaluate programme effectiveness, monitor students learning, adjust teaching strategies, engage students in self evaluation , understand students strengths and need each of these purposes and audiences may require different kinds of assessment and different types of information. One type of assessment cannot meet the needs of all audiences.

Formal normative tests or norm-reference assessment help teachers and administrators know how students are performing compared to other students across the nation or state. Informal classroom work samples and performances help teachers and students evaluate the application of skills to everyday learning, observations. Student self-assessment helps students become selfdirected learners. the very purpose of assessment shared by other experts and researchers, that is the ministry would like to introduce an assessment system that stimulates efforts to improve learning (Brindley,1997, National Centre for Educational Statistics, 1997, Carnegie Forum on Education and the Economy, 1986; McMillan, James H., 2000; Black, P. and Wiliam, D.1998; Interstate New Teacher Assessment and Support Consortium,1992, Banks, 2005). The case of assessments being formal or informal relates to how much planning and structure goes into it. Informal assessments consist of finding out the developmental level of students and are based on a variety of unofficial activities, for example, informal observations and oral diagnostic questions. Formal assessments on the other hand are preplanned, structured and developed for evaluative purposes. They are generally used to make judgements and as a basis for assigning grades (Harris,1994). Harris (Ibid.:26, Hanna, G.S and Dettmer, P.A, 2004, Banks, S.R., 2005) all remind teachers that although testing takes time, it should be seen as part of the teaching/learning process rather than something divorced from it. Swihart, P.A. (2007) in her Ph D dissertation on Life Histories of outstanding English Teachers noted how these teachers despite being caught in the era of summative evaluation always used alternative means to evaluate students knowledge of the four English skills to direct [their] teaching(p.130), that the purpose (of assessment) wasnt to punish or fail the students but to test and make a diagnosis, for them [students] to grow(pp135-136); to monitor their learning, identify areas that needed improvement, give them constructive feedback, and plan lessons to met their needs and expectations before undertaking summative assessments. A good teacher is continuously assessing students, whether those assessments are incidental or intended. Assessment procedures can also be categorized based on the instructional purpose. For example, formative assessments are planned assessments that act as guide and give direction to both teacher and students. Summative assessments are a type of formal assessment used to measure student outcomes at the end of the instructional programme or course. They can be used to award certification or to make decisions of the level the student has attained. Teachers in Malaysia are familiar with the more traditional type of grading standard, i.e. norm-referenced which is based on how well students do in comparison to each other. Norm-reference assessments are also sometimes called grading on the curve because the assignment of grades is based on certain reference points on the normal curve (Banks, 2005:31). At the same time there are instances when criterion-referenced assessment is used to report a performance standard. One such example would be the School-based Oral Assessment currently being used in Secondary schools. Assessments can also be categorized by degree of authenticity, whether it is performance or traditional. In recent years, with renewed interest in learning theories and

efforts to be fair to students and for a more holistic profile of students ability; alternatives to traditional assessment procedures have been proposed to include more realistic tools of measurements. Among them are portfolios, exhibitions and simulations. Performance-based assessments often measure skills that involve some type of observable activity for example, project work. Portfolios are an accumulated record of a students performances and exhibitions tend to be more summative because it usually indicates the mastery of a subject area (Banks, 2005). The principle feature about carrying out assessment is that there should be more than one. The article on Literacy and Numeracy: Towards More Effective Teaching Practice in Curriculum Update, Issue 47 August 2001, Ministry of Education, Wellington, New Zealand mentions that teachers should employ a range of high-quality assessment tools and procedures At classroom level, these tools should be teacher devised or obtained from central bodies but the tasks or assessment procedure should be the type that support informal and continuous teacher, peer, and self assessment based on observation, questioning, and feedback; and those that improve learning. They may include a variety of formal and informal assessment techniques e.g. observation, portfolios of student work, teacher-made tests, performance tasks, projects, student self-assessments, peer assessment, and standardized tests (Interstate New Teacher Assessment and Support Consortium 1992). The assessment tools used at the system level may be different because they are meant to provide information for different purposes. At the national level, the assessment programme will be designed to show what students know and can do and to provide information about national achievement trends and comparative information about students achievement in relation to other students. The selection and types of assessment procedures should be relevant to the purpose of the assessment as well as how the information is to be used. Generally, assessment in the classroom is diagnostic in nature. Teachers use the information obtained to address specific weaknesses with appropriate measures, establish how well students are learning by monitoring their progress against learning objectives as specified in the Curriculum Specifications as well as make decisions about the next learning steps, and to develop partnerships with parents when reporting to them about their childs performance. At the school level, the significance of the information assumes a broader aspect because not only does it aid in strategic planning and school development, it also helps to improve the achievement of individual students besides providing a base for norm-referenced assessment. At the Systems level, it provides reliability; assuring the quality of education; providing certification for the achievement. The shift away from examinations would decentralize the role of the Malaysia Examination Syndicate with implications for an active involvement of teachers at both school and state level. It is possible that the weight would shift to teachers who may be become more responsible for the design and conduct of assessment tasks. Teachers who until now have been merely administering tests prepared at the state level or using

exercises from commercially prepared workbooks will now find themselves in the position of having to develop tools and procedures for monitoring, recording, and assessing learners' progress and achievement in the classroom on a more systematic and formal basis (Brindley, 1997). Teachers must have assessment literacy(Fullan,1999) to move with this shift. SETTI G STA DARDS Given the more informed era we find ourselves in, educational accountability is being increasingly demanded at all levels of society considering the important decisions made about test-takers and the power it wields over ones future (Shohamy, E. 2001). The Ministry of Education plans to evaluate school performance via monitoring by the various government agencies such as the Schools Inspectorate, Schools Audit Division and Schools Division (Plan Induk Pembagunan Pendidikan 2006-2010:124) to encourage a culture of accountability. The same blueprint, Ibid.124 outlines steps to establish standards and indicators for the countrys new direction in education. Schools are expected to increase excellence via change management (Our translation). To this end, some assumptions are made about teachers that they all have the knowledge and skills to identify, select, design and administer tests and analyse data to inform teaching and learning. To what extent is this assumption true? Principle 8 in the Model Standards for Beginning Teacher Licensing, Assessment and Development: A Resource for State Dialogue developed by Interstate New Teacher Assessment and Support Consortium 1992 states: The teacher understands and uses formal and informal assessment strategies to evaluate and ensure the continuous intellectual, social and physical development of the learner. The teacher must know how to select, construct, and use assessment strategies and instruments appropriate to the learning outcomes being evaluated and to other diagnostic purposes

The road to assessment literacy is a long journey. In Malaysia, teacher institutions have taken the first step via formal teacher education courses where Language Testing and Language Assessment would have been a component in the Teacher Education syllabus. But teachers qualify through many pathways and this component may not have received the desired degree of attention necessary to assessment literacy. On the other hand, can this knowledge received so early in their career sustain them through the myriad changes that the curriculum, the syllabus and assessment procedures have undergone. The American Federation of Teachers, the National Council on Measurement in Education and the National Education Association (1990) established standards for teacher competence in student assessment because the associations believed that student assessment is an essential part of teaching and that good teaching cannot exist without good student assessment. We need to answer this question about standards in Malaysia by referencing our journey against the standards set by them because we

believe these standards are the minimum expected of any language teacher the world over. These standards published in Educational Measurement: Issues and Practices (1990) and cited in Cunningham, G.K., 1998; 3-5) are not copyrighted and the committee that wrote this document actually encourages their widespread distribution. They are: Teachers should be skilled in choosing assessment methods appropriate for instructional decisions. Teachers should be skilled in developing assessment methods appropriate for instructional decisions. The teacher should be skilled in administering, scoring and interpreting the results of both externally-produced and teacher-produced assessment methods. Teachers should be skilled in using assessment results when making decisions about individual students, planning teaching, developing curriculum, and school improvement. Teachers should be skilled in developing valid pupil grading procedures which use pupil assessments. Teachers should be skilled in communicating assessment results to students, parents, other lay audiences, and other educators. Teachers should be skilled in recognizing unethical, illegal, and otherwise inappropriate assessment methods and uses of assessment information.

WHAT TEACHERS EED TO K OW Let us convince you by referring to the literature out there: Teachers must have the theoretical knowledge of developing and using assessment tools (Brindley,G. 1997, Interstate New Teacher Assessment and Support Consortium 1992 ). Teachers need to develop tools and procedures for monitoring, recording, and assessing learners' progress and achievement in the classroom on a more systematic and formal basis (ibid). McMillan,J.H. (2000) too advocates the need for teachers to know the "essential" assessment concepts, principles, techniques, and procedures. Fullan (1999) expands on this qualification for teachers as assessors by defining assessment literacy as the capacity to examine student data and make sense of it; the ability to use the data to make effective changes in teaching and in schools; a commitment to engaging in external assessment discussions, that is, the active seeking out of external standards against which to test performance, knowledge about learners and learning; skills in devising assessment tasks; knowledge of criteria and appropriate standards; skills in the analysis and use of assessment information; expertise in giving appropriate, targeted feedback. The Interstate New Teacher Assessment and Support Consortium (INTASC, 1995) established 10 key principles it believes to be central tenets of effective

teaching among them being knowledge on how to establish proper assessment tools to measure student development. Maxwell (2004:6) ends his report on school-based assessment in Queensland by citing that the most important factor critical to the success of this approach to progressive assessment is the need for teachers to become skilled in conducting assessment programs and judging the quality of student performance against defined assessment standards. Doherty, Mangubhai, and Shearer (1996) as cited in Brindley (n.d.) in their discussion on the introduction of a new national assessment in Australia claim that although teachers had had a moderate level of assessment training; they were not convinced that their training equipped them for their assessment duties as delineated in explicit curricula .

Brindley reports that Chapelle, C.A. and Jamieson, J. aptly refer to these areas which teachers need to fully understand as thorny issues. They are indeed thorny because the educational system in Malaysia has come a long way and testing methods have undergone revolutionary changes. We now need to convince ourselves that teachers knowledge of assessment and professional development have kept pace with these changes. THE JOUR EY TO ASSESSME T LITERACY One of the objectives in the Plan Induk Pembangunan Pendidikan 2006-2010 Rancangan Malaysia Ke-9 (National Education Blueprint Ninth Malaysia Plan) is to Memartabatkan Profesion Keguruan (p.106). We translate this to mean to raise, the status of the teaching profession or to enhance the professionalism in teaching. Teachers are described ...as the most significant and costly resource in schools, teachers are central to school improvement efforts. Improving the efficiency and equity of schooling depends, in large measure, on ensuring that competent people want to work as teachers, that their teaching is of high quality, and that all students have access to high quality teaching. (Organization for Economic Co-operation and Development (OECD), 2005 (as cited in Plan Induk Pembangunan Pendidikan 2006-2010 If Malaysia plans to Memartabatkan Profesion Keguruan then we need ensure that teachers, among other professional skills, attain the standards proposed to prepare teachers to correctly use assessment techniques in their teaching, and better function in decision-making roles. We suggest that all concerned the Ministry of Education and its supporting agencies, State Education Departments, District Education Offices, School Principals and Head Teachers, Subject Panel Heads and Parents study the following areas to understand better how teachers are coping with current assessment practices before new challenges are added to their portfolio of responsibilities. They need to be aware that:

Change takes time Assessment requires more time than traditional tests Assessment procedures are different from traditional tests Assessment require new knowledge, new skills, new materials Implementation needs careful planning and ongoing management Professional development should be more focused and on-going Institutional support must come from all levels Assessment literacy needs to be demonstrated at all levels Allocate more time for this component in the Teacher Education Curriculum Teaching Practice criteria can focus on language assessment Frequent Professional Dialogues enhance teacher knowledge Current Assessment Practices need to be reviewed

CO CLUSIO Teaching, learning, assessment and evaluation impact all of us in many ways. In schools, assessment of student learning and classroom instruction serve many purposes including placement of students and qualifying them for entry to other forms of learning. When standards are involved and teachers shoulder the responsibility of making decisions, it is important to look at the preparedness of this crucial agent who determines a students future. We do not want to entertain questions on the validity and reliability of assessment practices. We cannot put our teachers in the line of fire when decisions about students are questioned. So let us pause to reflect on existing standards for teacher competence in educational assessment. (Hanna and Dettmer (2004:4) and let us plan for the future of language assessment. We cannot allow assessment literacy to be an afterthought.

10

REFERE CES Alderson, J.C. (1986) Innovations in Language Testing, in Portal (ed) Innovations in Language Testing. London: NFER/Nelson Alderson, J & et.al. (1995). Language Test Construction and Evaluation. Cambridge: Cambridge University Press. Banks, S.R. (2005) Classroom Assessment: Issues and Practices. Boston: Pearson Education, Inc. Black,P.and Wiliam, D. Inside The Black Box Raising Standards Through Classroom Assessment (1998) Brindley,Geoff.(1997)Assessment and the Language Teacher: Trends and Transitions. The Language Teacher. Retrieved October 12, 2006. http://www.jaltpublications.org/tlt/files/97/sep/brindley.html Brown, HD. (2004) Language Assessment: Principles and Classroom Practices. New York: Longman Cunningham, G.K. (1998) Assessment in the Classroom. London:The Falmer Press: Curriculum Update, Issue 47, August 2001 Discover Projects-Assessment (2004) Retrieved March 23, 2004. <http://info-center.ccit.arizona.edu/~discover/asessment.htm> Edelen P. and Kubanek-German, A.Teacher assessment: the concept of diagnostic competence, Language Testing, vol 21. no . 3 (July 2004):259-283 Education Shift Towards Quality. (2006, September 16) ew Straits Times.p.1 Falcchikov, N. (2005) Improving Assessment Through Student Involvement. London and New York:RoutledgeFalmer. Fullan , Michael. (2001) The ew Meaning of Educational Change (third edition) London: Routledgefalmer. Gronlund, N. E. (1985). Measurement and Evaluation in Teaching .London: Macmillan Gronlund,N.E.(2003) Assesssment of Student Achievement (7th Edn). New York: Pearson Education Enc. Hancock, Charles. R. (1994) Alternative Assessment and Second Language Study: and Why? Retrieved March 23, 2004. http://www.ericfacility.net/ericdigests/ed376695.html

11

Hanna, G. S. and Dettmer, P.A.(2004) Assessment for Effective Teaching. Boston: Pearson Education, Inc. Harris, M. and McCann, P.(1994) Handbooks for the English Classroom Assessment. Macmillan Henning,Grant (1987) A guide to language testing. Cambridge: Newbury House Publishers. Howell, K.W. and Nolet, V. (2000) Curriculum-based Evaluation.(3rd edn.) Belmont: Wadsworth. Lembaga Peperiksaan Malaysia, Kementerian PendiDIkan Malaysia (2002) Sijil Pelajaran Malaysia: Pengendalian Ujian Lisan Berasaskan Sekolah (ULBS) Major Teaching Overhaul .(2006, July 14) ew Straits Times.p.1 Maxwell, G.S (2004) Progressive assessment for learning and certification: Some lessons from school-based assessment in Queensland. Paper presented at the Third Conference of the Association of Commonwealth Examination and Assessment Boards, Redefining the Roles of Educational Assessment,March 2004, Nadi, Fiji Retrieved May 19, 2008. http://www.spbea.org.fj/aceab/GMaxwell.pdf McMillan, James H. (2000). Fundamental assessment principles for teachers and school administrators. Practical Assessment, Research & Evaluation, 7(8). Retrieved May 19, 2008. http://PAREonline.net/getvn.asp?v=7&n=8 Ministry of Education Malaysia. (2006) The Malaysian Smart School: Pilot Project Retrieved November 9, 2006. http://www.msc.com.my/smartschool/whatis/p_2.asp 9th Malaysia Plan: Good times ahead (2006, December 25) The Star Organization for Economic Co-operation and Development (OECD)(2005) in Plan
Induk Pembangunan Pendidikan 2006-2010 (2006) Bahagian Perancangan dan Penyelidikan Dasar Pendidikan <http://www.moe.gov.my/galeri_awam_manual/pipp2010/moe_pipp_08.pdf>

Rea-Dickins, Pauline (ed) (2004) Language Testing, Vol 21 (3) July 2004. London: Arnold Shohamy,E. (2001) The Power of Tests. London: Longman. Swihart, Patricia Alvarez, Life histories of outstanding English teachers , (Ph.D. dissertation. University of Puerto Rico, 2007) via ProQuest,

12

<http://proquest.umi.com.newdc.oum.edu.my/pqdweb?did=1317323371&sid=3& Fmt=2&clientId=56581&RQT=309&VName=PQD>. Ujian Pencapaian Sekolah Rendah (Primary School Assessment) Wikepedia. Retrieved November 9, 2006. http://en.wikipedia.org/wiki/Ujian_Pencapaian_Sekolah_Rendah

13