Name : Zakaria Bintang Pamungkas

Class : C/ Pendidikan Bahasa Inggris

NPM : 21602073050

SELF, PEER AND TEACHER ASSESSMENT IN JAPPANESE UNIVERSITY

EFL WRITING CLASSROOMS

Keywords : bias analysis, essay writing, FACETS, Jappanese University, language

testing, self-assessment, peer assessment, teacher assessment.

In general, teacher usually used teacher assessment as method to asess students task.

Whereas, it is needed in the field of education used alternative assessments such as peer

asessment and self asessment. It is applied to avoid bias assessment of the assessor.

Moreover, if teacher assessment is used in performance test such as oral presentation,writen

composition, and role play it will be unconfortable. For example, however, in traditional

classroom settings, which are often found in present-day Japan, the teacher acts as the sole

evaluator.When students take a test made up of items that have only one correct answer, the

traditional approach is generally appropriate; however, in performance tests, such as oral

presentations, written compositions, and role-plays, the use of a single assessor can result in

potentially biased evaluations. As a result of attempts to overcome the limitations of teacher

assessments, alternative assessments, such as self-assessments and peer-assessments, have

been the focus of increasing interest in the field of education (Hargreaves, Earl &
Literature review

1. SELF ASESSMENT

Self-assessment can be defined as ‘procedures by which the learners themselves

evaluate their language skills and knowledge’ (Bailey, 1998, p. 227). In first language

assessment, self-assessment has oftenbeen reported as an effective tool because self-assessme

nt helps students to develop a better understanding of the purpose of the assignment and the

assessment criteria (Orsmond & Merry, 1997),improves learning (Sullivan & Hall, 1997),

and softens the blow of a bad grade by helping students understand the reasons for their grade

(Taras, 2001).

2. PEER ASESSMENT

Peer-assessment can be defined as ‘an arrangement for peers to consider the

level, value, worth, quality or successfulness of theproducts or outcomes of learning of

others of similar status’ (Topping, Smith, Swanson & Elliot, 2000, p. 150). In the field offirst

language pedagogy, peer-assessment has also been considered as an effective tool in both

group and individual projects. Peer assessment has been found to help teachers assess each

person’s effort in group projects (Conway & Kember, 1993; Goldfinch, 1994;Goldfinch &

Raeside, 1990) and to help students learn more and work cooperatively in a group (Kwan &

Leung, 1996).

Validity and Reliability of Self and Peer Assessment

Inside of literature, the measurement of validity and reliability of self and peer

assessment were conducted by comparing self rating questionnaire result with estimates of

participant language proficiency (TOEFL score), but not comparing students’ performance
(presentation, essay) with their elf assessment. The result from some researcher found that

using self asessment makes ESL learner were generably able to perceive areas in which they

had difficulty.

Peer-assessment in ESL/EFL contexts has often been conducted qualitatively under

such names as peer-response and peer-review. On the other hand, few researcher conduct

study relate to the self and peer asessment in quantitative methods. All the research have

made limitations. As result, the limitation can be identified:

1. First, lthough extensive research into self- and peer-assessment in first language

pedagogy suggests that self- and peer-assessments are pedagogically beneficial.

2. Second, in both first and second language pedagogy, the question of whether self-

and peer-assessments can be used as part of formal classroom assessment has been

a point of contention.

3. Third, most previous researchers have been concerned with interrater reliability

and have measured this using simple correlations; the degree to which raters are

internally consistent

4. Fourth, most researchers have used the traditional true-score approach.

The Comparison of Self and Peer Assessment by using Multifaceted Rasch

Measurement (MRFM).

Relate to the teacher assessment, the issues is the bias of its in assessing students

achievement. Many student feels that there is unequal assessment

Nowadays, the application of Multifated Rasch Measurement (MRFM) is used to

investigete how self and peer assessment work in comparison with teacher asessment in
actual writing class. Multifated Rasch Measurement (MRFM) can show the degree to which

raters are internally and externally consistent. Moreover, MFRM can measure rater

severity/leniency, and through bias analyses, to whom and to which assessment criteria each

rater displays bias. Overall, MFRM can illuminate the inside of the workings of the

assessment process. Eventhough MFRM has been utilized in various rating studies, only one

study of self- and peer-assessment (Nakamura, 2002) was conducted using this approach.

MFRM was utilized to investigate self-, peer-, and teacher-assessments.This was

accomplished through an inspection of writers’ abilities, raters’severities, and assessment

criteria difficulties, and the ways in which writers’ abilities and assessment criterion

difficulties differed among self-, peer-, and teacher-assessments. The target of this research

was writing abilities. This was accomplished through an inspection of writers’ abilities,

raters’severities, and assessment criteria difficulties, and the ways in which writers’ abilities

and assessment criterion difficulties differed among self-, peer-, and teacher-assessments.

Then, the research question founded :

1. To what degree do writers’ abilities, raters’ severities, and assessment criteria’s

difficulties vary and fit the model?

2. How do self-assessors, peer-assessors, and teacher-assessors differ when assessing

writers’ abilities?

3. How do self, peer, and teacher-assessments compare in terms of assessment criterion

difficulty and its infit mean square value?

4. To what degree do self-assessors, peer-assessors, and teacher assessors exhibit bias

towards writers’ abilities, and what types of bias are they?

Method

1. Participant

The participant at the study was 91 students of second year Japanesse students. The

study was conducted in two university writing classes. The participants’ages ranged from

19 to 21 years old. In addition to the student participants, four native Japanese teachers (T1–

T4) were selected for this study.

2. Procedures

The study was conducted in two university writing classes. And between the first and

seventh weeks, the participants received instruction concerning essay writing such as essay

format, mechanics, organization and contents.

3. Instrument

4. Analyses

Multifated Rasch Measurement (MRFM) was conducted using the FACETS computer

program with version 3.22. In the analysis, writer, raters and assesment criteria were

specified as facets. The out put of FACETS analysis reffers :

 FACETS map

 Ability measures and fit statistics for each writer.

 A severity estimate and fit statistics for each rater.

 difficulty estimates and fit statistics for each assessment criterion

 a bias analysis for rater and writer interactions

The FACETS map provides visual information about different elements of FACET,

such as differences in severity among raters and ability among writers. Writer ability

logit measures are estimated concurently with rater severity logit estimates and

assessment criterion difficulty logit estimates. By placing them on the same linear

measurement scale, the result easily compared.

Result

The result was shown that peer assessments have the potential to make

important contributions to the overall assessment process. It was proved by the

analytical of Multifated Rasch Measurement (MRFM). It is mostly appropriate while it

use for assessments.

The use of self assessment shown a slow respon while its using for assessment.

The result of self assessment reffers high achieving writers, were overly critical toward

themselves. This result was probably caused by the tendency of many Japanese to

display a degree of modesty.

In the use of peer assessments, the result did not show much of variance. the

result was consistent and their rating/ assessment paterns were not dependent on their

own writing performance. It means that higher achieving writers were not more severe

raters and lower achieving writers were not more lenient raters.