Brandon McLeod

TSL 633 Testing and Assessment in ESL/EFL

Dr. Negueruela-Azarola
18 June 2015
Critique: Brent Green and Maureen Andrades
Guiding principles for language assessment reform: A model for collaboration
Green and Andrades (2010) article describes the process through which an academic
ESL department proceeded with an overhaul of its assessment practices. This reformation was
initiated by the program director and the reading coordinator, and was overseen by Green and
Andrade. The article outlines the situations in which reform is necessary, the steps that need to be
taken in order to ensure a smooth transition, satisfied stakeholders, and effective (reliable and
valid) assessments.
The reform process that expanded almost four years and utilized a qualitative case study
approach, since the process was on-going as opposed to historical (p. 324). The restructuring
of the assessment practices involved nineteen full-time and part-time faculty, as well as five
administrators (these numbers do not include the researchers themselves). According to the
researchers, the key factors in test reform are best represented in diagram form as three entities
(Stakeholder Factors, Contextual Factors, and Test Development Factors) with double arrows
pointing from each entity to the others while encircling Agents of Reform Factors (with double
arrows pointing from each entity to the central item (p. 325-327). In this way, the researchers
account for all of the elements involved in the reform process.
The process began with observations, interviews, reviewing of program documents, and
archival data (p. 324). Following this, the instructors (both full and part-time) were grouped

McLeod, Brandon 2
according to skill (Reading, Writing, or Listening/Speaking) and told to assess the current exams
and determine if the course objectives were being tested (validity assessment). This was a crucial
step in the reformation process in that it allowed the instructors, who were often invested in the
exams they had created, to see that the current system of tests were not valid. Additionally, the
research period revealed that level progression was the result of non-synchronous testing
methodologies and often up to the persuasiveness of individual instructors. By having the
teachers assess the exams together, and examine the process of level progression, the researchers
were able to get the teachers to realize that the existing tests were neither valid nor reliable. Once
the tests had been assessed, and the principles for effective testing had been thoroughly explained
(with research and sample questions given to all the instructors), the teachers were asked to
create new final exams. This time, however, collaboration was required (where it never was
before) and they were instructed on how to focus on authenticity, interactiveness, construct
validity, and reliability (p. 329).
One of the biggest things that the instructors noticed was that by revising these exams,
they also had to revise the course objectives to be more specific, and create rubrics that
compliment the curricular goals. Another thing realized was that it takes time for teachers and
students to adjust to the new system. This was helped in that the agents of change (the director,
the reading coordinator, and the researchers) presented a united front to any opposition and at all
opportunities utilized instructor input to guide the process. Additionally, they instituted a policy
that sent all of the students final exam grades to every instructor (for level progression analysis),
giving them all the ability to compare their students to the other classes and to make level
progression more uniform (p. 331-332). These scores were also put into a database where the

McLeod, Brandon 3
director, when confronted by students who felt they deserved a higher level, could access the
mean scores and easily compare the student to his or her peers.
In the end, the process seems to have taken much longer than necessary. Moreover, as the
researchers point out, instructor input is necessary for change to be long-term and effective.
However, the part-time instructors were not compensated for their time in this endeavor and were
expected to contribute significantly. While there is no clear solution to the budgetary issues that
constrain language programs, it is much more difficult to create and enforce assessment reform
in a system that has a high turnover of adjunct instructors or where instructors are limited by
their contracts (p. 332). The researchers offer no solutions to these issues. In the end, this was an
excellent overview of the methods utilized to initiate and complete comprehensive assessment
reform.
References
Green, B. and Andrade, M. (2010). Guiding principles for language assessment reform: A model
for collaboration. Journal of English for Academic Purposes, 9, 322-334.
http://dx.doi.org/10.1016/j.jeap.2010.06.003