A presentation by

Marcia Tadjuddin
 Which side are you on?

Grammar and vocabulary should be

tested in isolation. (direct testing of
grammar and vocabulary)

Pro or Con?
Develop your arguments!
Testing Grammar
Why Test Grammar?

As far as proficiency tests are concerned, there

has been a shift towards the view that since it is
language skills that are usually of interest, then
it is these which should be tested directly, not
the abilities that seem to underlie them.
Why test grammar?
Probably, most proficiency tests that are
administered on a large scale still retain a grammar
section.
One reason for this must be the ease with which
large number of items can be administered and
scored within a short period of time.
The question of content validity: if we decide to test
writing ability directly, then we are severely limited
in the number of topics, style of writing and the
grammatical elements that we can cover in any
version of the test.
So, should we test grammar?
Wherever the teaching of grammar is thought
necessary, then consideration should be given to
the advisability of including a grammar
component in achievement area.
Whether or not grammar has an important place
in an institutions teaching, it has to be accepted
that grammatical ability, or rather the lack of it,
sets limits to what can be achieved in the way of
skills performance.
Writing specifications
Specification of content for achievement tests:
From the syllabus, which list the grammatical
structures to be taught.
Inferring from textbooks and other teaching
materials
For placement test:
Includes all of the structures identified above
Also perhaps those structures even for the lowest
classes
Sampling
Reflect an attempt to give the test content
validity by selecting widely from the structures
specified.
It should also take into account of what are
regarded as the most important structures.
It should not deliberately concentrate on the
structure that happen to be easiest to test.
Writing items
Whichever techniques are chosen, it is important
for the text of the item to be written
grammatically correct and in natural language.
E.g. We cant work in this class because there isnt
enough silence.
To avoid unnatural language, it is recommended
to use corpus based examples.
4 techniques are presented here: gap filling,
paraphrase, completion and multiple choice. The
first 3 require the candidates production, the
last calls only for recognition.
Gap Filling
Ideally, gap filling items should have just one
correct response.
E.g. The council must do something to improve
transport in the city. ______________, they
will lose the next election. [otherwise]
An item with two possible correct responses may
be acceptable if the meaning is the same,
whichever is used.
E.g. My dad works in a company _________
makes Nike Shoes. [that/which]
Gap filling
An item is probably rejected if the different
possibilities give different meanings or involve quite
different structures, one of which is the one that is
supposed to be tested.
Patient: My baby keeps me awake all night. She wont
stop crying.
Doctor: __________ let her cry. Shell stop in the end.
[just, Id, Well, Then, etc.)

This item may be improved by including the words

then, and just so that it cannot fill the gap.
Doctor: Then _________ just let her cry. Shell stop in
the end. [Id , you]
Gap filling
Adding to the context can often restrict the
number of possible correct responses to a single
one.
This may be presented in a longer passage with
several gaps and can be used to test a set of
related structures (such as the articles) or a
variety of structures

We live in _____ old house in _______ middle

of the village. There is ________ beautiful
garden behind ________ house. ______ roof of
______ house is in very bad condition.
Paraphrase
Require the student to write a sentence
equivalent in meaning to one that is given.
It is helpful to give part of the paraphrase in
order to restrict the students to the grammatical
structure being tested.
When I came, my mom was setting up the dinner
table.
When I came, the dinner table____________
Completion
Can be used to test a variety of structures.
Usually in a form of conversational transcript.

See the example on pg. 177

Multiple Choice
There are times when gap filling will not test what
we want to test because there are too many
possibilities.

Gap filling:
They left at 7. They _________ be home by now.

Multiple Choice:
A: They left at 7. They _________ be home by now.
B: Yes, but we cant count on it, cant we?

a. can b. could c. will d. must

Multiple Choice
Can be used to test discontinuous elements.

A: Poor man, he ________at that for days now.

B: Why doesnt he give up?

a) was working
b) has been working
c) is working
d) had worked
Scoring Production Grammar Tests
The important thing when scoring is to be clear
about what each item is testing, and to award points
for that only.
Nothing should be deducted for non-grammatical
errors, or for errors in elements of grammar which
are not being tested by the item.
If two elements are being tested in an item, then
points may be assigned to each of them.
Alternatively, it can be predetermined that both
elements have to be correct for any points to be
awarded.
To ensure scoring is valid and reliable, careful
preparation of the scoring key is necessary.
Testing Vocabulary
Why test vocabulary?
If there is little teaching of vocabulary, it may be
argued that there is little call for achievement tests
of vocabulary.
For those who believe that systematic teaching of
vocabulary is desirable, vocabulary achievement
tests are appreciated for their backwash effect.
The usefulness and feasibility of a general diagnostic
test of vocabulary is not readily apparent.
In placement tests, we would be looking for some
general indication of the adequacy of the students
vocabulary. We would not normally require a
particular set of lexical items to be a prerequisite for
a particular language class.
Writing Specification
If vocabulary is being consciously taught, then
presumably all the items presented to the students
should be included in the specifications.
We can add all the new items that the students have
met in other activities (reading, listening, etc.)
Words should be grouped according to whether their
recognition of their production is required.
A subsequent step is to group the items in terms of
their relative importance.
For placement tests or proficiency tests ,
specification refer to one of the published word lists
that indicate the frequency of the words.
Sampling
Words can be grouped according to their
frequency and usefulness.
From each of these groups, items can be taken at
random, with more being selected from the
groups containing the more frequent and useful
words.
Writing Items:
Testing recognition ability

Multiple choice can be recommended for this

type of testing problem.
Distractors are usually readily available
It seems unlikely to be any serious harmful
backwash effect since guessing meaning of
vocabulary items is something that we would
probably wish to encourage.
Recognizing Synonyms
E.g.: which is the closest in meaning to gleam?
A. gather B. shine C. welcome D. clean

Which distractors do you think are likely to be

chosen?
Whether distractors would work as intended
would only discovered through trialling.
Note that all of the options are words that the
candidates are expected to know.
Recognizing Synonyms

Another way is to have a common word as the

stem with 4 less frequent words as options.

E.g.: which is the closest in meaning to shine?

A. malm B. gleam C. loam D. snarl

The drawback: what distractors to use? How do

we ensure that the distractors are unfamiliar
words to the test takers? If the test taker knows
them, they will not distract.
Recognizing Definitions
Loathe means
a) dislike intensely
b) become seriously ill
c) search carefully
d) look very angry

Note that all of options are of about the same length.

Test takers who are uncertain of which option is
correct will tend to choose the one which is
noticeably different from the others.
In the example above, the writer has included some
notion of intensity in all of the options.
Recognizing Definitions
The less frequent word or the difficult word could
also be used (although with the same concerns)

One word that means to dislike intensely is

A. growl B. screech C. sneer D. loathe

Thrasher (Internet) believes that vocabulary is best

tested in context. A better way to test knowledge of
loathe would be:

Bill is someone I loathe for the humiliation that he has

caused me and my family.
A. like very much C. respect
B. dislike intensely D. fear
Recognizing appropriate word
for context
Context, rather than a definition or a synonym,
can be used to test knowledge of a lexical item.
E.g. The strong wind _______ the mans efforts
to put up the tent.
a. disabled b. hampered c. deranged d.
regaled

Note that the context should not itself contain

words that the candidates are unlikely to know.
Recognizing appropriate word
for context
Since learners and language users in general
normally meet vocabulary in context, providing
context in an item makes the task more authentic
and perhaps result in a more valid measure of the
candidates ability.
The context may help activate a memory of the
word, in the sane way as meeting it when reading in
a non-test situation.
There could be some negative backwash when words
are presented in isolation.
However, when we test vocabulary by means of
multiple choice, the range of possible distractors will
be wider if words are presented in isolation.
Testing Production Ability
Using Pictures
The main difficulty in testing productive lexical
ability is the need to limit the candidate to the
lexical item that we have in mind using only
words. One way around this is to use pictures.
However, this method is obviously restricted to
concrete nouns that can be unambiguously
drawn.
Testing Production Ability
Definitions
May work for a range of lexical items.
But not all items can be identified uniquely from
a definition (excluding all synonyms)
Not all words can de defined entirely in words
more common or simpler than themselves.
Testing Production Ability
Gap filling
This can take the form of one or more sentences
with a single word missing.
Often there is an alternative word to the one we
have in mind. This problem can be solved by
giving the first letter of the word (possibly more)
and even an indication of the number of letters.
Reminders:
While grammar and vocabulary contribute to
communicative skills, they are rarely to be
regarded as ends in themselves.
It is essential that tests should not accord them
too much of importance, and so create a
backwash effect that undermines the
achievement of the objectives of teaching and
learning where these are communicative in
nature.