Principles of assessment

Aims
To look at the why, what and how of testing To examine the relationship between question types and the skills they test To examine how questions are written and produced and exams are put together To apply criteria for marking and assessing written and oral skills
2

Producing an exam: Question Paper Production Process

Commissioning of material for question papers Vetting and editing of material Pretest construction Revision Rejection Materials Bank Question paper construction
3

Pretesting Item analysis

Why do we test?
For diagnosis To establish what someone knows To establish what someone can do To provide motivation / goal To provide information for employers and academic institutions
4

What do language tests test?

Knowledge what language is - mcqs Skills what language can do various task types, including mcqs Communicative competence assessment criteria

Skills and subskills

Receptive skills reading and listening

Productive skills writing and speaking

What are some of the subskills?

Reading: Looking for specific information (scanning) Identifying gist and main point (skimming) Inferring Understanding text organisation Dealing with unknown vocabulary Guessing meaning through context
7

What is being tested? Question 9: option D

Mrs Hart was one of four theatre lovers chosen to judge all new drama productions last year. They were part of a panel that also included five professional theatre critics.

What is being tested? Question 11: option G

Being a judge was hard work, though. In one week alone, her duties involved her reviewing as many as seven plays. But I never got sick of it. Even the plays I didnt like always had some redeeming qualities.

What are some of the subskills?

Listening: For specific information/detail For gist Inferring For attitude and opinion
10

Sentence completion
Helen gives the example of . as the only typical sporting activity at the camp. For teenagers taking part, the camp lasts for .

11

Some writing subskills

Planning and crafting Editing Paragraphing Controlling register Using cohesive devices and coherence
12

Assessment of writing
General language criteria to establish level Task specific markscheme to establish Content Organisation Vocabulary/structures Register Required effect on target reader
13

Support in writing tasks

Writing tasks always provide: context content reason for writing target reader (and therefore style) number of words

14

Overview of BEC Vantage Writing

Part 1 Task Note, message, memo or email. Compulsory task Business correspondence, report or proposal. Compulsory task Input Context and instructions Output internal communication; 4050 words

One or more of: external business communication; correspondence, 120140 words internal communication, notice, advert, graph, chart, etc.

15

Both BEC Vantage and FCE writing papers (B2)

use functional language cover core skills: writing clearly and concisely planning and organising texts feature guided writing task with handwritten notes
16

General Mark Scheme criteria

Sets language criteria Interpreted at CEFR requirements for level

17

Task specific markscheme

content points for task range of structures and vocabulary appropriacy of organisation register and style effect on target reader
18

Part 1: content points

say who the client is give details of the printing order suggest what should be done about the situation
19

Assessment of scripts: Think about

content points use of language and accuracy range of structures and vocabulary organisation register and format effect on the reader

20

Marking writing
Differences between marking in the classroom and marking in an exam: providing support for students what to correct, and how diagnosis future course design Implications for marking of a skills based course
21

Examining procedures
Team System Teams mark papers

Team Leaders
22

Examiners co-ordinated

Overview of BEC Vantage Speaking

Part 1 Format Interlocutor asks each candidate in turn some questions. Each candidate in turn gives a short presentation. Timing about 3 minutes about 6 minutes Content personal information and opinions a business-related topic chosen from a set of three a business-related situation

Candidates have a about 5 discussion with each minutes other, using prompt card. Interlocutor asks both candidates questions.

23

Overview of FCE Speaking

Format Timing Content personal information and opinions Comparing two visual prompts collaborative task based on visual and written prompts based on topic of Part 3, and discussing wider issues interlocutor asks each 3 mins candidate in turn some questions individual long turns two-way conversation between candidates 4 mins 3 mins

three-way conversation 4 mins between candidates and interlocutor

24

Assessment of speaking
Format of the test provides support for candidates 2:2 format Different sections provides opportunity for different language

25

Assessment criteria
Grammar and vocabulary Discourse management Pronunciation Interactive communication

26

Round-up
Language testing need to use appropriate testing tools for different skills test language in use provide support for students test communicative competence

27