Language Testing http://ltj.sagepub.

com/

Formative assessment in ELT primary (elementary) classrooms: an Italian case study

Francesca Gattullo Language Testing 2000 17: 278 DOI: 10.1177/026553220001700210 The online version of this article can be found at: http://ltj.sagepub.com/content/17/2/278

Published by:
http://www.sagepublications.com

Additional services and information for Language Testing can be found at: Email Alerts: http://ltj.sagepub.com/cgi/alerts Subscriptions: http://ltj.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://ltj.sagepub.com/content/17/2/278.refs.html

>> Version of Record - Apr 1, 2000 What is This?

Downloaded from ltj.sagepub.com by emilia barbosa on October 5, 2012

Formative assessment in ELT primary (elementary) classrooms: an Italian case study

Francesca Gattullo Universita ` di Bologna and Universita ` di Trieste, Italy

I Introduction This research into classroom assessment has been largely shaped by the recent work in educational assessment of Torrance and Pryor (1998), in particular by their concept of the microsociology of classroom assessment and classroom learning which is dened as the study of how assessment of young children is carried out in classrooms, and with what possible consequences for their understanding of schooling and the development of their learning in particular subject areas (Torrance and Pryor, 1998: 3). A pilot study had been conducted over a two-year period (199798) in two primary schools in the Emilia-Romagna Region, Northern Italy.1 Four teachers, three specialists and one generalist, were observed during their lessons for a total of about 10 hours,2 as summarized in Table 1. They were teaching 86 pupils altogether, aged from 9 to 10 years, who had been studying English as a foreign language since the third grade (8 year olds). In addition, pupil questionnaires were administered, the participating teachers were interviewed, and an analysis of the assessment materials used for the teaching of English as a foreign language (EFL) was also undertaken. The ndings from the pilot work highlighted a number of areas of weakness and/or of teacher concern. These related to: 1) developing more structured ways of assessing oral abilities in pair and group work; 2) the provision of feedback and of action aimed at self-repair for individual pupils; and

Address for correspondence: Francesca Gattullo, via M. De Maria 2, 40129 Bologna, Italy; e-mail rav1905iperbole.bologna.it 1 For more details on the pilot phase of this research and on the overall project, see the Interim Reports presented at the Euroconferences 199799 (see Acknowledgements at the end of this article). For more information, contact the Euroconferences Coordinator, Dr P. Rea-Dickins, The Graduate School of Education, University of Bristol, 35 Berkeley Square, Bristol, BS8 IJA, UK. 2 An overview of the current foreign language-teaching situation in Italian primary schools can be found in Gattullo and Pallotti (in press).
Language Testing 2000 17 (2) 278288
0265-5322(00)LT181OA 2000 Arnold

Downloaded from ltj.sagepub.com by emilia barbosa on October 5, 2012

Francesca Gattullo 279

Table 1 Overview of pilot study
Type of teacher

Hours of observation
2.5 7.5

Generalist, who teaches English and L1, history, social studies Specialist, who teaches English only

1 3

3) the development of procedures for recording individual pupils performance for assessment purposes. This led to the preliminary conclusion that in the Italian primary foreign language classroom there was a need to investigate further issues of: 1) formative assessment in terms of information collection, feedback provision, and use of assessment results; and 2) the implicit and explicit presence of assessment processes in everyday classroom interaction. For the next phase in the research, a broad denition of classroom assessment for formative purposes was adopted as a starting point. The following are identied as the main traits of this type of assessment: it is an ongoing multi-phase process that is carried out on a daily basis through teacherpupil interaction; it provides feedback for immediate action; and it aims at modifying teaching activities in order to improve learning processes and results. II Work in progress 1 Aims The aims of the current study are dened in terms of both researcher and teacher perspectives, summarized as follows: 1) For the researcher to identify: (a) and describe how assessment is being interpreted and implemented by EFL teachers in the nal years of primary schools (i.e., children aged 810 years); (b) the different dimensions of formative assessment; (c) some examples of good practice of formative assessment. 2) For the teachers to have the opportunity to reect on:

Downloaded from ltj.sagepub.com by emilia barbosa on October 5, 2012

280 Formative assessment in ELT primary classrooms (a) the difference between formative and summative assessment in terms of information, collection procedures, feedback provision and use of their results; (b) the different dimensions of formative assessment; (c) the extent and range of assessment actions in the classroom, both implicit and explicit; (d) the possible development of improved strategies for formative classroom assessment. The focus of this article is on the processes of formative assessment so far identied from the data (see Section III below). 2 Methodology In order to identify different formative assessment processes such as questioning, correcting, counter-suggesting, marking, observing and so forth I have focused on particular assessment events or incidents as the unit of analysis (Torrance and Pryor, 1998: 5). With these authors I share a key theoretical interest in the social construction of assessment and learning in action, in ordinary classroom settings (p. 5). Data are gathered by classroom observation and recorded on audio tape. Three schools are involved, within the same school district ( circolo didattico ), and four teachers are observed: three generalists, and one specialist who was involved in the pilot study (see Table 1 above). They all volunteered to participate in the research, as this was part of a teacher development project I have been coordinating in the school district (for details, see Acknowledgements at the end of this article). A total of 70 children were involved. The transcribed observational data so far derive from about 15 hours of audio recordings in 3rd and 4th grade classrooms (children aged 8 to 10). From each recording relevant and interesting assessment events were selected and transcribed, ranging from one to ve minutes in length. Transcribed data have been qualitatively analysed, adding a commentary on each action which could be of interest for subsequent analysis. This rst level analysis and interpretation of assessment events will be developed further through discussion with individual teachers. Alongside the classroom observations, meetings with teachers have also taken place. The initial meeting at the beginning of the year introduced the teachers to the project and its aims. Intermediate meetings are being held with individual teachers in order to explore critically and analyse the classroom observation processes through the transcriptions and to gain a better insight into the assessment events.

Downloaded from ltj.sagepub.com by emilia barbosa on October 5, 2012

Francesca Gattullo 281 Further meetings are planned to implement a teacher development component for peer classroom observation, so that teachers may continue with peer observations on their own. A nal meeting at the end of the school year will evaluate progress to date and plan for the next phase of the research. It is envisaged that collaboration between the researcher and the teachers will continue to strengthen the teacherdevelopment dimension arising from this research, possibly through an action research methodology (i.e., a methodology focusing on a particular problem in a given context carried out only to solve the specic problem). III Preliminary ndings Torrance and Pryors (1998:160) framework of assessment processes assigns formative assessment events to one of 14 categories (see A O in Appendix 1). This provides the conceptual basis for the present study. Data collected so far can be assigned to the following nine categories, out of the original total of 14: A) B) C) D) E) F) G) H) J) Questioning / eliciting Correcting Judging Rewarding Observing process Examining product Clarifying Task criteria Metacognitive questioning

Appendix 1 orders the actions from the most frequent ones to those not used at all, as reected in the classroom data analysed thus far. The most common teacher action so far is A, that of asking questions to elicit evidence of what the pupils know, understand or can do, to which pupils respond (p. 160). A pupils response is (almost inevitably) followed by B: correction, counter-suggestion or information already supplied by the teacher. The two actions of questioning (A) and correcting or making counter-suggestions (B) represent the rst and last of the familiar threepart sequence of Input, Response and Feedback, a sequence which is extremely common in teaching settings (see Sinclair and Coulthard, 1975). However. this apparently simple IRF dynamic is in fact an oversimplication of what actually happens. An example is given in Transcript 1 (given in Appendix 2a), where the teachers question triggers a long sequence of interaction (here, as in Transcript 2 in

Downloaded from ltj.sagepub.com by emilia barbosa on October 5, 2012

282 Formative assessment in ELT primary classrooms Appendix 2b), the key assessment actions are highlighted in bold; the letters correspond to the assessment processes identied in Appendix 1). At the end of the exchange in Appendix 2a, the teacher claimed that she had gained a clearer idea of Marks stage of acquisition of spatial prepositions. As Mark was considered an average pupil, this represents a benchmark for the rest of the class. The teacher subsequently said that she would take a note of this and would rehearse prepositions with the whole class in a future lesson. However, she did not do so in the subsequent two months. From the above example there is some evidence that: the assumption that teachers can easily interpret pupils behaviour and ask clear questions that elicit clear and discrete answers is not well founded (Torrance and Pryor, 1998: 45). This episode shows that questions are asked for two purposes: revising the content of the lesson and establishing rules for teacherpupil interaction. Indeed, selecting Mark for responding to her questions not only feeds the language assessment process, but also reinforces guidance to other pupils on how to rehearse spatial prepositions and, more generally, on how to respond to language drills. The analyses so far reveal a high proportion of the three actions identied above which are mostly targeted at rehearsing knowledge and/or at enhancing motivation. There are assessment actions exemplied in Torrance and Pryors data (1998) that are rarely or never implemented at all by some of the teachers observed. Here, I refer mainly to those assessment actions that could develop and promote learning processes through, for example, metalinguistic and metacognitive activities. In other words, the data do not show teachers asking for clarication about what an individual pupil has said or done, or questioning why and how pupils have approached or achieved a task in the way they have. However, data from another study in progress illustrates assessment process J, that of metacognitive questioning.3 Here individual pupils had been asked to put some scrambled phrases in the correct order to reconstruct a known story. Once they had completed their task, the researcher spoke to some of the pupils at their desks to ask about how and why they had sequenced their sentences in a particular way. The conversation is transcribed in Appendix 2b. The key questions are indicated in bold. In this example, the researcher is trying to trigger from the pupil the thinking process that has led to the pupils
3 The data were collected for a small-scale action research study as part of an assessed work by Anna Luberto, a primary teacher, during a training course at the University of Bologna. I thank Anna Luberto for use of these data.

Downloaded from ltj.sagepub.com by emilia barbosa on October 5, 2012

Francesca Gattullo 283 ordering of the sentences. The hypothesis underlying this metacognitive questioning (see J in Appendix 1) is that through this kind of interaction pupils can better articulate their understanding and thus become more aware of their own linguistic processes. When asked about the absence of this kind of formative assessment action in their transcripts, the four teachers in the study said that they do not think it is viable in the foreign language-teaching context, although they believe it is important in other subjects, particularly for maths and other logical thinking activities. This, it is suggested, touches on a crucial issue for foreign language teaching in Italy, where the methodology for younger language learners has always been concerned with making learning fun and using the foreign language only, thereby neglecting completely the serious cognitive and linguistic aspects of this subject. (It is to be noted, though, that some teachers codeswitch from the second language to the rst language, as Transcript 1 in Appendix 2a shows, but this is more an exception than the rule.) Taking stock of the data collected so far, it would appear that teachers are often not able to make productive use of information they collect for formative purposes. In other words, language teachers questions do trigger responses that elicit an immediate feedback, but then such responses are not fully exploited for the potential insight that they may provide into the language learning process. Furthermore, teachers seem to make little or no use of some types of questioning and negotiations that could be fed into formative assessment and enhance the learning processes. Critical discussion with teachers and deeper data analysis could cast further light on this emerging looseness of formative assessment. IV Conclusions The focus of this research in progress is on formative, as opposed to summative, classroom assessment in a primary English as a foreign language context. This study has been motivated by the fact that we have relatively little knowledge at the level of classroom implementation of assessment (but see Rea-Dickins and Gardner, this issue) and that this is an area in which it appears that teachers may need to sharpen their professional skills. In the work in progress reported here, I have started to identify and describe how assessment is being interpreted and implemented by EFL teachers in the classes observed, based on the analytic framework of Torrance and Pryor (1998). This was used because it enables us to investigate formative assessment as a social process through a discourse analysis perspective. It also

Downloaded from ltj.sagepub.com by emilia barbosa on October 5, 2012

284 Formative assessment in ELT primary classrooms investigates formative assessment as a multifaceted phenomenon, taking into account many of the actions that teachers and pupils perform in their everyday classroom work. The classroom data analysed to date suggests that some formative assessment actions are more common than others (i.e., questioning, correcting, judging), at the expense of those that could be considered more benecial for learning (e.g., observing process, examining product, metacognitive questioning). In terms of teacher perspectives, the parallel discussions with them have highlighted the following as major points arising from the collaborative research: 1) an increased awareness of a wide range of formative assessment actions available, with particular reference to metacognitive questioning, communication of task criteria and individual pupil observation at work; 2) the importance of an open attitude towards learners, aimed at encouraging and establishing a dialogue with them; in particular, one teacher stressed the possibility of taking mistakes as a starting point for interaction rather than merely correcting them (see Zangl, this issue); 3) the importance of peer-teacher observations in developing new insights into ones own professional understanding and work. Furthermore, the teachers extremely positive responses and their willingness to continue this partnership in research serves to conrm the appropriacy of this type of collaborative classroom enquiry. As a result of this, a follow-up research proposal for further funding has been submitted to the Local Education Authority. Acknowledgements I would like to thank Pauline Rea-Dickins, Lyle Bachman and the anonymous reviewer for their guidance in helping me to revise earlier versions of this article. This study is part of a larger project developed within the three-year Euroconferences Programme Evaluating innovation and establishing research priorities in the teaching of foreign languages in European primary schools, and partly supported by the European Commission under Framework 4, DG XXII, Training and Mobility of Researchers. The research reported here has been undertaken as part of a wider initiative involving a group of researchers from other Mediterranean countries. The research has also been funded through a local project Progetto Lingue Comunitarie nella Scuola Elementare Azione 6 Progettazione e Attuazione di inizi` dellapprendimento linguistico. ative di valutazione della qualita

Downloaded from ltj.sagepub.com by emilia barbosa on October 5, 2012

Francesca Gattullo 285 V References

Gattullo F. and Pallotti G. in press: Foreign language teaching to young learners in Italy. In Nikolov, M. and Curtain, H., editors, An early start: young learners and modern languages in Europe and beyond. European Council for Modern Languages: Graz, Austria. Sinclair, J. and Coulthard R. 1975: Towards an analysis of discourse. Oxford: Oxford University Press. Torrance H. and Pryor, J. 1998: Investigating formative assessment: teaching, learning and assessment in the classroom. Buckingham: Open University Press.

Downloaded from ltj.sagepub.com by emilia barbosa on October 5, 2012

286 Formative assessment in ELT primary classrooms Appendix 1 The processes of formative assessment
Description Possible teacher intentions Possible positive effects for the pupil
Rehearsal of knowledge, understanding or skills; articulation of understanding to realize understanding Enhancement of knowledge and/or understanding Information about present achievement with respect to longer-term goals Enhanced motivation

A. T asks principled question (seeks to elicit evidence of what P knows, understands or can do). P responds B. T supplies information, corrects or makes counter-suggestions C. T assigns mark, grade or summary judgement on the quality of this piece of work D. T rewards or punishes the pupil or demonstrates approval or disapproval E. T observes P at work (process)

Insight or understanding of Ps knowledge, understanding or skills

Communication of alternative or more acceptable product Information for summative assessment

Improvement or maintenance of relationship with pupil; enhancement of motivation Gain in understanding of why/how the pupil has approached or achieved task Gain in understanding of what P has done Gain in understanding of what P has done and of Ps understanding of the task Communicating goals and success criteria; ensuring work is on target

Enhanced motivation due to Ts attention

F. T examines work done (product)

Enhanced motivation due to Ts attention Re-articulation of understanding; enhanced self-awareness Understanding of task and piece of work

G. T asks for clarication about what has been done or will be done; P replies H. T communicates task criteria (what has to be done in order to complete the task) or negotiates them with P J. T questions P about how and why specic action has been taken (meta-process and metacognitive questioning); P responds

Gain in understanding of why/how P has approached or achieved task

Articulation of thinking about thinking

K. T communicates quality criteria or negotiates them with P

Enhancement of quality of future work; promotion of greater independence Enhancement of quality of future work; promotion of greater independence Attributions and therefore motivation of P for further work

Understanding of notions of quality to aid future selfmonitoring Understanding of notions of quality to aid future selfmonitoring Enhanced motivation; development of learning goals

L.

T critiques a particular aspect of the work or invites P to do so

M. T gives and/or discusses evaluative feedback on work done with respect to task and/or effort and/or aptitude, ability N. T suggests or negotiates with P what to do next

Insight into ways forward for immediate further teaching of individual, refocusing P on curricular achievement Insight into ways forward for planning or group activities

Insight into ways to continue working and learning

O. T suggests or negotiates with P what to do next time

Deepening of understanding of principle/process

Source: adapted from Torrance and Pryor, 1988

Downloaded from ltj.sagepub.com by emilia barbosa on October 5, 2012

Francesca Gattullo 287 Appendix 2a: Transcript 1 Example of a sequence containing Actions A (questioning), B (correcting), D (rewarding)
Mark T Alf T Mark T T
look its / behind / the sofa /behind/? ( whispering ) behind are you sure? [ = Action A] behind mm mm do you remember whats his name? Spot the little dog spot? who is who is (.) behind the door be-hind the carpet? nella storia che abbiamo fatto con i aps? ti ricordi? [in the story with the aps? do you remember?] [ = Action A] behind come si dice? how do you say this (in English)? look (showing something above) [ = Action A] on ( showing something underneath ) un under [ = Action B] (moving Mark in front of his classmate) in in? on? under? [ = Action B] (. .) tu come sei rispetto a Paul? davanti [where are you relative to Paul? in front of] [ = Action B] ehm mmmm in front of and now invece come sei? [where are you now?] (moving Mark behind his classmate) [ = Action A & B] on ehe oh on? no not on [ = Action B] behind be-hind ok be-hind [ = Action D]

Mark pronounces the preposition incorrectly T prompts self-correction another pupil suggests the answer overlapping with the teachers next turn T questions Marks answer Mark repeats his mates suggestion T may not be satised with the answer since it was suggested by another pupil. She goes back to a well-known story to recall spatial prepositions; translation is given in square brackets Mark is only partly able to follow the teacher T now asks in Italian how do you say this in English with the help of a gesture correct answer T does not need to repeat the question M hesitates T completes the answer the quasi-drill goes on M answers T produces a different input M hesitates T rephrases the second question in Italian M is showing difculties T provides the answer and goes on asking M makes a new mistake T corrects nally, Mark provides the correct response T acknowledges stressing the pronunciation.

Mark T Mark T Mark T T Mark T Mark T

Mark T

Mark T Mark T

Downloaded from ltj.sagepub.com by emilia barbosa on October 5, 2012

288 Formative assessment in ELT primary classrooms Appendix 2b: Transcript 2 Example of a sequence containing Actions A (questioning) and J (metacognitive questioning)
Researcher dimmi una cosa, i pezzi nuovi come hai fatto a metterli? quando cerano quei pezzi nuovi che non ti aspettavi? [Tell me this: how did you sort the new bits, those new bits you were not expecting?] [ = Action J] Giulia perche mmm certe parole (.) dei pezzi nuovi mm certe parole le sapevo, allora sapevo come metterle [cause mmm some words (.) some new chunks mm I did know some words, so I knew how to sort them] R mm mm per esempio? [ = Action A] G (legge) The ice-cream is very cold and it is on Mr Hunts head R Si [ = Action D] Noi sul quaderno non avevamo scritto G che era molto freddo, allora io lo sapevo cosa voleva dire, allora [We hadnt written it in our exercise book that it was very cold, so I knew what it meant, so] R Lhai aggiunto, bene. E poi cosa hai dovuto aggiungere? Hai dovuto aggiungere anche unaltra parola? [You added it, well. And then what did you have to add? Did you have to add a new word?] [ = Action D & J] G si and R G
and mm, in modo da mettere insieme le due frasi [and mm, so that you could link the two sentences together] si

R questions P about how she has put the words in order; translation given in square brackets

P tries to articulate her thinking

R asks for an example P gives it R acknowledges P continues

R asks further, providing some scaffolding

P replies with and, the linking word needed to put two phrases together. R is helping P in the process of scaffolding P acknowledges.

Downloaded from ltj.sagepub.com by emilia barbosa on October 5, 2012