You are on page 1of 11

Multimodality in the EFL classroom

Multimodalidade na sala de aula de ingls

Fbio BEZERRA1

Abstract: Due to the increased multimodal nature of communication in todays globalized and
culturally diverse world (Christie, 2005), it is paramount to devote attention in the classroom to
how semiotic resources (van Leeuwen, 2005) other than verbal language have been used to create
identities and to position people socially, especially for the fact that the school plays or should
play a vital role in peoples individual, social, cultural and political development. In this paper, I
contextualize the work with images and present metalanguage from the Grammar of Visual Design
(Kress & van Leeuwen, 2006) in order to enable teachers to develop activities to foster their
students multimodal communicative competence (Royce, 2007; Heberle, 2010). Besides, I carry
out analysis of a movie poster so as to put the theory presented into practice.
Keywords: multimodality; classroom; grammar of visual design; multimodal communicative
competence

Resumo: Devido crescente natureza multimodal da comunicao no mundo globalizado e


culturalmente diverso de hoje (Christie, 2005), fundamental que dediquemos ateno na sala de
aula maneira como recursos semiticos (van Leeuwen, 2005) alm da linguagem verbal tm sido
utilizados para criar identidades e posicionar as pessoas socialmente, especialmente pelo fato de
que a escola desempenha - ou deveria desempenhar - um papel vital no desenvolvimento das
pessoas nos mbitos individual, social, cultural e poltico. Neste artigo, contextualizo o trabalho
com imagens e apresento metalinguagem da Gramtica Visual (Kress & van Leeuwen, 2006) a fim
de capacitar professores para desenvolverem atividades que promovam a competncia
comunicativa multimodal de seus alunos (Royce, 2007; Heberle, 2010). Alm disso, analiso um
cartaz de filme com o objetivo de colocar a teoria apresentada em prtica.
Palavras-chave: multimodalidade; sala de aula; gramtica visual; competncia comunicativa
multimodal

1 Introduction

In our globalized and culturally diverse world, communication is increasingly multimodal


(Christie, 2005, p.123), hence the importance of devoting attention in the classroom to how semiotic
resources (van Leeuwen, 2005) other than verbal language have been used to create identities and to
position people socially, especially for the fact that the school plays or at least should play a
vital role in peoples individual, social, cultural and political development.
In this context, more actions should be taken to foster students multimodal
communicative competence (Royce, 2007), which involves the knowledge and use of language
concerning the visual, gestural, audio and spatial dimensions of communication, including

1
I'm currently in the last year of my doctorate in English and Applied Linguistics at Universidade Federal de Santa
Catarina (UFSC) and my PhD in Linguistics at the University of Sydney (cotutelle).

BELT Journal Porto Alegre v.2 n.2 p. 167-177 julho/dezembro 2011 167
computer-mediated-communication (Heberle, 2010, p. 102) besides the other communicative
competences described by Hymes (1972) and Canale and Swain (1980).
Thus, this new reality demands immediate actions from all of us, teachers and students, in
order to develop a better understanding of how verbal language and images construe representations
of our experience and relationships between social actors as well as how these are brought together
in a textual manifestation as a cultural construct (Bezerra, Nascimento & Heberle, 2010; Motta-
Roth & Nascimento, 2009, p. 320). It should also be emphasized that the work with multimodal
texts in the EFL classroom can be used to develop not only students reading skills, but also their
writing, speaking and listening competence, which could help consolidate a holistic approach to the
teaching of the four skills (DAndrea, 2010).
Facing this challenge, the New London Group2 published an article in the Harvard
Educational Review, where they present a theoretical overview of the connections between the
changing social environment facing students and teachers and a new approach to literacy pedagogy
that they call multiliteracies (1996). In this new pedagogy, the following four components are
suggested for the work with the students: situated practice, overt instruction, critical framing and
transformed practice.
Situated practice refers to the need to approach whatever meaning-making resource from
the starting point of the personal experiences of students so that they can locate themselves in
relation to the study to be done, especially for the fact that meanings belong to culture, rather than
to specific semiotic modes (Kress and van Leeuwen, 2006, p.2). This term is actually closely
related to Gees discussion of situated meaning, where he states that meaning is always situated
in specific sociocultural practices and experiences (2000, p.195).
Overt instruction would be the moment to provide students with the metalanguage to carry
specific investigations. This is exactly the main objective of the present paper, since the analysis
carried out is done by using the metalanguage proposed by Kress and van Leeuwen (2006) in their
visual grammar, which we must teach our students in order to provide them with the tools to
understand that certain images and ideas about the world are favoured or presented as normal,
while other possibilities as excluded, either implicitly or explicitly (Callow, 1999, p.2).
Critical framing is fostered by having students interpret the contextual background and
values which inform whatever social practice and its related text. In other words, students are
engaged in the activity of critically deconstructing what is given to them, whether in the form of

2
For information about the New London Group and access to their seminal article on multiliteracies, see
http://wwwstatic.kern.org/filer/blogWrite44ManilaWebsite/paul/articles/A_Pedagogy_of_Multiliteracies_Designing_Soci
al_Futures.htm#11

BELT Journal Porto Alegre v.2 n.2 p. 167-177 julho/dezembro 2011 168
written texts, images, sound etc. Therefore, being literate means being able to understand, enjoy
and critique the kinds of messages that these images convey (Callow, 1999, p.2).
Transformed practice is the idea that students, having learned how texts of various kinds are
constructed, start designing their own practices based on the new knowledge in the same or in new
contexts. This is the moment when the teacher will really have the chance to see just how much
students could demonstrate an appropriation of not only the metalanguage which was taught, but
most importantly of a new mindset towards the reading of varied texts, which is especially
important in such a fast-changing world, where many of the texts presented to students in textbooks
are non-linear (Christie, 2005).

2 The grammar of visual design

Taking all the aforementioned points into consideration, especially the one about the
situated practice, as visual language is not () transparent and universally understood; it is
culturally specific (Kress and van Leeuwen, 2006, p.4), I aim to demonstrate the usefulness of the
categories for the analysis of images described in the Grammar of Visual Design (Kress & van
Leeuwen, 2006). For that matter, I analyze a movie poster used to promote the motion picture Sex
and the City, since it is based on the homonymous world famous3 TV series (Bezerra, 2008) and has
been strongly advertised by the media. Besides that, this can also be seen as an example of how to
use different genres in the EFL classroom, which may contribute to making the work with another
language more relevant for the students in the new communicative configuration that permeates
the learning environment nowadays (Lange, 2010).
Before starting the analysis, it is important to have an overview of the metalanguage used,
for it facilitate[s] systematic attention by teachers to the multimodal nature of texts in developing
critical literacy practices (Unsworth, 2001, p.72). It is important to state, though, that it is the
teachers responsibility to consolidate their knowledge of the categories used for analysis by doing
additional readings (see References).
Even though all meaning-making processes have always been multimodal, not much
attention has been paid to the meanings of regularities in the way image elements are used (Kress
and van Leeuwen, 2006, p.1). This is due to the longstanding misconception that images are
transparent representations which do not require instruction to be understood. It is precisely one of

3
Sex and the City is viewed in the following countries, as informed in Wikipedia The Free Encyclopedia
(http://en.wikipedia.org/wiki/Sex_and_the_city): U.S.A., Canada, Mexico, Brazil, Germany, United Kingdom, Ireland,
France, Netherlands, Sweden, Italy, Belgium, Spain, Finland, Russia, Romania, Bulgaria, Turkey, Australia, New
Zealand, Japan, Malaysia, Thailand, Singapore, Hong Kong, India, Pakistan, Philippines, Lithuania, Latvia, Denmark and
Hungary.

BELT Journal Porto Alegre v.2 n.2 p. 167-177 julho/dezembro 2011 169
the aims of a project for multiliteracies that students should have a chance to be introduced to
descriptions of images, and other semiotic resources, in much the same way that their verbal
literacy is fostered in schools.
Therefore, in the Grammar of Visual Design (GVD), Kress and van Leeuwen (2006)
propose a more systematic approximation of the image in a tri-functional perspective, as in Halliday
and Matthiessens (2004) Introduction to Functional Grammar, of which their work is an
extrapolation. In other words, they develop visual analysis having the following three metafunctions
as realizing meanings: the representational, the interactional and the compositional.

2.1 The representational metafunction

The representational metafunction concerns how participants and objects as well as events
(processes) and their associated circumstances are realized, which may be done through narrative or
conceptual representations.
The narrative representations concern actions, reactions, thought and
speech. Actions and reactions are represented by the presence of a vector
connecting the participants. In actions, the vector departs from the actor and is
directed towards the goal, in the case of the bears left arm touching the other bear
in Figure 1. In some images, the vector can be bi-directional, with both
participants being at the same time actor and goal, which would be the case if

Figure 1 both bears were touching each other.


In reactions, the vector is formed by the eye
line, that is, one participant (reactor) is looking at another
(phenomenon), or both are looking at each other (bi-directionality).
In Figure 2, for instance, some people are foregrounded and we see
that they are gazing at something. It is also important to add that Figure 2
actions and reactions can be either transactional or non-
transactional, depending on whether or not both represented
participants can be seen by the viewer. Thus, in Figure 2, it is
a non-transactional process of reaction, since we do not see
what those people are looking at. Figure 4
Figure 3

BELT Journal Porto Alegre v.2 n.2 p. 167-177 julho/dezembro 2011 170
Now, turning to thought and speech, one must know that
the former is depicted by the use of a thought cloud (Figure 3)
and the latter by means of a speech balloon or bubble (Figure
4), both being connected to the participants (senser and sayer,
respectively) also by a vector.
The conceptual representations, in
Figure 5
turn, refer to images that either classify
(covert or overt taxonomy), show part-whole relationships (analytical structured
or unstructured) or attribute/suggest values (symbolic). In Figure 5, we see a
covert representation of types of car. It is covert because it is not overtly stated
Figure 6
in the image what it is classifying. In Figure 6, the focus is only on part
of an aircraft (its engines), while in Figure 7 the golden hue arguably
adds a symbolic value of wealth to the city. I do not extend on these
categories in the present paper, since they are not found in the posters
analyzed. However, for further information, see Kress and van Leeuwen Figure 7
(2006).

2.2 The interactional metafunction

The interactional metafunction refers to the interrelation between the


image and the viewer. This relationship can be analyzed in terms of
interaction (contact), social distance, attitude, power and realism. Contact
can be defined as either demand or offer, depending on whether or not the
represented (human, human-like or animal) participant is looking directly at
Figure 8
the viewer. Figure 8 is an instance of offer. In the case of a demand, there is
the construal of a stronger relationship.
Also, such gaze directed at, or away from, the
viewers can be further emphasized by the participants
facial expression. Social distance may happen in one of Figure 9
three levels: intimate (close shot), social (medium shot) or
impersonal (long shot). In Figure 9, we see a long shot. Attitude may either show
Figure 10 involvement (frontal angle) or detachment (oblique angle as in Figure 10), that
is, it depends on whether or not the frontal angle of the represented participant and
the viewer coincide. Power may be attributed to the represented participant (low angle), to the

BELT Journal Porto Alegre v.2 n.2 p. 167-177 julho/dezembro 2011 171
viewer (high angle) or there may be a sense of equality (eye-level view). Finally, realism is
measured by how color, context, depth, detail and light interplay, with low modality as a
representation located away from a naturalistic view in a continuum, which happens, for instance, in
black and white images.

2.3 The compositional metafunction

The compositional metafunction has to do with the distribution of the elements and
information in the image, that is, the way in which the representational and interactive elements
are made to relate to each other, the way they are integrated into a meaningful whole (Kress and
van Leeuwen, 2006, p.176).
Three elements are analyzed in the compositional organization
of images: information value, framing and salience. When dealing
with the information value in the image, we observe the layout in
regard to the left-right, top-bottom and center-margin positioning. In
this perspective, while what is located on the left-hand side of the
page is taken as given information (like books in Figure 11), the right-
Figure 11 hand side brings the new information being introduced to the viewer
(computers in Figure 11).
Also, while the bottom of the page represents the real information, that is, what the viewer
identifies as more concrete, the top part brings the ideal, what is supposed to be aspired to by the
viewer. The layout may also foreground a central positioning, giving
prominence to certain elements, while having others as a marginal surrounding
structure.
Besides information value, an image also has framing, which can be
strong or weak, depending on whether the elements are shown as being
connected or disconnected. Finally, we can also analyze salience in the image,
by identifying which factors may give prominence to specific elements in the
Figure 12
image, which can be done, for instance, through the use of relative size, color
and foregrounding (like the glass of wine in Figure 12).

3 Analysis

BELT Journal Porto Alegre v.2 n.2 p. 167-177 julho/dezembro 2011 172
Having contextually situated the analysis and presented the metalanguage needed, we can
now focus on the poster chosen for the present paper (Figure 13). In this analysis, I aim to
demonstrate how the knowledge of the metalanguage can help the reader be aware of the
classificatory and constitutive nature of both language and images, which, hopefully, can be a step
towards having more teachers designing activities including the analysis of images in the classroom
based on sound terminology and theory.
In this figure, I bring not only the movie poster, but also a summary of the points discussed,
so as to provide an overview of the image construal.

Interactional
VISUAL GRAMMAR
(Contact)
Offer

Representational (Social Distance)


Impersonal/Long Shot

(Attitude)
Point of View/Oblique Angle
(Process) Detachment
Narrative: Acti onal >
Non-transac tional (Power)
Eye-level Vi ew
(Participant)
Actor: Carrie (Realism)
Medium Modality
(Circumstance)
SATC (skyscraper)
Compositional

SFL (Information Value)


Given/N ew
& Ideal/Real
Ideational
(Framing)
Connec ted
What: Get Carried Away
Where: Only in Theaters (Salien ce)
Size, Color and
When: May 30 Foreground

Figure 13

In the poster of Figure 13, there is a narrative representation of an actional process, as can be seen
through the vector formed by Carries dress and legs, showing her (actor) walking; however, this
action is non-transactional, since we dont see any other represented participant (goal). The name of
the movie (Sex and the City) can be seen as a circumstance of location, especially for its relative
size in comparison to Carrie, which suggests an idea that it might be a skyscraper behind her, being
integral to a glamorous and sparkling city. This focus on the action is also corroborated by a
transitivity analysis (Halliday and Matthiessen, 2004) of the verbal language used in the poster: Get

BELT Journal Porto Alegre v.2 n.2 p. 167-177 julho/dezembro 2011 173
Carried Away Only in Theatres May 30. There is the use of a material process aiming to entice
the viewer into watching the movie by a play of words with the main characters name (Carrie).
This way, a choice of action is being given to the viewer, but while watching the movie, this action
is consequently passed on to Carrie, who will then carry us away throughout the narrative.
However, it is highlighted that this encounter can only happen in theatres and on a specific day,
which are the circumstantial elements of the event.
Looking at the interactive meanings construed in the image, we note that there is not eye
contact with the viewer, which suggests only an offer. This offer being the possibility of seeing the
movie and having the opportunity to be carried away. This invitation is done from an impersonal
distance, as it is depicted in a long shot. This point is also reinforced by the detachment created
through the oblique angle between Carries body and the viewer. However, this detachment is
softened by the fact that there is an equal relation of power, created by the eye-level view, which
makes sense as the creator of the image probably wanted to foster proximity with the movie
audience. What calls attention, though, in this poster is the fact that there is use of medium to low
modality, as the representation of both Carrie and the city is not done in a very naturalistic manner
(both are placed against a black artificial background), which, as a matter of fact, does not seem to
work in favor of the portrayal of Carrie as a real woman who goes through the same ups and downs
as many others. Although we can highlight this possible contradiction, there is also the fact that it
might be important for the success of the movie to show its main character as someone to be
admired precisely for the fact that she represents something the audience may aspired to be like,
which is actually done compositionally.
Therefore, in a compositional perspective, we see Carrie on the right-hand side of the poster
and the play with her name in the clause Get Carried Away on the left-hand side, suggesting that
even though Carrie is someone we are already familiar with, she always has something new to
present, either in her appearance or in her life. The place where Carrie is inserted the city is
located at the top (the ideal) of the image, since New York City (where the story in the movie takes
place) is a much sought-after destination for its glamorous and sizzling life style. However, the
information about the movie itself (where and when it can be seen) is given at the bottom of the
poster, indicating that this is a real element, on which viewers can rely. The weak framing of the
image shows Carrie and the city completely connected, which is further indicated by the use of the
color pink in the name of the movie (the city skyscraper) and her dress, also suggesting a mixture of
romance and modernity, as the shade of pink used is really strong and bright. Also discussing the
salience in the image, we can notice that, even though Carrie is smaller in comparison to the name
of the movie, she is foregrounded. However, this foregrounding is not so salient, which may be

BELT Journal Porto Alegre v.2 n.2 p. 167-177 julho/dezembro 2011 174
understood as the portrayal of Carrie as an integral part to the city and vice-versa, a fact which was
strongly emphasized in the TV series of which the movie is a continuation.
In sum, we see the interconnection of the three metafunctions towards a representation of
Carrie as a woman who both acts and represents new things to come, which are to be seen by those
who will eventually watch the movie. The city of New York is portrayed as a glamorous and
shining place, basically though the use of relative size, color and light, that is, through
compositional elements. Again, it is worth stressing that the representation of Carrie and the city in
a medium to low modality seems to have been an element of contradiction in the whole message
conveyed by the poster, for a higher modality could have added to the idea of bringing the viewer
closer to her and the circumstance where she is situated. However, this fact is somehow softened by
the use of other representational, interactional and compositional resources.

4 Final Remarks

In conclusion, it is important to bring forth once more the need for systematic actions
towards the preparation of teachers to work with multimodal texts in their classes (Heberle &
Meurer, 2007), especially for the fact that, as previously put, our students will undoubtedly have
increased contact with texts which not only use varied semiotic resources, but whose access has also
changed, including to a greater extent the computer-mediated technologies.
Additionally, it is vital for teachers to be constantly reviewing their practice so as to keep
up with the new challenges, not only in regard to the myriad of semiotic resources and media
available in todays society, but also in respect to the role of the English language in the new world
order (Salles & Gimenez, 2010) and to their role to prepare [students] to function effectively in
such contexts (Ur, 2010). Therefore, the ultimate aim of this paper has been to provide teachers
with some theoretical basis and related metalanguage as well as to demonstrate that the work with
multimodal texts can be not only stimulating, but most importantly emancipating.

References

BEZERRA, F. SEX AND THE CITY: an investigation of womens image in Carrie Bradshaws
discourse as narrator. Dissertao de Mestrado. Ps-graduao em Ingls. Universidade Federal de
Santa Catarina, Florianpolis, SC, 2008.

BEZERRA, F.; NASCIMENTO, R. G.; HEBERLE, V. M. Anlise multimodal de anncios do


programa Na Mo Certa. Revista Letras (UFSM), Santa Maria, v. 20, n. 40, 2010, p. 9-26.

BELT Journal Porto Alegre v.2 n.2 p. 167-177 julho/dezembro 2011 175
CALLOW, J. Reading the visual: an introduction. In Callow, J. (Ed.). Image matters: visual texts in
the classroom. Newtown, NSW: Primary English Teaching Association (PETA), 1999, p. 1-13.

CANALE, M.; SWAIN, M. Theoretical bases of communicative approaches to second language


teaching and testing. Applied Linguistics, v. 1, n. 1, 1980, p. 1-17.

CHRISTIE, F. Language education in the primary years. Sydney: UNSW Press, 2005.

DANDREA, L. Using writing to develop communicative competence in the foreign language


classroom. BELT JOURNAL, Porto Alegre, v. 1, n. 2, 2010, p. 139-146.

GEE, J. P. Discourse and sociocultural studies in reading. In Kamil, M. L., Mosenthal, P. B.,
Pearson, P. D., & Barr, R. (Eds.), Handbook of reading research, vol. III. New Jersey, USA: LEA,
2000, p. 195-207.

HALLIDAY, M. A. K.; MATTHIESSEN, C. An introduction to functional grammar. 3. ed.


London: Edward Arnold, 2004.

HEBERLE, V. Multimodal literacy for teenage EFL students. Caderno de Letras (UFRJ), n. 27,
2010, p. 101-116.

HEBERLE, V., MEURER, J. L. Aspects of visual analysis for the EFL class. Anais do I Congresso
Internacional da ABRAPUI. Belo Horizonte: UFMG/FAPEMIG, 2007, p. 1-11.

LANGE, C. A sala de aula contempornea. BELT JOURNAL, Porto Alegre, v. 1, n. 2, 2010, p. 99-
105.

KRESS, G., van LEEUWEN, T. Reading images: the grammar of visual design. (2ed.).
London/NY: Routledge, 2006.

HYMES, D. On communicative competence. In: PRIDE, J.; HOLMES, J. (Eds.). Sociolinguistics.


Hasrmondsworth, UK: Penguin, 1972, p. 269-293.

MOTTA-ROTH, D.; NASCIMENTO, F. S. Transitivity in visual grammar: concepts and


applications. Linguagem & Ensino UCPel, Pelotas, v. 12, n. 2, 2009, p. 319-349.

NEW LONDON GROUP. A pedagogy of multiliteracies: designing social futures. Harvard


Educational Review, 66(1), 1996, p. 60-91.

ROYCE, T. Multimodal communicative competence in second language contexts. In: ROYCE, T.;
BOWCHER, W. (Ed.). New directions in the analysis of multimodal discourse. Mahwah, NJ:
Lawrence Erlbaum, 2007, p. 361-403.

SALLES, M.; GIMENEZ, T. Ensino de ingls como lngua franca: uma reflexo. BELT JOURNAL,
Porto Alegre, v. 1, n. 1, 2010, p. 26-33.

UNSWORTH, L. Teaching multiliteracies across the curriculum: changing contexts of text and
image in classroom practice. Buckingham, UK: Open University Press, 2001.

UR, P. English as a lingua franca: a teachers perspective. Caderno de Letras (UFRJ), n. 27, 2010,
p. 85-92.

BELT Journal Porto Alegre v.2 n.2 p. 167-177 julho/dezembro 2011 176
van LEEUWEN, T. Introducing social semiotics. London/New York: Routledge, 2005.

Received: September 20, 2012


Accepted: March 21, 2012

E-mail:
fabes10@yahoo.com.br

BELT Journal Porto Alegre v.2 n.2 p. 167-177 julho/dezembro 2011 177