Computers & Education 62 (2013) 102110

Contents lists available at SciVerse ScienceDirect

Computers & Education

journal homepage: www.elsevier.com/locate/compedu

Using clickers in class. The role of interactivity, active collaborative learning and
engagement in learning performance
Lorena Blasco-Arcas 1, Isabel Buil*, Blanca Hernndez-Ortega 1, F. Javier Sese 1
Departamento de Direccin de Marketing e Investigacin de Mercados, Universidad de Zaragoza, Mara de Luna s/n, Edicio Lorenzo Normante, 50018 Zaragoza, Spain

a r t i c l e i n f o

a b s t r a c t

Article history:
Received 22 August 2011
Received in revised form
23 October 2012
Accepted 26 October 2012

As more and more educational institutions are integrating new technologies (e.g. audience response
systems) into their learning systems to support the learning process, it is becoming increasingly
necessary to have a thorough understanding of the underlying mechanisms of these advanced technologies and their consequences on student learning performance. In this study, our primary objective is
to investigate the effect of clickers (i.e. audience response systems) on student learning performance. To
do so, we develop a conceptual framework in which we propose that interactivity, active collaborative
learning and engagement are three key underlying forces that explain the positive effects and benets of
clickers in enhancing student learning performance. We test these relationships empirically in
a university class setting using data from a survey answered by students in a social sciences degree. The
results provide strong support for our proposed framework and they reveal that the high level of
interactivity with peers and with the teacher that is promoted by the use of clickers positively inuences
active collaborative learning and engagement, which, in turn, improves student learning performance.
These results show the importance of clickers in improving the student learning experience and
recommend their use in educational settings to support the learning process.
2012 Elsevier Ltd. All rights reserved.

Keywords:
Clickers
Learning performance
Engagement
Collaborative learning
Interactivity

1. Introduction
Information technology (IT) has been recently considered a strategic resource in educational settings, offering educational institutions
a unique opportunity to increase student motivation and enhance learning outcomes (Roblyer & Wiencke, 2003). Examples of technologies
that have been adopted by educational institutions in recent years, and that have received considerable attention in prior research, include
WebCT, Blackboard, tablet PCs and instant messaging. In this study, we focus on clickers, which have increasingly become an integral part of
the student learning experience in some educational institutions and offer a high potential for learning performance improvements. Studied
under different names, such as audience response systems, voting machines, wireless keypad response systems, or classroom communication systems, clickers are interactive remote response devices that transmit and record student responses to questions providing
immediate feedback to both the students and the teacher about the learning process (Homme, Asay, & Morgenstern, 2004).
Despite recent interest in the role of clickers, several aspects prevent researchers from fully understanding their inuence on student
learning (see the meta-analysis of Kay & LeSage, 2009). First, existing knowledge comes primarily from qualitative analyses. These studies,
while offering sound guidance and advice about the use of clickers in the educational context, provide little direction for understanding the
mechanisms through which clickers inuence the student learning process. There is a lack of quantitative studies which, if based on sound
theory, would help us better understand the role of clickers in student learning and the underlying mechanisms that explain their ultimate
impact on performance outcomes (Fies & Marshall, 2006; Kaleta & Joosten, 2007). Second, there is a noticeable lack of reliability and validity
analysis of the measurement instruments used in prior studies, which makes it difcult to have a rigorous understanding of the
phenomenon (Kay & LeSage, 2009). Third, according to Fies and Marshall (2006), it is surprising that much research analyzes the efcacy of
clickers in individual mode. Given the emphasis on collaborative work in the US National Science Education Standards, the application of

* Corresponding author. Tel.: 34 976 761000; fax: 34 976 761767.

E-mail addresses: lorena@unizar.es (L. Blasco-Arcas), ibuil@unizar.es (I. Buil), bhernand@unizar.es (B. Hernndez-Ortega), javisese@unizar.es (F.J. Sese).
1
Tel.: 34 976 761000; fax: 34 976 761767.
0360-1315/$ see front matter 2012 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.compedu.2012.10.019

L. Blasco-Arcas et al. / Computers & Education 62 (2013) 102110

103

clickers in group mode would be a better avenue for research. Finally, prior research has been conducted on a limited set of educational
settings, mainly on technical and scientic subjects such as mathematics, chemistry, engineering and astronomy. Thus, it is necessary to
study social science subjects for a better understanding of the general educational impact of clickers.
Addressing these gaps, our primary objective is to investigate the impact of clickers on student learning performance. We develop
a conceptual framework in which we identify several mechanisms behind the effect of clickers on learning performance. We propose that
the level of interactivity among students and between the teacher and the students that results from the use of clickers affects student
collaborative learning and engagement and, in turn, learning performance. To test this framework empirically, we used clickers in peer
group practices based on class-wide discussion, which offers an opportunity to improve students attention and stimulate collaborative
work. We collect real data from undergraduate students in a social science degree, study the reliability and validity of the scales and apply
a quantitative approach. In this way, we aim to bridge the gaps identied and contribute to existing research with an empirical application
that provides interesting implications about the use of clickers for student learning. Moreover, we contribute to the existing discussion
about the generation and enhancement of engagement, recently identied as a research priority in the learning arena (Oncu & Cakir,
2011).
2. What are clickers? Characteristics, advantages and disadvantages
Clickers are small transmitters that look similar to a television remote control. They are advanced technological devices that allow
students to quickly answer questions that are presented in class. When the students answer the questions, the clickers codes appear onscreen and students know that their responses have been recorded. A computer summarizes the responses and the results are automatically
displayed in chart form, usually a histogram. Responses can be anonymous or linked to specic students through the clicker unit ID, allowing
the teacher to know who gave correct and incorrect responses. Clickers are interactive and can speed up didactic lectures when teaching
adults and/or active learners.
Clickers provide signicant benets to both the teacher and students (Bergtrom, 2006; Bullock et al., 2002; Simpson & Oliver, 2007). For
the teacher, clickers provide immediate feedback about the student learning process and allow him/her to gauge the overall comprehension
of the concepts involved in the material. Clickers are also very effective at engaging students in the class, promoting interactions among
students, providing immediate feedback on their understanding of the lessons, and facilitating the active participation of students in the
learning process by discussing the answers given to the questions. These features of clickers stimulate the development of studentteacher
relationships and lead students to perceive the activity as entertaining, which, in turn, increases their willingness to participate in the class
(Caldwell, 2007). In sum, this technology improves student understanding of complex subjects, individual progress and comprehension and
teacher awareness of learning problems (Caldwell, 2007; Knight & Wood, 2005).
Despite these signicant benets of clickers, some disadvantages should also be noted. First, although the price of this technology has
decreased in recent years, clickers may still represent a signicant economic cost for some educational institutions, which may become
a barrier to adopt and integrate them into the learning process. Second, despite the ease of use of the technology and the benets that they
provide, faculty members may be reluctant to introduce new technologies in class and may perceive high costs in terms of time and effort
investments (Kay & LeSage, 2009). In addition, similar to other advanced technologies, clickers can generate frustration and unsatisfactory
situations due to technical issues like failures or bugs.
Overall, the signicant benets identied lead us to expect a positive effect of clickers on student learning performance.
3. Conceptual framework and hypotheses
In this study, we provide a conceptual framework that identies key mechanisms through which the use of clickers inuences student
learning performance (see Fig. 1). We propose that student perceptions of the interactivity with peers and with the teacher that result from
using clickers promote active collaborative learning and engagement, which ultimately leads to enhanced learning performance.
As noted previously, by stimulating two-way communication during the process of answering questions and in the discussions about the
correct answers, clickers increase the degree of perceived interactivity in the classroom both among students (interactivity with peers) and
between the students and the teacher (interactivity with teacher) (Banks, 2006; Bergtrom, 2006; Caldwell, 2007; Mayer et al., 2009).
Interactivity is conceived as a critical element in the learning process. It stimulates students to participate in the classroom -active
collaborative learning- (Guthrie & Carlin, 2004; Thalheimer, 2003), and to develop a sustained behavioral involvement in learning activities
-engagement- (Carnaghan & Webb, 2007; Kay & LeSage, 2009). The presence of these two elements is instrumental in enhancing students
learning performance. Only when students actively collaborate in the learning process can the teacher adapt the pace, style and topic of the
lecture to better t the students needs, identify any misunderstandings so as to clarify them properly and punctually, and make sure that
they have understood all the materials before continuing with the next learning step. In addition, active learning potentially promotes

H1

Interactivity with
Peers

Active Collaborative
Learning

H7

H2

LEARNING
PERFORMANCE

H6
H3

Interactivity with
Teacher

H4

Engagement

Fig. 1. Conceptual framework.

H5

104

L. Blasco-Arcas et al. / Computers & Education 62 (2013) 102110

student engagement. Engaged students have a high level of involvement that leads them to prepare themselves better for the class, pay
more attention, take good notes, think and be able to recall material from previous lectures (Caldwell, 2007). Eventually, all this will result in
a successful learning. Below, we offer the conceptual rationale for the proposed relationships.
3.1. Interactivity
Traditional learning methods can hinder interactions in the classroom (Cotner, Fall, Wick, Walker, & Baepler, 2008). Limited class time,
rigid seating arrangements and students reservations about speaking out in class have been identied as important barriers to high levels of
interactivity (Draper & Brown, 2004; Liu, Liang, Wang, Chan, & Wei, 2003). However, advanced technology has changed how students and
the teacher interact in the classroom and has provided new opportunities to enhance interactivity.
Promoting interactivity is crucial as it leads to better and more effective learning (Bannan-Ritland, 2002; Erickson & Siau, 2003). Thus, it
becomes a key source of success in education (Chou, 2003; Siau, Sheng, & Nah, 2006). This concept has long been considered one of the main
pedagogical issues in the classroom, especially for larger classes and technology-related courses. When interactivity is present in the
learning activity, students are not only more motivated to learn, but also more attentive, participative and more likely to exchange ideas
with others (Liu et al., 2003; Sims, 2003). Consequently, interactivity inuences student learning outcomes such as attitude and achievement (Haseman, Polatoglu, & Ramamurthy, 2002).
It is important to distinguish between two types of interactivity: (1) interactivity between students -Interactivity with peers- and (2)
interactivity between the students and the teacher -Interactivity with the teacher.
Interactivity with peers results from participation, discussion and peer instructions, and it improves the active processing of course
material and higher-order learning (Crouch & Mazur, 2001; Michaelson, Knight, & Fink, 2004). Some students prefer hearing explanations
from their peers because they use a similar language and, therefore, can explain the problems and solutions more effectively, than the
teacher (Caldwell, 2007; Nicol & Boyle, 2003). Regarding interactivity with the teacher, building interactions with students in class is an
important part of the teachers task and a critical component of the learning process (Mayer et al., 2009). This interactivity allows the teacher
to provide feedback during the class, assess students understanding of materials and concepts, identify the doubts and problems of
students, and develop closer relationships with them (Trees & Jackson, 2007). Indeed, studentteacher interaction is ranked highly among
the factors inuencing learning performance (Bullock et al., 2002; Hake, 1998). In sum, both types of interaction involve students actively in
the classroom (Sims, 2003), provide information feedback to both parties and ultimately improve learning performance (Draper & Brown,
2004; Higgins, Hartley, & Skelton, 2002; Trees & Jackson, 2007).
The use of clickers improves existing interactions during the learning process because they foster communication among the students
and between the teacher and the students (Banks, 2006; Beatty, 2004; Bergtrom, 2006; Caldwell, 2007; Mayer et al., 2009). These two
interactivities lead to signicant improvements in the learning process including greater articulation of student thinking (Beatty, 2004) and
effective peer-to-peer discussion (Bergtrom, 2006; Kennedy, Hyland, & Ryan, 2006). Moreover, they promote both active collaborative
learning (Elliot, 2003; Kennedy et al., 2006) and engagement (Anderson, 2003; Fredricks, Blumenfeld, Friedel, & Paris, 2002; Marks, 2000).
The interactions among the teacher and students encourage the latter to collaborate more actively during the class and to develop a high
engagement with the subject matter, fostering the interpersonal dimension of this variable (Gallini & Moely, 2003; Kay & LeSage, 2009).
On the basis of the preceding discussion, we consider that student engagement and active collaborative learning are two important
consequences of interactivity. Thus, we hypothesize:
H1
H2
H3
H4

Interactivity
Interactivity
Interactivity
Interactivity

with
with
with
with

peers as a result of using clickers increases students active collaborative learning.

peers as a result of using clickers increases students engagement.
the teacher as a result of using clickers increases students active collaborative learning.
the teacher as a result of using clickers increases students engagement.

3.2. Engagement
Engagement has been identied in the learning literature as a research priority and it has been addressed from different perspectives.
Despite the attention that this variable has received, there are still weaknesses in its conceptualization, which in some cases have lead to
a manifest overlap, duplication and lack of differentiation between engagement and other existing concepts (Fredricks, Blumenfeld, & Paris,
2004). Recently, several authors have acknowledged the multifaceted nature of engagement. While Gallini and Moely (2003) consider the
dimensions community, academic and interpersonal engagement, Fredricks et al. (2004) identify behavioral, emotional and cognitive
components. This study builds on both approaches, dening engagement as the perception of the student that results from his/her
interactions with peers and teacher during the learning experience and which generates involvement with the topic studied (Anderson,
2003; Fredricks et al., 2002; Gallini & Moely, 2003).
Engagement is determined by the interactions between the environment and the individual, so that social and academic changes in class
modify students perceptions and engagement (Finn & Rock, 1997). Guthrie and Wigeld (2000) suggest that engagement mediates the
inuence of curricular and instructional changes on student performance and achievements. Moreover, as mentioned in section 3.1, the
interactions between students and with the teacher (i.e. the social environment) inuence the engagement developed in the learning
experience.
Students learn better when they engage in appropriate cognitive processes, so their engagement is an important explanatory variable of
their success (Mayer et al., 2009). High engagement is considered an accurate predictor of continuing motivation, commitment and overall
performance (Shernoff & Hoogstra, 2001), and it is a reliable predictor of learner achievement (Baker, Spiezio, & Boland, 2004; Kuh, 2003;
Marks, 2000). Ahlfeldt, Mehta, and Sellnow (2005) highlight the importance of developing engagement not only for student motivation but
also to increase the richness of the students learning environment that leads to better student performance.
Several researchers point out that the engagement achieved from using clickers has a direct effect on individual outcomes (Brewer, 2004;
Caldwell, 2007; Hu et al., 2006; Kenwright, 2009). Engagement is positively related to higher scores and other achievement measures

L. Blasco-Arcas et al. / Computers & Education 62 (2013) 102110

105

(Stipek, 2002). On the basis of this discussion, we consider that the use of clickers generates student engagement, which is a key driver of
learning performance. Thus, we hypothesize:
H5 Engagement as a result of using clickers improves students learning performance.
3.3. Active collaborative learning
Active learning is dened as the result of a deliberate and conscious attempt on the part of a teacher to cause students to participate
overtly in a lesson (Pratton & Hales, 1986, p. 211). It refers to practices that engage students in the learning process and includes techniques
where students do more than passively listen to a lecture. Active learning involves talking and listening, writing, reading and reecting, and
it can increase exam scores over traditional formats (Knight & Wood, 2005; Yoder & Hochevar, 2005). According to the generative theory of
learning, students learn better when they participate in active cognitive processing (Mayer & Wittrock, 2006; Prince, 2004), so good
practices use active learning techniques (Chickering & Ehrmann, 1996).
One of the most benecial methods of active learning is collaborative learning, which occurs when students work together in small
groups toward a common goal, creating meaning, exploring a topic or improving skills (Prince, 2004). Collaborative learning is dened as
a philosophy of learning that involves sharing knowledge, experiences and authority, in which students teach and learn from each other and
develop a positive interdependence (Panitz, 1996). It increases the ability to think critically (Angeli, Valanides, & Bonk, 2003; Garrison,
Anderson, & Archer, 2001) and encourages students to participate in giving the answer, explaining and justifying their opinion (Lantz,
2010). In this context, students turn into active agents in their learning process and collaborate in the creation of their own knowledge.
Therefore, this method allows students to delve deeper into the subject matter and helps them to build new associations from previous
knowledge (Draper, Cargill, & Cutts, 2002; Kennedy & Cuts, 2005). Some positive consequences of collaborative learning are student
involvement, satisfaction, engagement and higher-order learning (Hiltz, Coppola, Rotter, & Turoff, 2000; Khan, 2000; Prince, 2004).
Moreover, it enhances the likelihood that all group members will learn the subject matter and decreases the likelihood that only a few
students will understand the material (Soller, 2001).
The effect of active collaborative learning on student performance is further enhanced when it is combined with the use of
technology (Stowell & Nelson, 2007). Kryder (1999) offers support for this view by suggesting that, when students use technologies,
they are more collaborative in their learning process. Similarly, Fowler, Armarego, and Allen (2001) note that students who are skilled
technology users have a learning style that is both sensory and visual and that 80% of all these students are active learners. The growth
in the use of technology to promote collaborative learning has attracted a rapidly growing number of studies (Resta & Laferrire,
2007), which collectively provide support for the view that technology enhances learning processes (Kreijns, Kirschner, & Jochems,
2003).
With respect to the technology that we study, clickers enable students to cognitively process questions asked by the teacher and to
increase participation (Caldwell, 2007; Ribbens, 2007). They introduce important changes in the class format because they foster the
processing of new concepts and the integration with prior knowledge (Mayer et al., 2009), encouraging students to discuss ideas and debate
points of view critically (Guthrie & Wigeld, 2000). Clickers facilitate the development of active learning and students contribution to
knowledge creation, so that students feel that are participating in their own learning (Guthrie & Carlin, 2004; Thalheimer, 2003). We
consider that using clickers improves the degree of active collaboration perceived by the students during the learning process, which
enhances engagement with the topic and their overall performance (Ryan, 2000; Yourstone, Kraye, & Albaum, 2008). Therefore, we propose
the following hypotheses:
H6 Active collaborative learning as a result of using clickers improves students engagement.
H7 Active collaborative learning as a result of using clickers improves students learning performance.

4. Method
4.1. Participants
We tested the proposed hypotheses using a sample of 198 undergraduate business students enrolled in an introductory marketing course
during the 20102011 academic year at a major university in Spain. Participants belonged to four different classes and were in their rst year
of study. They attended classes 2 days a week for 4 h during the rst semester. Their ages ranged from 18 to 36. The sample consisted of 89
males and 109 females.

4.2. Procedure and materials

The three teachers in charge of this introductory marketing course collaborated in the study (two teachers taught one class and the third
taught two classes). Similar to prior studies in educational contexts (e.g. Yourstone et al., 2008), teachers standardized their course material,
lectures and PowerPoint slides to ensure that all four classes covered the same material in an identical way. The lesson plans, delivery
methods and approach to teaching in the classrooms were also coordinated. In addition to this coordination, the teachers had signicant
personal experience working with each other, as well as an identical experience with the clickers.
During the semester, each of the four classes was given 7 multiple-choice tests, with 10 questions per test, using clickers to respond.
Specically, a total of 6 tests were administered after each unit was covered (the introductory marketing course includes six units). These 6
tests were not cumulative, each covering only the assigned material in each unit. In addition, a nal test covering all the material of the
course was carried out at the end of the semester. The quizzes accounted for 5% of a students mark.

106

L. Blasco-Arcas et al. / Computers & Education 62 (2013) 102110

The method of instruction used during the test sessions with clickers was class-wide discussion (Nicol & Boyle, 2003). As such, students
were clustered in small groups (45 people). The total number of groups across the four classes was 51. Each group was given a clicker with
an identication number for use throughout the semester.
At the beginning of the sessions in which clickers were used, groups picked up their assigned clicker and the teacher presented the
multiple-choice questions on Power-Point slides. After each question, the teacher required the groups to click on the correct answer. Once
the clicker system was activated, a timer appeared on the screen. The system allowed a time limit to be set for responses. According to the
nature and difculty of the questions, groups of students were given 6090 s to discuss among themselves and answer each question using
their clicker. The software recorded each groups response. The number of groups who had responded at any point in time was visible on the
screen. When the time was up, the student responses were displayed as a bar graph with the distribution of answers (shown as
a percentage). Each group was then encouraged to explain its answer, ask questions to other groups and discuss the alternative answers.
Finally, the correct answer was shown.
The materials used during the test sessions consisted of ten Power-Point slides for each of the seven tests. Each slide contained
a multiple-choice question that tried to check the students understanding of the material of each unit and their ability to apply the
knowledge to new situations. Clicker questions were prepared by the groups of students. At the end of each of the six units of the marketing
course, the groups were asked to prepare three multiple-choice questions with answers. As the effectiveness of clickers to increase learning
depends to a great extent on the design of the questions, special attention was paid to this stage. Students were provided with guidelines and
suggestions proposed by the literature for designing the questions (e.g. give a term or concept and identify the correct denition from a list,
and vice versa, or apply a familiar idea or knowledge to new contexts). Moreover, before choosing the nal test for each unit, teachers
carefully analyzed and adapted questions following the cited guidelines (Beatty, 2004; Beatty, Gerace, Leonard, & Dufresne, 2006; Beekes,
2006; Wit, 2003).
Clickers from Hyper-Interactive Teaching Technology (H-ITT) were employed. All signals were collected by a receiver plugged into
a laptop at the front of the classroom.
4.3. Data collection and measures
This paper is part of a larger study that examines the students opinions about the use of clickers and the perceived benets of using this
technology. A questionnaire consisting of 80 items was designed, and a pilot test was conducted with 15 students. Following minor revisions
to the layout, the survey was administered at the end of the semester (January 2011). Each participant was provided with a questionnaire
and a brief background to the study. It was made clear that participation in the survey was voluntary and anonymous. Of the 280 students
enrolled in the course, 198 completed the survey, yielding a response rate of 71%.
In addition to the variables of interest in our model, the survey included questions about the students overall impressions of clickers (e.g.
perceived usefulness, perceive ease of use, enjoyment, overall attitude, satisfaction) and the benets of using this technology (e.g. sharing
knowledge, sense of belonging, recognition). A literature review was carried out to measure the constructs. In all cases, seven-point Likert

Table 1
Constructs, items and conrmatory factor analysis results.
Constructs and items
Interactivity with peers (Liu, 2003; McMillan & Hwang, 2002)
Using the clickers in class...
INT_P1. Facilitates interaction with peers
INT_P2. Gives me the opportunity to discuss with peers
INT_P3. Facilitates dialog with peers
INT_P4. Allows the exchange of information with peers
Interactivity with the teacher (Liu, 2003; McMillan & Hwang, 2002)
Using the clickers in class...
INT_T1. Facilitates interaction with the teacher
INT_T2. Gives me the opportunity to discuss with the teacher
INT_T3. Facilitates dialog with the teacher
INT_T4. Allows the exchange of information with the teacher
Active collaborative learning (So & Brush, 2008)
In this course...
ACL1. I felt that I actively collaborated in my learning experience
ACL2. I felt that I have co-created my own learning experience
ACL3. I felt that I had free reign to co-create my own learning experience
ACL4. I felt that I had freedom to participate in my own learning experience
Engagement (Gallini & Moely, 2003; Medlin & Green, 2009)
Using the clickers...
ENG1. I felt that my opinions have been taken into account in this course
ENG2. In this course, my peer and faculty interactions made me feel valuable
ENG3. This course has favored my personal relationships with my peers and teachers
Learning performance (MacGeorge et al., 2008)
The use of clickers...
LP1. Has improved my comprehension of the concepts studied in class
LP2. Has led to a better learning experience in this module
LP3. Has allowed me to better understand the concepts in this module

Standardized factor
loading (>.60)

.846
.902
.855
.707

.891
.899
.900
.843

.807
.861
.856
.812

.859
.875
.726

.880
.911
.947

R2 (>.50)

CRC (>.70)

AVE (>.50)

.90

.69

.93

.78

.90

.70

.86

.68

.94

.83

.716
.814
.731
.500

.794
.790
.810
.711

.651
.742
.732
.659

.737
.765
.528

.775
.829
.896

Note: Cut-off values recommended in the literature are shown in brackets; R2: coefcient of determination; CRC: composite reliability coefcient; AVE: average variance
extracted.

L. Blasco-Arcas et al. / Computers & Education 62 (2013) 102110

107

Table 2
Descriptive statistics and correlations.

1.
2.
3.
4.
5.

Interactivity (peers)
Interactivity (teacher)
Active collaborative learning
Engagement
Learning performance

SD

5.78
5.09
5.61
5.65
5.73

.92
1.08
.94
.89
1.00

.69
.29
.15
.20
.19

.78
.24
.27
.27

.70
.61
.48

.68
.39

.83

Note: M: Mean; SD: Standard deviations. Means and standard deviations are based on scale averages. Values on the diagonal are the AVE. Off-diagonal elements are the
squared correlations among constructs.

scale items were used that ranged from 1 (strongly disagree) to 7 (strongly agree). The survey also included questions about demographic
information (age and gender).
Table 1 shows the 18 items used to measure the constructs included in our model. Interactivity with peers and the teacher was measured
using a subset of four items from Liu (2003) and McMillan and Hwang (2002). Measures of active collaborative learning were adapted from
So and Brush (2008). Engagement was measured using three items adapted from Gallini and Moely (2003) and Medlin and Green (2009).
Finally, learning performance was assessed following MacGeorge et al. (2008).
5. Analyses and results
5.1. Conrmatory analysis
Conrmatory factor analysis (CFA) using the robust maximum-likelihood estimation method was performed to test the dimensionality,
reliability and validity of the constructs in the model. The goodness-of-t indices exceeded the optimal levels recommended by Hair, Black,
Babin, Anderson, and Tatham (2006): NFI .919; NNFI .958; CFI .966; IFI .966; RMSEA .058; X2 normed 1.66, thus providing
evidence that the measurement model ts the data appropriately.
Factor loadings of the indicators for each construct were statistically signicant and sufciently large (see Table 1). Moreover, the
coefcients also had a clear relation with the underlying factor (R2 > .50). The internal validity of the measurement model was examined by
calculating the composite reliability coefcient (CRC) and the average variance extracted (AVE) (Bagozzi & Yi, 1988; Fornell & Larcker, 1981).
As can be seen in Table 1, for all factors, CRCs were above the recommended .70 and AVEs exceeded .50.
The discriminant validity of the measures was examined in two ways. First, the AVE was compared with the squared correlation among
the latent variables (Fornell & Larcker, 1981), testing whether, for any two constructs, it was always greater than the squared correlation
estimate (see Table 2). Furthermore, all the condence intervals around the correlation estimate between any two factors were tested. None
of the condence intervals included the value 1, suggesting that discriminant validity is supported (Anderson & Gerbing, 1988).
5.2. Structural model
We tested the structural relationships presented in the proposed model with the use of EQ (6.1) (see Fig. 2). The model yielded a good
overall t: NFI .896; NNFI .932; CFI .943; IFI .944; RMSEA .073; X2 normed 2.05.
The results indicate that interactivity with peers positively and signicantly inuences both collaborative learning (b1 .203, p < .01) and
engagement (b2 .130, p < .01), which provides strong support for H1 and H2, respectively. This result highlights the importance of
promoting interactivity among students in the class through the use of clickers in order to increase students engagement and their active
collaborative learning. The estimation results show that interactivity with the teacher has a positive and signicant effect on collaborative
learning and engagement (b3 .426, p < .01 and b4 .136, p < .05, respectively), providing support for H3 and H4. Thus, the higher level of
interactivity between the students and the teacher promoted by the use of clickers enhances the students level of involvement and their

0.203***
(2.56)

Interactivity with
Peers
0.130**
(2.08)

Active Collaborative
Learning

0.517***
(4.29)

LEARNING
PERFORMANCE

0.673***
(8.11)

0.426***
(5.18)

Interactivity with
Teacher

Engagement

0.221*
(1.87)

0.136**
(1.98)

Note: Standardized beta coefficients are in bold; t-value in brackets.

Levels of significance: ***p<.01; **p<.05; *p<.1
Fig. 2. Results for the proposed model.

108

L. Blasco-Arcas et al. / Computers & Education 62 (2013) 102110

collaborative learning. The indirect effects of interactivity with peers on learning performance are .16 and those of interactivity with the
teacher are .31.
We also nd that active collaborative learning and engagement positively and signicantly affect student learning performance, which
provides strong support for H5 (b5 .517, p < .01), and only weak support for H7 (b7 .221, p < .1). Consistent with H6, we nd that active
collaborative learning has a positive and signicant effect on engagement (b6 .673, p < .01). These results emphasize the critical role
played by engagement and active collaborative learning in improving student learning performance. We should also note that engagement
and collaborative learning both mediate the inuence of interactivity on learning performance. Overall, the explanatory power of the model
is .49 and active collaborative learning has the biggest impact on nal behavior (.67).
In sum, the results demonstrate that the interactivity with peers and with the teacher that results from using clickers is critical for
promoting active collaborative learning and increasing engagement. By favoring the development of closer relationships among students
and between the students and the teacher and by promoting the development of active collaborative learning and student engagement,
clickers are shown to be a powerful technological tool to enhance student learning performance.
6. Conclusions
As more and more educational institutions integrate clickers into their learning systems to enhance the learning process, it becomes
increasingly necessary to have a thorough understanding of the mechanisms behind the effect of clickers on learning performance. In spite
of recent research on the impact of clickers in the student learning process, several gaps remain that prevent us from having a complete
understanding of the phenomenon and that offer researchers new opportunities to advance in its study.
In this study, our primary objective was to identify and understand the mechanisms that underlie the effect of clickers on learning
performance. Drawing upon sound theories of student behavior and learning as well as on existing empirical research, we develop
a conceptual framework in which we propose that the interactivity with peers and with the teacher that results from the use of clickers
inuences student engagement and active collaborative learning, which ultimately determine student learning performance. We test the
framework using a sample of 198 undergraduate students of a business degree. The results of our empirical study provide strong support for
the proposed model and they enable us to contribute to existing research in several important ways.
First of all, it is important to highlight that the descriptive results from our study reveal that the mean of all the constructs is higher than
the average scale level. This nding is revealing in itself, as it shows that students perceive high levels of the constructs when using the
clickers and especially high levels of learning performance. This result also suggests that students perceive that using clickers in the class
facilitates the understanding of the concepts and class materials and signicantly improves their learning process.
A more rigorous analysis of the data using econometric techniques allows us to understand more deeply the processes behind the
positive association between clickers and learning performance suggested by previous descriptive analyses. The results from the causal
model indicate that interactivity plays a critical role in explaining the effect of clickers in student learning. By fostering student communication with their peers and teachers and promoting social and collaborative exchanges among them, clickers help students develop
communication abilities and a collaborative spirit. This may be because clickers involve students in sharing ideas, in searching for the correct
answer to questions and in explaining their decisions, all of which contributes to increasing their interactions with peers and the teacher
and, through this process, to better understand the course materials. Likewise, by using the clickers, students perceive that their answers
and opinions are taken into account by the teacher and their peers, and understand better that this process helps them improve their
learning performance. At the same time, teachers can see the percentage of students that understand the concepts, which helps them
identify any misunderstanding and enables them to clarify it properly and punctually, as well as to adapt pace, style and topic of the lecture
to better t the student needs. These results strongly recommend the use of clickers in educational contexts as a means of promoting
interactivity and of enhancing the learning experience. This technology can also help in breaking educational barriers and bringing equal
opportunities to the participants of the learning process.
Notable results from our study are the consequences of interactivity for improving and enhancing student learning performance.
Interactivity with peers and with the teacher is an important determinant of a students active collaborative learning and engagement. As
students interact with their peers and with the teacher during the use of the clickers, they feel their active role in the learning process
and perceive their contribution to building new knowledge. Active collaborative learning has proven critical in enhancing the student
learning experience, as our results indicate that it is a central determinant of learning performance and engagement (although we only
found a weakly signicant effect for this variable). This is because it allows students to think critically about the material and to
understand the alternative answers, thus achieving a deeper processing of knowledge. Reection and review require processing the
material in some depth (Lantz, 2010) and, according to our results, students consider that clickers allow them to apply these techniques in
class.
It is important to highlight that the improvement in learning performance may not be exclusively derived from the use of this technology,
but also from the development of peer group practices. We consider that the interplay of clickers and peer group practices, based on classwide discussion, enhances positive outcomes and supports the relationships between the concepts included in our conceptual framework.
Clickers allow the individual and the group to share the response to questions and allow everybody to participate in the discussion of the
correct answer. They affect the classroom dynamic through involving students in the material presented and improve the collaboration of all
the agents involved in the activity. The benets perceived by the students may be the result of their use of clickers for the development of an
active collaborative learning process. Therefore, when teachers decide to introduce clickers in the class, they must also encourage discussion
across peer groups by asking them to explain the reasons for their answers to the class.
Our study presents some limitations that need to be addressed in further research. First, our sample consists only of students who have
used clickers and, thus, we cannot compare the results that we obtain with a control group of non-users. This prevents us from ruling out
other potential explanations for our results such as the liking or afnity of the student toward the subject. Thus, a promising avenue for
further research would be to develop a quasi-experiment to test the proposed framework across two different student groups: clicker users
and non-users. This comparison can help to better understand not only the benets provided by clickers, but also the effects that the barriers
to adopt this technology would have on student learning performance.

L. Blasco-Arcas et al. / Computers & Education 62 (2013) 102110

109

Moreover, given that the marketing course in our empirical application is the pioneer in the introduction of this technology in the
university under study, we acknowledge that our results may reect, in some way, novelty effects. For all the students, in the empirical study,
this was the rst time they have used clickers. Future research should examine the effect of clickers longitudinally in order to determine
whether the effects obtained diminish with an increase in clicker experience. In addition, given that we found a weakly signicant support
for the effect of engagement on student learning performance, more research is required to understand whether this positive result can be
generalized to other educational contexts and situations, or whether there are any contextual factors (such as the mentioned novelty effect)
that may be affecting (moderating) the direction and strength of this association.
In sum, we can conclude that clickers enhance students learning performance by increasing interactivity with peers and the teacher. This
interactivity, subsequently, promotes individual active participation and collaborative learning, which increases student engagement in the
learning process. Overall, these results provide strong support for the use of clickers in the class as a tool to enhance the learning experience.
Acknowledgments
The authors would like to thank the following sources for their nancial help: projects PESUZ 10_6_117 and PIIDUZ 11_1_059 from the
University of Zaragoza, IDI projects (Ref: ECO2011-23027 and ECO2009-08283) from the Government of Spain, Catedra Telefnica, and
the project GENERES (Ref: S-09) from the Government of Aragon and the European Social Fund.
References
Ahlfeldt, S., Mehta, S., & Sellnow, T. (2005). Measurement and analysis of student engagement in university classes where varying levels of PBL methods of instruction are in
use. Higher Education Research and Development, 24(1), 520.
Anderson, T. (2003). Modes of interaction in distance education: recent developments and research questions. In M. G. Moore, & W. G. Anderson (Eds.), Handbook of distance
education (pp. 129144). Mahwah, New Jersey: Lawrence Erlbaum Associates Inc.
Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modelling in practice: a review and recommend two-step approach. Psychological Bulletin, 103(3), 411423.
Angeli, C., Valanides, N., & Bonk, C. J. (2003). Communication in a web-based conferencing system: the quality of computer-mediated interaction. British Journal of Educational
Technology, 34(1), 3143.
Bagozzi, R. P., & Yi, P. R. (1988). On the evaluation of structural equation models. Academy of Marketing Science, 16(1), 7494.
Baker, K. Q., Spiezio, K. E., & Boland, K. (2004). Student engagement: transference of attitudes and skills to the workplace, profession, and community. The Industrialorganizational Psychologist, 42(2), 101107.
Banks, D. A. (2006). Reections on the use of ARS with small groups. In D. A. Banks (Ed.), Audience response systems in higher education (pp. 373386). Hershey, PA: Information Science Publishing.
Bannan-Ritland, B. (2002). Computer-mediated communication, e-learning, and interactivity: a review of the research. Quarterly Review of Distance Education, 3(2), 161169.
Beatty, I. (2004). Transforming student learning with classroom communication systems. Boulder, Colo: EDUCASE Center for Applied Research, available. http://www.educase.
edu/LibraryDEtailPage/666?IDERB0403 3.
Beatty, I. D., Gerace, W. J., Leonard, W. J., & Dufresne, R. J. (2006). Designing effective questions for classroom response system teaching. American Journal of Physics, 74(1), 3139.
Beekes, W. (2006). The millionaire method for encouraging participation. Active Learning in Higher Education, 7, 2536.
Bergtrom, G. (2006). Clicker sets as learning objects. Interdisciplinary Journal of Knowledge and Learning Objects, 2, Available in. http://ijklo.org/Volume2/v2p105-110Bergtrom.pdf.
Brewer, C. A. (2004). Near real-time assessment of student learning and understanding in biology courses. Bioscience, 54(11), 10341039.
Bullock, D. W., LaBella, V. P., Clingan, T., Ding, Z., Stewart, G., & Thibado, P. M. (2002). Enhancing the student-instructor interaction frequency. The Physics Teacher, 40, 535541.
Caldwell, J. E. (2007). Clickers in the large classroom: current research and beast-practice tips. CBE Life Sciences Education, 6(1), 920.
Carnaghan, C., & Webb, A. (2007). Investigating the effects of group response systems in student satisfaction, learning, and engagement in accounting education. Issues in
Accounting Education, 22(3), 391409.
Chickering, A., & Ehrmann, S. C. (1996). Implementing the seven principles: technology as lever. AAHE Bulletin36, Available. http://www.tltgroup.org/programs/seven.html
October.
Chou, C. (2003). Interactivity and interactive functions in web-based learning systems: a technical framework for designers. British Journal of Educational Technology, 34(3),
265279.
Cotner, S., Fall, B., Wick, S., Walker, J., & Baepler, P. (2008). Instant feedback assessment methods: can we improve engagement, enjoyment, and preparation for exams in
large-enrollment biology courses? Journal of Science Education and Technology, 17, 437443.
Crouch, C. H., & Mazur, E. (2001). Peer instruction: ten years of experience and results. American Journal of Physics, 69(9), 970977.
Draper, S. W., & Brown, M. I. (2004). Increasing interactivity in lectures using an electronic voting system. Journal of Computer Assisted Learning, 20, 8194.
Draper, S. W., Cargill, J., & Cutts, Q. (2002). Electronically enhanced classroom interaction. Australasian Journal of Educational Technology, 18, 1323.
Elliot, C. (2003). Using a personal system in economics teaching. International Review of Economics Education, 1(1), Available in. http://www.economicsnetwork.ac.uk/iree/i1/
elliot.htm.
Erickson, J., & Siau, K. (2003). E-ducation. Communications of the ACM, 46(9), 134140.
Fies, C., & Marshall, J. (2006). Classroom response systems: a review of the literature. Journal of Science Education and Technology, 15(1), 101109.
Finn, J. D., & Rock, D. A. (1997). Academic success among students at risk for school failure. Journal of Applied Psychology, 82, 221234.
Fornell, C., & Larcker, D. (1981). Structural equation models with unobserved variables measurement error. Journal of Marketing Research, 36(3), 3950.
Fowler, L., Armarego, J., & Allen, M. (2001). CASE tools: constructivism and its application to learning and usability of software engineering tools. Computer Science Education,
11(3), 261272.
Fredricks, J. A., Blumenfeld, P. B., Friedel, J., & Paris, A. (2002). Increasing engagement in urban settings: an analysis of the inuence of the social and academic context on
student engagement. In Paper presented at the annual meeting of the American Educational Research Association. New Orleans.
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: potential of the concept, state of the evidence. Review of Educational Research, 74, 59108.
Gallini, S. M., & Moely, B. E. (2003). Service-learning and engagement, academic challenge and retention. Michigan Journal of Community Service Learning514, Fall.
Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance
Education, 15(1), 723.
Guthrie, R. W., & Carlin, A. (2004). Waking the dead: using interactive technology to engage passive listeners in the classroom. In Proceedings of the AMCIS, paper 358. New
York: August.
Guthrie, J. T., & Wigeld, A. (2000). Engagement and motivation in reading. In M. Kamil, & P. Mosenthal (Eds.). Handbook of reading research, vol. 3 (pp. 403422). Mahwah, NJ:
Lawrence Erlbaum.
Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate data analysis (6th ed.). Upper Saddle River, New Jersey: Prentice Hall.
Hake, R. R. (1998). Interactive-engagement versus traditional methods: a six-thousands student survey of mechanics test data for introductory physics courses. American
Journal of Physics, 66, 6474.
Haseman, W. D., Polatoglu, V. N., & Ramamurthy, K. (2002). An empirical investigation of the inuences of the degree of interactivity of user-outcomes in a multimedia
environment. Information Resources Management Journal, 15(2), 3148.
Higgins, R., Hartley, P., & Skelton, A. (2002). The conscientious consumer; reconsidering the role of assessment feedback in student learning. Studies in Higher Education, 27(1),
5364.
Hiltz, S. R., Coppola, N., Rotter, N., & Turoff, M. (2000). Measuring the importance of collaborative learning for the effectiveness of ALN: a multi-measure, multi-method
approach. Journal of Asynchronous Learning Networks, 4(2), 103125.
Homme, J., Asay, G., & Morgenstern, B. (2004). Utilisation of an audience response system. Medical Education, 38(5), 575.

110

L. Blasco-Arcas et al. / Computers & Education 62 (2013) 102110

Hu, J., Bertol, P., Hamilton, M., White, G., Duff, A., & Cutts, Q. (2006). Wireless interactive teaching by using key-pad ARS. In D. A. Banks (Ed.), Audience response systems in
higher education (pp. 209221). Hershey, PA: Information Science Publishing.
Kaleta, R., & Joosten, T. (2007 May 8). Student response systems: a University of Wisconsin study of clickers. Educause Center for Applied Research Bulletin, At. http://www.blog.
utoronto.ca/in_the_loop/les/ClickersERB0710.pdf 10.
Kay, R.-H., & LeSage, A. (2009). Examining the benets and challenges of using audience response systems: a review of the literature. Computers & Education, 53, 819827.
Kennedy, G., & Cuts, Q. (2005). The association between students use and electronic voting system and their learning outcomes. Journal of Computer Assisted Learning, 21, 260268.
Kennedy, D., Hyland, A., & Ryan, N. (2006). Writing and using learning outcomes: A practical guideIn The Bologna handbook, Available in. http://www.bologna.msmt.cz/les/
learning-outcomes.pdf.
Kenwright, K. (2009). Clickers in the classroom. TechTrends, 53(3), 7477.
Khan, B. H. (2000). Discussion of resources and attributes of the web for the creation of meaningful learning environments. Cyber Psychology & Behavior, 3(1), 1723.
Knight, J. K., & Wood, W. B. (2005). Teaching more by lecturing less. Cell Biology Education, 4(4), 298310.
Kreijns, K., Kirschner, P. A., & Jochems, W. (2003). Identifying the pitfalls for social interaction in computer supported collaborative learning environment: a review of the
research. Computers in Human Behavior, 19(3), 335353.
Kryder, L. G. (1999). Integrating computer literacy: why and what can be done. Business Communication Quarterly, 62(2), 8186.
Kuh, G. D. (2003). What were learning about student engagement from NSSE. Change, 35(2), 2432.
Lantz, M. E. (2010). The use of clickers in the classroom: teaching innovation or merely an amusing novelty? Computers in Human Behavior, 26, 556561.
Liu, Y. (2003). Developing a scale to measure the interactivity of websites. Journal of Advertising Research, 43(3), 207216.
Liu, T., Liang, J., Wang, H., Chan, T., & Wei, L. (2003). Embedding educlick in classroom to enhance interaction. In Proceedings international conference computers in education
(ICCE) (pp. 117125). Hong Kong, China.
MacGeorge, E. L., Homan, S. R., Dunning, J. B., Jr., Elmore, D., Bodie, G. D., Evans, E., et al. (2008). The inuence of learning characteristics on evaluation of audience response
technology. Journal of Computing in Higher Education, 19, 2546.
Marks, H. M. (2000). Student engagement in instructional activity: patterns in the elementary, middle, and high school years. American Educational Research Journal, 37(1),
153184.
Mayer, R. E., Stull, A., DeLeeuw, K., Almeroth, K., Bimber, B., Chun, D., et al. (2009). Clickers in collage classrooms: fostering learning with questioning methods in large lecture
classes. Contemporary Educational Psychology, 34, 5157.
Mayer, R. E., & Wittrock, M. C. (2006). Problem solving. In P. A. Alexander, & P. H. Winne (Eds.), Handbook of educational psychology (2nd edn.) (pp. 287304).
McMillan, S. J., & Hwang, J. (2002). Measures of perceived interactivity: an exploration of the role of direction and communication, user control and time in shaping
perceptions of interactivity. Journal of Advertising, 31(3), 2942.
Medlin, B., & Green, K. W., Jr. (2009). Enhancing performance through goal setting, engagement, and optimism. Industrial Management & Data Systems, 109(7), 943956.
Michaelson, L. K., Knight, A. B., & Fink, L. D. (2004). Team-based learning: A transforming use of small groups in college teaching. Sterling, VA: Stylus.
Nicol, D., & Boyle, J. (2003). Peer instruction versus class-wide discussion in large classes: a comparison of two interaction methods in the wired classroom. Studies of Higher
Education, 28, 457473.
Oncu, S., & Cakir, H. (2011). Research in online learning environments: priorities and methodologies. Computers & Education, 57, 10981108.
Panitz, T. (1996). A denition of collaborative versus cooperative learning. Deliberations, Available. http://www.londonmet.ac.uk/deliberations/collaborative-learning/panitzpaper.cfm (online).
Pratton, J., & Hales, L. (1986). The effects of active participation on student learning. Journal of Educational Research, 79, 210215.
Prince, M. (2004). Does active learning work? a review of the research. Journal of Engineering Education, 93, 223231.
Resta, P., & Laferrire, T. (2007). Technology in support of collaborative learning. Education Psychology Review, 19, 6583.
Ribbens, E. (2007). Why I like personal response systems. Journal of College Science Teaching, 37(2), 6062.
Roblyer, M. D., & Wiencke, W. R. (2003). Design and use of a rubric to assess and encourage interactive qualities in distance courses. American Journal of Distance Education,
17(2), 7798.
Ryan, A. M. (2000). Peer groups as a context for the socialization of adolescents motivation, engagement, and achievement in school. Educational Psychologist, 35, 101111.
Shernoff, D. J., & Hoogstra, L. (2001). Continuing motivation beyond the high school classroom. New Directions in Child and Adolescent Development, 93, 7387.
Siau, K., Sheng, H., & Nah, F. F.-H. (2006). Use of classroom response system to enhance classroom interactivity. IEEE Transactions on Education, 49(3), 398403.
Simpson, V., & Oliver, M. (2007). Electronic voting systems for lectures then and now: a comparison of research and practice. Australasian Journal of Educational Technology,
23(2), 187208.
Sims, R. (2003). Promises of interactivity: aligning learner perceptions and expectations with strategies for exible and online learning. Distance Education, 24(1), 87103.
So, H.-J., & Brush, T. A. (2008). Students perceptions of collaborative learning, social presence and satisfaction in a blended learning environment: relationships and critical
factors. Computers & Education, 51(1), 318336.
Soller, A. (2001). Supporting social interaction in an intelligent collaborative learning system. International Journal of Articial Intelligence in Education, 12(1), 4062.
Stipek, D. (2002). Good instruction is motivating. In A. Wigeld, & J. Eccles (Eds.), Development of achievement motivation. San Diego, CA: Academic Press.
Stowell, J. R., & Nelson, J. M. (2007). Benets of electronic audience response systems on student participation, learning and emotion. Teaching of Psychology, 34, 253258.
Thalheimer, W. (2003). The learning benets of questions (white papers). Somerville, MA: Work Learning Research.
Trees, A. R., & Jackson, M. H. (2007). The learning environment in clicker classrooms: student processes of learning and involvement in large university-level courses using
student response systems. Learning, Media and Technology, 32, 2140.
Wit, E. (2003). Who wants to be. the use of a personal response system in statistics teaching. MSOR Connections, 3, 1420.
Yoder, J. D., & Hochevar, C. M. (2005). Encouraging active learning can improve students performance on examinations. Teaching of Psychology, 32, 9195.
Yourstone, S. A., Kraye, H. S., & Albaum, G. (2008). Classroom questioning with immediate electronic response: do clickers improve learning? Decision Sciences Journal of
Innovative Education, 6(1), 7588.