Computers & Education 56 (2011) 3643

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier.com/locate/compedu

Effective learning in science: The use of personal response systems with a wide
range of audiences
Karen Moss*, Mark Crowley
Centre for Effective Learning in Science (CELS), School of Science and Technology, Nottingham Trent University, Clifton Site, Nottingham NG11 8NS, United Kingdom

a r t i c l e i n f o

a b s t r a c t

Article history:
Received 14 December 2009
Received in revised form
30 March 2010
Accepted 31 March 2010

This paper describes the exibility of Personal Response Systems (PRSs), (also known as clickers or
electronic voting systems (EVS)), as part of strategies to support students learning in science. Whilst
variants of this technology began to appear 12 years ago, there is now a steadily increasing adoption of
these systems within higher education, including science programmes, and this use has grown signicantly in the last six years. They have previously been shown to offer a measurable learning benet.
Typically, someone at an institution buys these systems for learning support and they never make it out
of their cases. Far less work has been done with these systems at school level. In this practitioner based
paper, the broad range of practical uses for these systems is described in a variety of formal and informal
learning situations from testing the understanding of science concepts (from primary aged school
children up to physics undergraduates), to undertaking evaluation of events as well as public participation in data collection for research on attitudes to careers. In addition, the data collected on such
handsets can be mapped to demographic factors such as gender and age yielding further layers of
analysis. Overall this is a highly exible and transferable approach to the use of interactive technology for
engaging learners of all ages as well as carrying out research.
2010 Elsevier Ltd. All rights reserved.

Keywords:
Improving classroom teaching
Post-secondary education
Elementary education
Secondary education
Applications in subject areas

1. Introduction
The issue of students response to challenging scientic concepts is not news. Any staff common room in a higher education establishment provides stories of what students dont know or dont understand being swapped over cups of coffee. What can be done about this?
One option is to consider the model of content delivery used. Lectures remain a mainstay of delivery of content in UK undergraduate science
degrees and many lectures are based on a passive transmission model (Flowers, 2000) where the academic is the central gure, the sage on
the stage. In this model the lecturer has the knowledge and transmits that knowledge to the students. This process may not involve active
understanding by the students; instead it treats them as empty vessels waiting to be lled with knowledge.
Educational theory suggests that there are better ways of helping students to learn; by introducing a more constructivist based approach
whereby students build a knowledge network in their heads and learn best when they can relate what they are doing to things they already know.
Students also need to be actively engaged in the learning process through a range of activities in the classroom from simple problems to case
studies and projects. These can require a shift in the role of the academic/teacher from the sage on the stage to the guide on the side, (King,1993).
According to Bates, Howie, and Murphy (2006)
In active learning the student acts out the higher level cognitive processes of questioning, reasoning, organising and integrating within
the subject context. The inclusion of peers in the process, through discussion, generates interactivity. On a supercial level, such interactivity
can address the attention span limit and can make the lecture a more enjoyable experience for students. On a more substantive level,
engagement with the material and its underlying concepts has been shown to have a profoundly positive effect on student learning
Hake has carried out surveys with six thousand students assessing the efcacy of differing types of active learning approaches in teaching
as compared to more traditional methods of instruction and comments.
While it is now well known that large numbers of students arrive at college with large educational and cognitive decits, many faculty
and administrative colleagues are not aware that many students lost all sense of meaning or understanding in elementary school.. are in
* Corresponding author. Tel.: 44 0 115 848 8063; fax: 44 0 115 848 8077.
E-mail address: karen.moss@ntu.ac.uk (K. Moss).
0360-1315/$ see front matter 2010 Elsevier Ltd. All rights reserved.
doi:10.1016/j.compedu.2010.03.021

K. Moss, M. Crowley / Computers & Education 56 (2011) 3643

37

consonance with physics education research showing that passive student lectures, recipe labs, and algorithmic problem sets are nearly
useless in increasing students conceptual understanding of physics.(Hake, 2000)
However, exploring what students have grasped of key concepts is still illusive if you ask a large group questions in class you may get
correct answers by the keen student, but what does that tell the teacher about the other students? In an ideal world we would know what
our students are thinking and be able to adjust our teaching accordingly. Some indications may appear on test or exam papers but by then
they have moved onto the next module or course with a possibly shaky set of foundations on which to build their scientic understanding of
the world i.e. this is often too late in the learning cycle to be of use in terms of positive learning interventions for the individual. Ongoing
formative feedback is recommended, but can often add to the assessment burden of staff and students alike. Strategies that allow active
learning with immediate formative feedback on conceptual understanding would, therefore, be helpful in supporting effective student
learning.
Difculties with understanding science concepts are not, however, a purely undergraduate phenomenon, as exemplied by Tabers work
on chemistry misconceptions in secondary schools (Taber, 2002). As a university centre involved in outreach events to promote science with
young people, a wide range of age groups and different abilities are involved in our activities. This provides an additional challenge in
effectively communicating scientic concepts to such mixed and unknown audiences. One approach that can be used is to introduce
interactive technology into the learning environment for all age groups: in this case voting systems.
Members of the public have seen Personal Response Systems (PRS) used on e.g. in the game show Who wants to be a millionaire? These
systems are seen as having the wow factor when introduced to groups and users are always keen to have a go. Going beyond this fun
aspect is a piece of solid educational technology. Having been introduced to the learning potential of this technology in action, through the
work of Simon Bates (Bates et al., 2006), PRS seemed to offer an opportunity to tackle both of the above challenges. A trial of their use in
a range of learning situations and outreach activities in science was undertaken in the Centre for Effective Learning in Science (CELS). This
paper explores the uses of this technology and considers the following questions:
* Are personal response systems useful in exploring the understanding of science concepts in higher education?
* How can personal response systems be used to enhance outreach activities? [NB Did they allow session leaders to rapidly gauge the depth
of a students understanding of scientic concepts or their attitudes to science in a range of formal and informal learning situations?]
To address these questions a range of experiences with these systems are described and evaluated.
2. Methods and materials
Personal response systems are actually known by a wide range of acronyms and labels such as Electronic voting systems (EVS),
Interactive voting systems, Classroom assessment systems and clickers [NB because of the sound they make when keys are pressed.]. This
area of educational technology has grown signicantly over the last decade. In 2005, U.K market research rm DTC Worldwide, which tracks
the global market for education technology, produced data that showed that schools and universities most in the United States bought
nearly a million clickers in 2004 (Gilbert, 2005) whilst their more recent commentary on Voting Systems (Messenger, 2009) suggests
continuing optimism as to the future potential of this technology.
The worldwide market for Voting Systems showed a 36% increase year-on year with a 2009 forecast at 6 million handsets with a value of
over $320M. In USA by the end of 2009 over 8% of K-12 classrooms will have a voting system.
For a robust overview of what is available the reader is referred to the excellent online resource maintained by Steve Draper from the
Department of Psychology at the University of Glasgow (Draper, 2009) as well as an excellent survey of their use in the secondary classroom
by Falkirk Council Education Services (Falkirk, 2006). There are multiple suppliers the Falkirk review carried out comparative classroom
tests on eleven types of handsets from seven suppliers.
In general the physical PRS set up consists of:
*
*
*
*

A set of remote handsets

One teacher control
A receiver unit (commonly as a USB device)
Software to make your own slides and present them using a PC or laptop.

A critical decision to be taken by users relates to the way the handsets transmit to the receivers; these are usually either based on infrared (IR) or on radio-frequency (RF) transmitters and receivers.
The infra-red hardware, whilst the cheaper option, is reported as being less reliable, needs more receivers (e.g. one per 50 or so handsets),
and the receivers are more sensitive to positioning in a lecture theatre, if you want to collect all the votes as quickly as possible. Simon Bates
reviews their experiences at Edinburgh when they invested in such an IR system in 2005 (Bates & Howie, 2006) in his paper The use of
electronic voting systems in large group lectures: challenges and opportunities.
Radio-frequency technology has gained much ground lately as the cost of this technology has dropped in general. The range is greater
than IR signals (typical 300 m against an IR range of 25 m), enabling a greater number of participants per base unit. There is much more
exibility over positioning of the receiver (base unit) i.e. doesnt need to be in line-of-sight with the remotes as radio waves are omnidirectional. The current view of a number of reviews is that the greater cost of RF over IR should be balanced against the ease of operation and
reliability.
Two RF handsets in use at Nottingham Trent University are Qwizdom Q5 and TurningPoint ResponseCard. The former offers a wide
range of functions and data entry types whilst the latter is much simpler and cheaper.
Both systems are packaged with similar software which embeds an extra tool bar into Microsoft PowerPoint (Fig. 1). There are a wide
range of question types possible including the option of collecting demographic data about gender, age, geographical origins etc. These can
then be used later as part of the data analysis of the answers. Types of questions possible include:

38

K. Moss, M. Crowley / Computers & Education 56 (2011) 3643

Fig. 1. The Quizdom ActionPoint add-in tool bar.

*
*
*
*
*
*
*
*
*
*
*

True/False
Yes/No
Multiple choice
Numeric (decimals/fractions/negative numbers)
Multiple mark (more than 1 correct answer)
Sequence
Rating scale
Spontaneous/Verbal question
Paper based test facility
Fill in the blank
Short answer specic (text based)

The questions or problems are displayed on a screen and the students vote with their handsets. Each handset has a unique identier, so
all answers from one handset can be tracked. The data can be anonymous or traceable the software can be linked to class lists and students
can be asked to use unique codes to activate their response unit.
Additional options allow users to change their answer or limit the time period for registering the vote/answer. Once votes are all cast the
software allows you to display results immediately as graphs or numbers. One key feature of these systems is that the graphical and
numerical data from their answers can be saved and exported into a spreadsheet for further analysis. The potential for instant feedback is
enormous. Practically it helps if one person locally can become the expert and train others to use the system. Once its all been installed, the
system takes about half day to learn how to use it in basic manner.
2.1. PRS and investigating conceptual understanding of physics in higher education
PRS have become useful tools to support interactive engagement and feedback. In addition to testing students understanding or their
current knowledge, PRS can support self-reective and collaborative activities.
To what extent can we say that PRS is a useful tool for effective learning? MacArthur and Jones (2008) analysis of 56 studies that show
clear learning gains through the use of this technology (called clickers in this paper), under certain conditions. They conclude:
Peer Instruction is an effective method of fostering student collaboration that appears to be easy to adopt. For any method to be successful,
the students must believe in the potential of the clickers to improve their learning environment, and therefore a continual process of metaanalysis on the ways in which the clickers are used is appropriate. That is, the students need to understand that you are using clickers to
empower the students ability to gauge their understanding of the content, rather than in an attempt to control their behaviour.
In terms of science learning, a recent review of the use of PRS systems (or clickers) in chemistry courses observes
Clickers have a number of perceived benets that make them a promising educational tool. The primary advantage offered is the option
that they allow students to submit either identied or anonymous responses, providing opportunities for both formative and summative
assessment.. the most successful uses of clickers have been observed with applications involving both formative assessment and student
collaboration. These results are consistent with the prevailing modern educational theories of constructivism and student collaboration.
MacArthur and Jones (2008)
The near instant responses that PRS provide should enable teaching staff to adapt their sessions based on the feedback from their students.
Such systems can be seen to support community needs analysis whereby a lecturer can determine and then address the needs of their cohort.
This needs assessment approach forms part of the Test Teach Test model combining assessment for learning with assessment of learning.
In the example tested here (results in Section 3.1) the aim was to use a variant of this model to rstly identify how 31 rst year physics
students judged their own understanding of Newtonian mechanics (using a paper based questionnaire) and then compared this with their
attainment in a physics theory test delivered using PRS.
2.2. PRS in science outreach work
Voting systems have been mainly in undergraduate programmes a recent review by Kay and LeSage pointed out that few studies of
educational benets have taken place in secondary or primary arenas, (Kay & LeSage, 2009). The work of our centre involves engaging

K. Moss, M. Crowley / Computers & Education 56 (2011) 3643

39

Fig. 2. Text-based multiple choice question.

around 5000 young people (aged 519) annually with science through a wide range of Outreach activities (CELS 2009). School groups
attending activities or events have unknown levels of knowledge or conceptual understanding. Here a key application of the PRS system is to
gather a picture of their current knowledge which is done using quizzes with a range of question types. The resulting session is then adapted
to their current knowledge and then at the end their understanding is checked to see if they have improved their understanding. This
approach is built around the Test Teach Test model.
In terms of learning gains the validity of the Test Teach Test model used here has been investigated formally by the Mazur group when
they adapted their Peer Instruction (PI) method for use with PRS. In this approach, new material was presented in lectures, followed by the
students entering individual answers to a multiple choice question, discussing these answers in small groups before selecting answers again
as a group. They found that the second set of answers was more accurate, (Crouch & Mazur, 2001).
This approach is equally valid with any age group or level of academic study. It has been used it with around 200 primary school pupils
annually as part of a demonstration talk called Flames, Forces and Foams for Key Stage 2 primary school pupils (typically 811 years old). It is
equally suitable for use with post 16 students as part of university master classes in chemistry, Synthesis and Spectroscopy: here it has been

Fig. 3. Image-based sequencing question.

40

K. Moss, M. Crowley / Computers & Education 56 (2011) 3643

Fig. 4. Image-based evaluation question.

used annually with more than 120 students and has been so popular that school teachers specify the use of PRS as one of the main reasons
for attending these sessions.
PRS offers the advantage of being able to encourage large scale audience participation and to accurately gauge their understanding,
(Gilbert, 2005). In a large group it is difcult to control input from many students, to track who said what, and to assess whether everyone
actually understands the concepts. In Flames, Forces and Foams, to check their understanding, the PRS has been used to get the audience to
vote on the potential outcome of a demonstration before it happens, in this case a chip pan re.
If we were to pose questions to the group en masse we may get answers from dominant or over keen types whilst peer pressure and
embarrassment keeps the others quiet (particularly with teenage groups). PRS systems offer a very enabling way to overcome this as they:
*
*
*
*
*
*

Allow an anonymous response

Can be used to check understanding, particularly with unfamiliar people
Gauge reactions to questions or situations
Allow active participation from large audiences
Can be fun (Games included!)
Collect and analyse responses live

One of the challenges in working with school pupils is in how best to phrase a question to aid the most effective measure of conceptual
understanding. This is shown below in Figs. 2 and 3 regarding checking understanding in an activity for KS4 pupils (1416 year olds) on
Rates of Reactions in science. The question to be answered is a predication by the students of which reaction they think is fastest. In
traditional multiple choice format the answers are given in text; this requires digestion of all the words rst. However, as not all learners are
text focused one option is to exploit the full range of the PRS system for visual learners through the use of images. The images are shown in
Fig. 3. Using these images it is possible to use a different question mode of the PRS system by asking the students to rank the reactions
starting with the fastest this sort of questioning removes the guess element and explores a deeper level of understanding. By viewing all
the responses anonymously in class the teacher gets instant feedback as to whether the class has understood the theory. This then allows an
exploration of any misconceptions held by the group ensuring they have rm foundations on which to build the next stage. They can also
then check the impact of an intervention with a post-event question or test.
Anybody engaged in outreach/public engagement work will be familiar with the requirement to evaluate the impact of the activities,
which usually results in a pile of forms that are both time consuming to ll in and to analyse. In addition, when such paper feedback forms
where used with younger age groups, it was particularly felt that children as young as ve struggled with this format it became more a test
of their literacy than their opinions. PRS offered the chance to get answers to evaluation questions to the whole cohort in a short timeframe.
As part of our outreach work a PRS based quiz was devised using with graphics (See Fig. 4). This has been used with KS1 & 2 children
attending activity days at the university where they undertake a circus of hands-on science activities.
Finally, this system has been used in a range of public engagement events such as county shows and nanotechnology exhibitions. Short quizzes and data collection activities were carried out by family groups. Most recently a PRS quiz to test knowledge
of science content of careers was held at a two day Whats Next Aim higher Careers fair for 1516 year olds. (Sample results are
given in Section 3.2)
All test data and evaluation studies presented here have been subject to ethical approval processes.

K. Moss, M. Crowley / Computers & Education 56 (2011) 3643

41

Table 1
Student perception of concept difculty (Rockcliffe, 2008).
Concept area

I nd it very
difcult and do
not understand
it at all
F

Kinematics
Newton 1
Newton 2
Newton 3
Gravity

I nd it fairly
difcult and
dont think I fully
understand it

I nd it fairly easy
and have a fairly
good
understanding of it

I nd it very easy
and have a very
good
understanding of it

I dont know what

it is

2
1
2
1
2

6.5
3.2
6.5
3.2
6.5

20
22
17
20
13

64.5
71.0
54.8
64.5
41.9

5
8
12
10
16

16.1
25.8
38.7
32.3
57.6

12.9

3. Results
3.1. PRS: a tool for pedagogical research into conceptual understanding of physics
In 2006, the House of Lords Select Committee on Science and Technology Tenth Report (House of Lords, 2006), referred to a serious and
fundamental problem for science education in the perception of science & maths subjects as more difcult than other subjects. Why? There
are a lot of abstract ideas in the study of physics from which arise a number of common misconceptions, for example Newtonian Mechanics.
Newtons theories of Mechanics relate to how objects move. Newtons Laws of Motion can appear to contradict our everyday observations of
the world as they contain counter intuitive ideas. For example, Newtons rst law states:
A body at rest remains at rest and a body in motion will continue to move at a constant velocity unless acted upon by an external force.
Most peoples experience, however, is that when an object such as a table or chair is pushed along the oor, it does not continue to move
when the force pushing it is removed. The common sense explanation of this would be that a force is needed to keep objects moving. This
common sense explanation is not consistent with Newtons laws of motion, the table or chair stops moving not because the force has been
removed but due to the presence of additional frictional forces.
The pedagogical study using PRS was designed to investigate whether these problems with mechanics could be classied as a type of
misconception called troublesome knowledge based on the innovative work of David Perkins (2006). The methodology required an
investigation of the students understanding of selected concepts. These concepts were then analysed to see if they t the descriptions of

Fig. 5. An example of a multiple choice question on Newtons second law.

42

K. Moss, M. Crowley / Computers & Education 56 (2011) 3643

Table 2
PRS results from question on Newtons second law in Fig. 2 (Rockcliffe, 2008).

Valid
A
B
D
E
Total

Frequency

Percent

Valid Percent

Cumulative Percent

10
13
6
2
31

32.3
41.9
19.4
6.5
100.0

32.3
41.9
19.4
6.5
100.0

32.3
74.2
93.5
100.0

< correct answer

troublesome knowledge or the related concept of threshold concept (Meyer & Land, 2006). To probe this two methods were used: i)
a paper-based questionnaire on students perceptions of their understanding of Newtonian Mechanics and ii) a quiz using PRS systems.
Results from the questionnaires are in Table 1 (Rockcliffe, 2008).
Quiz questions were selected from a proven tool the Force Concept Inventory (Hestenes, Wells & Swackhamer, 1992), to cover Newtons
three laws, Kinematics and Gravity.
[NB The Force Concept Inventory (FCI) is an instrument designed to probe student understanding of force in its different dimensions. It
was designed for introductory physics but had been evaluated as appropriate at University level for the identication of common
misconceptions in physics students. ]
A Qwizdom pre-test with a small number of questions was used prior to the quiz to allow the students to practice using the handsets.
This ensured that all handsets were working correctly and that the students felt comfortable using them. The quiz was set up to display the
number of students that had answered each question to ensure that all students were able to use the voting devices and answered each of
the questions. The results from the question in Fig. 5 are shown in Table 2
18 Out of the 31 (58.1%) students who completed this quiz question failed to correctly apply Newtons second law (a resultant force
causes an acceleration) to this question. This result suggested that these students were not discriminating between velocity and acceleration
and therefore did not fully understand Newtons second law.
The most interesting outcome of the results, from these two methods, was that students judged their own understanding to be good in
areas that the least number of students answered correctly on in the quiz. The analysis of the full data shows that Newtons second law can
indeed be viewed as an example of troublesome knowledge as it matches four of the descriptors Perkins gives. It is conceptually difcult,
counter intuitive, difcult to link to real life situations (i.e. an example of inert knowledge), and includes concepts that may not be easily
discriminated, e.g. in this case velocity and acceleration.
The challenge of this sort of area is having appropriate questions to use with the handsets in order to test conceptual understanding.
There are banks of questions now becoming available interestingly there has been more work done in physics than in chemistry. An
overview of resources of this sort can be found in reviews, such as that of MacArthur & Jones, (MacArthur & Jones, 2008) for chemistry. Bates
has been doing extensive work in physics (Bates et al., 2006) and there are electronic question banks available e.g. through the Higher
Education Academy Physical Sciences Subject Centre in the UK (Physical Sciences Subject Centre Question Bank, 2009).
3.2. PRS: assessing attitudes with unknown school groups
As part of outreach work locally, much consideration is being given to how young people view careers in science and technology and how
careers in such areas could be better promoted. A local careers fair is organised annually for school children in 1416 age group and offered
an opportunity to use the PRS system to collect data for a pilot study on young peoples knowledge about careers, some of which include
science, and to map that data to gender and age. Of over 300 pupils who interacted with us at the event, 131 secondary pupils in year 11 (15
16 year olds) from local schools agreed to take part in the voluntary survey.
The questions were designed to provide some evidence to help better focus the careers advice given as part of outreach activities. It was
based on previous experience of asking young people informally How much science do you think there is in career X? The demographic
questions revealed that 47 males and 84 females took part i.e. twice as many females as males agreed to take part. The questions were
mostly based on 5 point Lickert-type rating scale with some multiple choice/multiple response.
In response to a multiple response question about which general career areas interested them, there was evidence of some stereotypical
behaviour eg 23% of females interested in hair/beauty cf to 4% of males. However both sexes voted medical/nursing careers as their most
popular options. (See Table 3)
In terms of their future plans over 70% of both sexes wanted to go on to study A-levels; around 14% were considering vocational programmes such as BTEC; and 6% intended to get a job. Interestingly, around 10% of both sexes were interested in apprenticeships, a rather
unexpected result.(Table 3)
In another section, they were asked how much science they thought were in various careers (on a Lickert scale of 15, with 1 being none)
For beauty related careers an intermediate rating was given (2.7 male, 3.2 female) whilst both sexes felt work in emergency services (police/
re) contained a lot of science, scoring 4. For sport based careers both sexes rated biology content as 3.9 with chemistry and physics scoring 3
and 3.3. This was a simplistic survey, which needs further development, what it did prove though was the potential of large scale survey data
capture through the use of PRS.
Table 3
Sample data from careers questionnaire with year 11 pupils.
% By area

Hairdressing/beauty

Building trades

Emergency services

Medicine/health

Sport related

Female
Male

23.3
3.9

6.7
11.8

13.3
17.7

46.7
43.1

10.0
23.5

K. Moss, M. Crowley / Computers & Education 56 (2011) 3643

43

4. Conclusion
This practitioner based study has shown that with some creative thinking PRS systems can be used by a wide range of people in a wide
range of settings. It can certainly be used with school aged pupils as readily as undergraduates. There is a learning curve initially to learning
how to use the system but half a day is sufcient to pick it up. As with all technology these systems they are not perfect and can stall
occasionally. However, once set up, these systems have been shown to be highly exible and adaptable to a wide range of learning styles.
They have enhanced our ability to deliver effective learning to young people at outreach events, evaluate activities, engage members of the
public with live experiments and they have shown their worth in carrying out research in the scholarship of learning and teaching with
undergraduate students. Their use in the educational arena is limited only by the imagination of the teacher or presenter whose hands they
are in. In particular, it is well worth going beyond the straight multiple choice questions and exploiting the full range of questions types on
offer. Our recommendation is that if your institution has a set of these devices languishing in a cupboard then they are well worth getting to
grips with in terms of the enhancement of learning they offer, the research they can do and the range of audiences they can be used with.
Acknowledgements
This work was carries out under funding from the Higher Education Funding Council for England (HEFCE) Centres for Excellence in
Teaching & Learning Initiative (CETL). We acknowledge the contributions of Anne Rockcliffe, Clare Greenall, Natasha Neale and Robert
Francis who collected and processed sections of the data at a range of events and activities. We are grateful for the loan of the Qwizdom PRS
handsets from the HEFCE funded initiative Chemistry for our Future.
References
Bates, S. P., Howie, K., & Murphy, A. St. J. (2006). The use of electronic voting systems in large group lectures: challenges and opportunities. New Directions in the Teaching of
Physical Sciences, 2, 18.
CELS. (2009). Centre for effective learning in science outreach activities. Available at www.ntu.ac.uk/cels/outreach Accessed 30.09.09.
Crouch, C. H., & Mazur, E. (2001). Peer instruction: ten years of experience and results. American Journal of Physics, 69, 970977.
Draper, S. (2009). Electronic voting systems and interactive lectures. Available at http://evs.psy.gla.ac.uk/ Accessed 28.09.09, Also JISC support available at https://www.
jiscmail.ac.uk/cgi-bin/webadmin?A0ELECTRONIC-VOTING-SYSTEMS.
Falkirk Council Education Services. (2006). Evaluation of classroom response systems/voting tools. Available at http://www.abernet.org.uk/bigdownloads/Falkirk.pdf
Accessed 29.09.09.
Flowers, W. (2000). Why change, Been doin it this way for 4000 years!. ASME mechanical engineering education conference: drivers and strategies of major program
change, Fort Lauderdale, Florida, reported in Bahner, B., (2000). Web invites new competitors into unversities realm in ASME News 19(5), May. Available at http://www.
asmenews.org/archives/backissues/may/features/educonf.html Accessed 28.09.09.
Gilbert, A. (August 5, 2005). New for back-to-school: Clickers. CNET News. Available at http://news.cnet.com/New-for-back-to-school-clickers/2100-1041_3-5819171.html
Accessed 28.09.09.
Hake, R. R. (2000). The general populations ignorance of science related societal issues:a challenge for the university. AAPT Announcer, 30(2), 105, Available at http://www.
physics.indiana.edu/whake/GuelphSocietyG.pdf Accessed 29.09.09.
Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force concept inventory. The Physics Teacher, 30 March, 141158.
House of Lords Select Committee on Science and Technology. (2006). Tenth report [online]. The United Kingdom Parliament. Available at http://www.publications.parliament.
uk/pa/ld200506/ldselect/ldsctech/257/25705.htm.
Kay, R. H., & LeSage, A. (2009). Examining the benets and challenges of using audience response systems: a review of the literature. Computers & Education, 53(3), 819827.
King, A. (1993). From sage on the stage to guide on the side. College Teaching, 41, 3036.
MacArthur, J. R., & Jones, L. L. (2008). A review of literature reports of clickers applicable to college chemistry classrooms. Chemical Education, Research and Practice, 9, 187195.
Messenger, C. (2009). Voting systems (clickers, response systems): state of the market report, abstract only. Available at www.futuresource-consulting.com/reports.htm
Accessed 29.09.09.
Meyer, J. H. F., & Land, R. (Eds.). (2006). Overcoming barriers to student understanding, threshold concepts and troublesome knowledge. Abingdon: Routledge.
Perkins, D. (2006). Constructivism and troublesome knowledge. 2006. In J. H. F. Meyer, & R. Land (Eds.), Overcoming barriers to student understanding, threshold concepts and
troublesome knowledge (pp. 3347). Abingdon: Routledge.
Physical Sciences Question Bank. (2009). Available on line at. https://secure.eps.surrey.ac.uk/psscqb/spad/psqbank.php Accessed 7.10.09.
Rockcliffe, A. (2008). The problem is, physics is difcult: An investigation of student understanding of physics concepts, PGDip Thesis, Nottingham Trent University.
Taber, K. (2002)Chemical misconceptions, Vols. 1 & 2. London, UK: Royal Society of Chemistry.