Professional Documents
Culture Documents
on Student Engagement
A look at the use of a clicker-based presentation tool
in the K-12 classroom
Brichaya Shah and Benjamin Harris
4/25/2011
Contents
Executive Summary ...................................................................................................................................... 4
Introduction to the Report ............................................................................................................................. 6
Purpose of the Evaluation ......................................................................................................................... 6
Audiences for the Evaluation Report ........................................................................................................ 6
Limitations of the Evaluation .................................................................................................................... 6
Overview of Reports Contents ................................................................................................................. 6
Focus of the evaluation ................................................................................................................................. 7
Description of the Evaluation Object ........................................................................................................ 7
Evaluative Questions Used to Focus the Study......................................................................................... 7
Information Needed to Complete the Evaluations ................................................................................... 7
Brief overview of evaluation plan and procedures ....................................................................................... 8
Presentation of the evaluation results.......................................................................................................... 10
Summary of Evaluation Findings ............................................................................................................. 10
Participants and Setting ...................................................................................................................... 10
Data Sources ....................................................................................................................................... 10
Methods .............................................................................................................................................. 10
Interpretation of Evaluation Findings ..................................................................................................... 11
Qualitative Analysis ............................................................................................................................. 13
Classroom Observations ..................................................................................................................... 14
Conclusions and Recommendations ........................................................................................................... 16
Criteria and Standards Used to Judge Evaluation Object ....................................................................... 16
Judgments about Evaluation Object ....................................................................................................... 16
Recommendations .................................................................................................................................. 17
Works Cited ................................................................................................................................................. 18
Appendices.................................................................................................................................................. 19
Appendix A .............................................................................................................................................. 19
Appendix B .............................................................................................................................................. 22
Appendix C .............................................................................................................................................. 23
Teacher Interview Transcripts ............................................................................................................ 23
Appendix D .............................................................................................................................................. 24
Evaluability Checklist........................................................................................................................... 24
2
Executive Summary
The study focused on TurningPoint, a clicker based presentation tool used in the K-12 setting. It
evaluated the use of Turning Point, and determined its level of effectiveness at keeping students on
task. Using this technology required a digital classroom environment that consists of a presentation
computer, a projector, and handheld clickers. Nelson Cue (1998) who determined that there was a
significant increase of student engagement by using Personal Response Systems bound the study in
research. The study also follows other research pertaining to engaging students in conversation with the
use of technology in the classroom (Cutts, Q., et al, 2004).
Teachers usedclassroom clickers to obtain feedback from students who in ordinary circumstances
choose to shy away or disengage from classroom instruction. Learning becomes fun and all students
have a voice in the classroom. For the purposes of this study, the evaluators synonymously used the
word clickers and classroom response system developed by Turning Technologies, which is a particular
student response system used in the Clayton County Public School district to implement audience
response technologies in the digital classroom environment. The antithesis to this study would be a
teacher that uses the traditional classroom questioning approach that either favors particular students
or favors learners that are more aggressive or have personalities that are more vocal. Clicker-based
presentation toolshave the potential to embed in various technologies while prompting students to
anonymously respond with a clicker remote control; therefore,teachers can use the technology as a tool
for formative assessment while acquiring data from shy as well as aggressive learners. In order for this
to become an effective technology, students must be engaged in the learning process. Juxtaposed
against traditional lecture instruction, using TurningPoint should engage learners with its interactivity in
the learning process. Even though the technology is interactive, it cannot compete with the level of
interaction that occurs in sophisticated video game consoles. The emphasis of this evaluation focused
on student behavior and the veracity that the technology engages student learning and helps teachers
provide effective instruction. The study focused on off task or disengaged student behavior. This
behavior can include sleeping, playing with peers, daydreaming, and general distractedness. Other
significant foci of this study were to look at student collaboration and discussion behavior while using
TurningPoint, as well as the teachers ability to differentiate instruction while using the classroom
clickers. The goal of this study was to discover if TurningPoint was a viable method of combating
student disengagement, which has been determined as a key factor for low student achievement in
4
Figure 1
Information
Required
Grade-level,
level of
familiarity
with using
TurningPoint
Information
Source
Survey
K-12
Classroom
Teachers
(n=66)
Method for
Collecting
Information
Survey,
Observations,
and Interviews
By Whom
Conditions
When
Internal
Evaluator for
Surveys,
External
Evaluator for
Observations.
Electronic
Surveys,
Live
Observations,
and Video
Observations
3/1
3/31
Observations
1 Elem
1 Middle
Analysis
Procedures
Grade-level,
level of
familiarity
with using
TurningPoint
Survey
K-12
Classroom
Teachers
(n=66)
Observations
1 Elem
1 Middle
1 High
Survey,
Observations,
and Interviews
Internal
Evaluator for
Surveys,
External
Evaluator for
Observations.
Electronic
Surveys,
Live
Observations,
and Video
Observations
3/1
3/31
Reporting of Information
To Whom
How
Technical
Report to
administrators
to discuss their
interpretation
of results.
Technology
Showcase
presentation to
share with
parents,
teachers, and
students
Technical
Report due
Technical
Report to
administrators
to discuss their
interpretation
of results.
Technology
Showcase
presentation to
share with
parents,
teachers, and
students
Technical
Report due
Results
from survey
will be
imported in
Microsoft
Excel to
chart data.
Results
from
checklist
will be
charted in
Excel.
Do 80% of
responders
believe that
clickers
positively
impact student
engagement
and on-task
behavior? Do
classroom
observations
reveal positive
student
behavior?
Technology
Director,
Administrators,
Parents,
Students, and
Teachers
Results
from survey
will be
imported in
Microsoft
Excel to
chart data.
Results
from
checklist
will be
charted in
Excel.
Do 80% of
responders
believe that
clickers
positively
impact student
student
questioning
and discussion?
Do classroom
observations
reveal
discussion?
Technology
Director,
Administrators,
Parents,
Students, and
Teachers
1 High
Will the use of
TurningPoint
Clickers affect
an increase in
student
questioning
and
discussion?
Interpretation
Procedures and
Criteria
When
4/25
Presentation to
Technology
Showcase
4/25
4/25
Presentation to
Technology
Showcase
4/25
Information
Required
Grade-level,
level of
familiarity
with using
TurningPoint
Information
Source
Survey
K-12
Classroom
Teachers
(n=66)
Method for
Collecting
Information
Survey,
Observations,
and Openended
Interviews
By Whom
Internal and
External
Evaluator
Conditions
When
Electronic
Surveys,
Live
Observations,
and Video
Observations
3/1
3/31
Observations
1 Elem
1 Middle
Analysis
Procedures
Interpretation
Procedures and
Criteria
Reporting of Information
To Whom
How
Technical
Report to
administrators
to discuss their
interpretation
of results.
Technology
Showcase
presentation to
share with
parents,
teachers, and
students
Technical
Report due
Technical
Report to
administrators
to discuss their
interpretation
of results.
Technology
Showcase
presentation to
share with
parents,
teachers, and
students
Technical
Report due
Results
from survey
will be
imported in
Microsoft
Excel to
chart data.
Results
from
checklist
will be
charted in
Excel.
Do 80% of
responders
believe that
clickers
positively
impact taskrelated student
interaction and
collaboration?
Do
observations
reveal
collaboration?
Technology
Director,
Administrators,
Parents,
Students, and
Teachers
Results
from survey
will be
imported in
Microsoft
Excel to
chart data.
Results
from
checklist
will be
charted in
Excel..
Qualitative
responses
will be
analyzed in
Nvivo9
software.
Do 80% of
responders
believe that
clickers
positively
impact
differentiation
of instruction?
Technology
Director,
Administrators,
Parents,
Students, and
Teachers
When
4/25
Presentation to
Technology
Showcase
4/25
1 High
Will the use of
TurningPoint
Clickers help
teachers to
better
differentiate
instruction?
Grade-level,
level of
familiarity
with using
TurningPoint,
Qualitative
information
about
differentiation.
Survey
K-12
Classroom
Teachers
(n=66)
Observations
1 Elem
1 Middle
1 High
Survey, OpenEnded
Interviews,
and
Observations
Internal and
External
Evaluator
Electronic
Surveys
3/1
3/31
Do survey
responses
contain
references to
differentiation?
4/25
Presentation to
Technology
Showcase
4/25
Figure 2
For the remainder of the survey, there were 59 potential participants because seven participants were
disqualified by never using TurningPoint in their classrooms. It was discovered that the majority of users
either used TurningPoint in conjunction with PowerPoint or used TurningPoint with the popular
classroom management system known as Study Island.
11
91.3% of participants determined that students were either usually or very engaged when using
TurningPoint in the classroom (Figure 3). 86.3% of teachers believed that using TurningPoint has
increased either Teacher-Student or Student-Student discussion in the classroom (Figure 4). 80.7% of
teachers believed that TurningPoint had a much or great deal of improvement in student interaction and
collaboration (Figure 5). Overall, teachers believed, at 96.4%, that TurningPoint helped them to
differentiate instruction. The differentiation question encouraged teachers to explain how TurningPoint
helped or hindered the differentiation instruction (Appendix A).
Figure 4
Figure 5
12
Figure 6
Figure 7
13
Figure 8
Figure 9
Classroom Observations
Three classrooms were recorded from each grade area - elementary, middle, and high. The recordings
were submitted to an external evaluator to determine student behavior based on an engagement
checklist (Appendix B). The results of the evaluation were positive for elementary and high school levels
showing a high level of engagement to include classroom discussion and collaboration. The middle
school observation received very low scores particularly in the areas of collaboration and fun and
excitement (Figure 11).
14
5
4
3
2
1
Series1
Positive Body
Language
Consistent
Focus
Verbal
Participation
Student
Collaboration
Fun and
Excitement
Figure 10
5
4
3
2
1
Series1
Positive Body
Language
Consistent
Focus
Verbal
Participation
Student
Collaboration
Fun and
Excitement
Figure 11
15
5
4
3
2
1
Series1
Positive Body
Language
Consistent
Focus
Verbal
Participation
Student
Collaboration
Fun and
Excitement
Figure 12
The elementary observation showed the highest levels of engagement with ratings in the high and very
high categories (Figure 10). The middle school observation showed the lowest levels of engagement
with ratings of medium, low, and very low ratings (Figure 11). Finally, the high school posted three high
ratings and two very high ratings (Figure 12). The evaluators must note, however, that the middle
school observation consisted of a different teacher delivery where verbal participation, student
collaboration, and fun and excitement were discouraged. The activity was perceived to be more of a
summative assessment rather than a formative assessment.
Recommendations
The evaluators recommend that the district explore funding avenues to give teachers one-to-one access
to a classroom set of clickers. Several teachers in the evaluation expressed a need to have their own set
of clickers. Almost 70% of responders revealed that they must share or reserve a classroom set. The
evaluators recommend that teachers undergo professional development to explore ways to increase
student collaboration while using the classroom response system and explore ways to implement
TurningPoint in other applications beyond PowerPoint and Study Island. The evaluators also suggest
training for administrators so that they may know the value of student competition and verbal
interaction while using clickers in instruction.
17
Works Cited
Cue, N. (1998, December 10-12). Hong Kong SAR, China. Retrieved March 31, 2011, from A Universal
Learning Tool for Classrooms? In: Proceedings of the "First Quality in Teaching and Learning
Conference": http://celt.ust.hk/ideas/prs/pdf/Nelsoncue.pdf
Cutts, Q., Kennedy, G., Mitchell, C., & Draper, S. (2004, August 16-18). Maximizing Dialogue in Lectures
Using Group Response Systems. Retrieved March 30, 2011, from
http://www.dcs.gla.ac.uk/~quintin/papers/cate2004.pdf
18
Appendices
Appendix A
19
20
Appendix B
The instrument was adapted from Student Engagement Teacher Handbook by the International Center
for Leadership in Education.
Works Cited
Jones, R. D. (2009). Student Engagement Teacher Handbook. Retrieved March 9, 2011, from
leadered.com: http://www.leadered.com/pdf/Student%20Engage%20handbook%20excerpt.pdf
22
Appendix C
Teacher Interview Transcripts
Interview 1
I am [name omitted] 7th grade math teacher at Elite Scholars and I would just like to say that
turning point and study island both have helped my students increase their scores statewide
testing TurningPoint facilitates easy assessing and it also grades it for me so that's wonderful
less time for the teacher... Study Island just makes sure ... makes ... ensures that all everything
that we do in the classroom is aligned with the state standards so the two of them together have
been valuable to me and have made my first year as a teacher excellent.
Interview 2
[name omitted] TurningPoint is a good program to use with students keeps them... engaged
and very excited using the clickers to put in their responses as opposed to pencil and
paper.Sometimes they get tired of using pencil and paper ...They like to see the immediate results
showing how many people voted for each answer and if their answer was correct....I enjoy using
it. It helps me out a lot with the grading. ...It grades it for you. If you decide to use it as an
assessment ... if you just wanna use it as a review ... preparing for a test. It is also good so you
can see ... what skills each child is having difficulty with so that you can address those skills.The
before assessment - many different types of reports whichever one you want depending on the
detail that you want for that particular Turning Point. TurningPoint Anywhere is great because
you're not limited to just Powerpoint ... You can use Turningpoint in any program at all
..internet, Word document, Adobe file, it doesn't matter.You can use TurningPoint Anywhere
and get the same outcome.You still can pull your reports.Each clicker is attached to a student and
you can see how each student is doing individually.
Interview 3
The TurningPoint clickers have truly improved instruction simply because it allows us to
govern instruction based on data. Immediately we can see if the students have mastered the skill
by setting a baseline such as eighty percent and if less ...if the percentage drops then we know
that's a skill that we need to reteach ... It allows us to correct wrong thinking immediately and it
also engages the kids ... We can get feedback from kids we may not necessarily hear from by
show of hands ... It allows the kids who might be shy to also give us immediate feedback ... And
we also sometimes see those kids who are not apt to be verbal to get involved in the tactile nature
and the kinesthetic motions of Turning point, so it has improved all around.
23
Appendix D
EvaluabilityChecklist
Step 7. Are the stakeholders in a position to Yes the stakeholders can truly benefit from and use
use the information productively?
( If yes, go to step 8; if no, discontinue or focus on
other stakeholders who can use the information to
make decisions or take action.)
24
Yes
Yes,
Yes
No
Appendix E
Copy of the Georgia Class Keys Evaluation Document
25
Appendix F
Survey Data Results
26
27
28