Evaluation of the Usefulness and Effectiveness of Study

Guides for Online Algebra 2 Courses

Kim Hefty
EdTech 505
M.E.T. Program
December 11, 2013

Table of Contents
Summary.................................................................................................................... 3
Program Overview...................................................................................................... 3
Program Description................................................................................................ 3
Program Objectives................................................................................................. 4
Program Components.............................................................................................. 4
Evaluation Method...................................................................................................... 5
Program Participants............................................................................................... 5
Program Procedures................................................................................................ 5
Data Sources........................................................................................................... 6
Results........................................................................................................................ 6
Pre-Test & Post-Test Results...................................................................................... 6
Survey Results......................................................................................................... 6
Interview Results..................................................................................................... 8
Discussion.................................................................................................................. 9
Project Expenses...................................................................................................... 10
Appendices.................................................................................................................. 11
Appendix A: Study Guide for Online Algebra 2 Students.......................................................11
Appendix B: Algebra 2 Pre-Test & Post-Test.....................................................................16
Appendix C: Student Survey......................................................................................... 18
Appendix D: Parent Survey I........................................................................................ 20
Appendix E: Student Interview Questions.........................................................................21
Appendix F: Parent Interview Questions..........................................................................22
Appendix G: Post Student Survey................................................................................... 23
Appendix H: Parent Post Survey.................................................................................... 25

EdTech 505

Summary
This report is an evaluation of the usage in study guides for students participating in an online
mathematics course in the Sammamish School District. [note: this district name is fictitious] The
school district is in its third year of allowing students to complete state high school graduation
requirements online. The district is exploring ways to improve student performance and
retention of material learned in a non-traditional way.
Traditionally, in a classroom, students will listen to a lecture while simultaneously taking notes,
then review through a textbook before attempting an assignment. While completing an
assignment, the students have the ability to use their lecture notes and to flip back to the
examples in the book for reference. However, in an online course, the material is solely presented
on the computer screen. Some of the information may be interactive but there is not a traditional
teacher watching the students making them take notes or even insuring they are viewing the
information on the computer screen. Students often skip past the lesson and go straight to the
online assignments. How can online programs help students read through the computer slides
interactively and improve academic performance? Study guides!
This evaluation utilized pre and post-tests, surveys, and interviews to assess whether or not study
guides would compel students to reflect on and comprehend material, actively view the material
online, improve overall grades, and create a permanent reference for future usage. The results of
this evaluation are that study guides are a perfect tool for capturing the information students need
to complete assignments, prepare for quizzes and study for exams. Study guides also provide an
easily accessible and convenient reference.

Program Overview
Program Description
In a traditional brick and mortar classroom a students classroom teacher evaluates a student. A
teacher provides instruction and guides students in content and study skills. A traditional teacher
would instruct students how and when to take notes and provide opportunities for students to ask
questions. However, students taking online courses have a much different experience. There is no
one guiding the students each step of the way. So how can a district provide support to
students? Can the usage of study guides assist the technology with this process?
The purpose of this project is to evaluate the usage and effectiveness of study guides given to
high school juniors taking an online algebra 2 course in the Sammamish School District during
the Fall of 2013. This evaluation will consist of both quantitative and qualitative data and
analysis of the usage of study guides. The key question is: Do study guides improve student
performance on online assignments and exams.
This evaluation will examine the implementation of study guides for the twelve students
currently enrolled in the online algebra 2 course. The specific provider of the course is not being

EdTech 505

evaluated. The purpose is not to criticize or complement a specific program but to see if study
guides can enhance the experience and performance of students already enrolled in online math.
Students in the Sammamish School District participating in online courses currently have
minimum requirements for their course. They are required to complete all assignments, activities
and examinations on their own. They are required to take a district supervised midterm and final.
Online students have the option to hire a district approved tutor at their own expense. A district
supervisor monitors their progress but does not provide any support or guidance. The online
course and grade appear on the students transcript and are a part of the students permanent
record. When and how a student completes the course is completely up to the student and/or
tutor, as long as the final date does not exceed the school districts calendar.
The objectives for this evaluation were developed by Sammamish Mathematics Evaluators
(SME), an evaluation firm and private tutoring company that provides academic support to
student district wide that participates in online math courses. SME is a district approved
company that works closely with the districts online coordinator. Consulting fees for the
evaluation are paid by the school district and tutoring fees are paid by the families utilizing this
company. SME has created the study guides evaluated in this report.
Program Objectives
The purpose of this evaluation is to determine whether implementing or requiring the usage of
study guides in online courses is effective. The specific objectives are:

Obligate students to reflect on and comprehend concepts

Encourage students to actively view online material

Improve academic performance of online math students

Create a permanent reference for students

Program Components
Online learning is a rapidly expanding program offered throughout the state of Washington. This
is the third year that all students in the state are allowed to complete state required high school
mathematics courses online. Since the program is relatively new, the expectations are
requirements for students are constantly evolving.
This evaluation is observing whether adding the component of study guides to online learning
will improve academic performance and individual study skills and comprehension.
The components of this evaluation include a diagnostic pre-test and post-test of math skills
related to unit 2 in algebra 2, student and parental surveys, and student and parent interviews.
The intent is to decide if study guides are beneficial to students participating in an online math
course.

EdTech 505

The study guide is 5 pages long and consists of key topics and concepts covered in the unit. The
guide is open-ended prompting the students to create notes for the unit. Currently, there are no
requirements for online students to take or make notes. Students, in theory, could completely
skip through any provided lessons and go directly to the required assignments, activities, quizzes
and tests.

Evaluation Method
Program Participants
In Sammamish School District there are currently 152 students participating in the state approved
DLD (Digital Learning) program. The state covers the costs of the program for the students as
long as it is considered to be part of a students regular schedule. For example, local high school
students take 6 classes each year, the online course must be one of the students 6 courses in order
for the state to consider it to be part of the students regular schedule. Students are allowed to
complete the course on or off campus. Students completing the course on campus are provided a
classroom and a supervisor (not a teacher). Students completing the course off-campus, which is
the majority of the students, have a free period each day to work on the course; however, they
are not allowed on campus. Many students seek the assistance of tutors such as SME and work
on the course there.
There are twelve juniors at Plateau High School currently taking online algebra 2 with the
assistance of SME. All twelve of these students were selected to participate in the evaluation of
the study guides. In addition, all of their parents also agreed to participate in the evaluation. The
online district coordinator for Sammamish School District also participated in the content of the
study guide as well as the survey and interview questions.
Program Procedures
Prior to the start of unit 2 all students were given a pre-test and a survey. All parents were also
given a survey. The students and parents were given 48 hours to complete the pre-test and
surveys. The students were then given the study guide for unit 2 developed by SME in
partnership with Sammamish School District. The students were instructed to complete the study
guide prior to completing any assignments, activities, quizzes or tests. No instructions were
provided on how to complete the study guide.
Students were given 3 weeks to complete all requirements for unit 2 (assignments, activities,
quizzes, and tests) as usual. After completing all of unit 2, students were given the post-test
[same test as the pre-test.] The students were also given another survey specific to the study
guides. In addition, each student was interviewed.
Parents were also given an additional survey relating to the study guide. Parents were given the
interview questions but were not directly interviewed. Instead parents were instructed to fill out
their responses on paper.

EdTech 505

After all of the pre and post-tests, surveys, and interviews were completed, the data was analyzed
to determine whether or not the initial outlined objectives had been met.
Data Sources
The data used for this evaluation included the pre and post-tests as well as surveys and
interviews. The pre and post-tests evaluated skills specific to the unit of study. The pre-survey
focused on the students attitudes and behaviors in their online math course. The post-survey and
interview focused on the students attitudes and opinions about the study guide. The pre-survey
for the parents focused on the observed behavior of the students in their online course. The
parents post-survey and interview focused on the observed and perceived behavior of the
students after using the study guide.
The interview questions asked about whether or not the guides were helpful and if there were any
noticeable improvements with understanding or academic performance. All questions asked the
respondents to elaborate. In addition, everyone was asked whether or not they would recommend
the usage of study guides.

Results
Pre-Test & Post-Test Results
Pre
Post

Question 1
4
11

Question 2
2
12

Question 3
4
12

Question 4
1
11

Question 5
3
11

The data above shows the number of students out of twelve correctly answering each question.
This data illustrates that the students knew very little of the topics in unit 2 prior to starting the
unit. After completing the unit the students showed remarkable improvement.
Survey Results
Student Pre-Survey

EdTech 505

Always
1
1
1

Sometimes
4
3
4

Rarely
5
6
5

Never
2
2
2

1
1
Daily

4
1
Several
Times per
Week

5
8
Couple
Times
per Week

2
2
Almost
Never

Question
I read/review the material regularly
I take notes for my online class
I review my notes and online materials
prior to beginning graded work
I review the material before I take an exam
I keep up to date so I dont have to cram
Question

How often do you view your online course

Question
How much time per week do you spend on
your online math course

1
0-4 hrs
4

6
5-9 hrs
6

5
10-14 hrs
2

1
15+ hrs

The purpose of the pre-survey to determine the study habits and attitudes of students prior to
being given the study guides. It is evident that the overwhelming majority students are not
reviewing or studying the material. In a traditional math course, students would spend
approximately 4.5 hours per week in seat time and about 4-8 hours on homework. It is evident
that the students are not spending an appropriate amount of time on their math course.
Parent Pre-Survey
The purpose of this survey was to determine the parental perception of their students study
habits. There was a clear difference between the students stated of time spent on the course vs.
parental perception of that time.
Question
My student regularly views online material
My student takes notes for their online
class
My student reviews their notes regularly
My student spends an adequate amount of
time working on their online math course

Always

Sometimes

Rarely

6
6

4
4

2
2

Dont
Know
0
0

3
2

5
8

2
2

2
0

Student Post-Survey
The purpose of the post-survey to determine the study habits and attitudes of students after being
given the study guides. There was a marked increase in the amount of time spent studying,
preparing and generally working on the students online math course while using the study
guides.

I used the study guide for this unit

The study guide helped me make notes for
my online class
I reviewed the study guide before I took the
exams or quizzes
I reviewed the study guide before I
completed assignments or activities
The study guide helped me to not cram
Question

EdTech 505

Strongly
Agree
4
4

Agree

Disagree

7
7

1
1

Strongly
Disagree
0
0

8
Daily

3
Several

1
Couple

0
Almost

Question

While using the study guide how often did

you view your online course
Question
How much time per week do you spend on
your online math course

Times
per Week
1

Never

Times per
Week
9

0-4 hrs
1

5-9 hrs
4

10-14 hrs
7

15+ hrs
0

Parent Post-Survey
The purpose of this survey was to determine the parental perception of their students study
habits after using the study guides. There was still some disconnect between the parents
perceptions vs. the actual results reported by the students, but the gap was closer. The parents
did notice an increased amount of time spent by the students on the course.
Question
My student read/review material regularly
My student take notes for my online class
My student review notes and online
materials prior to beginning graded work
My student spends an adequate amount of
time working on their online math course

Always

Sometimes

Rarely

6
6
3

4
4
5

2
2
2

Dont
Know
0
0
2

Interview Results
All twelve students were interviewed in person; for each question they were asked to elaborate.
Students were asked if they used the study guide. All of the students indicated that they had used
the study guide. Approximately three fourths of the students said they were able to complete the
entire guide without any outside help. The remaining students stated they needed some
assistance in completing the guide. The students were asked if the guide was helpful. All of the
students claimed the guide was very helpful. The students stated that it helped them to know
what to focus on while looking through the materials. Several students said that previously they
felt there was too much information in the online course and couldnt distinguish between
relevant and non-relevant material. One student commented that they had never understood what
teachers meant by take notes and the study guide was like someone telling him what to take
notes on. Students were asked if they noticed any improvement in their understanding of the
material. All but one student indicated that the guide improved their knowledge of unit 2. The
common comment was that they were able to narrow their focus to material that they were
specifically tested on. Several students commented that it felt like they were being told what was
going to be on the test. One student stated it did not improve their knowledge, but could not
elaborate. The students were also asked if they saw any improvement in their grades. Ten of the
twelve students said they noticed a dramatic improvement; their grades went from low Bs to
high Bs or As. They said that it was because they used the guide as notes and this helped them
while completing assignments and activities. It also became a great reference tool for them to

EdTech 505

use while preparing for the exam. Two of the students indicated their grades were already good.
The final question was whether or not they would recommend using study guides in the future.
Eleven of the twelve students said they would! One student thought it was too much work to do
if its not graded.
The parents were given the interview questions in paper form and asked to return them. Nine out
of twelve parents did so. All of the respondents indicated they had noticed their students using
the guides and all stated they noticed grade improvement. Many of the parents stated they were
not sure if their student understood the material better or not, but assumed so since the grades
were better. All but one parent noticed a significant increase in the amount of time spent
working on online math. Every parent said they would highly recommend using study guides in
the future. Several parents felt that it forced the students to be interactive with the material.
Some parents also felt the guides should be graded as an extra incentive to make the students do
a good job on them. Overall, the parent responses were highly favorable.

Discussion
The purpose of this study was to evaluate whether implementing or requiring the usage of study
guides in an online algebra 2 course was effective. The overwhelming evidence gathered from
students and parents suggests that all of the objectives were met. Based on the data students
spent more time on the material and showed academic improvement.
Objective one was to obligate the students to reflect on and comprehend the concepts in their
unit. Based on the pre and post-test results there was a dramatic improvement in the overall
scores on this unit. It should be noted that there may have been other factors that could have
influenced these scores.
Objective two was to encourage the students to actively view the online materials. The students
and parents survey results show that the students spent significantly more time online in order to
complete the guides. Therefore, the primary objective was met based on the survey results.
Objective three was to improve academic performance. The majority of students and parents
both indicated vastly improved academic performance while using the guides. This was
confirmed on both the post-survey and the interviews.
Objective four was to create a permanent reference for students. During the student interviews
the students indicated that the majority of them had used the guides to help them prepare for their
exams and quizzes. Although not the explicit objective, this does indicate that the guides were
used as a reference material.
Based on the data gathered it appears that study guides have met all the outlined objectives. It is
recommended that their usage be expanded to additional units, topics and courses. The guides
compelled the students to reflect on and comprehend concepts.
SMEs summative assessment is that students can create a permanent reference with study guides
that they can use in their current unit, course, and in the future. They will have compelled

EdTech 505

themselves to reflect on and absorb the material. The study guide can help reinforce what they
have learned in their subconscious through brief periodic review sessions. The study guide can
serve as a tremendous reference for problem solving and reinforcing concepts that can be used
over and over again.

Project Expenses
The project expenses for this evaluation were based on an initial 30 minute meeting with the
Sammamish School District online coordinator, Nancy Drew. Based on the meeting with Ms.
Drew it was determined that in order to properly evaluate study guides a pre and post-tests as
well as surveys and interviews were necessary. Four hours were necessary to both conduct and
score the interviews, surveys, and tests. An additional four hours were needed to analyze the
extensive data and write the included report. It was a please working with Ms. Drew and the
Sammamish School District.

EdTech 505

Appendices
Appendix A: Study Guide for Online Algebra 2 Students
Functions and Relations
Study Guide
Using this study guide will help you prepare for your exam and ensure that you learn the most
important content. Keep it in your notebook or computer and take notes as you progress through
the unit.
Define Key Terms:
1. Relation

2. Function

3. Vertical Line Test

4. Increasing Function

5. Decreasing Function

EdTech 505

6. Constant Function

7. Continuous Function

8. Discontinuous Function

9. Even Function

10. Odd Function

11. Inverse of a Relation

EdTech 505

12. What are some other letters used for function notation?

13. Horizontal Line Test

Key Concepts:
1. How is a mapping of a relation used to show ordered pairs?

2. How can you determine if a relation is a function?

3. What does f ( x ) represent when graphing functions?

EdTech 505

4. What must be true for a function to be considered an even function?

5. What must be true for a function to be considered an odd function?

6. What is meant by the term continuous function?

7. What is the process to determine if a graph is symmetric with respect to the origin?

8. What are the steps for finding the inverse of an equation?

EdTech 505

9. What line is the graph of a function or relation reflected across to find its inverse?

10. What is function notation? How is it read?

EdTech 505

Appendix B: Algebra 2 Pre-Test & Post-Test

Question 1
Which of the following relations is a function?
A. {(-2,-1),(2,5),(-2,7), (8,7)}
B. {(0,3),(2,5),(0,7), (5,12)}
C. {(2,3),(2,5),(4,7), (8,10)}
D. {(1, 4), (8, 9), (10, 11), (12, 12)}
Question 2
Determine if the function is even,odd, or neither.
f(x) = x7 + x5
A. Even
B. Odd
C. Neither
Question 3
Determine if the function continuous or discontinuous.

A. Continuous
B. Discontinuous

Question 4
To determine if the following graph is a function, which test should you use?

EdTech 505

A. Vertical Line Test

B. Horizontal Line Test
C. Neither
Question 5
Is the following function increasing, decreasing or constant?

D.
A. Increasing
B. Decreasing
C. Neither

EdTech 505

Appendix C: Student Survey

Student Pre-Survey:
Study strategies for the class
I review the online materials regularly.
always / almost always
sometimes
almost never
never even thought of it
I take notes for my online class.
always / almost always
sometimes
almost never
never even thought of it
I review my notes and online materials prior to beginning the graded work.
always / almost always
sometimes
almost never
never even thought of it
Math tests
I review the material before I take an exam
always / almost always
sometimes
almost never
never even thought of it

I keep up-to-date, so I don't have to "cram" the night before the test.
always / almost always

EdTech 505

sometimes
almost never
never even thought of it
Time Spent on Online Course
How often do you log on, view your online course
daily / almost daily
several times per week
a couple of times a week
almost never
How much time do you spend on your online math course.
Please write in approximate amount in hours ________

EdTech 505

Appendix D: Parent Survey I

Parent Pre-Survey:
Study strategies for the class
My student regularly views online materials.
always / almost always
sometimes
almost never
I dont know
My student takes notes for their online class.
always / almost always
sometimes
almost never
I dont know
My student reviews their notes regularly
always / almost always
sometimes
almost never
I dont know
My student spends an adequate amount of time working on their online math
course
always / almost always
sometimes
almost never
I dont know

EdTech 505

Appendix E: Student Interview Questions

Describe your thoughts and attitudes about the following questions.
1.

Did you use the study guide? Explain

2.

Was the study guide helpful? Explain.

3. Did you notice any improvement in your understanding of the material after using the
study guides? Explain.

4. Did you notice any improvement in your grades on assignments, quizzes, or tests?
Explain.

Would you recommend using study guides in the future? Explain.

EdTech 505

5.

Appendix F: Parent Interview Questions

Describe your thoughts and attitudes about the following questions.
1. Did your student use the study guides? Explain

2. Did you notice any improvement in your students understanding of the material after
using the study guides? Explain.

3. Did you notice your student spending more time working on their online math course
while using the study guides? Explain.

4Did you notice any improvement in your students grades on assignments, quizzes, or tests?
Explain.

5 Would you recommend using study guides in the future? Explain.

EdTech 505

Appendix G: Post Student Survey

Study strategies for the class
I used the study guide for this unit.
strongly agree
agree
disagree
strongly disagree
The study guide helped me make notes for my online class.
strongly agree
agree
disagree
strongly disagree
I reviewed the study guide before I took the exams or quizzes.
strongly agree
agree
disagree
strongly disagree
I reviewed the study guide before I completed an assignment or activity.
strongly agree
agree
disagree
strongly disagree
The study guide helped me keep up-to-date, so I didnt have to cram the
material.
strongly agree
agree
disagree
strongly disagree
The study guide helped me keep up-to-date, so I didnt have to "cram" the
night before the test.
completely agree

EdTech 505

somewhat agree
disagree
totally disagree
Time Spent on Online Course
While using the study guide, how often did you log on, view your online course
daily / almost daily
several times per week
a couple of times a week
almost never
While using the study guide how much time do you spend on your online math
course.
Please write in time in hours _______________

EdTech 505

Appendix H: Parent Post Survey

Parent Pre-Survey:
Study strategies for the class
My student regularly views online materials.
always / almost always
sometimes
almost never
I dont know
My student takes notes for their online class.
always / almost always
sometimes
almost never
I dont know
My student reviews their notes regularly
always / almost always
sometimes
almost never
I dont know
My student spends an adequate amount of time working on their online math
course
always / almost always
sometimes
almost never
I dont know

EdTech 505