Professional Documents
Culture Documents
Richards Middle School Reading Institute Program Evaluation Georgia Southern University Spring 2010 Betsy Beach Scott Price Casey Manning
2 Table of Contents I. II. III. Focus of the Evaluation IV. V. VI. VII. Appendices pg 16 Recommendations and Conclusions pg 14 Presentation of Results pg 11 Overview of Evaluation Plan and Procedures pg 9 pg 8 Introduction pg 6 Executive Summary pg 3
Executive
Summary
The
Reading
Institute
focuses
on
moving
the
students
who
did
not
meet
on
the
Reading
CRCT
to
the
meets
category.
Although
Richards
Middle
School
made
Adequate
Yearly
Progress
last
year,
the
school
did
not
make
AYP
in
years
prior.
Meeting
AYP
is
crucial
for
Richards
Middle
this
year
as
it
will
ensure
that
the
school
does
not
receive
N12
status.
In
years
past,
Richards
Middle
has
missed
AYP
by
very
few
percentage
points;
every
single
child
that
is
currently
enrolled
in
Reading
Institute
has
a
direct
impact
on
Richard
Middles
ability
to
make
or
not
make
AYP.
Does
the
Richards
Middle
Reading
Institute
effectively
increase
student
fluency,
comprehension,
and
vocabulary?
Using
weekly
fluency
tests,
coach
connected
post
tests,
and
domain
specific
vocabulary
acquisition
and
other
standardized
tests
administration
should
be
able
to
monitor
the
effectiveness
of
the
Reading
Institute.
With
written
reports
to
administrators
and
oral
discussions
about
results,
interpretations
about
the
program
and
possible
changes
can
be
made
to
ensure
positive
results.
What
are
the
overall
perceptions
of
the
Richards
Middle
Reading
Institute
and
is
there
sufficient
support
for
the
program?
The
program
has
been
well
funded
and
supported
by
school
administration.
Teachers
were
not
forced
to
participate
in
the
program;
all
feel
strongly
enough
about
the
program
to
give
up
their
planning
periods
and
attend
a
great
deal
of
professional
development.
The
parents
and
students
are
also
in
consensus.
Students
choose
to
attend
the
Reading
Institute
and
parents
must
agree
to
have
their
child
participate.
The
students
choose
to
give
up
a
more
exciting
connections
class
to
attend
the
Reading
Institute.
4 What are the primary benefits of the program experienced by the students? The students will gain confidence in their reading ability and pass the Reading and ELA CRCT. Hopefully, setting them up for future success in the classroom and continued growth in their reading ability. To what extent has the program been successful in meeting AYP for Richards Middle? Every child that is currently enrolled in Reading Institute has a direct impact on Richard Middles ability to make or not make AYP. Since the program is still in progress, and the results of the CRCT are not yet available, the outcome has not been determined. Evaluation Results Fluency Data Baseline Fluency Reading 96 Final Fluency Reading 139.76 Comprehension Data Coach Connected Coach Pretest Score Connected Post Test Score 60 77 Percentage Increase On or Above Grade Level Before (Y/N) 2 Y, 28 N On or Above Grade Level After (Y/N) 24 Y, 6 N Percentage Increase 46.71 On or Above Grade Level (Y/N) 26 Y, 4 N
Average
Average
36.81
5 Recommendations Hold a roundtable discussion with instructors to determine their needs. Offer a variety of professional development courses for instructors. Target the lowest students in each area, creating specific learning activities for those students. Utilize flexible grouping to continuously move students based on weekly fluency scores. Create and implement a mid-institute test to monitor comprehension.
Introduction
The
Richards
Middle
Reading
Institute
program
is
designed
to
increase
student
reading
fluency,
comprehension
and
vocabulary.
The
program
has
approximately
70
students
enrolled
and
focuses
on
moving
the
students
form
the
did
not
meet
status
on
the
Reading
CRCT
to
the
meets
category.
This
will
help
Richards
Middle
to
meet
AYP
for
this
year.
Purpose
The
evaluation
was
designed
to
determine
if
the
Middle
Reading
Institute
is
effectively
increasing
student
reading
fluency,
comprehension
and
vocabulary.
The
principle
goal
of
this
report
is
to
enlighten
on
the
following
questions:
Does
the
Richards
Middle
Institute
effectively
increase
student
fluency?
Does
the
Richards
Middle
Institute
effectively
increase
student
reading
comprehension?
Does
the
Richards
Middle
Institute
effectively
increase
student
vocabulary?
Setting up a model created at the beginning of the year, the evaluation will conclude with results of either success or failure of the program. Audience The evaluation will involve the stakeholders; school administrators, program recipients, and program coordinators. The administration directly will oversee the RMRI and make the final
7 decision on the future of the program. The teachers along with the administration, are the creators of the program, and will determine the criteria and procedures. The students and parents will be directly affected by the outcome of the evaluation. Limitations The program relies upon students actively participating and applying themselves. Students that choose to drop out of the program may not see improved reading skills or improvement on their CRCT scores. Parents and teachers that do not invest the necessary time and devotion to the program can also adversely affect the overall results of the program. Overview The following sections are included in this report: Focus of the evaluation- which includes a description of the Richards Middle Reading Institute and the questions used for the study Brief overview of evaluation plan and procedures on how the data was collected and the methods, criteria used to interpret data Presentation on evaluation result Conclusions and recommendations
The data was then collected from these test results along with descriptive statistics to determine the effectiveness of the program. For additional details refer to the Appendix.
10 B. Information Source a. Students who are in the Richards Middle School Reading Institute (program recipients) C. Information Collection Arrangements a. Information collected by RMRI instructors b. Pretest collected January 5th c. Posttest collected April 15th D. Analysis Procedures a. Descriptive statistics E. Criteria a. A score of 75 or above (Level 2) for those who scored a level 1 in 6th grade. A score of 85 or above (level 3) for those who scored a level 2 in 6th grade. b. Level 2 CRCT: 800 F. Reporting of Information a. To whom: school administrators, program recipients, and program coordinators b. Context: Help answer the question: How effective is the RMRI? c. How: Written report to administrators with oral discussion to discuss results, their interpretations, and possible changes d. When: May
11
Average
The seventh grade fluency range is between 130 and 160. The average fluency score for all students at the beginning of the Reading Institute was 96. The average fluency score for all students at the end of the Reading Institute was 139.76 The average percentage increase was 46.71%. At the beginning of the Reading Institute, 0 students had a baseline fluency score on grade level. At the end of the Reading Institute, 26 students had a fluency score on grade level.
Question 2: Does the Richards Middle Reading Institute effectively increase student reading comprehension? Coach Connected Coach Percentage On or Above On or Above Pretest Score Connected Post Increase Grade Level Grade Level Test Score Before (Y/N) After (Y/N) Average 60 77 36.81 2 Y, 28 N 24 Y, 6 N
12 Stakeholder Perceptions Student Participants At the beginning of the program, students (along with teachers) filled out goal sheets. Students made goals for fluency scores, Coach Connected scores, and grades in each class. Each week, students filled out Fluency Checklists at the end of their fluency tests. At the end of the program, students filled out comment sheets. At the end of the program, 96% of student participants believe that what they learned in this program will be valuable for [their] future. At the end of the program, 96% of student participants believe that the instructors cared about the progress of their students. At the end of the program, 100% of student participants were satisfied with the learning experiences in this program. Instructor Participants At the end of the program, instructors filled out evaluation surveys and left comments. A 75% is considered on grade level for the Coach Connected test. The average Coach Connected Pretest score was a 60. The average Coach Connected Posttest score was a 77. The average percentage increase was 36.81%. Before the Reading Institute began, 2 students scored on grade level, and 28 students score below grade level. After the Reading Institute, 24 students scored on grade level, and 6 students scored below grade level.
13 At the end of the program, 100% of instructors believe that they had adequate time to work with [their] students. At the end of the program, 96% of instructors believe that their students made progress throughout the course of this program At the end of the program, 100% of instructors believe that this program should be continued and/or expanded into next year. 50 % of instructors believe that the facilities were of good quality. 64% of instructors believe that the professional development provided [them] with the information [they] needed to adequately teach reading comprehension and fluency.
14
15 34% of the instructors believed their professional development did not aid in their ability to teach reading comprehension and fluency. Recommendations Hold a roundtable discussion with instructors to determine their needs. Offer a variety of professional development courses for instructors. Target the lowest students in each area, creating specific learning activities for those students. Utilize flexible grouping to continuously move students based on weekly fluency scores. Create and implement a mid-institute test to monitor comprehension. Based on the aforementioned data, it is the recommendation of the evaluation study that the positive impact of the Reading Institute on its participants fluency levels and reading comprehension scores justifies its continuation into the future. And overwhelming majority of the students have progressed to reading on grade level from the beginning to the current date of the Reading Institute. In order to improve this program and ensure that all targeted students are reading on grade level, it is our recommendation that the administrators of the program hold detailed conversations with instructors to determine their professional needs and needs for resources. The evaluation study also recommends that the instructors institute flexible grouping to move the students on a weekly basis, determined by their fluency scores. Although there are weekly fluency tests, there are no weekly assessments for monitoring comprehension. It is the recommendation of the study that the Reading Institute creates a comprehension tests to monitor comprehension more frequently throughout the course of the program.
16
Appendices
I.
Student
fluency
data
Student
Number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Average
Baseline
Fluency
Reading
80
94
115
104
89
96
92
104
109
76
97
92
80
90
95
115
104
113
102
86
89
94
91
99
92
85
86
94
97
120
96
Final
Fluency
Reading
130
132
169
137
138
145
125
132
141
128
142
129
140
134
149
153
138
143
157
140
136
136
123
148
139
138
133
139
149
150
139.76
Increase
Percentage
Increase
62.50
Y
40.43
46.96
31.73
55.06
51.04
35.87
26.92
29.36
68.42
46.39
40.22
75.00
48.89
56.84
33.04
32.69
26.55
53.92
62.79
52.81
44.68
35.16
49.49
51.09
62.35
54.65
47.87
53.61
25.00
46.71
Y
Y
Y
Y
Y
N
Y
Y
N
Y
N
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
N
Y
Y
Y
Y
Y
Y
Y
26
Y,
4
N
On
or
Above
Grade
Level
(Y/N)
50 38 54 33 49 49 33 28 32 52 45 37 60 44 54 38 34 30 55 54 47 42 32 49 47 53 47 45 52 30 43.76
17 II. Student Comprehension Data Student Number Coach Connected Pretest Score Coach Connected Post Test Score Increase Percentage Increase On or Above Grade Level Before (Y/N) Y Y N N Y Y Y Y Y Y Y Y Y Y Y Y N N Y Y Y N Y Y Y N Y Y Y Y 24 Y, 6 N N N N N Y N Y N N N N N N N N N N N N N N N N N N N N N N On or Above Grade Level After (Y/N)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Average
30 36 52 57 49 74 28 74 68 64 59 63 65 69 73 42 62 64 68 71 67 49 59 54 58 64 68 62 67 73 60
76 79 67 69 84 74 78 74 84 89 76 84 81 79 74 78 63 69 75 79 75 69 75 79 80 73 81 78 82 76 77
46 43 15 12 35 0 50 0 16 25 17 21 16 10 1 36 1 5 7 8 8 20 16 25 22 9 13 16 15 3 17
153.33 N 119.44 28.85 21.05 71.43 0.00 178.57 0.00 23.53 39.06 28.81 33.33 24.62 14.49 1.37 85.71 1.61 7.81 10.29 11.27 11.94 40.82 27.12 46.30 37.93 14.06 19.12 25.81 22.39 4.11
36.81 2 Y, 28 N
Yes
No
1. The instructors were available for consultation and advice when I needed to speak with them. 2. The instructors in the program inspired me to do my best. 3. The instructors in the program gave me helpful feedback on my work. 4. The instructors in the program had thorough knowledge of the content of the courses they taught. 5. The instructors were enthusiastic about the program. 6. The instructors cared about the progress of their students. 7. What I have learned in this program will be valuable for my future. 8. Overall I was satisfied with the quality of my learning experiences at this institution.
Yes
No
1. All facilities (computers, classrooms, resources) were of good quality. 2. Adequate facilities (computers, classrooms, resources) were available to use. 3. I had adequate time to work with my students during the Institute. 4. Administration was willing to work with instructors to obtain resources, facilities and appropriate time. 5. My professional development provided me with the information I needed to adequately teach reading comprehension and fluency. 6. I believe that my students made progress throughout the course of this program. 7. I believe that this program should be continued and/or expanded into next year. 8. Overall I was satisfied with facilities, resources, time, and the presentation of the program as a whole.
Yes
No