Read Naturally:

A Structured Reading Intervention

Program Evaluation

FRIT 8435 Spring 2011 Georgia Southern University Dr. Randal D Carlson, Instructor

Team Members: Willony Barclay Carol Ann Dunlap Aubrey Thornton

Contents
Executive Summary ............................................................................................................................................................. 3 Recommendations .............................................................................................................................................................. 3 Introduction ........................................................................................................................................................................ 3 Focus of the Evaluation ....................................................................................................................................................... 4 Brief Overview of Evaluation Plan and Procedures ............................................................................................................ 4 Presentation of Evaluation Results ..................................................................................................................................... 5 Appendix ................................................................................................................................................................................. 6 Student AIMS Scores ........................................................................................................................................................... 6 Students in Reading EIP Receiving the Read Naturally Program ........................................................................................ 6 Read Naturally Average ...................................................................................................................................................... 8 Interview: ............................................................................................................................................................................ 8 Read Naturally Versus EIP Alone......................................................................................................................................... 9

Executive Summary
The purpose of this evaluation is to describe the effectiveness of the reading intervention program, Read Naturally. The program evaluation focused on the increase in reading fluency among students currently struggling with reading fluency, resulting in their placement in the Reading Early Intervention Program (EIP). In order to evaluate the successfulness of Read Naturally the evaluation sought to answer the following questions:

How do the benchmark scores of reading fluency among 3rd graders receiving the Read Naturally intervention program increase from the beginning of the year until the end of the year? How do third grade students in the reading Early Intervention Program (EIP) receiving the Read Naturally remediation compare to the student in reading EIP not receiving the Read Naturally intervention? What measurable growth have the target students exhibited? What unique challenges were encountered when integrating Read Naturally into the normal curriculum? What are the current instructional techniques and applications within this program?

Recommendations
While the students enrolled in Read Naturally exhibited significant gains in reading speed, the stated goals for success were not reached. Therefore, we cannot recommend the continuation of the program.

Introduction
Becoming a skilled reader is a multifaceted task which involves the mastery of many concepts, such as word recognition, reading fluency, and comprehension skills. The reading process is compiled of layered building blocks in which one must master word recognition before mastering reading fluency; consequently, they must master reading fluency before the student can understand or comprehend a story. When a student struggles with any of these tasks the reading process becomes tedious and frustrating for the reader. If a student is not reading fluently, or at a decent pace, then the attention of the student is lost thus making it nearly impossible to comprehend or understand a story. No longer can the student focus on reading for meaning, because he or she must concentrate on tackling each individual word. The Read Naturally (SE Version 2.1) program is an AYP initiative used to assist struggling readers and help develop reading fluency skills and comprehension skills. Read Naturally, Inc., was created in 2004, and this program offers a proven strategy to help struggling readers increase the pace of words read per minute, thus assisting with the comprehension or understanding of the story. A rural school in Forsyth County chose to use funds from the Parent-Teacher Association (PTA) to adopt the Read Naturally program with hopes to assist struggling readers. This school has decided to use Read Naturally in the Reading EIP program in third grade through fifth grade. This evaluation will formally assess the effectiveness in increasing reading fluency among struggling readers receiving the Read Naturally program. It

will also compare the increase in reading rate of students in Reading EIP receiving the Read Naturally program, and those in Reading EIP not receiving the program. The results will evaluate the overall effectiveness and usefulness of Read Naturally and will also highlight the opinions of the Reading EIP teachers that administer the program.

Focus of the Evaluation

The Read Naturally program is administered 5 days a week by the Reading EIP teacher. Students receiving Read Naturally will receive the intervention for a total of 45 minutes a day. During a session, the student will choose from a list of stories on their particular reading level and will first read along on the computer while the story is read aloud. Then, the program defines any difficult vocabulary words. Next, the student practices reading aloud with a recording and then tries to read the passage aloud by themselves. Finally, the administrator of the program, EIP teacher, listens to the student read aloud and notes any errors and evaluates the inflection in the students voice. The student will receive an overall score that will go towards their individualized goal. The purpose of the evaluation is to judge the effectiveness of this program and document the overall opinion of the program held by the teachers using administering it.

Brief Overview of Evaluation Plan and Procedures

The first priority of the evaluation was to obtain baseline data from the beginning of the year in order to document the growth of the students receiving Read Naturally (Appendix A). The baseline data was collected from a progress monitoring system called AimsWeb. AimsWeb testing is a way for teachers to provide benchmark scores to track students success in reading and math throughout the school year. For this evaluation, we used the reading fluency scores, or how many words read per minute on a grade level passage, in order to document the success of the Read Naturally program. The AimsWeb scores were collected once a month from September 2010 to March 2011. We were able to use these scores to judge the effectiveness of the Read Naturally program. Our main concern was to illustrate the growth of the students in reading fluency to the stakeholders and compare it to the progress in which the stakeholders hoped to discover. The stakeholders in this evaluation are the schools administration team and the PTA board members. The stakeholders agreed that an acceptable goal for increasing reading fluency would be for a third grade student to read 120 words per minute on a third grade passage by the end of the school year. The second priority of the evaluation was to compare the reading rates of struggling readers receiving the Read Naturally program to struggling readers not receiving the program (Appendix A). Again, we used the monthly benchmark scores from the AimsWeb testing to obtain this information. We wanted to find out if the Read Naturally program provided a substantial difference in the reading fluency rate compared to students simply receiving daily reading interventions and not the Read Naturally program. The goal set forth, and agreed upon by the stakeholders, in this evaluation was for the students receiving Read Naturally to increase their reading rate higher than 20% more than students not receiving Read Naturally. Our final piece of information we wanted to provide to the stakeholders is the overall views and opinions of the Read Naturally program provided by the teachers who actually use the program on a daily basis. These teachers were asked the following questions (Appendix B):

1. How many times a week do you implement RN or for what duration? 2. How convenient and user friendly is the program? 3. Have you seen a change in the students since starting the program? 4. What are the program's strengths? 5. What are the program's weaknesses? 6. What unique challenges are encountered when integrating Read Naturally with the existing curriculum? 7. What are the current instructional techniques and applications within the program? The results from this interview and opinions of the EIP teachers administering Read Naturally (Appendix B) combined with the Aims Web results (Appendix A) will provide a clear picture for the stakeholders as to whether Read Naturally is an effective and useful program.

Presentation of Evaluation Results

How do the benchmark scores of reading fluency among 3rd graders receiving the Read Naturally intervention program compare in the beginning, middle, and end of the year? The students increased their reading fluency using the Read Naturally program. However, only four of the sample of seven students in the program achieved the specified goal of 120+ words per minute of a third grade text. This equates to a 57% achievement rate. From a strict observance of our goal, since all students in Read Naturally did not reach the goal of 120, the program cannot be termed a success. How do third grade students in the reading Early Intervention Program (EIP) receiving the Read Naturally remediation compare to third grade students in reading EIP not receiving the Read Naturally intervention? The students receiving the Read Naturally program increased their reading speed by an average of 30.6 words per minute for a 44% increase from their initial score. The students enrolled only in the EIP program increased their speed by an average of 18 words per minute for a 36% improvement. The difference between the two sets of data is less than ten percent. As the goal specified that the students in Read Naturally would increase at a rate 20% greater than EIP alone, the program did not succeed. What measurable growth have the target students exhibited? All of the Read Naturally students exhibited growth. The average increase in reading speed was 30.6 words per minute compared to 18 for students not in the program. Four out of seven students reached a final AIMS score of 122 or higher compared to three students of eight in EIP alone. Despite the successful increase of speed, it was not enough to meet the program goal of 120 words per minute minimum for each student.

Work Cited
Naturally Read, Inc. (2004). Read naturally se version 2.1: Teachers guide [computer software and manual]. Retrieved from http://www.readnaturally.com/index.htm.

Appendix
Student AIMS Scores
Students in reading EIP receiving the Read Naturally Program Name Andrew Blakelyn Daniel Hannah Madison Mehmed Summer Name Briana Brooks Caleb Daniel Garrett Jonathan Logan Olav 9/16/10 56 100 78 44 57 111 103 9/16/10 18 65 70 114 90 59 101 83 10/8/10 47 114 109 53 58 109 98 10/8/10 19 67 70 114 109 87 104 87 11/5/10 46 114 74 60 48 119 115 11/5/10 13 57 67 112 75 68 109 95 12/3/10 39 100 97 60 62 120 88 12/3/10 15 49 90 122 92 83 104 83 1/7/11 45 110 114 60 58 135 98 1/7/11 23 34 124 120 91 80 103 98 2/4/11 72 112 120 74 60 140 114 2/4/11 27 98 130 125 98 117 110 91 109 3/4/11 19 50 80 109 88 80 111 88 122 3/18/11 35 68 110 124 84 123 120 92 3/4/11 63 120 110 77 67 3/18/11 72 131 136 82 80

Students in reading EIP NOT receiving the Read Naturally Program

Students in Reading EIP Receiving the Read Naturally Program

56 72 16 55 29% Fails

Initial AIMS Score Final AIMS Score WPM Improvement AVG WPM Gain 120 WPM

100 131 113 31 31% Exceeds

Initial AIMS Score Final AIMS Score AVG WPM WPM Improvement Gain 120 WPM

78 136 105 58 74% Exceeds

Initial AIMS Score Final AIMS Score AVG WPM WPM Improvement Gain 120 WPM

44 82 64 38 86% Fails

Initial AIMS Score Final AIMS Score AVG WPM WPM Improvement Gain 120 WPM

57 80 61 23 40% Fails

Initial AIMS Score Final AIMS Score AVG WPM WPM Improvement Gain 120 WPM

111 140 122 29 26% Exceeds

Initial AIMS Score Final AIMS Score AVG WPM WPM Improvement Gain 120 WPM

103 122 106 19 18% Exceeds

Initial AIMS Score Final AIMS Score AVG WPM WPM Improvement Gain 120 WPM

Read Naturally Average

30.6
Average WPM Increase from September 16 to March 18 Interview:

44%
Average Percentage Increase from September 16 to March 18

1. How many times a week do you implement RN or for what duration? 4 times a week, 30 minutes each session 2. How convenient and user friendly is the program? Very easy to use once someone has been trained properly. Doesnt demand much teacher instructional time/student works mostly independent. 3. Have you seen a change in the students since starting the program? Most students show growth in about 4-6 weeks of using the program. For those students that do not show growth, we use the RN data as a means of support for screening for other disabilities that may be present. 4. What are the program's strengths? Fluency practice, repetitive reading, choral reading, echo reading, and tracking data points 5. What are the program's weaknesses? Comprehension and the lack of instructional comprehension pieces that could be used with each non-fiction passage. 6. What Unique challenges are encountered when integrating Read Naturally with the existing curriculum? Finding time is the hardest challenge to face. Most students use RN outside of their classroom with support staff. This is the easiest option. For those teachers that use it in their classroom, they must organize time around whole group and small group lessons to implement this intervention on a regular basis. This can be hard to maintain for a long period of time. 7. What are the current instructional techniques and applications within the program? Fluency practice, repetitive reading, vocabulary practice, summarization, and prediction

Read Naturally Versus EIP Alone

Read Naturally
78.4 103.8 30.6 44% Average Score 9/16/10 Average Score 3/18/11 Average WPM Increase Average % Increase

EIP
75 79.3 18 36%