Effective Reading Programs for Middle and High Schools: A Best-Evidence Synthesis

Robert E. Slavin Johns Hopkins University -andUniversity of York Alan Cheung Hong Kong Institute of Education Cynthia Groff University of Pennsylvania Cynthia Lake Johns Hopkins University Version 1.0 August, 2007
______________ This research was funded by the Institute of Education Sciences, U.S. Department of Education (Grant No. R305A040082). However, any opinions expressed are those of the authors and do not necessarily represent IES positions or policies.

We thank Michele Victor, Lucretia Brown, and Susan Davis for their help with the review, and John Nunnery, Carole Torgerson, and Jon Baron for comments on an earlier draft.

Abstract
This article systematically reviews research on the achievement outcomes of three types of approaches to improving the reading achievement of students in middle and high schools: Alternate reading textbooks, computer-assisted instruction (CAI), and instructional process programs. Study inclusion criteria included use of randomized or matched control groups, a study duration of at least 12 weeks, and valid achievement measures independent of the experimental treatments. A total of 26 studies met these criteria. No studies of textbooks qualified. Across 12 studies of CAI, the median effect size was +0.23. Across 14 studies of instructional process programs, the median effect size was +0.18, with particularly strong evidence for three similar cooperative learning programs: The Reading Edge, Student Team Literature, and Student Team Reading (median ES = +0.33). The review concludes that programs designed to change daily teaching practices or supplement instruction with technology have substantially greater research support than those that focus on curriculum alone.

The middle and high school years are a time of great promise as well as great peril. Unfortunately, in high-poverty schools, there often seems to be more peril than promise. In particular, students who enter high school with poor literacy skills face long odds against graduating and going on to postsecondary education or satisfying careers. The Alliance for Excellent Education (2002, 2003) reports that roughly 6 million secondary students read well below grade level and that every day across the United States approximately 3,000 students drop out of high school. The secondary years provide a last chance for many at-risk students, who must quickly improve their reading skills to be able to succeed in their demanding courses (Biancarosa & Snow, 2004). Further, the problems of low achievement in secondary reading are most acute among disadvantaged and minority students. On the 2005 National Assessment of Educational Progress (NAEP, 2005), 43% of eighth graders who qualified for free lunch scored below the basic level, compared to only 19% of nonpoor eighth graders. Among African-American eighth graders, 48% scored below basic, and among Hispanic students, 44% scored this poorly. Only 18% of white students scored below basic. This situation is not new, but by any standard, the reading scores of disadvantaged and minority students in middle and high schools are unacceptable, as this poor performance translates directly into the high dropout rates and diminished future potential that are all too common for adolescents in highpoverty schools. Even among students who do graduate from high school, inadequate reading skill is a key impediment to success in postsecondary education (American Diploma Project, 2004). Students who struggle with reading are often blocked from taking the
4

academically challenging coursework that could lead to more wide reading and thus exposure to advanced vocabulary and content ideas (Au, 2000). The 2006 report by ACT, Reading Between the Lines: What the ACT Reveals About College Readiness in Reading, describes even more troubling trends. Only 51% of ACT-tested students were ready for college-level reading demands. Minority students and students whose families earned less than $30,000 per year were up to two and a half times less likely to be ready for college-level reading and success in basic college courses (ACT, 2006). While almost half of disadvantaged and minority students score below the basic level on NAEP, even those who do read at the basic level have difficulty understanding the increasingly complex narrative and expository texts they are faced with in high school and beyond. For example, one of the major hurdles in acquiring science literacy is the conceptual density of math and science materials (Barton et al., 2002). Performance on these more difficult texts, which include context-dependent vocabulary, concept development, and graphical information, is the clearest differentiator between students who are ready to succeed in college and work and those who are not (ACT, 2006). Clearly, there is a need for well-evaluated programs capable of enabling middle and high school students with poor reading skills to develop the facility they need with complex text to succeed in their high school coursework and to graduate ready for college and work-related reading demands. Improvements in reading skills are needed in all schools, of course, but the crisis is in schools serving many poor and minority students.

Due in large part to accountability programs focusing on reading, U.S. schools are increasingly providing instruction in reading to a large proportion of middle and high school students. Once seen only in remedial or special education programs, reading courses are now common in middle schools, and remedial courses are becoming more widespread in high schools. Yet there is little understanding of which particular programs are likely to be effective in middle and high schools. Remarkably, a comprehensive review of the research on middle and high school reading programs has never been done. The federal What Works Clearinghouse is doing a review of research on elementary reading programs but does not even have secondary reading in its long-term plans. The purpose of the present article is to review research on middle and high school reading programs, applying consistent methodological standards to the research. The scope of the review includes all types of programs that teachers, principals, or superintendents might consider as a solution to their secondary students reading problems: textbooks, computer-assisted instruction, and professional development programs. The review is not limited to high-poverty schools, but looks at all sources of evidence for programs that might be applied by schools to help students improve their reading performance. The review uses a form of best evidence synthesis (Slavin, 1986), adapted for use in reviewing what works literatures in which there are generally few studies evaluating each of many programs (see Slavin, 2007). Similar methods have been used previously to review research on elementary mathematics programs (Slavin &

Lake, 2006), middle and high school mathematics (Slavin, Lake, & Groff, 2007), and reading programs for English language learners (Cheung & Slavin, 2005). Even though they involved a different subject, the two mathematics reviews (Slavin & Lake, 2006; Slavin, Lake, & Groff, 2007) provide important background for the current review. In both cases, median effect sizes across many qualifying studies were quite low for textbook programs as different as NSF-funded constructivist programs such as Everyday Mathematics and the algorithmic Saxon Math. Median effect sizes for alternative textbooks were +0.05 for elementary studies and +0.07 for middle/high studies. Larger but still modest effects were found in both reviews for computer-assisted instruction programs such as Jostens/Compass Learning and SuccessMaker. Median effect sizes were +0.19 for elementary studies, +0.16 for middle/high. The largest impacts in both reviews were for instructional process programs, such as cooperative learning, classroom motivation and management programs, and other approaches focused on changing teacher and student behaviors in daily lessons. Median effect sizes for cooperative learning, for example, were +0.29 for elementary studies and +0.32 for middle/high. Studies of these instructional process programs were also more likely to have used random assignment to treatments. The Cheung & Slavin (2005) review of research on (mostly elementary) studies of reading programs for ELLs also found that effective programs were ones that emphasized professional development and changed classroom practices, such as cooperative learning and comprehensive school reform. Recognizing that reading is not the same as math, and secondary reading is not the same as elementary, we nevertheless hypothesized that in secondary reading,
7

programs focusing on reforming daily instruction would have stronger impacts on student achievement than would programs focusing on innovative textbooks or computer-assisted instruction alone.

Focus of the Current Review The present review uses procedures essentially identical to those used in the mathematics reviews to review research on reading programs for middle and high schools, grades 6-12 (sixth graders appear in the current review if they were in middle schools). The purpose of the review is to place all types of programs intended to enhance the reading achievement of middle and high school students on a common scale, both to inform theory in a consistent way and to provide educators with meaningful, unbiased information that they can use to select programs most likely to make a difference with their students. The review also seeks to identify common characteristics of programs likely to make a difference in student reading achievement. This synthesis was intended to include all kinds of approaches to reading instruction, and groups them in three categories: reading curricula, computerassisted instruction, and instructional process programs. Reading curricula focus primarily on alternative textbooks. Computer-assisted instruction (CAI) refers to programs that use technology to enhance reading achievement. CAI programs can be supplementary, as when students are sent to computer labs for additional practice, or they can be core, substantially replacing the teacher with self-paced instruction on the computer. CAI is the one category of secondary reading programs that has been reviewed in the past, as a few secondary reading studies were included in reviews by
8

Kulik (2003), Murphy et al. (2002), and Chambers (2003). The third category, instructional process programs, is the most diverse. All programs in this category rely primarily on professional development to give teachers effective strategies for teaching reading. These include programs focusing on cooperative learning, strategy instruction, and comprehensive school reform.

Review Methods The review methods are the same as those used by Slavin & Lake (2006) and Slavin, Lake, & Groff (2007), who used a technique called best evidence synthesis (Slavin, 1986), which seeks to apply consistent, well-justified standards to identify unbiased, meaningful information from experimental studies, discussing each study in some detail, and pooling effect sizes across studies in substantively justified categories. The method is very similar to meta-analysis (Cooper, 1998; Lipsey & Wilson, 2001), adding an emphasis on description of each studys contribution. It is also very similar to the methods used by the What Works Clearinghouse (2007), with a few exceptions noted in the following sections. (See Slavin (2007) for an extended discussion and rationale for the procedures used in all of these reviews).

Literature Search Procedures A broad literature search was carried out in an attempt to locate every study that could possibly meet the inclusion requirements. This included obtaining all of the middle and high school studies cited by other reviews of reading programs, including technology programs that teach reading (e.g., Chambers, 2003; Kulik, 2003; Murphy
9

et al., 2002). Electronic searches were made of educational databases (JSTOR, ERIC, EBSCO, PsychInfo, Dissertation Abstracts), web-based repositories, and education publishers websites. Besides searching by key terms (e.g. reading, intervention; reading, program), we conducted searches by program name and attempted to contact producers and developers of reading programs to check whether they knew of studies that we had missed. Citations of studies appearing in the studies found in the first wave were also followed up. Unlike the What Works Clearinghouse, which excludes studies more than 20 years old, studies meeting the selection criteria were included if they were published from 1970 to the present, as several high-quality studies of direct relevance to todays schools took place in the 1970s and early 80s.

Effect Sizes In general, effect sizes were computed as the difference between experimental and control individual student posttests after adjustment for pretests and other covariates, divided by the unadjusted posttest control group standard deviation. If the control group SD was not available, a pooled SD was used. Procedures described by Lipsey & Wilson (2001) and Sedlmeier & Gigerenzor (1989) were used to estimate effect sizes when unadjusted standard deviations were not available, as when the only standard deviation presented was already adjusted for covariates or when only gain score SDs were available. School- or classroom-level SDs were adjusted to approximate individual-level SDs, as aggregated SDs tend to be much smaller than individual SDs. If pretest and posttest means and SDs were presented but adjusted

10

means were not, effect sizes for pretests were subtracted from effect sizes for posttests.

11

Criteria for Inclusion Criteria for inclusion of studies in this review were as follows. 1. The studies evaluated programs for middle and high school reading. Studies of variables, such as use of ability grouping, block scheduling, or single-sex classrooms, were not reviewed. 2. The studies involved middle and high school students in grades 7-12, plus sixth graders if they were in middle schools. 3. The studies compared children taught in classes using a given reading program to those in control classes using an alternative program or standard methods. 4. Studies could have taken place in any country, but the report had to be available in English. 5. Random assignment or matching with appropriate adjustments for any pretest differences (e.g., analyses of covariance) had to be used. Studies without control groups, such as pre-post comparisons and comparisons to expected scores, were excluded. Studies in which students selected themselves into treatments (e.g., chose to attend an after-school program) or were specially selected into treatments (e.g., gifted or special education programs) were excluded unless experimental and control groups were drawn after selections were made. 6. Pretest data had to be provided, unless studies used random assignment of at least 30 units (individuals, classes, or schools) and there were no indications of initial inequality. Studies with pretest differences of more than 50% of a standard deviation were excluded because, even with analyses of covariance,
12

large pretest differences cannot be adequately controlled for as underlying distributions may be fundamentally different (Shadish, Cook, & Campbell, 2002). 7. The dependent measures included quantitative measures of reading performance, such as standardized reading measures. Experimenter-made measures were accepted if they were comprehensive measures of reading, which would be fair to the control groups, but measures of reading objectives inherent to the program (but unlikely to be emphasized in control groups) were excluded. The exclusion of measures inherent to the experimental treatment is a key difference between the procedures used in the present review and those used by the What Works Clearinghouse. 8. A minimum treatment duration of 12 weeks was required. This requirement is intended to focus the review on practical programs intended for use for the whole year, rather than brief investigations. Brief studies may not allow programs to show their full effect. On the other hand, brief studies often advantage experimental groups that focus on a particular set of objectives during a limited time period while control groups spread that topic over a longer period. Appendix 1 lists studies that were considered but excluded according to these criteria, as well as the reasons for exclusion. Appendix 2 lists abbreviations used throughout the review.

13

Categories of Research Design Four categories of research designs were identified. Randomized experiments (RE) were those in which students, classes, or schools were randomly assigned to treatments, and data analyses were at the level of random assignment. When schools or classes were randomly assigned but there were too few schools or classes to justify analysis at the level of random assignment, the study was categorized as a randomized quasi-experiment (RQE) (Slavin, 2007). Several studies claimed to use random assignment because students were assigned to classes by a scheduling computer, but scheduling constraints (such as conflicts with advanced or remedial courses taught during the same period) can greatly affect such assignments. Studies using scheduling computers or other random-appearing assignment methods under the control of school administrators were categorized as matched, not random. Matched (M) studies were ones in which experimental and control groups were matched on key variables at pretest, before posttests were known, while matched post-hoc (MPH) studies were ones in which groups were matched retrospectively, after posttests were known. For reasons described by Slavin (2007), studies using fully randomized designs are preferable to randomized quasi-experiments, but all randomized experiments are less subject to bias than matched studies. Among matched designs, prospective designs were strongly preferred to post-hoc or retrospective designs. In the text and in tables, studies of each type of program are listed in this order (RE, RQE, M, MPH). Within these categories, studies with larger sample sizes are listed first. Therefore, studies discussed earlier in each section should be given greater weight than those listed later, all other things being equal.
14

Reading Curricula No studies of secondary reading curricula met the standards of this review.

Computer-Assisted Instruction The effectiveness of computer-assisted instruction has been extensively debated over the past 20 years and there is a great deal of research on the topic. Kulik (2003) concluded that research did not support use of CAI in elementary or secondary reading, although Chambers (2003) came to a somewhat more positive conclusion. A large study of technology immersion, in which Texas middle schools received laptops for every student, extensive software, and significant amounts of professional development, found no significant effects on reading or math achievement in comparison to schools with ordinary levels of technology (Texas Center for Educational Research, 2007). A large randomized evaluation of various computer software programs by Dynarski et al. (2007) found no effects on the reading achievement of first and fourth graders or the math achievement of sixth graders or students taking algebra. None of these studies or reviews focused specifically on secondary reading, but they nevertheless provide context for the review of effects of CAI on reading in middle and high schools. Twelve studies of computer-assisted instruction (CAI) met the standards for the review. These were divided into three categories. Core CAI programs, such as Read 180, were those intended for use as the primary approach to reading instruction. These programs also typically incorporated non-technology elements. Supplemental
15

CAI programs, such as Compass Learning and other integrated learning systems, were ones that provided additional instruction at students assessed levels of need to supplement traditional classroom instruction. Computer-Managed Learning Systems included only Accelerated Reader. This program uses computers to assess students reading levels, assign reading materials at students levels, score tests on those readings, and chart students progress, but students do not work directly on the computer. Descriptions and outcomes of all studies of CAI in secondary reading that met the inclusion criteria appear in Table 1. ================ TABLE 1 HERE ================ Core CAI Programs Read 180 Read 180 is an intervention program for upper-elementary, middle, and high school students who are struggling in reading. It was originally developed by Hasselbring & Goin (2004) at Vanderbilt University and is currently marketed by Scholastic. Stage B, for students in grades 6 and above reading at grade levels 1.5 to 8, provides students with 90 minutes a day of instruction in groups of 15. Each period begins with a 20-minute shared reading and skills lesson, and then students in groups of 5 rotate among three activities: computer-assisted instructional reading, modeled or independent reading, and small-group instruction with the teacher. The Read 180 software includes videos, mostly on science and social studies topics, and students
16

read about the videos and engage in comprehension, vocabulary, fluency, and word study activities around them. In addition, audiobooks model comprehension, vocabulary, and self-monitoring strategies used by good readers. Students read leveled paperbacks in many genres. Teachers are given materials and professional development to support instruction in reading strategies, comprehension, word study, and vocabulary. A large matched evaluation of Read 180 was carried out in Little Rock, Arkansas middle and high schools by Mims, Lowther, Strahl, & Nunnery (2006), who were third-party evaluators. About 1000 mostly African-American students in 5 middle and 5 high schools used Read 180. They were individually matched with a non-Read 180 student in the same school and grade level on 2005 ITBS Reading scores and demographics. Spring, 2006 scores on ITBS Reading and Arkansas Benchmark exams were used as outcome measures. On ITBS, differences favored the control group at all grade levels: 6 (ES=0.15), 7 (ES=-0.23), 8 (ES=-0.12), and 9 (ES=-0.16), for an overall mean of ES=0.17. Differences were only statistically significant at grades 7 and 9, however. On the Arkansas Benchmarks, patterns were similar. Effect sizes were -0.19 at grade 6, -0.05 at grade 7, and +0.02 at grade 8, for an overall mean of ES=-0.07. Averaging ITBS and benchmark effect sizes gives an overall effect size of -0.12. White, Haslam, and Hewes (2006), under contract with the publisher, carried out a large-scale evaluation of Read 180 in the Phoenix (AZ) Union High School District. Low-achieving students using Read 180 across the district were matched with non-participants using propensity matching, and were nearly identical on pretest
17

measures (SAT-9). There were two cohorts that had control groups: 1) ninth graders in 2003-04, most of whom were followed for 1 years, and 2) ninth graders in 200405. The 2003-04 cohort used Read 180 for a full year and then most students continued for an additional semester. The 2004-05 group was tested after a semester. At the end of the 2003-04 school year, Cohort 1 students who experienced Read 180 scored 1.3 NCEs higher on SAT-9 than controls (ES=+0.12, p<.05). Effects were particularly positive for ELLs (ES=+0.32). However, the Cohort 1 ninth graders who continued on to tenth grade had scores identical to those of non-participants on the states AIMS reading test (ES=.00, n.s.). Among these 1 1/2 -year participants, directionally positive effects were seen for ELLs (ES=+0.28) and low achievers (ES=+0.14), but not for students with pretest scores above 35 NCEs (ES= -0.11). Ninth graders in Cohort 2 scored 2.9 NCEs higher than controls on Terra Nova (ES=+0.24, p<.05). Again, effects were particularly positive for ELLs (ES=+0.41) and low achievers (ES=+0.30) but not for students scoring above 40 NCEs at pretest (ES=+0.05). Averaging effect sizes across the SAT-9 and AIMS outcomes for Cohort 1 and Terra Nova outcomes for Cohort 2 yields an effect size of +0.12 overall and +0.34 for ELLs. Papalwis (2004) carried out a study of low achieving, mostly Hispanic 8 th graders in a large urban district. Most students had been retained and about half were ELLs. The study compared students using Read 180 to well-matched comparison students from across the district. Read 180 students made substantially greater gains on SAT-9 Reading (ES=+0.68, p<.05).

18

The Council of Great City Schools and Scholastic commissioned an evaluation of Read 180 in three urban districts (Interactive, 2002). The study focused on Grade 6 in Boston, Grade 8 in Dallas, and Grades 7 and 8 in Houston. In each case, Stanford 9 was administered as a pre- and posttest. Students in schools using Read 180 were compared to those in schools not doing so, with students matched on pretests and demographic factors. Across the three cities, there were 387 students in Read 180 and 323 in the control group. On adjusted posttests, effect size averaged +0.24, p<.001. Haslam, White, & Klinge (2006) evaluated Read 180 in the Austin (TX) Independent School District. Low-achieving seventh and eighth graders using Read 180 district-wide were matched on demographic factors and TAKS pretests with a control group. At posttest, adjusting for pretests, students who had experienced Read 180 gained 1.9 NCEs more than controls (ES=+0.18, p<.05). Across five studies of Read 180, the median effect size was +0.18.

Supplemental CAI Jostens Two studies in rural schools evaluated the Jostens integrated learning system. Roy (1993) evaluated the program in a junior high and a middle school in different rural areas of Texas. Both served primarily Anglo populations. At Hallsville Middle School, 150 seventh and eight graders using Jostens were matched with 150 control students. Adjusting for pretests, there were nonsignificant NAPT effects among seventh graders (ES=+0.10, n.s.) and eighth graders (ES=+0.04 n.s.), for a mean ES = +0.04. At Midway Junior High, there were 54 sixth graders using Jostens matched
19

with 54 controls. There were significantly positive effects on NAPT Reading (ES = +038, p< .05). The mean ES across the two schools was +0.21. Hunter (1994) evaluated Jostens in grades 2-8 reading and math in rural Jefferson County, Georgia. The reading evaluation in grades 6-8 is described here. Chapter 1 students who received 30-minute daily sessions with Jostens for 28 weeks were compared to those who did not receive CAI. Three experimental and three control schools were compared. Fifteen students at each grade level were randomly selected for measurement. Effect sizes were estimated at +0.37 for sixth grade, +0.37 for seventh grade, and +0.19 for eighth grade, for a mean of +0.31. Across the two studies of Jostens, the median effect size was +0.26.

Other Supplemental CAI Metrics Associates (1981) carried out an evaluation of the use of a variety of supplemental CAI programs in six Massachusetts school districts. Two of the districts, Billerica and Woburn, included junior high schools (grades 7-9) in the experiment. In one school in each district, Title I students in the CAI conditions (N=70) spent 10 minutes of their daily 30-minute remedial reading period using drilland-practice courseware. Matched students (N=35) participated in 30-minute remedial classes without CAI. Students were pre- and posttested on the MAT. Adjusted posttests indicated an effect size of +0.56, p<.001.

Computer-Managed Learning Systems Accelerated Reader

20

Accelerated Reader is a supplemental program in which students reading levels are assessed on a computer, which then prints out reading suggestions at students levels. Students read books or other materials, and then take tests on the computers to show their comprehension of what they have read. Students can earn recognition or rewards based on reading many books and passing tests on them. A small matched study by Hagerman (2003) evaluated Accelerated Reader with sixth graders in a suburban middle school near Portland, Oregon. Students in the middle school using Accelerated Reader (N=64) were compared with matched students in another middle school in the same district (N=57). Students were pre- and posttested on the TORC-3. On posttests adjusted for pretests, the Accelerated Reader group scored significantly higher (ES=+0.53, p<.001). A small study by Vollands, Topping, & Evans (1999) evaluated Accelerated Reader in high-poverty schools in Scotland. Sixth graders (N=27) in one class that used AR for six months were compared to 12 children of the same age in a control class. Adjusting for pretests, AR students gained more on the Shortened Edinburgh Comprehension Test (ES=+0.51), the Neale Accuracy Test (ES=+0.33), and the Neale Comprehension Test (ES=+0.64), for a mean ES of +0.49. By far the largest evaluations of Accelerated Reader in grades 6-8 were carried out in two Mississippi districts, Pascagoula and Biloxi. Data on two cohorts of students were analyzed by third-party evaluators. Ross & Nunnery (2005) compared one-year gains for schools using Accelerated Reader (N=2106) to those in matched schools using traditional methods (N=1124). The schools using Accelerated Reader were also using Accelerated Math. The following year, the same comparisons were
21

made in the same schools by Ross, Nunnery, Avis, & Borek (2005), with Ns of 2419 for Accelerated Reader and 1666 for control. Some students were of course in their treatment groups for both years, but the data are presented as two cross-sectional studies, not a longitudinal study. On Mississippi Curriculum Test (MCT) Reading scores, adjusted for pretests, effect sizes for the 2003 cohort were +0.11 for sixth grade, +0.16 for seventh grade, and +0.12 for eighth grade, for a mean of +0.13 (p<.05). For the 2004 cohort, effect sizes were -0.04 for sixth grade, +0.04 for seventh grade, and +0.10 for eighth grade, for a mean of +0.03 (n.s.). Combining across both cohorts, the effect size was +0.08. The median effect size across all four studies of Accelerated Reader was +0.31, but the two large studies reported much smaller effect size estimates, averaging ES=+0.08.

Conclusions: Computer-Assisted Instruction A total of 12 studies evaluated various forms of computer-assisted instruction. Overall, the median effect size was +0.23. This is similar to the median effect size of +0.18 for CAI in secondary mathematics reported by Slavin, Lake, & Groff (2007). Previous reviews of research on CAI (e.g., Kulik, 2003) found greater effects in math than in reading, so this is a surprising finding. However, the amount and quality of evidence in math and reading is quite different. The secondary reading studies were all matched, not randomized, and many had very small sample sizes. In contrast, Slavin et al. (2007) identified 27 secondary studies of CAI in math that met the same

22

standards, and eight of these used randomized or randomized quasi-experimental designs. Only three CAI programs had multiple qualifying studies. Five studies of Read 180, a core CAI model, had a median effect size of +0.18. Accelerated Reader, a computer-managed learning system, was evaluated in four studies, which had a median effect size of +0.31, but as noted above this median was strongly influenced by two very small studies. Finally, there two were studies of Jostens, an earlier form of Compass Learning, which had a median effect size of +0.26.

Instructional Process Programs Instructional process programs are methods that focus on providing teachers with extensive professional development to implement specific instructional methods. These fell into three categories. Cooperative learning programs use methods in which students work in small groups to help one another master academic content. Strategy instruction programs incorporate methods in which students are taught to use specific study strategies, such as paraphrasing, summarization, and prediction, to improve their reading comprehension. Comprehensive school reform (CSR) programs are ones that attend to instruction, curriculum, assessment, classroom management, parent involvement, and other factors. Only CSR programs that incorporate specific reading approaches are reviewed here (for others, see CSRQ, 2006). Descriptions and outcomes of all studies of instructional process programs meeting the inclusion criteria appear in Table 2.

23

================= TABLE 2 =================

Cooperative Learning Peer-Assisted Learning Strategies (PALS) Peer-Assisted Learning Strategies, or PALS, is a cooperative learning program in which students work in pairs, taking turns doing partner reading and engaging in summarization and prediction activities. PALS has primarily been used in the early elementary grades, where it has been successfully evaluated (Fuchs, Fuchs, Mathes, & Simmons, 1997), but it is also used in remedial and special education programs in upper-elementary and secondary grades. Calhoon (2005) evaluated an application of PALS with middle school students reading at or below the third grade level. The treatment combined PALS with a linguistic skills training approach in which students took turns tutoring each other on specific phonological and spelling skills. Four special education teachers and their intact classes of students (N=38) with learning disabilities were randomly assigned to PALS or control conditions, making this a randomized quasi-experiment. Most students were sixth graders, but there were a few seventh graders and one eighth grader. Students were pre- and posttested on four scales from the Woodcock-Johnson III. Adjusting for pretests, there were significant differences on Letter-Word Identification (ES=+0.84, p<.05), Passage Comprehension (ES=+0.66, p<.05), and Word Attack (ES=+0.46, p<.05), but not Reading Fluency (ES= -0.13, n.s.). The mean effect size was +0.46.
24

Fuchs, Fuchs, & Kazdan (1999) evaluated PALS among special education and remedial classes in 10 high schools. Eighteen teachers were non-randomly assigned to PALS or control classes in a 16-week study. The experimental group used PALS procedures on alternating days, 2 times per week. Students were pre- and posttested on an experimenter-made measure called the Comprehensive Reading Assessment Battery, or CRAB, an oral reading measure not aligned with the PALS intervention. Controlling for pretests, differences were statistically significant on comprehension questions (ES=+0.33, p<.05) but not on words read correctly (ES=+0.04, n.s.), for a mean effect size of +0.19. Hankinson & Myers (2000) evaluated PALS in a suburban middle school near Pittsburgh. A total of 51 eighth graders experienced PALS and 32 served as a matched control group in a 12-week study. Students were pretested on the Gates McGinitie and the Pennsylvania Reading Assessment comprehension measure, and 12 weeks later they were posttested. Adjusting for pretests, PALS students gained nonsignificantly more than controls on Gates Vocabulary (ES = +0.10) and Comprehension (ES =+0.44), for a mean of +0.27. On the Pennsylvania Reading Test, control schools gained non-significantly more (ES = -0.34), although the report notes that the control group received special practice on this measure. The median across the two measures was -0.04. The median effect size across three studies of PALS was +0.19, but the one randomized quasi-experiment had the strongest positive effects.

25

Student Team Reading Student Team Reading, or STR (Stevens & Durkin, 1992), is a cooperative learning program for middle schools in which students work in 4-5 member teams to help one another build reading skills. Based on Cooperative Integrated Reading and Composition, or CIRC (Stevens, Madden, Slavin, & Farnish, 1987), used in upper elementary grades, STR has students engage in partner reading, story retelling, storyrelated writing, word mastery, and story structure activities to prepare themselves and their teammates for individual assessments, which form the basis for team scores. Instruction focuses on explicit teaching of metacognitive strategies. Stevens & Durkin (1992, Study 1) carried out a large-scale matched evaluation of STR in five high-poverty, mostly African-American middle schools in Baltimore. Two STR schools with 72 classes in grades 6-8 were matched on demographic characteristics and CAT pretests with three control schools with 88 classes in grades 6-8. Students in the STR classes also experienced a component called Student Team Writing (STW). On reading measures, using z-scores to combine across grades 6-8, STR classes scored significantly higher than controls, adjusting for pretests, with effect sizes of +0.46 (p<.05) for CAT Reading Vocabulary and +0.34 (p<.05) for CAT Reading Comprehension, a mean effect size of +0.40. There were also positive effects on CAT Language Expression, but this is ascribed to Student Team Writing, not STR. In a similar study, Stevens & Durkin (1992, Study 2) evaluated STR in six high-poverty, mostly African-American middle schools in Baltimore. Three schools with 20 sixth-grade classes were compared to three schools with 34 sixth grade
26

classes. On CAT posttests, controlling for CAT pretests, there were small but significant differences favoring STR on Reading Comprehension (ES=+0.13, p<.05), but there were no differences on Reading Vocabulary (ES= -0.02, n.s.), for a mean effect size of +0.06. Separate analyses for students with special needs found much larger impacts, with effect sizes of +0.60 for reading comprehension and +0.28 for vocabulary.

Student Team Literature Mac Iver, Plank, & Balfanz (1997) created an adaptation of Student Team Reading called Student Team Literature (STL), which added to STR a focus on great books, more high-level questions, and additional background information for students. The evaluation of STL took place in a Philadelphia middle school with 21 participating classes in grades 6-8. A similar school with 25 classes, matched on demographic factors and pretests, served as a control group. CTBS tests from the previous spring were used as pretests, and Stanford-9 tests were used as posttests. Based on hierarchical linear modeling, the difference between the schools was substantial (ES=+0.51, p<.05).

The Reading Edge In an adaptation of Student Team Reading, Slavin, Daniels, & Madden (2005) created a program called The Reading Edge to serve as the reading component of the Success for All Middle School. The Reading Edge uses the same cooperative learning

27

structures and basic lesson design as Student Team Reading, but regroups students for reading instruction according to their reading levels across grades and classes. An evaluation of The Reading Edge by Chamberlain, Daniels, Madden, & Slavin (2007a, b) randomly assigned two successive cohorts of sixth graders within two high-poverty, majority-White middle schools in rural West Virginia and Florida to The Reading Edge or control classes. Combining across cohorts, there were 1164 students. On Gates MacGinitie posttests, controlling for pretests, students in The Reading Edge scored significantly higher than controls on Reading Total (ES=+0.15, p<.005). On subtests, The Reading Edge students scored significantly higher on Vocabulary (ES=+0.17, p<.002), and there were smaller significant differences on Comprehension (ES=+0.11, p<.05.). There were no differences in outcomes between the two cohorts. A large-scale matched study of The Reading Edge was carried out by Slavin, Daniels, & Madden (2005). Seven high-poverty schools in six states implemented The Reading Edge over a three-year period. A school matched on prior achievement and demographic factors in the same state (usually the same district) was designated, and state test scores (percent scoring proficient or better) were compared at pre- and posttest. Using arcsine transformations (Lipsey & Wilson, 2001), effect sizes were estimated for each pair of schools. One of the schools, on an Indian reservation in Washington State, made extraordinary gains, going from 0% passing the WASL to 96%, while its control group, also a reservation school, gained 18 percentage points, an effect size of +2.29. Because of this positive outlier, a median rather than a mean

28

was computed across all seven school pairs on their respective state tests, yielding a median effect size of +0.33. Across eight studies of cooperative learning approaches to middle school reading, the median effect size was +0.26. The five studies of the similar Reading Edge, Student Team Reading, and Student Team Literature approaches had a mean effect size of +0.33.

Strategy Instruction Programs Benchmark Detectives Gaskins (1994) evaluated a form of strategy instruction in a Pennsylvania school called the Benchmark School, attended by poor readers of normal or superior intelligence, in which middle school teachers were given professional development in the use of metacognitive reading strategies across the curriculum. In monthly inservice sessions taught by a variety of national experts on the use of cognitive strategy instruction, as well as within-school coaching, co-teaching, and conference attendance, teachers learned several comprehension strategies as well as methods for introducing them. An evaluation compared students in three cohorts entering the middle grades to those in a previous cohort that did not experience strategy instruction. The cohorts were similar in WISC-R IQ measures. On MAT-Reading, adjusted for WISC-R, the strategy group scored non-significantly higher than the baseline group after one year (ES=+0.21, n.s.) and significantly higher after two years (ES=+0.52, p<.01).

29

Strategy Intervention Model The Strategy Intervention Model (SIM; Schumaker, Denton, & Deshler, 1984) is a method in which low-achieving secondary students are taught metacognitive reading strategies, especially paraphrasing, to help them comprehend text. A small study of SIM by Losh (1991) involved junior high school students with learning disabilities. Students in a SIM group (n=32) were individually matched with students in a control group (n=32) based on CAT reading scores, handicapping condition, gender, and grade. On spring CAT scores, controlling for prior spring scores, SIM students scored nonsignificantly higher on CAT composite (ES = +.11, n.s.). There were directionally positive effects on Comprehension (ES = +0.24), but not Vocabulary (ES = -0.01). Mothus (1997) carried out a small matched post-hoc evaluation of SIM in two middle class, mostly White junior high schools in central British Columbia, Canada. One school had used SIM for two years with two cohorts of low-achieving eighth graders (total N=33). These students were compared to students in the same school and in a neighboring school (total N=34) who received conventional learning assistance and were well matched on Stanford Diagnostic Reading Comprehension Tests (SDRCT) given at the beginning of eighth grade. They were also compared to matched low achievers in both schools who did not receive either SIM or conventional learning assistance, but were similarly low achieving. On SDRCT posttests at the end of the year, SIM students scored significantly higher than both the learning assistance group (ES=+0.39, p<.05) and the unserved group (ES=+0.32, p<.05), for a mean ES of +0.36.
30

Comprehensive School Reform Comprehensive school reform (CSR) programs are whole-school models that include extensive professional development in instructional methods, curriculum, school organization, classroom management, parent involvement, and other issues. Only CSR models with specific approaches to reading are included here, but for broader reviews of middle and high school CSR, see CSRQ, 2006; Borman et al., 2003.

Talent Development High School The Talent Development High School, or TDHS, is a comprehensive reform model that focuses on improving reading and mathematics performance in highpoverty high schools. A key element of the approach is a ninth grade academy, which provides a double dose of reading and math instruction (90 minutes of each per day) in a 4 x 4 block schedule. The reading program, called Strategic Reading, is used in the first semester. It emphasizes teacher modeling of comprehension processes, mini-lessons on comprehension strategies and writing, cooperative learning with paired reading and discussion groups, and self-selected reading. In the second semester, students experience the districts English I curriculum, supported by TDHS discussion guides and writing supplements that continue Strategic Reading methods with the district curriculum. Balfanz, Legters, & Jordan (2004) evaluated the TDHS Strategic Reading approach in three inner-city, very low achieving high schools in Baltimore, mostly
31

composed of African-American students. The three TDHS schools, with 20 general education reading classes taught by eight teachers, were compared to three control schools, well matched on pretest scores and demographic factors. Control schools also used a double dose of reading and math in a 4 x 4 block schedule, so instructional time was similar. On district-administered Terra Nova scores, adjusted for pretests, TDHS students scored significantly better than controls at the end of the year (ES=+0.17, p<.01). A third-party evaluation of the TDHS model was carried out in five highpoverty, mostly African-American schools in Philadelphia by MDRC (Kemple, Herlihy, & Smith, 2005). Six high schools matched on eighth grade PSSA scores served as controls. Eleventh grade PSSA-Reading scores served as posttests. Due to high mobility over the three-year experiment, only 399 students from the original sample were still present at posttest, but attrition was similar in the two groups. Among this group, effect sizes were estimated at -0.04, n.s. However, there were significantly positive effects of TDHS on outcomes such as the percent of students promoted to tenth grade, total credits earned, and attendance rates.

Talent Development Middle School The Talent Development Middle School (TDMS) is a comprehensive reform model designed to help high-poverty, urban middle schools improve outcomes for their students. It organizes schools into small, interdisciplinary learning communities, and introduces teaching methods in English/Language Arts, Math, Science, and U.S. History that emphasize cooperative learning. Catch-up courses in reading and math
32

are also provided for struggling students. Extensive professional development and coaching are given to all teachers. A third-party evaluation of TDMS was carried out by Herlihy & Kemple (2004, 2005). Using a comparative interrupted time series design, six middle schools in a large urban district were compared over three baseline years and four to six implementation years to matched comparison schools in the same district. For reading, eighth grade scores on a State Standards Assessment (SSA) for successive cohorts of students were compared in terms of each schools deviation from its own three-year baseline average. Different schools had different numbers of follow up years, but differences were small in all years. Effect sizes were as follows: Year 1, ES = -0.07, n.s.; Year 2, ES = +0.16, p<.01; Year 3, ES = .00, n.s.; Year 4, ES = -0.06, n.s.; Year 5, ES = +0.15, n.s.; Year 6, ES = +0.06, n.s. The average across all years was +0.04.

Conclusions: Instructional Process Programs As was true in the Slavin & Lake (2006) elementary math review and the Slavin, Lake, & Groff (2007) secondary math review, the largest numbers of rigorous studies were ones that evaluated instructional process programs. Across 14 studies, two of which used random assignment to conditions, the median effect size was +0.18. The two randomized studies had a larger effect size, +0.30. Eight of the studies (two of which were randomized) evaluated various forms of cooperative learning. These had a median effect size of +0.26. This corresponds with findings for the math reviews, which reported median effect sizes of +0.29 for
33

cooperative learning at the elementary level (Slavin & Lake, 2006) and +0.32 at the middle and high school level (Slavin et al., 2007). Effect sizes across five studies of three similar programs, Student Team Reading, Student Team Literature, and The Reading Edge, were +0.33. Three studies found positive effects of programs that teach metacognitive strategies to students, Benchmark Detectives and SIM (median ES=+0.36).

Overall Patterns of Outcomes Across all categories, there were 26 qualifying studies of middle and high school reading programs, of which two used random assignment. The median effect size was +0.20. These studies were identified from among more than 200 studies initially reviewed, and represent those that used rigorous experimental procedures. The most surprising finding is the fact that no studies of secondary reading textbooks met the inclusion criteria. Widely used programs such as McDougal Littel Reading, Soar to Success, Wilson Reading, and Language! have not been studied in experimental-control comparisons that met the standards of this review. This contrasts with the situation in secondary math, where Slavin et al. (2007) found 38 qualifying studies of mathematics curricula and 100 qualifying studies overall. Of course, reading was traditionally not taught in middle and high schools except to students in remedial and special education programs, so the dearth of studies in this area is not surprising, but it is nevertheless distressing to find so little evidence behind the curricula used with hundreds of thousands of secondary students who are struggling in reading.
34

The two categories in which qualifying studies did exist were computerassisted instruction and instructional process programs. Both approaches found robust positive effects on achievement in mostly matched quasi-experiments (only two were randomized). The median effect sizes were +0.23 for CAI and +0.18 for instructional process programs. There were no discernable patterns of differential effects according to factors such as ethnicity, poverty, or middle vs. high schools.

Sample Size Matters One factor that did differentiate among studies was sample size. Studies with total sample sizes of 250 or more (125 per treatment), or 10 classes, were considered large. Previous research (e.g., Slavin, 2007; Sterne, Gavaghan, & Egger, 2000; Taylor & Tweedie, 1998) has shown that studies with small sample sizes included in quantitative syntheses report larger effect sizes than those with large samples, due primarily to two factors. First, small studies produce much more variable outcomes than large studies, and second, studies that produce zero or negative effects, especially small, underpowered studies, are less likely to be published or to be findable in any format. Authors are reluctant to even write up the results of small studies that find zero or negative effects, and journal editors are unlikely to publish such studies. As a result, small studies are only likely to be findable when their effects are so large that they are statistically significant despite their small sample sizes. In contrast, large studies finding zero or negative effects are more likely to be published, and because large studies were likely to have been funded or to have been dissertations, they are more likely to produce a report, even if they are not published.
35

Also, studies with statistically significant differences are more likely to be published and otherwise reported, and small studies only have significant differences if effect sizes are large. In the present review, large studies clearly produce lower effect sizes than small studies. Among CAI studies, nine large studies had a median effect size of +0.18, while three small studies had a median of +0.53. Among instructional process programs, eight large studies had a median effect size of +0.17, while six small ones had a median of +0.28.

Summarizing Evidence of Effectiveness for Current Programs For many audiences, it is useful to have summaries of the strength of the evidence supporting achievement effects for programs educators might select to improve student outcomes. Slavin (2007) proposes a rating system intended to balance methodological quality, median effect sizes, sample sizes, and other factors, and this system was applied by Slavin & Lake (2006) and Slavin, Lake, & Groff (2007). Using the same procedures, secondary reading programs were categorized as follows.

Strong Evidence of Effectiveness At least two studies, one of which is a large randomized or randomized quasiexperimental study, or multiple smaller studies, with a median effect size of at least +0.20. A large study is defined as one in which at least ten classes or schools, or 250

36

students, were assigned to treatments. Smaller studies are counted as equivalent to a large study if their collective sample sizes are at least 250 students. Effect sizes from randomized studies take precedence over those from matched studies. Moderate Evidence of Effectiveness At least two studies of any design with a collective sample size of 250 students, with a median effect size of at least +0.20. Limited Evidence of Effectiveness At least one qualifying study of any design with a statistically significant effect size of at least +0.10. Insufficient Evidence of Effectiveness One or more qualifying study of any design with non-significant outcomes and a median effect size less than +0.10.
N No Qualifying Studies

Table 3 summarizes currently available programs falling into each of these categories (within categories, programs are listed in alphabetical order).

=============== Table 3 Here ===============

None of the programs qualified for the strong evidence of effectiveness category,

37

which requires at least one large randomized experiment with an effect size of at least +0.20. However, five programs met the criteria for moderate evidence of effectiveness. Four are cooperative learning programs: PALS, The Reading Edge, Student Team Literature, and Student Team Reading. An early CAI program, Jostens, a precursor to the current Compass Learning program, also had good evidence of effectiveness from two matched studies The limited evidence of effectiveness category required only a single study with an effect size of at least +0.10. Two programs, SIM and Benchmark Detectives, which provide strategy instruction to students, met this standard, as did Read 180, Accelerated Reader, and the Talent Development High School, a comprehensive reform program.

Discussion The most important conclusion of the research reviewed in this article is that there are fewer large, high-quality studies of middle and high school reading programs than one would wish for. There were no methodologically adequate studies of alternative reading texts or curricula. Although 26 studies did qualify for inclusion, there were small numbers of studies on any particular program, and only two studies involved random assignment to conditions. Despite these limitations, there are several important patterns in the findings that are worthy of note. First, this article finds that most of the programs with strong evidence of effectiveness have cooperative learning at their core. These are all forms of cooperative learning in which students work in small groups to help one another
38

master reading skills, and in which the success of the team depends on the individual learning of each team member, the elements that previous reviewers (e.g., Rohrbeck et al., 2003; Slavin, 1995, in press; Webb & Palincsar, 1996) have identified as essential to the effectiveness of cooperative learning. The finding of positive effects of cooperative learning programs is consistent with the findings of reviews of elementary and secondary math programs (Slavin & Lake, 2006; Slavin et al., 2007). Less consistent with previous research, however, is the finding in the present study that forms of computer-assisted instruction generally produced positive effects. An earlier review of CAI in math and reading by Kulik (2003) did not find positive effects for reading. The findings of this review add to a growing body of evidence to the effect that what matters for student achievement are approaches that either fundamentally change what teachers and students do every day (such as cooperative learning) and those that supplement teachers instruction with additional time on targeted skills (such as CAI). In earlier reviews, these strategies had outcomes that were clearly and consistently more positive than those found for alternative textbooks or curricula alone. More research and development of reading programs for secondary students is clearly needed, but we already know enough to take action, to use what we know now to improve reading outcomes for at-risk students in their critical secondary years.

39

References

ACT, Inc. (2006). Reading between the lines: What the ACT reveals about college readiness in reading. Iowa City, IA: Author. Alliance for Excellent Education (2003). Left out and left behind: NCLB and the American high school. Washington, DC: Author. Alliance for Excellent Education. (2002). Every child a graduate. Washington, DC: Author. American Diploma Project. (2004). Ready or not: Creating a high school diploma that counts. Washington, DC: Achieve, Inc. Au, K.H. (2000). A multicultural perspective on policies for improving literacy achievement: Equity and excellence. In M.L. Kamil, P.B. Mosenthal, P.D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. III) (pp. 835851). Mahwah, NJ: Lawrence Erlbaum Associates. Balfanz, R., Jordan, W., & Legters, N. (2004). Catching Up: Impact of the Talent Development ninth grade instructional interventions in reading and mathematics in high-poverty high schools. Center for Research on the Education of Students Placed at Risk, Johns Hopkins University. Barton, M.L. et al. (2002). Teaching reading in mathematics and science. Educational Leadership, 60 (3), November, 2002. Biancarosa, G., & Snow, C. (2004). Reading Next: A vision for action and research in middle and high school literacy. New York: Carnegie Corporation of New York. Borman, G.D., Hewes, G.M., Overman, L.T., & Brown, S. (2003) Comprehensive school reform and achievement: A meta-analysis. Review of Educational Research, 73 (2), 125-230. Calhoon, M. (2005). Effects of a peer-mediated phonological skill and reading comprehension program on reading skill acquisition for middle school students with reading disabilities. Journal of Learning Disabilities, 38(5), 424433. Chamberlain, A., Daniels, C., Madden, N., & Slavin, R.E. (2007a). A randomized evaluation of the Success for All Middle School reading program. Middle Grades Research Journal, 2 (1), 1-21. Chamberlain, A., Daniels, C., Madden, N.A. & Slavin, R.E. (2007b). The randomized evaluation of the Success for All Middle School reading program: Secondyear findings. Baltimore: Johns Hopkins University, Center for Data-Driven Reform in Education. Chambers, E. A. (2003). Efficacy of educational technology in elementary and secondary classrooms: A meta-analysis of the research literature from 199240

2002. Unpublished doctoral dissertation, Southern Illinois University at Carbondale. Cheung, A., & Slavin, R.E. (2005). Effective reading programs for English language learners and other language minority students. Bilingual Research Journal, 29 (2), 241-267. Comprehensive School Reform Quality Center (2006). CSRQ center report on middle and high school comprehensive school reform models. Washington, DC: American Institutes for Research. Cooper, H. (1998). Synthesizing research (3rd ed.). Thousand Oaks, CA: Sage. Dynarski, M., Agodini, R., Heaviside, S., Novak, T., Carey, N., Campuzano, L., Means, B., Murphy, R., Penuel, W., Javitz, H., Emery, D., & Sussex, W. (2007). Effectiveness of reading and mathematics software products: Findings from the first student cohort. Washington, DC: Institute of Education Sciences. Fuchs, L.S., Fuchs, D., & Kazdan, S. (1999). Effects of peer-assisted learning strategies on high-school students with serious reading problems. Remedial and Special Education, 20, 309-318. Fuchs, D., Fuchs, L., Mathes, P.H., & Simmons, D.C. (1997). Peer-assisted strategies: Making classrooms more responsive to diversity. American Educational Research Journal, 34, 174-206. Gaskins, I.W. (1994). Classroom applications of cognitive science: Teaching poor readers how to learn, think and problem solve. In K. McGilly (Ed.).Classroom lessons (pp. 129-154). Cambridge, MA: MIT Press. Hagerman, T. E. (2003). A quasi-experimental study on the effects of Accelerated Reader at middle school. Unpublished doctoral dissertation, University of Oregon. Hankinson, R.D., & Meyers, D.L. (2000). Effectiveness of the Middle School PALS in Reading program on the reading comprehension of middle school students. Unpublished doctoral dissertation, Duquesne University. Haslam, B. M., White, R. N., Klinge, A. (2006). Improving student literacy: READ 180 in the Austin independent school district 2004-05. Washington, DC: Policy Studies Associates. Hasselbring, T., & Goin, L. (2004). Literacy instruction for older struggling readers: What is the role of technology? Reading & Writing Quarterly, 20, 123144. Herlihy, C. M. & Kemple, J. J. (2004) The Talent Development Middle School Model: Context, components, and initial impacts on students performance and attendance. New York: MDRC. Herlihy, C. M. & Kemple, J. J. (2005). The Talent Development Middle School Model: Impacts though the 2002-2003 school year. An update to the December 2004 report. New York: MDRC.
41

Hunter, C.T.L. (1994). A study of the effect of instructional method on the reading and mathematics achievement of chapter one students in rural georgia. Unpublished Doctoral Dissertation, South Carolina State University. Interactive, Inc. (2002). Final report: Study of READ 180 in the Council of Great City Schools. New York, NY: Interactive Inc. Kemple, J.J., Herlihy, C.M., & Smith, T.J. (2005). Making progress toward graduation: Evidence from the Talent Development High School model . New York: MDRC. Kulik, J. A. (2003). Effects of using instructional technology in elementary and secondary schools: What controlled evaluation studies say. SRI Project Number P10446.001. Arlington, VA: SRI International. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage. Losh, M.A. (1991). The effect of the strategies intervention model on the academic achievement of junior high learning-disabled students. Unpublished doctoral dissertation, University of Nebraska. Mac Iver, D. J., Plank, S.B., & Balfanz, R. (1997). Working together to become proficient readers: Early impact of the Talent Development Middle School's Student Team Literature Program. Baltimore, MD: Center for Research on the Education of Students Placed At Risk (CRESPAR), Johns Hopkins University. Metrics Associates. (1981). Evaluation of the Computer Assisted Instruction Title I Project, 1980-81. Research Report. Chelmsford, MA: Merrimack Education Center. Mims, C., Lowther, D., Strahl, J.D., & Nunnery, J. (2006). Little Roch School District Read 180 evaluation: Technical Report. Memphis, TN: The Center for Research in Educational Policy. Mothus, T.G. (1997). The effects of strategy instruction on the reading comprehension achievement of junior secondary school students. Masters Abstracts International, 42(01), 44. Murphy, R., Penuel, W., Means, B., Korbak, C., Whaley, A., & Allen, J. (2002). EDESK: A review of recent evidence on discrete educational software . Menlo Park, CA: SRI International. National Assessment of Educational Progress (2005). The nations report card. Washington, DC: National Center for Education Statistics. Nunnery, J., Ross, S.M. & Goldfeder, E. (2003). The effect of School Renaissance on TAAS scores in the McKinney ISD. Memphis: University of Memphis, Center for Research in Educational Policy.

42

Papalwis, R. (2004). Struggling middle school readers: Successful, accelerating intervention. Sacramento, CA: California State University. Rohrbeck, C. A., Ginsburg-Block, M. D., Fantuzzo, J. W., & Miller, T. R. (2003). Peer-assisted learning interventions with elementary school students: A metaanalytic review. Journal of Educational Psychology, 94 (2), 240-257. Ross, S., Nunnery, J., Avis, A., & Borek, T. (2005). The effects of School Renaissance on student achievement in two Mississippi school districts: A longitudinal quasi-experimental study. Memphis: TN: Center for Research in Education Policy. Roy, J.W. (1993). An investigation of the efficacy of computer-assisted mathematics, reading, and language arts instruction. Unpublished doctoral dissertation, Bylor University. Schumaker, J. B., Denton, P. H., & Deshler, D. D. (1984). The paraphrasing strategy. Lawrence, KS: The University of Kansas. Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105, 309-316. Shadish, W.R., Cook, T.D., & Campbell, D.T. (2002). Experimental and quasiexperimental designs for generalized causal inference . Boston: HoughtonMifflin. Slavin, R. E. (1986). Best-evidence synthesis: An alternative to meta-analytic and traditional reviews. Educational Researcher, 15, (9), 5-11. Slavin, R.E. (1995). Cooperative learning: Theory, research, and practice (2nd Ed.). Boston: Allyn & Bacon. Slavin, R. (2007). What works? Issues in synthesizing education program evaluations. Manuscript submitted for publication. Slavin, R.E. (in press). Cooperative learning. In G. McCulloch & D. Crook (Eds.) International Encyclopedia of Education. Abington, UK: Routledge. Slavin, R.E., Daniels, C., & Madden, N.A. (2005). The Success for All Middle School: Adding content to middle grades reform. Middle School Journal, 36 (5), 4-8. Slavin, R.E., & Lake, C. (2006). Effective programs in elementary math: A best evidence synthesis. Manuscript submitted for publication. Slavin, R.E., Lake, C., & Groff, C. (2007). Effective programs in middle and high school math: A best evidence synthesis. Manuscript submitted for publication. Sterne, J., Gavaghan, D., & Egger, M. (2000). Publication and related bias in metaanalysis: power of statistical tests and prevalence in the literature. Journal of Clinical Epidemiology, 53 (11), 1119-29.
43

Stevens, R.J., & Durkin, S. (1992). Using student team reading and student team writing in middle schools: Two evaluations. Baltimore, MD: Center for Research on Effective Schooling for Disadvantaged Students. Stevens, R.J., Madden, N.A., Slavin, R.E., & Farnish, A.M. (1987). Cooperative Integrated Reading and Composition: Two field experiments. Reading Research Quarterly, 22, 433-454. Taylor, S., & Tweedie, R. (1998). A non-parametric trim and fill method of assessing publication bias in meta-analysis. Denver, CO: University of Colorado Health Sciences Center. Texas Center for Educational Research (2007). Evaluation of the Texas Technology Immersion Pilot: Findings from the second year. Austin, TX: Author Vollands, S. R., Topping, K. J., & Evans, R. M. (1999). Computerized selfassessment of reading comprehension with the Accelerated Reader: Action research. Reading and Writing Quarterly, 15, 197211. Webb, N.M., & Palincsar, A.S. (1996). Group processes in the classroom. N D.C. Berliner & R.C. Calfee (Eds.), Handbook of Educational Psychology. New York: Simon & Schuster Macmillan. What Works Clearinghouse (2007). Beginning reading. Washington, DC: U.S. Department of Education. Retrieved June 18, 2007, from http://www.whatworks.ed.gov/Topic.asp?tid=01&ReturnPage=default.asp White, R. N., Haslam, M. B., Hewes, G. M. (2006). Improving student literacy in the Phoenix Union High School District, 2003-04 and 2004-05. Final Report . Washington, DC: Policy Studies Associates.

44

TABLE 1 Computer-Assisted Instruction: Descriptive Information and Effect Sizes for Qualifying Studies Design (Large/Small) Sample Grade Characteristics Core CAI Programs Evidence of Initial Equality Effect Size Overall Effect Size

Study

Duration

Posttest

Read 180 Mims, Lowther, Strahl, & Nunnery (2006) Matched (L) 1 year 1000 students 1652 students (826T, 826C) 1630 students (815T, 815C) 1073 students 700 students (387T, 323C) 614 students 6th-9th Mostly AfricanAmerican students in Little Rock, Arkansas Well matched on ITBS reading scores and demographics ITBS Arkansas Benchmark AIMS 9th -10th Students with low reading scores in Phoenix, AZ 9th Well matched on pretest, gender, ethnicity, and language proficiency Well matched on pretest Well matched on demographics and pretest (TAKS) Well matched on pretest measures (SAT-9) ELLs Low scorers High scorers Terra Nova ELLs Low scorers High scorers SAT-9 Reading Language Arts -0.17 -0.12 -0.07 .00 +0.28 +0.14 -0.11 +.24 +0.41 +0.30 +0.05 +0.68 +0.30

1.5 years White, Haslam, & Hewes (2006) Matched (L) 1 semester

+0.12

Papalewis (2004)

Matched (L)

1 year

8th (mostly), retained

Low performing students in Los Angeles Two middle schools each in Boston, Houston, Dallas, and Columbus Low performing students in Austin, TX

+0.68

Interactive, Inc (2002) Haslam, White, & Klinge (2006)

Matched (L)

1 year

6th - 8th

SAT-9

+0.24

Matched (L)

1 year

7th - 8th

TAKS

+0.18

Supplemental CAI Programs Jostens/Compass Learning Roy (1993) Matched (L) 1 year 408 students (204T, 204C) 270 students (135T, 135C) in 6 schools (3T, 3C) 6th-8th Schools in rural northeast and central Texas Schools in rural Jefferson County, Georgia Well matched on achievement pretest ANCOVA was used to adjust for pretest (ITBS) differences NAPT 6th (MISD) th th 7 /8 (HISD) ITBS 6th 7th 8th +0.38 +0.04 +0.37 +0.37 +0.19 +0.21

Hunter (1994)

Matched (L)

28 weeks

6th - 8th

+0.31

Other Supplemental CAI Metrics Associates (1981) Matched (S) 1 year 105 students (70T, 35C) 7th - 9th Two Massachusetts school districts Scores were adjusted for any pretest (MAT) differences MAT reading +0.56

Computer-Managed Learning Systems Accelerated Reader Hagerman (2003) Matched (S) 12 weeks 121 students (64T, 57C) 39 students (27T, 12C) 6th Two suburban middle schools in Oregon High-poverty schools in Scotland Well matched on demographics and achievement Scores were adjusted for any pretest differences Schools were matched on student ethnicity, poverty, mobility, school location, grades served, size, prior achievement. Pretest differences were controlled using ANCOVA. Well matched on pretest (MCT) scores and demographics TORC-3 Short. Edinburgh Comprehension Neale Accuracy Neale Comp MCT 6th 7th 8th MCT 6th 7th 8th -0.04 +0.04 +0.10 +0.03 +0.11 +0.16 +0.12 +0.13 +0.51 +0.33 +0.64 +0.53

Vollands, Topping & Evans (1999)

Year 1
+0.49

Matched (S)

6 months

6th

Ross & Nunnery (2005)

Matched Post Hoc (L)

1 year

3230 students (2106T, 1124C) in 10 schools

6th - 8th

Schools in southern Mississippi

Ross, Nunnery, Avis, & Borek (2005)

Matched Post Hoc (L)

1 year

4085 students (2419T, 1666C)

6th - 8th

Schools in southern Mississippi

46

TABLE 2 Instructional Process Programs: Descriptive Information and Effect Sizes for Qualifying Studies Study Design (Large/Small) Duration N Grade Sample Characteristics Evidence of Initial Quality Posttest Effect Size Overall Effect Size

Cooperative Learning Peer-Assisted Learning Strategies (PALS) WJ-III Randomized quasiexperiment (S) 38 students taught by 4 teachers in 2 schools 102 students (52T, 50C) in 18 classes (9T, 9C) 83 students (51T, 32C) 6th 8th Special education classes at two middle schools in the southwest Special education and remedial classes in ten high schools Suburban middle school near Pittsburgh Scores were adjusted for any pretest differences Letter-Word Identification Passage Comprehension Word Attack Reading Fluency CRAB Comprehension Correct words read GMRT (Gates) Vocabulary. Comprehension. PSSA Reading Comp. CAT Stevens & Durkin (1992), Study 1 Five middle schools in Baltimore Well matched on ethnicity and SES background; Control group scores higher than the treatment group at pretest Reading Vocabulary Reading Comprehension All students CAT Vocabulary CAT Comp Special Ed CAT Vocabulary CAT Comp +0.46 +0.33 +0.04 +0.10 +0.44 -0.34 +0.84 +0.66 +0.46 -0.13

Year 1

Calhoon (2005)

31 weeks

+0.46

Fuchs, Fuchs, & Kazdan (1999)

Matched (S)

16 weeks

HS

Scores were adjusted for any pretest differences

+0.19

Hankinson & Myers (2000)

Matched (S)

12 weeks

8th

Well matched on reading pretest scores

-0.04

Student Team Reading (STR)

Matched (L)

1 year

3986 students

6-8th

+0.40 +0.34

Stevens & Durkin (1992), Study 2

Matched (L)

1 year

1233 students (455T, 768C)

6th

Twenty classes in six middle schools in an urban district in Maryland

Well matched on CAT pretest scores

-0.02 +0.13 +0.28 +0.60

Year +0.06

Year 1

Student Team Literature

47

MacIver, Plank, & Balfanz (1997) The Reading Edge Chamberlain, Daniels, Madden, & Slavin (2007a,b)

Matched (L)

1 year

2000 students in 46 classes

6-8th

Two low SES schools with 50% LEP students in Philadelphia

Well matched on student population and characteristics of the teaching staff

CTBS Reading comp (class as unit of analysis) +0.51

Randomized (L)

1 year

759 students in 2 cohorts (389T, 370C)

6th

Two majority White, high poverty rural middle schools in West Virginia and Florida High-poverty schools throughout the US

Well matched on pretest, race, SES and special education status Schools were matched on prior achievement and demographic factors.

GMRT (Gates) Total Comprehension Vocabulary

+0.15 +0.11 +0.17

+0.15

Slavin, Daniels, & Madden (2005)

Matched (L)

3 years

Students in 14 schools (7T, 7C)

6th 8th

State assessments

+0.33

Year 1

Strategy Instruction Benchmark Detectives Reading Program MAT reading Memory for text information Interpretation of text information Summarizing paragraph Outlining Integrating text Taking notes CAT Composite Comprehension Vocabulary +0.11 +0.11 +0.24 -0.01 +0.43 +0.35 +0.17 +0.52 +0.60 +0.49 +0.19 -0.09

Gaskins (1994)

Matched (S)

1-2 years

107

6th

Benchmark School

ANCOVA was used to adjust for IQ and age

Year 1

Strategy Intervention Model (SIM) Students with learning disabilities in a Nebraska junior high school Students were individually matched based on CAT scores, handicapping condition, gender, and grad

Losh (1991)

Matched (S)

1 year

64 Students (32T, 32C)

7th-9th

48

Mothus (1997)

Matched Post Hoc (S)

2 years

67 students (33T, 34C)

8th

Low-performing students at two middle class, mostly White junior high schools in central British Columbia, Canada

Students were well matched at pretest (SDRCT)

SDRCT

+0.36

Comprehensive School Reform Talent Development High School Balfanz, Legters, & Jordan (2004) Matched (L) 1 year 457 students (257T, 200C) in 6 schools (3T, 3C) 399 students HS Inner-city high schools in Baltimore High-poverty, mostly AfricanAmerican schools in Philadelphia Scores were adjusted for pretest differences Students were matched on 8th grade PSSA scores Terra Nova +0.17

Kemple, Herlihy & Smith (2005)

Matched (L)

3 years

9-11

PSSA-Reading

-0.04

Talent Development Middle School SSA Herlihy & Kemple (2004-2005) 4 or more years 18 schools (6T, 12C) 8th Middle schools in a large urban district TD schools were matched with set of comparison schools on demographics and test scores Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 -0.07 +0.16 0.00 -0.06 +0.15 +0.06

Matched (L)

+0.04

49

Table 3 Strength of Evidence for Secondary Reading Programs Strong Evidence of Effectiveness None Moderate Evidence of Effectiveness Jostens (CAI) PALS (IP, Cooperative Learning) The Reading Edge (IP, Cooperative Learning) Student Team Literature (IP, Cooperative Learning) Student Team Reading (IP, Cooperative Learning) Limited Evidence of Effectiveness Accelerated Reader (CAI) Benchmark Detectives (IP, Strategy) Read 180 (CAI) Strategy Intervention Model (IP, Strategy) Talent Development High School (IP, CSR) Insufficient Evidence of Effectiveness Talent Development Middle School (IP, CSR) N No Qualifying Studies 100 Book Challenge ABD's of Reading Academy of Reading Alphabetic Phonics AMP Reading System Barton Reading & Spelling System Be a Better Reader Boys Town Reading Curriculum Bridges to Literacy Caught Reading Charlesbridge Reading Fluency Classworks Compass Learning (current version) Comprehension Upgrade Concept-Oriented Reading Instruction (CORI) Corrective Reading CRISS / Project CRISS

Cross-Aged Literacy Program Direct Instruction Electronic Bookshelf Essential Learning Systems Exemplary Center for Reading Instruction (ECRI) Expeditionary Learning Outward Bound Failure Free Reading Fast ForWord Fast Track Reading First Step Fluent Reader Glass-Analysis method Great Leaps HOSTS IndiVisual Reading Intensive Reading Strategies Instruction (IRSI) Model Intensive Supplemental Reading Jamestown Education Junior Great Books Kaplan SpellRead K-W-L strategy Language! Learning Experience Approach Like to Read Lindamood-Bell method Literacy First Literacy Seminar McDougal-Littel McRAT Merit Software Multicultural Reading and Thinking My Reading Coach Open Book Anywhere Phonics for Reading Phono-Graphix PLATO Project Read Questioning the Author QuickReadsSecondary Quicktionary Reading Pen II Rave-O

REACH ReadAbout Read Naturally Read Now Read On! Read Right Read XL Reading Apprenticeship Reading in the Content Areas Reading Horizons Reading Is Fame Reading Power in the Content Areas Reading Plus Reciprocal Teaching REWARDS Rosetta Stone Literacy Second Chance at Literacy Learning Second Chance Reading Slingerland Soar to Success Solilioquy Reading Assistant Sound Sheets Spalding Method Strategic Literacy Initiative SuccessMaker Enterprise Supported Literacy Approach Text mapping strategy Thinking Works Voyager Passport Reading Journeys Wilson Reading System Wisconsin Design for Reading Skills Development (WDRSD)

APPENDIX 1 Studies Not Included in the Review Reason not included/comments Cited by

Author READING CURRICULA

Corrective Reading Airhart, K. (2005). The effectiveness of Direct Instruction in reading compared to a state mandated language arts curriculum for ninth and tenth graders with specific learning disabilities. Unpublished dissertation, Tennessee State University. Grossen, B., Burke, M., & Hagen-Burke, S. (2002). An experimental study of the effects of considerate curricula in language arts on reading comprehension and writing. Institute for Academic Access. Harris, R., Marchand-Martella, N., & Martella R. (2000). Effects of a Peer-Delivered Corrective Reading Program. Journal of Behavioral Education, 10(1), 21-36. Kalisek, A.M. (2004). The effects of a middle school corrective reading intervention on high school passage rate. Unpublished doctoral dissertation, University of La Verne. Kasendorf, S. J., & McQuaid, P. (1987). Corrective reading evaluation study. ADI News, 7(1), 9. Lingo, A., Jolivette, K., & Slaton, D. (2006). Effects of Corrective Reading on the Reading Abilities and Classroom Behaviors of Middle School Students with Reading Deficits and Challenging Behavior. Behavioral Disorders, 31(3), 265283. Sommers, J. (1995). Seven-year overview of Direct Instruction programs used in basic skills classes at Big Piney Middle School. Effective School Practices, 14(4), 2932. Strong, A., Falk, K., Lane, K., & Wehby, J. (2004). The Impact of a Structured Reading Curriculum and Repeated Reading on the Performance of Junior High Students with Emotional and Behavioral Disorders. School Psychology Review, 33(4), 561581. Shippen, M., Houchins, D., Sartor, D., & Steventon, C. (2005). A Comparison of Two Direct Instruction Reading Programs for Urban Middle School Students. Remedial and Special Education, 26(3), 175 182. Thorne, M. T. (1978). Payment for reading: The use of the Corrective Reading scheme with junior maladjusted boys. Remedial Education, 13, 87-90. Fluent Reader Raile, C., & Seekal, P. (2004). Curriculum-based measurements show improved fluency after only 12 weeks (Scientific Research: QuasiExperimental series). Madison, WI: Renaissance Learning, Inc. Available online: <http://research.renlearn.com/research/pdfs

pretest differences > 0.5 SD on TORC-3

duration < 12 weeks

no control group

. inadequate outcome measure no control group multiple probe design, seven participants

no control group

multiple baseline design, six participants

duration < 12 weeks

no control group

Lingo et al (2006)

inadequate control group: lower ability group received treatment

LANGUAGE! Greene, J. F. (1996). LANGUAGE! Effects of an individualized structured language curriculum for middle and high school students. Annals of Dyslexia, 46, 97-121. Lawrence, A.J. (2003). The effectiveness of the "Language!" program in improving the word recognition skills of middle school students with learning disabilities. Unpublished doctoral dissertation, California State University, Fullerton. Read XL Holly, T.M. (2004) Analyzing the effectiveness of reading intervention strategies on reading achievement in an urban West Tennessee school district. Unpublished doctoral dissertation, Union University. inadequate control group duration <12 weeks, pretest, posttest only

inadequate outcome measure

COMPUTER-ASSISTED INSTRUCTION
Accelerated Reader Gibson, M. (2002). An Investigation of the Effectiveness of the Accelerated Reader Program Used with Middle School At-Risk Students in a Rural School System. Unpublished doctoral dissertation, Mississippi State University. Goodman, G. (1999). The Reading Renaissance/ Accelerated Reader Program: Pinal County School-to-Work Evaluation Report. Phoenix, AZ: Creative Research Associates, Inc. Kohel, P.R. (2003). Using Accelerated Reader: Its impact on the reading levels and Delaware state testing scores of 10th grade students in Delaware's Milford High School. Unpublished doctoral dissertation, Wilmington College. Lewis, S. (2005). Evaluating Alternative Methodologies to Teaching Reading to Sixth-Grade Students and the Association with Student Achievement. Dissertation Abstracts International, 66(10), 3599A. (UMI No. 3195378) McDurmon, A. (2001). The effects of guided and repeated reading on English language learners. Unpublished masters thesis, Berry College. Nunnery, J., Ross, S.M. & Goldfeder, E. (2003). The effect of School Renaissance on TAAS scores in the McKinney ISD. Memphis: University of Memphis, Center for Research in Educational Policy. Nunnery, J. A., Ross, S. M., McDonald, A. (2006). A randomized experimental evaluation of the impact of Accelerated Reader/Reading Renaissance implementation on reading achievement in grades 3 to 6. Journal of Education for Students Placed at Risk, 11(1), Paul, T.D. (2003). Guided independent reading: An examination of the Reading Practice Database and the scientific research supporting guided independent reading as implemented in Reading Renaissance. Madison, WI: Renaissance Learning, Inc. Available onlin Peak, J. & Dewalt, M. (1994). Reading achievement: Effects of computerized reading management and enrichment. ERS spectrum, 12(1), 31-34. inadequate control group

no control group no adequate control group - STAR pretest differences > 1/2 SD no pretests for Terra Nova outcome measure (STAR) inherent to treatment Ceiling effect on TAAS

STAR Reading assessment inherent to treatment

no control group

insufficient information

Scott, L.S. (1999). The Accelerated Reader program, reading achievement, and attitudes of students with learning disabilities. Unpublished masters thesis, Georgia State University. (ERIC No. ED434431). Sims, S.P. (2002). The effects of the Accelerated Reader program and sustained silent reading on reading attitude and reading achievement of eighth grade students. Unpublished doctoral dissertation, Georgia State University. Smith, I. (2005). Can Accelerated Reader and cooperative learning enhance the reading achievement of Level 1 high school students on the Florida Comprehensive Assessment Test? Unpublished doctoral dissertation, Nova Southeastern University. Topping K. J. and Sanders, W. L. (2000). Teacher effectiveness and computer assessment of reading. School effectiveness and school improvement. 11 (3), 305-337. Vollands, S. R., Topping, K. J., & Evans, R. M. (1999). Computerized self-assessment of reading comprehension with the Accelerated Reader: Action research. Reading and Writing Quarterly, 15, 197211. Walberg, H.J. (2001). Final evaluation of the reading initiative (Report to the J.A. & Kathryn Albertson Foundation Board of Directors). Boise, ID: J.A & Kathryn Albertson Foundation. Available online: <http://jkaf.org/publications/pdfs/readevw.pdf>. Walker, G.A. (2005). The impact of Accelerated Reader on the reading levels of eighth-grade students at Delaware's Milford Middle School. Unpublished doctoral dissertation, Wilmington College. Compass Learning/Jostens Compass Learning. (2005). Compass Learning Odyssey School Effectiveness Report: Daniel Boone Area School District, Birdsboro, Pennsylvania. San Diego CA: Compass Learning. Failure Free Reading Algozzine, B., Lockavitch, J. F., & Audette, R. (1997). Effects of failure free reading on students at-risk for serious school failure. Australian Journal of Learning Disabilities, 2, (3), 14-17. Gum, L. (2003). Collateral effects of computer-assisted reading instruction on the classroom behaviors of learners with emotional and/or behavioral disorders. Unpublished doctoral dissertation, Tennessee Technological University. Rankhorn, B., England, G., Collins, S., Lockavitch, J. F., Algozzine, B. (1998). Effects of the Failure Free Reading program. Journal of Learning Disabilities, 31, 307-312. Slate, J., Algozzine, B., & Lockavitch, J.F. (1998). Effects of intensive remedial reading instruction. Journal of At-Risk Issues, 5, 30-35. Merit Jones, J., Staats, W., Bowling, N., Bickel, R., Cunningham, M., & Cadle, C. (2004). An Evaluation of the Merit Reading Software Program in the Calhoun County (WV) Middle/High School. Journal of Research on Technology in Education, 37(2), 177195. MultiFunk

inadequate control group: large pretest differences between groups inadequate outcome measure

no control group (pretest-posttest growth)

no control group

large pretest differences

Pearson et al

program evaluations: insufficient data presented no untreated control group

no control group

no control group multiple baseline design, eight participants no control group no control group

FFR

FFR FFR

duration < 12 weeks

Fasting, R. B. & Lyster, S. H. (2005). The effects of computer technology in assisting the development of literacy in young struggling readers and spellers. European Journal of Special Needs Education, 20(1), 2140. Peabody Literacy Lab Hasselbring, T., & Goin, L. (2004). Literacy Instruction for Older Struggling Readers: What is the Role of Technology? Reading & Writing Quarterly, 20, 123144. PLATO Barnett, T. L. (1986). A comparative analysis of the PLATO computerassisted instructional delivery system and the traditional individualized instructional program in two juvenile correctional facilities owned by the Commonwealth of Pennsylvania. Disserta Brush, T. (2002, May). PLATO evaluation series: Terry High School, Lamar Consolidated ISD, Rosenberg, TX. Bloomington, MN: PLATO Learning Inc. (ERIC Document Reproduction Service No. ED 469 375) Elliott, E. L. (1985). The effects of computer-assisted instruction upon the basic skill proficiencies of secondary vocational education students. Dissertation Abstracts International, 46 (11), 3329A. (UMI No. 8600439) Quicktionary Reading Pen II Higgins, E. L., & Raskind, M. H. (2005). The compensatory effectiveness of the Quicktionary Reading Pen II on the reading comprehension of students with learning disabilities. Journal of Special Education Technology, 20(1), 3140. Read 180 Admon, N. (2003). READ 180 Stages A and B:Iredell-Statesville schools, North Carolina. New York, NY: Scholastic Inc. Admon, N. (2005). READ 180 Stage B: St. Paul School District, Minnesota. New York, NY: Scholastic Inc. Brown, S.H. (2006). The effectiveness of READ 180 intervention for struggling readers in grades 6--8. Unpublished doctoral dissertation, Union University. Campbell, Y.C. (2006). Effects of an integrated learning system on the reading achievement of middle school students. Unpublished doctoral dissertation, University of Miami. Daviess County Public Schools, Assessment, Research and Curriculum Department. (2005). READ 180 Implementation year study. Owensboro, KY: Author. Denman, J.S. (2004). Integrating technology into the reading curriculum: Acquisition, implementation, and evaluation of a reading program with a technology component. (READ 180) for struggling readers. Newark, DE: University of Delaware. Dunn, C.A. (2002). An investigation of the effects of computer-assisted reading instruction versus traditional reading instruction on selected high school freshmen. Unpublished doctoral dissertation, Loyola University of Chicago.

duration < 12 weeks

Pearson et al

no adequate control group - pretest differences > 1/2 SD

duration < 12 weeks

no control group

large pretest differences (>0.5 SD) in reading and math

no pretest; duration < 12 weeks

Pearson et al

no control group, pre and post only no control group, pre and post only no control group (pretest-posttest growth) inadequate control group - pretest differences > 1/2 SD no control group, pre and post only no control group, pre and post only no adequate control group - pretest differences > 1/2 SD

Ferguson, J.M. (2005). The implementation of technology in reading classrooms and the impact of technology integration and student perceptions on reading achievement, Commerce, TX: Texas A&M University-Commerce. Gentry, L. (2006). An evaluation of Read 180 in an urban secondary school. Unpublished Doctoral Dissertation, American University. Goin, L., Hasselbring, T., & McAfee, I. (2004). Executive summary, DoDEA/Scholastic READ 180 project: An evaluation of the READ 180 intervention program for struggling readers. New York, NY: Scholastic Inc. Hasselbring, T. S., Goin, L. I,. Taylor, R., Bottge. B., & Daley, P. (1997). The computer doesnt embarrass me. Educational Leadership, 55(3), 30-33 Hewes, G. M., Palmer, N., Haslam, M. B., Mielke, M. B. (2006). Five years of READ 180 in Des Moines: Improving literacy among middle school and high school special education students. Des Moines, IA: Scholastics, Inc. & Des Moines Independent School Distr Holyoke School District. (2005). READ 180 Stage B: Holyoke School District, Massachusetts. New York, NY: Scholastic Inc. Kratofil, M.D. (2006). A comparison of the effect of Scholastic Read 180 and traditional reading interventions on the reading achievement of middle school low-level readers. Unpublished doctoral dissertation, Central Missouri State University. Newman, D., Leuer, M., Jaciw, A. (2006). Effectiveness of Scholastics READ 180 as a remedial reading program for ninth graders: Report of an implementation in Anaheim, CA. Palo Alto, CA: Empirical Education, Inc. Palmer, N. (2003). READ 180 middle-school study: Des Moines, Iowa 2000-2002. Research report. Available at: http://teacher.scholastic.com/products/read180/research/pdf/DesMoine s _Study.pdf Papalewis, R. (2003). Final Report: A study of READ 180 in middle schools in Clark County School District, Las Vegas, Nevada. New York, NY: Scholastic Inc. Pearson, L. M., & White, R. N. (2004). Study of the impact of READ 180 on student performance in Fairfax County Public Schools. Washington, DC: policy Studies Associates, Inc. Scholastic Research and Evaluation. (2004). READ 180 Stage C: Shiprock High School, Central Consolidated School District, New Mexico. New York, NY: Scholastic Inc. Scholastic Research and Evaluation. (2005). Special education students: Selbyville middle and Sussex Central middle schools, Indian River School District (Delaware). New York, NY: Scholastic Inc. Thomas, D.M. (2005). Examining the academic and motivational outcomes of students participating in the READ 180 program. Unpublished doctoral dissertation, University of Kentucky.

no control group, pre and post only no adequate control group - large pretest differences no control group

descriptive article

no adequate control group no control group, pre and post only no adequate control group pretest differences > SD no control group, pre and post only

no control group

no control group, pre and post only no control group, pre and post only no control group, pre and post only no control group, pre and post only pretest equivalence not established

Thomas, J. (2003). Reading program evaluation: READ 180, grades 48, November, 2003. Kirkwood, MO: Kirkwood School District. White, R. N., Williams, I. J., & Haslam, M. B. (2005). Performance of District 23 students participating in Scholastic READ 180. Washington, DC: Policy Studies Associates, Inc. Witkowski, P.M. (2004). A comparison study of two intervention programs for reading-delayed high school students. Unpublished doctoral dissertation, University of Missouri - Saint Louis. Zvoch, K., & Letourneau, L. (2006). Closing the achievement gap: An examination of the status and growth of ninth grade Read 180 students. Las Vegas, NV: Clark County School District. Reading Partner Salomon, G., Globerson, T. & Guterman, E. (1989). The computer as a zone of proximal development: Internalizing reading-related metacognitions from a reading partner. Journal of Educational Psychology, 81(4), 620627. Reading Plus Marrs, H. & Patrick, C. (2002). A return to eye-movement training? An evaluation of the Reading Plus program. Reading Psychology, 23, 297322. Reading Renaissance Renaissance Learning. (2002). Results from a three-year statewide implementation of Reading Renaissance in Idaho. Madison, WI: Renaissance Learning, Inc. Available online: http://research.renlearn.com/research/pdfs/106.pdf>. Roland Reading Method Hardiman, M. (2003). Teaching Adolescents with Reading Deficits: The Effects of a Phonics-Based Approach. Unpublished doctoral dissertation, Johns Hopkins University. Student Assistant for Learning from Text (SALT) MacArthur, C.A. & Haynew, J.B. (1995). Student assistant for learning from text (SALT): A hypermedia reading aid. Journal of Learning Disabilities, 28, 150-159. SuccessMaker & talking books Underwood, J. (2000). A comparison of two types of computer support for reading development. Journal of Research in Reading, 23(2), 136 148.

no control group, pre and post only pretest differences >0.5 SD no adequate control group no control group, pre and post only

duration < 12 weeks

Pearson et al

no adequate control group

no adequate control group

inadequate control group: large differences on free lunch % and some pretests

duration < 12 weeks

Berkeley et al.

insufficient information on pretest scores

Pearson et al

OTHER CAI
Arroyo, C. (1992). What is the effect of extensive use of computers on the reading achievement scores of seventh grade students? ERIC document Reproduction Service No. ED35344. Retrieved June5, 2007 from http://www.edrs.com Cicchetti, G., Sandagata, A., Suntag, M., & Tarnuzzer, J. (2003). The effects of web-based instruction in digital classrooms on math and reading performance on the CT Academic Performance test (CAPT) and related outcomes for a 10th grade cohort of CT urban vocationaltechnical school students. insufficient information

no control group (pretest-posttest growth)

Gentry, M. M., Chinn, K. M., & Moulton, R. D. (2005). Effectiveness of multimedia reading materials when used with children who are deaf. American Annals of the Deaf, 149(5), 394403. Kim, A. (2002). Effects of computer-assisted collaborative strategic reading on reading comprehension for high-school students with learning disabilities. Unpublished doctoral dissertation, University of Texas at Austin. Kim, A-H, Vaughn, S., Klingner, J.K., Woodruff, A.L., Reutebuch, C.K., & Kouzekanani, K. (2006). Improving the reading comprehension of middle school students with disabilities through computer-assisted collaborative strategic reading. Remedial and Specia Koza, J.L. (1989). Comparison of the achievement of mathematics and reading levels and attitude toward learning of high-risk secondary students through the use of computer-aided instruction. Unpublished doctoral dissertation, University of Minnesota. Kramarski, B. & Feldman, Y. (2000). Internet in the classroom: Effects on reading comprehension, motivation and metacognitive awareness. Educational Media International, 37(3), 149155. Lynch, L., Fawcett, A., Nicolson, R. (2000). Computer-assisted reading intervention in a secondary school: an evaluation study. British Journal of Educational Technology, 31 (4), 333348. Reinking, D. (1988). Computer-mediated text and comprehension differences: The role of reading time, reader preference, and estimation of learning. Reading Research Quarterly, 23, 484498. Traynor, P. L. (2003). Effects of computer-assisted instruction on differet learners. Journal of Instructional Psychology, 30(2), 137-143.

no adequate control group inadequate control group: large differences on % free lunch

Pearson et al Berkeley et al

average duration < 12 weeks

duration < 12 weeks, no adequate control group

duration < 12 weeks

Pearson et al

duration < 12 weeks Pearson et al

duration < 12 weeks

no control group

INSTRUCTIONAL PROCESS PROGRAMS

AMP Reading System Mid-continent Research for Education and Learning. (n.d.). Final Evaluation Report, AGS Globes AMP Reading System Efficacy Study. Boston, MD: Pearson Education. BIG Accomodation Model Grossen, B. J. (2002). The BIG Accomodation Model: The direct instruction model for secondary schools. Journal of Education for Students Placed at Risk, 7(2), 241-263. Career Academies Elliot, M.N., Hanser, L.M., & Gilroy, C.L. (2002). Career Academies: Additional evidence of positive student outcomes. Journal of Education for Students Placed at Risk (JESPAR), 7, (1), 71-90. Classwide Peer Tutoring Neddenriep, C.E.(2003). Classwide peer tutoring: Three experiments investigating the generalized effects of increased oral reading fluency to silent reading comprehension. Unpublished doctoral dissertation, University of Tennessee. duration < 12 weeks inadequate outcome measure no adequate control group - pretest differences > 1/2 SD inadequate control groups

Stevens, M.L. (1998). Effects of classwide peer tutoring on the classroom behavior and academic performance of students with ADHD. Unpublished doctoral dissertation, Alfred University. Veerkamp, M. B. (2001). The effects of ClassWide Peer Tutoring on the reading achievement of urban middle school students. Unpublished doctoral dissertation, University of Kansas. Corrective Reading Grossen, B. (2004). Success of a direct instruction model at a secondary level school with high-risk students. Reading & Writing Quarterly, 20: 161-178. CRISS Allen, R. (2000, Summer). Before its too late: Giving reading a last chance. ASCD Curriculum Update, 1-3, 6-8. Havens, L. (1993). Project CRISS: Reading, writing, and studying strategies for literature and content strategies. Research Report Pearson, J., & Santa, C. (1995). Students as researchers of their own learning. Journal of Reading, 38(6). 462-469. Santa, C. M. (2004). Project CRISS: Evidence of effectiveness. Research Report. www.projectcriss.com/research Direct Instruction Corrective Reading Program Maggs, A. & Murdoch, R. (1979). Teaching low performers in upper primary and lower secondary to read by direct instruction methods. Reading Education, 4(1), 35-39. Exemplary Center for Reading Instruction (ECRI) Reid, E. R. (2000). Exemplary center for reading instruction (ECRI) validation study. Research Report. Salt Lake City, UT: Reid Foundation. Fluent Reader Palumbo, T.J. (2004). Effects of the Fluent Reader program on reading performance. Unpublished masters thesis, University of Minnesota. Available online: <http://www.tc.umn.edu/~samue001/papers.htm>. Great Leaps Dudley, A.M. (2005). Effects of two fluency methods on the reading performance of secondary students. Unpublished doctoral dissertation, University of Arizona. Mercer, C,.Campbell, K., Miller, W., Mercer, K., & Lane, H. (2000). Effects of a reading fluency intervention for middle schoolers with specific learning disabilities. Learning Disabilities Research and Practice, 15(4), 179-189. Pruitt, B.A. (2000). The effects of "Great Leaps Reading" on the reading fluency of students served in special education. Unpublished doctoral dissertation, University of Kentucky. IRSI Model

duration < 12 weeks inadequate outcome measure

no control group

inadequate outcome measure: uncertain validity and reliability inadequate information on outcome measure validity inadequate outcome measure: uncertain validity and reliability inadequate information on outcome measure validity

no control group

one study with control group, but pretest differences >0.5 SD

duration < 12 weeks

no adequate control group no adequate control group

no control group

Seybert, L. (1998). The development and evaluation of a model of intensive reading strategies instruction for teachers in includive, secondary-level classrooms. Unpublished doctoral dissertation, University of Kansas. McRAT Hoskyn, J. (1992). Multicultural Reading and Thinking: A three year report1989-92. (ERIC Document Reproduction Service No. ED 380 416) Multicultural Reading and Thinking Program Hoskyn, J. et al. (1993). Multicultural reading and thinking program (McRAT). Paper presented at the Annual Meeting of the American Educational Research Association, Atlanta, GA, April 12-16, 1993. Lindamood-Bell Kennedy, K.M. & Backman, J. (1993). Effectiveness of the lindamood auditory discrimination in depth program with students with learning disabilities. Learning Disabilities Research & Practice, 8(4), 253-259. PHAB/DI, WIST Lovett, M., Lacerenza, L., Borden, S., Frijters, J., Steinbach, K. & De Palma Maria (2000). Components of Effective Remediation for Developmental Reading Disabilities: Combining Phonological and Strategy-based Instruction to Improve Outcomes. Journal of E Phono-Graphix Endress, S.A., Holly, W., Marchand-Martella, N.E., & Simmons, J. (2007). Examining the effects of Phono-Graphix on the remediation of reading skills of students with disabilities: A program evaluation. Education and Treatment of Children, 30 (2), 1-20. McGuinness, C., McGuinness, D., & McGuinness, G. (1996). PhonoGraphix: A new method for remediating reading difficulties. Annals of Dyslexia, 46, 73-96. Read Now Algozzine, B. (2003). Effects of Read Now on adolescents with reading difficulties. Charlotte, NC: University of North Carolina at Charlotte. Available online: http://education.uncc.edu/rfalgozz/rnfinal.pdf Read Right Litzenberger, J. (2001). Reading research results: WASL 2001, Using READ RIGHT as an intervention program for at-risk 10th graders. Final report prepared for Read Right Systems and Kent School District. Read Right (1998). Project Report: READ RIGHT Juvenile Detention Pilot Project, Mission Creek Youth Camp, Belfair, Washington. Shelton, WA: Read Right Systems. Mercer, C. D., Campbell, K. U., Miller, M. D., Mercer, K. D., & Lane, H. B. (2000). Effects of a reading fluency intervention for middle schoolers with specific learning diabilities. Learning Disabilities Research & Practice, 15, 179-189. Reading Apprenticeship

no adequate control group - pretest differences > 1/2 SD

no reading outcomes

Inadequate outcome measure: Writing, not reading

duration < 12 weeks

inappropriate control group (not studying reading)

duration < 12 weeks

no control group

duration < 12 weeks; STAR Reading assessment inherent to treatment

insufficient information

no control group no adequate control group: compares groups receiving the treatment for different durations

Greenleaf, C. L., Mueller, F. L., & Cziko, C. (1997). Impact of the Pilot Academic Literacy Course on ninth grade students reading development: Academic year 19961997. A Report to the Stuart Foundation. San Francisco, CACA: WestEd. Reciprocal Teaching Alfassi, M. (2004). Reading to learn: Effects of combined strategy instruction on high school students. Journal of Educational Research, 97(4), 171184. Alfassi, Miriam, "Reading for Meaning: The Efficacy of Reciprocal Teaching in Fostering Reading Comprehension in High School Students in Remedial Reading Classes," American Educational Research Journal, Vol. 35, No. 2, 1998, p. 309-332. Brady, P. L. (1990). Improving the reading comprehension of middle school students through reciprocal teaching and semantic mapping strategies. Unpublished doctoral dissertation, University of OregonEugene. Levin, M. C. (1989). An experimental investigation of reciprocal teaching and informed strategies for learning taught to learningdisabled intermediate school learners. Unpublished doctoral dissertation, Teachers College, Columbia University. Lovett, Maureen W., Susan L. Borden, Patricia M. Warren-Chaplin, Lea Lacerenza, Teresa DeLua, and Rosa Giovinazzo, "Text Comprehension Training for Disabled Readers: An Evaluation of Reciprocal Teaching and Text Analysis Training Programs," Brain & Lang Lysynchuk, L. M., Pressley, M., & Vye, N. J. (1990). Reciprocal teaching improves standardized reading-comprehension performance in poor comprehenders. The Elementary School Journal, 90, 469484. Lysynchuk, L., Pressley, M., & Vye, N. J. (1989, March). Reciprocal Instruction Improves Standardized Reading Comprehension Performance in Poor Grade-School Comprehenders. Paper presented at the Annual Meeting of the American Educational Research Associat Serran, G. (2002). Improving Reading Comprehension: A Comparative Study of Metacognitive Strategies. Unpublished masters thesis, Kean University. Westera, Julia, and Dennis W. Moore, "Reciprocal Teaching of Reading Comprehension in a New Zealand High School," Psychology in the Schools, Vol. 32, No. 3, 1995, p. 225-232. Talent Development High School McPartland, J., Balfanz, R., Jordan, W., & Legters, N. (1998). Improving climate and achievement in a troubled urban high school through the Talent Development model. Journal of Students Placed at Risk, 3, 337-361. REWARDS Gowan, D.W. (2006). REWARDS: Structural analysis as an effective means for decoding multisyllabic words. Unpublished doctoral dissertation, California State University. Spalding Method

no control group

duration < 12 weeks

duration < 12 weeks

duration < 12 weeks

duration < 12 weeks

duration < 12 weeks

Berkeley et al

duration < 12 weeks

duration < 12 weeks

duration < 12 weeks

duration < 12 weeks

no adequate control group

no adequate control group - pretest differences > 1/2 SD

Mohler, G.M. (2002). The effect of direct instruction in phonemic awareness, multisensory phonics, and fluency on the basic reading skills of low-ability seventh grade students. Unpublished doctoral dissertation, University of Nebraska. Thinking Works Lynch, M.T. (2001). The effects of strategy instruction on reading comprehension in junior high students. Unpublished doctoral dissertation, University of Toledo. Wilson Reading System Moats, Louisa C. (1998). Reading, Spelling and Writing Disabilities in the Middle Grades in Learning About Learning Disabilities, Wong, B (ed.) Orlando FL; Academic Press. Moccia, J. (2005). The influence of multi-sensory, multi-component reading intervention strategies with middle school poor readers . Unpublished doctoral dissertationl University. OConnor, J. R., & Wilson, B. A. (1995). Effectiveness of the Wilson Reading System used in public school training. In C. McIntyre & J. Pickering (Eds.), Clinical studies of multisensory structured language education (pp. 247254). Salem, OR: Internationa Reuter, H.B. (2006). Phonological awareness instruction for middle school students with disabilities: A scripted multisensory intervention. Unpublished doctoral dissertation, University of Oregon. Wilson, B.A. & O'Connor, J.R. (1995). Effectiveness of the Wilson Reading System used in public school training. In McIntyre, C. and Pickering, J. (Eds.). Clinical Studies of Multisensory Structured Language Education. Salem, OR: International Multisensor

no control group

no adequate control group

no control group, pre and post only no adequate control group

no control group no adequate control group - pretest differences > 1/2 SD no control group

OTHER INSTRUCTIONAL PROCESS STRATEGIES

Alfassi, M. (2000). The Use of Technology (ICT) as a Medium for Fostering Literacy and Facilitating Discourse within the Classroom, Educational Media International, 37(3), 137148. Anderson, V. & Roit, M. (1993). Planning and implementing collaborative strategy instruction for delayed readers in grades 6-10. The Elementary School Journal, 94(2), 121-137. Bakken, J. P., Mastropierri, M. A., & Scruggs, T. E. (1997). Reading comprehension of expository science materials and students with learning disabilities: A comparison of strategies. Journal of Special Education, 31, 300-325. Bhattacharya, A., & Ehri, L. C. (2004). Graphosyllabic analysis helps adolescent struggling readers read and spell words. Journal of Learning Disabilities, 37, 331348. Bos, C. S., Anders, P. L., Filip, D., & Jaffe, L. (1989). The effects of an interactive instructional strategy for enhancing reading comprehension and content area learning for students with learning disabilities. Journal of Learning Disabilities, 22, 384 Boyle, J. R. (1996). The effects of cognitive mapping strategy on the literal and inferential comprehension of students with mild disabilities. Learning Disability Quarterly, 12, 86-99. no control group quantitative data not provided Berkeley et al Pearson et al

duration < 12 weeks

duration < 12 weeks

duration < 12 weeks

duration < 12 weeks

Berkeley et al

Bryant, D. P. Vaughn, S., Linan-Thompson, S., Ugel, N., Hamff, A., & Hougen, M. (2000). Reading outcomes for students with and without reading disabilities in general education middle-school content area classes. Learning Disability Quarterly, 23, 238-252 Campbell, B.W. (2002). Genre Studies: Temporary homogeneous grouping to improve reading or merely another form of tracking? Unpublished doctoral dissertation, University of San Diego. DiCecco, V.M. & Gleason, M.M. (2002). Using graphic organizers to attain relational knowledge from expository text. Journal of Learning Disabilities, 35(4), 306-320. Dickson, S. & Bursuck, W. (1999). Implementing a Model for Preventing Reading Failure: A report from the field. Learning Disabilities Research & Practice, 14(4), 191-202 Dole, J. A., Brown, K. J., & Trathen, W. (1996). The effects of strategy instruction on the comprehension performance of at-risk students. Reading Research Quarterly, 31, 62-88. Erickson, E.A. (2006). A comparison of reading growth between Intensive Supplemental Reading and Second Chance Reading for the Des Moines Public Schools, 2002--2005. Unpublished doctoral dissertation, Drake University. Esser, M.S. (2001). The effects of metacognitive strategy training and attribution retraining on reading comprehension in African-American students with learning disabilities. Unpublished doctoral dissertation, University of Wisconsin-Milwaukee. Jitendra, A., Hoppes, M. K., & Xin, Y. P. (2000). Enhancing main idea comprehension for students with learning problems: The role of a summarization strategy and self-monitoring instruction. The Journal of Special Education, 34, 127139. Katims, D.S. & Harris, S. (1997). Improving the reading comprehension of middle school students in inclusive classrooms. Journal of Adolescent and Adult Literacy, 41, 116-123. Klingner, J.K. & Vaughn, S. (1996). Reciprocal teaching of reading comprehension strategies for students with learning disabilities who use English as a second language. The Elementary School Journal, 96, 275-293. Ligas, M. R. (2002). Evaluation of Broward County Alliance of Quality Schools Project. Journal of Education for Students Placed at Risk, 7(2), 117-139. Malmgren, K. W. & Leaone, P. E. (2000). Effects of a shortterm auxiliary reading program on the reading skills of incarcerated youth. Education and Treatment of Chidren, 23, 239-247. Manset-Williamson, G. & Nelson, J.M. (2005). Balanced, strategic reading instruction for upper-elementary and middle school studnets with readig disabilities: A comparative study of two approaches. Learning Disability Quarterly, 28, 59-74. Mastropieri, M.A., Scruggs, T.E., Hamilton, S.L., Wolfe, S., Whedon, C., & Canevaro, A. (1996). Promoting thinking skills of students with learning disabilities: Effects on recall and comprehension of expository prose. Exceptionality, 6, 1-11.

no adequate control group pretest equivalence not established inadequate outcome measure no control group, pre and post only duration < 12 weeks no adequate control group - pretest differences > 1/2 sd

Berkeley et al

Berkeley et al

duration < 12 weeks

Berkeley et al COI / Berkeley et al Berkeley et al Berkeley et al

duration < 12 weeks

duration < 12 weeks

duration < 12 weeks

no control group Lingo et al (2006) Berkeley et al.

no control group

duration < 12 weeks

duration < 12 weeks

Berkeley et al.

Mastropieri, M.A., Scruggs, T.E., Mohler, L., Beranek, M., Boon, R., Spencer, V., & Talbott, E. (2001). Can middle school students with serious reading difficulties help each other and learn anything? Learning Disabilities Research & Practice, 16, 18-27. Nesbitt, J. S. & Wang, N. (2007, April). An evaluation of a high school reading remediation program. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL. Newborn, S.L. (1998). The effects of instructional settings on the efficacy of strategy instruction for students with learning disabilities. Dissertation Abstracts International, 59(05A), 1527. Penney, C. G. (2002). Teaching decoding skills to poor readers in high school. Journal of Literacy Research, 34, 99118. Peverly, S.T. & Wood, R. (2001). The effects of adjunct questions and feedback on improving the reading comprehension skills of learningdisabled adolescents. Contemporary Educational Psychology, 26, 2543. Reeve, J., Jang, H., Carrell, D., Jeon, S., & Barch, B. (2004). Enhancing students engagement by increasing teachers autonomy support. Motivation and Emotion, 28, 147169. Sandora, C.A.(1995). A comparison of two discussion techniques: Great Books (post-reading) and Questioning the Author (on-line) on students' comprehension and interpretation of narrative texts. Unpublished doctoral dissertation, University of Pittsburgh. Sandora, C., Beck, I., & McKeown, M. (1999). A comparison of two discussion strategies on students comprehension and interpretation of complex literature. Journal of Reading Psychology, 20, 177212. Topping K. J. and Fisher, A. M. (2003). Computerized formative assessment of reading comprehension: Field trials in the UK. Journal of Research in Reading. 26 (3), 2267-279. Ugel, N.S. (1999). The effects of a multicomponent reading intervention on the reading achievement of middle school students with learning disabilities. Dissertation Abstracts International, 61(01A), 118. Wilder, A., & Williams, J. (2001). Students with severe learning disabilities can learn higher order comprehension skills. Journal of Educational Psychology, 93, 268278. Williams, J.P. (1998). Improving the comprehension of disabled readers. Annals of Dyslexia, 48, 213-238.

duration < 12 weeks

Berkeley et al.

insufficient information on control group pretests duration < 12 weeks pretest differences > 0.5 SD duration < 12 weeks duration < 12 weeks; inadequate outcome measure inadequate outcome measure Berkeley et al. COI Berkeley et al.

COI

duration < 12 weeks

no control group

duration < 12 weeks

Berkeley et al.

inadequate outcome measure duration < 12 weeks, inadequate outcome measure Berkeley et al.

Appendix 2 Table of Abbreviations ACT- American College Testing AIMS- Arizona's Instrument to Measure Standards ANCOVA- Analysis of Covariance AR- Accelerated Reader C- Control CAI- Computer-Assisted Instruction CAT- California Achievement Test CIRC- Cooperative Integrated Reading Composition CRAB- Comprehensive Reading Assessment Battery CSR- Comprehensive School Reform CTBS- Comprehensive Test of Basic Skills ELL- English Language Learner ERIC- Educational Resources Information Center ES- Effect Size GMRT Gates-MacGinitie Reading Tests HISD Hallsville Independent School District IES Institute of Education Sciences IP- Instructional Processes ITBS- Iowa Tests of Basic Skills LEP- Limited English Proficient M- Matched MANCOVA- Multivariate Analysis of Variance MAT- Metropolitan Achievement Test MCT- Mississippi Curriculum Test MISD- Midway Independent School District MPH- Matched Post-Hoc NAEP- National Assessment of Educational Progress NAPT- Norm-Referenced Assessment Program NCE- Normal Curve Equivalence NSF- National Science Foundation PALS- Peer-Assisted Learning Strategies PSSA- Pennsylvania System of School Assessment RE- Randomized Experiment RQE- Randomized Quasi-Experiment SAT- Stanford Achievement Test SD- Standard Deviation SDRCT- Stanford Diagnostic Reading Comprehension Test SES- Socioeconomic Status SIM- Strategy Intervention Model SSA- State Standards Assessment STL- Student Team Literature

STR- Student Team Reading STW- Student Team Writing T- Treatment TAKS- Texas Assessment of Knowledge and Skills TDHS- Talent Development High School TDMS- Talent Development Middle School TLI- Texas Learning Index TORC- Test of Reading Comprehension WASL- Washington Assessment of Student Learning WISC- Wechsler Intelligence Scale for Children WJ- Woodcock-Johnson