Reading & Writing Quarterly, 17: 5–23, 2001

Copyright Ó 2001 Taylor & Francis

1057-3569 /01 $12.00 1 .00

TE XTB OOK E VA LUA TION A ND A DOPTION

PRA CTICE S

Ma r c y S te in , P h .D.
Un iv e r s ity o f Wa s h in g to n , Ta c o m a , US A

C a r o l S tu e n , Ed .D.
S e a ttle P a c i c Un iv e r s ity , US A

Do u g la s C a r n in e , P h .D.
Un iv e r s ity o f Or e go n , US A

R o g e r M. Lo n g , M.Ed .
S e a ttle P a c i c Un iv e r s ity , US A

G iv en th e p r ev a le n ce o f co m m e rc ially d ev elo p e d in s tr u ctio n a l m a ter ials in cla ss -

r o om s th ro u g h o u t th e Un ited S tate s , th e p u rp os e o f th is a r ticle is to e n co u ra g e
e d u c a to rs to ca re fu lly e x a m in e th e tex tbo ok a d op tion p r o ce ss , e sp ec ially th e w a y
in w h ic h a d o p tio n co m m ittee s e va lu a te a n d se lec t in stru c tion a l m a te ria ls. Wh ile
th e e va lu a tio n a n d a d op tion o f th e se m a te ria ls is d icta ted in p a r t b y sta te a n d
loc a l p olicy , th e p ro ce ss o f tex tb oo k a d o p tio n is cr itica l to th e se lec tion o f h ig h -
q u a lity m a ter ials . In th is a r ticle , w e re v iew th e a va ilab le re se a r ch liter a tu re o n
th e tex tbo ok a d o p tio n p ro ce s s a n d in clu d e r ec o m m e n d a tio n s fo r im p r ov in g th e
p ro ce s s. In a d d ition , w e d is cu s s g u id e lin es fo r d e sig n in g e va lu a tio n c riteria
th a t c a n b e u se d in th e se lec tion of in stru c tion a l m a ter ials . S p ec i c ex a m p les o f
s c re en in g in str u m e n ts a n d a n a d op tion tim elin e a r e p ro vid ed a n d d isc u ss ed .

Given the prevalence of commercially developed ins tructional

materials and their potential impact on s tudent achievement, the
purpos e of this article is to encourage educators to examine carefully
the proces s es by which they s elect materials and, more s peci cally,
the procedures they us e to evaluate the materials . The examples we
us e in this article are drawn primarily from the area of reading.

Address correspondence to Marcy Stein, Univers ity of Was hington, Tac oma, B ox
358435, 1900 Commerce Street, Tacoma, WA 98402-3100. E-mail : mstein@
u.was hington.edu.

5
6 M. S te in et a l.

However, the dis cus s ion of the textbook adoption and evaluation pro-
ces s es pertains to other content areas as well.
Es timates vary regarding the current us e of commercially devel-
oped ins tructional materials in American clas s rooms . Indeed, es ti-
mates s ugges t that textbook s s erve as the bas is for 75 to 90 percent of
clas s room ins truction (Farr, Tulley & Powell, 1987 ; Miller 1986 ;
Tyson & Woodward, 1989). Chall and Squire (1991) reported that
expenditures for bas al reading programs accounted for at leas t two-
thirds of the total dollars allocated for reading ins truction, and that
bas al reading programs were us ed in more than 95 percent of all
s chool dis tricts . At that time, the authors ack nowledged that the
wides pread acceptance of a whole language approach to beginning
reading might have reduced teachers Ä us e of bas al programs . More
recently, in a s urvey of 1000 members of the International Reading
As s ociation, B aumann and Heubach (1996) found that only 12 percent
of the 563 res ponding members held a philos ophical orientation that
precluded the us e of publis hed bas al reading materials . These nd-
ings s upport thos e of Canney and Neuenfeldt (1993), who found that
des pite the movement toward more literature-bas ed clas s room
ins truction in reading, 66 percent of the (pr edominantly elementary)
teachers s urveyed preferred to teach reading us ing a combination of
bas al materials and trade book s .
In light of recent educational reform movements , many educators
are reexamining the role that commercially developed materials play
in the clas s room (B all & Cohen, 1996) and the role of bas al reading
materials in particular (B ak er, KameÄenui, Simmons , & Stahl, 1994).
The above evidence s uggests that commercially developed materials
remain predominant in mos t clas s rooms today.
It s hould be noted that this article is not written with a s peci c
s tudent population in mind. However, readers s hould unders tand that
an as s umption underlying our work is that better des igned ins truc-
tional materials would mos t lik ely have their greates t impact on low-
performing s tudents or thos e s tudents with dis abilities who are
receiving mos t, if not all, of their ins truction in the general education
clas s room. While we unders tand that no program will meet the needs
of all s tudents , we feel s trongly that teachers s hould have acces s to
ins tructional materials that as s is t them in being e ective with all of
their s tudents , not jus t the average and high-performing ones .
On the following pages , we rs t outline what is k nown about how
mos t textbook adoptions are conducted and provide recommendations
for improving the process . We then dis cus s at length the critical
components of any textbook adoption, that is , the ins tructional
evaluation.
 Te x tb oo k Ev a lu a tion a n d A d o p tio n P ra ctices 7

TH E TEXTB OOK AD OP TION P R OC ESS

Relatively little res earch has been conducted on the textbook adop-
tion proces s , which is s urpris ing given that commercially developed
ins tructional materials have an impact on a large number of teachers
and s tudents acros s the United States . We found that mos t res earch
about the adoption proces s was written between 10 and 15 years ago.
However, our more recent experiences with local textbook adoption
committees s ugges t that ndings from that res earch literature are
cons is tent with current practice.
The adoption of ins tructional materials is partially dictated by the
policies of individual s tates and local dis tricts. Currently, 22 s tates
conduct textbook adoptions at the s tate level, which involves a cen-
tralized evaluation and s election proces s , while 28 s tates are con-
s idered ÃÃfree.ÄÄ That is , individual s chool dis tricts are free to s elect
textbook s they deem to bes t meet the needs of their local com-
munities . While adoption procedures at the s tate and local levels
vary cons iderably (Tulley & Farr, 1990), the impact of s tate-level
adoptions on the development of ins tructional materials s hould not
be minimized. In particular, California and Texas , the two larges t
adoption s tates , together account for 11% of the total amount s pent
on textbook s and related materials in the United States (American
As s ociation of Publis hers , 1987). The s igni cance of two s tates
holding s uch a large s hare of the mark et is widely recognized. Publi-
s hers have been k nown not only to time the production of the newes t
editions of their textbook s to coincide with the adoption cycles of
Texas and California, but als o to des ign their ins tructional programs
to align with the curriculum objectives or s tandards identi ed in
thos e s tates .
Generally, s tate adoptions involve two tiers of review and s elec-
tion. The rs t is by members of a s tate level panel who review the
s ubmitted materials and s elect a limited number for inclus ion on an
approved lis t. Since local dis tricts mus t choos e ins tructional
materials from this lis t, a s econd tier of review and s election is con-
ducted by an adoption committee at the dis trict or s chool level. This
adoption committee reviews textbook s on the lis t and s elects thos e
materials that bes t meet dis trict or s chool needs . Local adoption in
free s tates , on the other hand, is a one-level proces s in that dis trict
adoption committees s elect materials without the res triction of a
s tate-impos ed lis t.
Over the years , proponents of s tate adoptions have o ered s everal
arguments in s upport of a s tatewide s ys tem of textbook evaluation
and adoption. Thes e advocates s ugges t that purchas ing ins tructional
8 M. S te in et a l.

materials in large quantities res ults in lower prices , that adoption is

done on a more regular bas is when it is regulated by the s tate, and
that teaching and learning are more cons is tent acros s s chool dis tricts
when dis tricts are limited in the number of options they have.
However, Farr, Tulley, and Rayford (1987) found that as ide from the
cos t, (non-adoption s tates paid approximately $1.00 more per book ),
there were no s igni cant dis advantages to adoptions done at the
local level in thes e free s tates .
In fact, Tulley and Farr (1990) s ugges t that there may even be
certain advantages to adoptions done at the local level. For example,
when compared to a two-tiered adoption proces s , adoptions only at
the local level may encourage teachers and adminis trators engaged
in the s election of ins tructional materials to more thoroughly analyze
the materials . Regardles s of whether the adoption is carried out at
the s tate and then local level, or only at the local level, res earchers
agree that the evaluation process its elf is critical to the s election of
high quality and relevant textbook s and related materials (Chall &
Squire, 1991 ; Farr, Tulley, & Rayford, 1987 ; Miller, 1986). The remain-
der of this article will addres s is s ues relevant to conducting a text-
book adoption at the local level.

W eaknesses in the Adoption P rocess

Res earchers have identi ed s everal weak nes s es in the adoption
proces s that may potentially limit the validity of nal s election deci-
s ions (Farr, Tulley, & Powell, 1987; Ros s , 1989 ; Tulley & Farr, 1990).
Perhaps the mos t s igni cant weak nes s they identi ed is the lack of
training of educators s erving on adoption committees . According to
Farr, Tulley, and Powell (1987), teachers are generally not o ered
any training in the evaluation of ins tructional materials , either in
their teacher preparation programs or as members of adoption com-
mittees . Adoption committee members , therefore, tend to evaluate
commercially developed materials with only limited attention to
res earch (Ros s , 1989) and without s ys tematic s tandards and pro-
cedures (Farr, Tulley, & Powell, 1987 ; Tulley & Farr, 1990). Thus,
rather than participating in informed, rigorous ins pections of
materials , committee members are often left to mak e adoption deci-
s ions bas ed on intuition, familiarity with k nown publis hers , and per-
s onal preferences (Ros s , 1989).
A s econd, frequently dis cus s ed weak nes s in the textbook adoption
proces s is the lack of time allocated to it. Textbook adoptions are
generally conducted over the cours e of one s chool year, although
only a fraction of this time, perhaps two months , is actually s pent
 Te x tb oo k Ev a lu a tion a n d A d o p tio n P ra ctices 9

carefully evaluating materials (Farr, Tulley, & Powell, 1987). The

problem is accentuated in that teachers and adminis trators are often
not provided releas e time to do the s erious and time cons uming
bus ines s of ins tructional evaluation, but rather are expected to do
this in addition to their regular teaching res pons ibilities . Such time
cons traints can lead to decis ions bas ed on the notorious ÃÃ ip tes t,ÄÄ
that is , teachers mus t s ettle for a brief and s uper cial examination of
the materials .
Related to the is s ue of lack of allocated time is the is s ue of the ÃÃall
teacher vote.ÄÄ The all teacher vote is a practice that allows all tea-
chers in a s chool dis trict to vote on the nal s election of an ins truc-
tional program or textbook . Res earch s ugges ts that in s uch s ituations
teachers tend to vote after only brief and limited examination of
materials . The res ult is that nal recommendations are made bas ed
not on the quality, but rather on the quantity of evaluations . Farr,
Tulley, and Powell (1987) noted that the ÃÃall teacher voteÄÄ not only
limits the validity of the nal decis ion, but als o compromis es the role
of the committee.
Finally, a s erious weak nes s in the adoption proces s s eems to be the
lack of res earch-bas ed criteria available for evaluating and s electing
ins tructional materials (Farr, Tulley, & Powell, 1987 ; Ros s , 1989 ;
Tulley & Farr, 1990). The topic of evaluation criteria is dis cus s ed in
the next s ection, but it is important to note here that textbook adop-
tion is often conducted without s peci c guidelines on which to bas e
the s election. Rather, criteria often tak e the form of predetermined
checklis ts s upplied by publis hers or adaptations of check lis ts us ed by
other committees . Such checklis ts do not encourage comprehens ive
examination of materials ; ins tead, they limit reviews to brief and
rather generic evaluations .
Certainly, there are numerous other factors that may advers ely
a ect the work of adoption committees. Some dis tricts may face
nancial constraints that preclude them from providing s ufficient
releas e time for members of the committee to examine textbook s .
Other cons traints may include pres s ure from publis hers Ä repres enta-
tives , pres s ure from concerned citizens , and lack of direction due to
ine ective leaders hip.

R ecommendations to Improve the Adoption P rocess

When appr oaching a textbook adoption cycle, a s chool dis trict has
the opportunity to s trengthen the evaluation and s election proces s by
s erious ly s tudying and implementing the recommendations of
res earchers in the eld. Tulley and Farr provide a concis e and us eful
10 M. S te in et a l.

District Decision to Develop Curriculum and/or Select Instructional Materials

Administration Establishes Parameters for Development and/or Adoption

· Curriculum Adoption Cycle
· B udget Cons iderations
· Timeline for Adoption and Implementation
· Staf ng
· General Adoption Policies

Administrator Ass igned to Facilitate Project

· Establish Adoption Procedures
· Determine Committee Members hip
· De ne B udget
· B egin Work with Principals /Other Involved Administrators
· Establish Communication Procedures
· Establish Ground Rules with Publisher Representatives

Curriculum Committee Membership Determined

· Determine Mak e-up of Committee
· Identify Individual Committee Members
· Educate Committee
Committee Review of Research
· Curriculum Wri tten or Instructional Materials Selected B ased on Criteria
Established from Research

Curriculum Committee Mak es Recommendation

to Administrator and B oard for Approval

Purchase of Instructional Materials and/or Publication of Curriculum

Sta Development and In-Service for Teachers & Administrators

Implementation of Curriculum and/or Instructional Materials

Ongoing Review of Implementation

Evaluation of Curriculum and New Instructional Materials

Modi cations of New Program

F IGURE 1 The textbook adoption process.

s ummary of s ome recommendations in their 1990 dis cus s ion of text-

book evaluation and s election. Their s ugges tions include, among
others , allocating s ufficient time for a thorough review of all
materials , organizing committees to promote communication both
within and acros s grade level, and es tablis hing committee
res pons ibility for the s election decis ion. The following s ection con-
tains a dis cus s ion of these s uggestions .

A do pt io n Tim e lin e
Figure 1 pres ents an example of the s equence of events compris ing
a textbook adoption. Note that a s ubs tantial amount of planning
 Te x tb oo k Ev a lu a tion a n d A d o p tio n P ra ctices 11

occurs prior to the establis hment and work of the committee. For
example, budget cons iderations , adoption timeline, criteria for com-
mittee members hip, communication procedures , and ground rules for
work ing with publis hers are determined before the committee begins
its review.
Committee members mus t be given adequate releas e time to review
the materials . Without s uch provis ions , teachers are often forced to
examine materials in a hurried and haphazard fas hion. Recommen-
dations bas ed on s uch reviews are certainly s ubject to ques tion and
may have a negative impact on the nal adoption decis ion. One
recent elementary reading adoption in which two of the pres ent
authors were involved included s ix days of releas e time for teachers
and adminis trators s peci cally intended for the review and evalu-
ation of materials (s ee Figure 2).
The number of days varies depending on the s ubject being evalu-
ated. Review of textbook s for a phys ics , foreign language, or health
clas s , for example, will involve fewer choices , and therefore fewer
days for evaluation. However, it mus t be noted that meaningful
examination of materials requires large block s of uninterrupted time.
Such an allocation of time by the dis trict s ugges ts a s erious com-
mitment to the review proces s and to the e orts of committee
members .

Co m m it t e e Me m be r s h ip
Adoption committee members are often s elected on the bas is of
years of experience and are us ually grouped according to grade level.
While experience is certainly not a factor to be ignored, we have
identi ed s everal additional criteria to cons ider when s electing com-
mittee members . Firs t, committee members s hould not only have an
academic interes t in the curricular area being addr ess ed, but they
s hould als o exhibit excellent interpers onal s k ills , including the
ability to communicate e ectively and hones tly to the group they rep-
res ent. Additionally, it is important that committee members under-
s tand, s upport, and tak e res pons ibility for group proces s activities
and decis ions . Moreover, the committee s hould be repres entative of
the entire s chool/dis trict s tudent and s ta population in terms of
gender, ethnicity, experience, s pecial populations (e.g., s tudents with
dis abilities or s pecial talents ), and community members , when appr o-
priate. Finally, the work of the committee is enhanced when members
are given the opportunity to dis cus s s peci c as pects of the adoption,
both within and acros s grade levels . When this k ind of dialogue is
encouraged, is s ues s uch as development and s equencing of s k ills can
12 M. S te in et a l.

1995–96–S ch o o l Yea r
Day 1 : Octo b e r 27
· Review of district curriculum process
Þ B oard policies
Þ State laws
Þ Time al locations
· What our dis trict students are expected to learn
Þ Ess ential learnings—state as sess ment
Þ Grade level objective
· Discuss ion of research and instructional methodologies
Þ Invited Guest(s): research-based practices
· Completion of District Survey/Questionnaire
Day s 2–3 Dece m b er 5–6
· Review Day 1
· Review s urvey results
· Establish à rs t draftÄ of s creening criteria
Day 4 Dece m b er 20
· Finalize screening criteria
· Field test screening criteria
Day s 5–6–7 Ja n u a ry 22–24
· Initial s creening of 13 programs
· Identify no fewer than 2 or more than 5 Ã nalistsÄ programs
· ÃFirst draftÄ of evaluation tool
Day s 8–9–10 F eb ru a ry 12–14
· Evaluate 2–5 Ã nalistÄ programs
· Reach tentative decision on dis trict adoption
· Correlation of grade level objectives to s elected program
Day 11 F eb ru a ry 28
· Invite à nalistÄ consultants to work with committee
Day s 12–13 Ma r ch 12–13
· Finalize Ãoriginal draftÄ of grade level of objectives to à nalistÄ
program
· Develop communication plan to announce recommendation
to s ta
· Prepare for s chool board presentation/adoption
Day s 14–15
· Develop individual school orders
· Develop sta development program
· Develop plan for program implementation
· Oth e r A ctiv ities
· 3 sta /community open houses for public review of
recommendations
· Community Curriculum Advisory Council (CCAC)—
mid-April
· Cons ultants with Title I teachers —early April
· Cons ultants with special education/k indergarten teachers—
early April
· School B oard—April 22
· Purchase orders prepared—May 1
· Sta development plan nalized—May 15
· Implementation of new adoption—September, 1996
· Summer sta development days —August 27 and August 28

F IGURE 2 Sample adoption timeline for reading curriculum adoption com-

mittee.
 Te x tb oo k Ev a lu a tion a n d A d o p tio n P ra ctices 13

be cons idered in the larger context of a s tudentÄs educational experi-

ence. Such thoughtful and purpos eful s election of committee members
is one way to s igni cantly s trengthen the adoption proces s , thereby
increas ing the lik elihood that the nal decis ion will be unders tood
and s upported by the entire educational community.

Co m m it t e e R e s po n s ib ilit ie s
A critical component of the adoption proces s is the direction pro-
vided by the chairpers on. The work of an adoption committee can be
s everely jeopardized without s trong, res pons ible leaders hip. Such
leaders hip includes clearly de ning the res pons ibilities of the com-
mittee at the beginning of the process , de ning parameters for the
committee in terms of dis trict policies , community politics , outlining
budget cons traints , and es tablis hing the lines of authority, that is ,
articulating the decis ion-mak ing proces s . We agree with Tulley and
Farr (1990) that the s election decis ion s hould res t with the adoption
committee. Furthermore, we believe that if appropriate training is
provided and s ufficient time is allowed for review of materials ,
members of the committee will be prepared to mak e an informed
decision.
Tulley and Farr, among others , have emphas ized the need for
improving s election criteria and evaluation procedures . In the next
s ection, we provide s peci c recommendations for evaluating the
ins tructional integrity of the textbook s and ins tructional programs .

EVALUATING TH E INSTR UC TIONAL INTEGR ITY OF

INSTR UC TIONAL M ATERIALS

The central activity of the textbook adoption process is the evalu-

ation of the textbook materials . The adoption committee is charged
with the s election of materials that are bas ed both on a coherent
body of educational research in a given content area (i.e., literacy,
mathematics , s cience) and on s ound principles of ins tructional
des ign. Therefore, prior to the examination of ins tructional materials ,
the committee s hould be given adequate time and as s is tance to
review relevant res earch in the content area.
Many teacher preparation programs do not include courses in the
unders tanding of educational res earch or even include the analys is of
res earch in their coursework (Stanovich, 1993/1994). As s uch, the
adminis trator in charge of the adoption may elect to hire a cons ul-
tant to as s is t the committee with reading and interpreting the
res earch literature. The cons ultant s hould be s omeone with expertis e
14 M. S te in et a l.

in the targeted content area who als o has experience reading and
interpreting research literature. Moreover, the cons ultant s hould
unders tand that his or her role is that of an impartial participant
available to help committee members review the res earch and impli-
cations of res earch ndings on the des ign of the content of ins truc-
tional materials and clas s room ins truction.

D esigning Screening Instruments

After reviewing relevant res earch and dis cus s ing how that res earch
s hould be re ected in ins tructional materials , the committee needs to
generate two s ets of criteria for evaluation. The rs t s et of criteria is
the s creening criteria. In areas s uch as reading, the number of com-
mercially developed ins tructional programs can be s o overwhelming
that evaluating all of them would be untenable. In textbook adop-
tions where numerous programs have been s ubmitted for con-
s ideration, the us e of s creening criteria can help reduce the
number of programs that the committee needs to evaluate more
comprehens ively.
To generate s creening criteria, a committee needs to agree on 2–3
critical content components to examine in all programs . In the
reading adoption in which two of the pres ent authors participated,
the primary grade teachers agreed, bas ed on the beginning reading
res earch literature, that the predominant ins tructional approach to
beginning reading ins truction evident in the beginning levels was
critical to the s ucces s of their s tudents . As a res ult, this committee
des igned a s creening ins trument for the primary grade levels that
look ed for evidence that the program employed an explicit phonics
approach s uppor ted by cons iderable res earch. If the reading program
promoted an explicit phonics approach, the text s elections would be
more lik ely to corres pond to the phonics les s ons provided in the
teacher manual. In addition, the text s elections would contain a
higher percentage of decodable words (i.e., words that can be s ounded
out) than s ight words . As a res ult of their analys is , the primary tea-
chers s elected two items for their beginning reading s creening ins tru-
ment. One item focus ed on the programÄs ins tructional approach, and
the s econd item required examination of the relations hip between the
identi ed approach and the text materials provided for s tudents to
read. (See Figure 3 for an example of an initial s creening ins trument
us ed by primary grade teachers in a reading curriculum adoption.)
In this adoption, the intermediate grade teachers on the committee
were concerned about ins truction in the areas of s tudy s k ills and
 INITIA L S CRE E NING INS TRUME NT—PRIMA RY
Ma te ria ls Need ed : TeacherÄs Manual for First and Third Grades

A. Approaches to B eginning Reading

Name of Program

1. Do the early text selections in the

rs t grade readers corres pond to
phonics ins truction in the teacher-
directed les sons ?

2. Are the words in the early text

selections decodable* words or
sight words ?

B. Decoding Instructions

1. Are there teacher-directed

decoding instructions in third
grade ?

No te . * 5 Able to be sounded out

FIGURE 3 Sample textbook adoption initial s creening ins trument for primary grades.

15
16 M. S te in et a l.

content area reading, as well as the balance of ction and non ction
text s elections . As a res ult of their dis cus s ions , thos e teachers
des igned a s creening ins trument that included examining the avail-
able ins truction in the areas of s tudy s k ills and content area reading
(s peci cally, vocabulary and main idea ins truction). In addition, the
intermediate teachers counted the number of ction and non ction
s elections at a given grade level to determine balance of text s elec-
tions . Since mos t major bas al reading programs have a general des ign
for the introduction and review of s k ills , teachers predicted that the
examination of these carefully s elected s k ills would lik ely re ect how
other s k ills were addres s ed in the program. (See Figure 4 for an
example of an initial s creening ins trument us ed by intermediate
grade teachers in a reading curriculum adoption.)

D esigning Evaluation Instruments

The goal of the s creening activity is to generate a more manageable
number of textbook s or ins tructional programs to be evaluated. Once
the number of programs to be evaluated is reduced, committee
members are ready to begin a more comprehens ive evaluation
proces s . The rs t tas k in this proces s involves es tablis hing the nal
evaluation criteria and des igning evaluation work s heets . The evalu-
ation criteria s hould be generated in the s ame manner as the s creen-
ing criteria. That is , the criteria s hould be bas ed on the available
res earch literature in the content area as well as on s ound principles
of ins tructional des ign. Moreover, committee members s hould s trive
to des ign criteria that are objective in nature and veri able. For
example, counting the number of di erent types of text s elections is
preferable to commenting on whether the programs provide a balance
of ction and non ction text s elections . Des igning objective evalu-
ation criteria and requiring evaluators to document their evaluations
by citing actual page numbers from the programs helps reduce evalu-
ator bias and maintain a profes s ional atmos phere during what can
s ometimes prove to be a s tress ful proces s .
It is important to note that an as s umption underlying the textbook
evaluation proces s is that the proces s is conducted to determine the
ins tructional integrity of ins tructional materials being cons idered for
purchas e. Teachers are cons tantly faced with modifying and adapting
materials to meet the needs of their s tudents . We unders tand that the
perfect program does not exis t and that good teaching involves moni-
toring s tudent progres s and adjus ting ins truction to meet individual
needs . However, the goal of the evaluation proces s s hould be to objec-
 INITIA L S CRE E NING INS TRUME NT—INTE RME DIA TE
Ma ter ia ls Ne ed ed : TeacherÄs Manual for Fourth, Fifth, and Sixth Grade Levels

A. S tudy S kills Instruction

List the study sk ills taught in each

of the grade levels . (Verify
instruction for tw o study s k ills.)

B. Type of Text S elections

Count the number of ction and

non- ction text selections in the
fth grade teacherÄs manuals .

C. Instruction in Content Area Reading

Count the number of activities

designed to teach vocabulary
related to the content text
selections .

Count the number of main idea

activities available that are related
to content area reading.

F IGURE 4 Sample textbook adoption initial s creening ins trument for intermediate grades.

17
18 M. S te in et a l.

tively s elect thos e materials that will be the mos t us eful, requiring
the leas t amount of modi cation and change.
Since it is beyond the s cope of this article to dis cus s the res earch
literature in s peci c content areas , we will dis cus s ins tead a s et of
guidelines for evaluating materials that are bas ed on empirically
derived principles of ins tructional des ign (Stein, Carnine, & Dixon,
1998). Thes e guidelines include content organization around big
ideas ; the pres ence of explicit, generalizable s trategies ; opportunities
for s ca olded ins truction ; the s trategic integration of s k ills and con-
cepts ; and judicious review. On the following pages , we dis cuss each
of thes e guidelines , as well as provide examples of how to apply them
to evaluating ins tructional materials in s peci c content areas .

Gu ide lin e 1: Is Co n t e n t O r g an ize d A r o u n d B ig Ide as ?

Porter (1989) obs erved that a relatively large number of topics
received brief coverage in many publis hed programs . He reported that
teachers referred to this phenomenon as ÃÃteaching for expos ure.ÄÄ
When analyzing the content organization or coverage in any of the
given ins tructional materials , adoption committee members s hould be
cautioned to examine carefully the s cope and s equence charts provid-
ed in the materials . Scope and s equence charts often illus trate the
phenomenon that PorterÄs teachers labeled as ÃÃteaching for expo-
s ure.ÄÄ The number of di erent topics included in any one given level
of a publis hed program often appears extens ive. However, a clos er
examination of the materials often reveals limited ins truction on any
s ingle topic.
An alternative to limited expos ure to many di erent topics is the
organization of content around ÃÃbig ideas .ÄÄ Thes e big ideas are criti-
cal concepts that are es s ential for content mas tery in a given s ubject.
For example, the authors of a recently publis hed American his tory
text organized the content of the text us ing the big idea of the
problem/s olution text s tructure. In this his tory text, s tudents are
introduced to the big idea that people are often faced with two k inds
of problems : economic problems and rights of people. Then, s tudents
are taught another big idea that people tend to s olve their problems
in one of ve ways : move, dominate, invent, tolerate, and accommo-
date (Carnine, Crawford, Harnis s , & Hollenbeck , 1995).
Once s tudents learn this framework , they can apply the problem/
s olution s trategy to the major events in his tory. In doing s o, they
gain an unders tanding of the underlying causes of major his toric
events as well as ins ights into the interrelations hips among events .
Determining whether the ins truction in a given textbook or program
 Te x tb oo k Ev a lu a tion a n d A d o p tio n P ra ctices 19

is focus ed and comprehens ive s hould be one of the rs t criteria in

evaluating ins tructional materials .

Gu ide lin e 2: D o t h e Cu r r ic u lu m Mat e r ials Co n t ain

E x plic it S t r at e g ie s ?
After determining whether the content is organized in a reas onable
way, evaluators need to examine whether the materials provide tea-
chers with explicit s trategies to us e in teaching their s tudents impor-
tant content. These s trategies s hould als o be des igned to promote
generalization of concepts and s k ills . The initial s trategy of s ounding
out words to decode in an example of an explicit s trategy in a begin-
ning reading program. Given that s tudents have the prerequis ite
letter/s ound k nowledge, s tudents can us e this s trategy to decode new
words . In the area of written expres s ion, a s trategy for s elf-editing is
an explicit s trategy, provided s tudents have the prerequis ite k now-
ledge required for editing.

Gu ide lin e 3: D o t h e Cu r r ic u lu m Mat e r ials P ro v ide

O ppo r t u n it ie s f o r Te ac h e r s t o S c a o ld In s t r u c t io n ?
An explicit s trategy contains a s et of overt s teps that s upport s tu-
dents as they move toward mas tery of the s k ills and concepts . Once
the s trategy is identi ed, the proces s for teaching s tudents to apply
that s trategy mus t involve an ins tructional s equence that provides
maximum s upport in the early s tages of acquis ition while gradually
encouraging more independent application of the s k ill or concept.
Ideally, commercially developed materials s hould s erve as a res ource
for teachers that provides them with the means and s trategies for
s upport during initial teaching and a means for monitoring how
much s upport s tudents continue to need.
Mos t commercially developed materials contain an as s es s ment
component. Evaluating the degree to which the as s es s ment rec-
ommendations provide s ufficient information regarding s tudent
progres s s hould als o be an important part of any ins tructional
evaluation.

Gu ide lin e 4: To W h at D e g r e e are t h e S k ills an d Co n c e pt s

In t e n t io n ally an d S t r at e g ic ally In t e g rat e d ?
The degree to which the s k ills and concepts in any given program
are s ys tematically integrated is a critical yet difficult guideline to
implement. A common criticis m agains t the teaching of s peci c s k ills
20 M. S te in et a l.

has been that s k ill ins truction is often pres ented in an is olated, frag-
mented manner. In res pons e to this criticis m, many educators have
chos en to teach s k ills only in the context of a more wholis tic
approach. In s ome mathematics programs , for example, computation
s k ills are taught only in the context of word problems that depict
real-life s ituations . In s ome written express ion programs , grammar is
introduced during the writing proces s , and only as needed. In s ome
reading programs , word identi cation is taught only in the context of
reading literature. A s erious problem with providing only contex-
tualized ins truction is that the nature of that ins truction is not
always s ufficient to ens ure that s tudents will s ucces s fully learn the
concept or s trategy being presented.
An alternative to teaching s k ills only in context is teaching the
s k ills as prerequis ite k nowledge that is later integrated into the
appropriate context. Few advocates of s k ill-bas ed ins truction have
ever cons idered s tudent performance in is olated s k ill exercis es as a
nal s tudent outcome. Thes e educators ack nowledge that s k ills are of
no us e to s tudents if s tudents are not taught explicitly when and how
to apply them in the appropriate context. We have found that the
primary problem with mos t s k ill ins truction is that while the ins truc-
tion is initially pres ented in is olation, the textbook s or ins tructional
programs provide very few examples of teaching s tudents how and
when to appropriately integrate their s k ills .
Sound ins tructional des ign s hould follow a s equence of ins truction
that includes teaching prerequis ite k nowledge, teaching explicit s tra-
tegies that integrate k nowledge and s k ills , and providing
opportunities that encourage s tudents to become automatic in the us e
of the s trategies . A good example of s trategic integration can be
found by examining writing ins truction. B efore s tudents can apply
s elf-editing s trategies , they mus t have the prerequisite k nowledge
that allows them to identify problems with their own writing. Self-
editing is a s trategy that allows the integration of both creative
e orts (i.e., s tructure and organization of content) and more mechani-
cal s k ills (grammar , punctuation, and s pelling). As s tudents begin to
acquire thes e s peci c writing s k ills , they are encouraged to evaluate
their own work by examining the extent to which they have us ed
both creative e orts and mechanical s k ills during their compos ition.
To read expos itory text critically, s tudents mus t integrate numer-
ous as pects of comprehens ion, including determining what the author
wants the reader to unders tand, s earching for evidence that what the
author s ays is true, as s es s ing the credibility of the author, and iden-
tifying contradictions in the text. A s trategy for reading critically
that involves a proces s of s elf-ques tioning can be taught fairly eas ily
 Te x tb oo k Ev a lu a tion a n d A d o p tio n P ra ctices 21

to s tudents . However, prior to the introduction of s uch a s trategy

that integrates a variety of s k ills , each of the concepts s hould be
introduced in is olation.

Gu ide lin e 5: Is t h e R e v ie w P r o v ide d in t h e In s t r u c t io n al

Mat e r ials S u f f ic ie n t , Cu m u lat iv e , an d V arie d ?
The value of review is rarely dis puted among educators. However,
in our experience, rarely is the amount or type of review related to
the value of the ins tructional s trategies pres ented. If an ins tructional
s trategy is of limited us e, then reviewing that s trategy extens ively is
a was te of valuable ins tructional time. Moreover, the type of review
s hould vary as s tudents become more pro cient. The review available
in ins tructional materials can be evaluated by examining the extent
to which the review is s ufficient, cumulative, and varied. Teachers
can begin to compare the extent to which di erent programs provide
adequate review s imply by counting the number of opportunities for
review (e.g., the number of review examples ).
Determining the extent to which the review is cumulative over
time is more difficult to as certain. If s k ills are introduced as prerequi-
s ite k nowledge to be integrated at a later time, teachers need to
determine whether the programs do, in fact, provide s tudents with
opportunities to integrate and practice the s k ills at a later time.
Finally, the practice opportunities s hould vary enough to give s tu-
dents the opportunity to generalize the application of their newly
acquired k nowledge and s k ills to les s s tructured contexts .

C ONC LUSION

Given the prevalence of thes e materials in the clas s room, the impact
of the us e of commercially developed materials on s tudent achieve-
ment is es timated to be quite s ubs tantial. The textbook adoption
proces s is the primary means educators have of ens uring that they
have acces s to well-de s igned ins tructional materials . In this paper, we
have s ugges ted that prior to participating in s uch a proces s , educa-
tors cons ider how that proces s might be organized to yield the bes t
pos s ible s election of ins tructional materials for the s tudents they
s erve.
Central to the implementation of an e ective adoption process is a
procedure for evaluating commercially developed materials . We have
s ugges ted that evaluation criteria us ed in reviewing thes e materials
re ect the current educational res earch literature as well as s ound
principles of ins tructional des ign. We als o s ugges ted that adoption
22 M. S te in et a l.

committee members be provided adequate training and s ufficient time

to thoroughly analyze the textbook s or programs to mak e better-
informed recommendations .
Finally, we believe that the process of evaluating and s electing
textbook s and ins tructional programs can become an important
conduit for communication between educators and program devel-
opers . This communication, hopefully, will encourage the education
community to develop commercially developed materials that s erve to
enhance teacher expertis e and fos ter s tudent growth.
This article on textbook s election clearly emphas izes the role of
res earch and the application of that res earch to ins tructional prac-
tice through the us e of s ound ins tructional des ign principles.
However, we ack nowledge that there exis t other very important
topics that may be addres s ed in the textbook s election proces s . Edu-
cational philos ophy, divers ity of s tudent population (e.g., gender,
race, economic s tatus), as well as s peci c community values all may
play a role in the s election of ins tructional materials to meet the
needs of a s peci c s chool dis trict. We have chos en to focus on thos e
features of the evaluation and s election proces s that we feel are mos t
lik ely to have the greates t impact on s tudent achievement. Clearly,
dis tricts involved in textbook adoption need to cons ider our rec-
ommendations and des ign a proces s that tak es into account their
individual needs .

R EFER ENC ES

American As s ociation of Publishers . (1987). A n n u a l in d u s tr y sta tistics . Was hington,

DC : Author.
B ak er, S. K., KameÄenui, E. J ., Simmons, D. C., & Stahl, S. A. (1994). B eginning read-
ing : Educational tools for diverse learners. S c h oo l P sy c h olo g y R e v iew , 23, 372–391.
B all, D. L., & Cohen, D. K. (1996). Reform by the book : What is —or might be—the role
of curriculum materials in teacher learning and instructional reform ? Ed u ca tion a l
R e s ea r ch e r , 25(9), 6–8.
B aumann, J . F., & Heubach, K. M. (1996). Do bas al readers desk ill teachers ? A nation-
al s urvey of educatorÄs use and opinion of basals . Th e Ele m e n tar y S c h oo l Jo u rn a l,
96, 511–526.
Canney, G., & Neuenfeldt, C. (1993). Teachers Ä preferences for reading materials .
R e a d in g Im p r ov em en t, 30, 238–245.
Carnine, D. W., Crawford, D. B ., Harniss, M. K., & Hollenbeck , K. (1995). Un d er sta n d -
in g U.S . Histo r y : V olu m e 1—Th r ou g h th e C iv il Wa r . Eugene, OR: Considerate Publi-
shing.
Chall, J ., & Squire, J . (1991). The publishing industry and textbook s . In R. B arr, M.
Kamil, P. Mos enthal, & P. D. Pearson (Eds .), Ha n d b o o k of R ea d in g R es ea r ch , Vol. 2,
(pp. 120–146). New York : Lawrence Erlbaum.
Farr, R., Tulley, M., & Powell, D. (1987). The evaluation and selection of bas al readers .
Th e Ele m e n tar y S c h o ol Jo u rn a l, 87, 267–281.
 Te x tb oo k Ev a lu a tion a n d A d o p tio n P ra ctices 23

Farr, R., Tulley, M., & Rayford, L. (1987). Selecting basal readers : A comparison of
school dis tricts in adoption and non-adoption s tates. Jo u rn a l o f R e s ea r ch a n d Dev e l-
op m e n t in Ed u ca tion , 20(4), 60–72.
Miller, J . (1986). Evaluation and selection of bas al reading programs. Th e R e a d in g
Tea ch e r, 40, 12–18.
Porter, A. (1989). A curriculum out of balance : The cas e of elementary school mathe-
matics . Ed u c a tio n a l R e se a r ch e r, 18(5), 9–15.
Ros s, E. (1989, April). A m o d el fo r b a s a l r ea d er a d o p tio n . Paper presented at the annual
meeting of the International Reading As sociation, New Orleans, LA.
Stanovich, K. E. (1993/94). Romance and reality. Th e R e a d in g Te a c h er , 47, 280–291.
Stein, M. L., Carnine, D., & Dixon, R. C. (1998). Direct instruction : Integrating curric-
ulum design and e ective teaching practice. In ter ve n tio n in S c h oo l a n d C lin ic, 33,
227–234.
Tulley, M., & Farr, R. (1990). Textbook s and s chooling in the United States. In Elliott,
D. L., & Woodward, A. (Eds .), Th e 89th Yea r b o ok of th e Natio n a l S o c iety fo r S tu d y
a n d Ed u c a tio n , Part 1 (pp. 162–177). National Society for Study in Education.
Tys on, J . & Woodward, A. (1989). Why s tudents arenÄt learning very much from text-
book s. Ed u c a tio n a l Lea d e r sh ip , 47(3), 14–17.