ASSESSING ONLINE READING COMPREHENSION:

A PRACTICALITY SURVEY

Purpose:
The purpose of this survey is to obtain information from leaders in education about how best
to design an assessment of online reading comprehension that is practical. These assessments
of online reading comprehension are being designed for use in the middle schools in a
research grant from the Institute of Educational Sciences (IES) of the U.S. Department of
Education.

Online reading comprehension:

takes place when readers have an important problem and seek to use Internet information
to solve this problem;
includes reading to locate, evaluate, synthesize, and communicate information;
includes both traditional reading comprehension skills and novel comprehension skills that
are required during online information use;
does not always correlate with state reading comprehension assessments, since it
measures additional elements of reading comprehension that are required on the Internet.

An example of online reading comprehension would take place when a student reads on the
Internet to determine why some people need glasses and then shares this information online
with another person who has asked for it.

Directions:
Please complete each question below, to the best of your ability, from the special vantage
point that your position provides. Your responses will be invaluable in helping us develop a
practical assessment of online reading comprehension for middle school students.

Demographic information:

1. Please indicate your current position.

State Commissioner/Superintendent of Education
State Assistant Commissioner/Superintendent of Education
Superintendent of Schools
Asst. Superintendent of Schools
Principal
Other. Please indicate: ________________________________________________

2. Please indicate your primary level of responsibility.

Elementary
Secondary
Tertiary
Multiple. Please indicate: ________________________________________________
Other. Please indicate: ___________________________________________________

1
General Importance:

3. How important is it for a school district/state to gather assessment information about their
students’ ability to read and comprehend online information as defined in the introduction?
Not at all important
Somewhat important
Very important
Essential

Please explain your thinking in a single sentence:

_________________________________________________________________________________

Availability of Technology Requirements:

4. Would a typical school district/state, with which you are familiar, have adequate
technology resources to enable students, in grades 6-8, to take an assessment online?
Yes
No
Uncertain

Please explain your thinking in a single sentence:

_________________________________________________________________________________

Administration Time:

5. How much time should a school district/state devote, each year, to assessing online reading
comprehension at the middle school level?
15 minutes per year?
40 minutes per year.
80 minutes per year.
Other. Please indicate: ___________________________________________________

Please explain your thinking in a single sentence:

_________________________________________________________________________________

Ease of Scoring and Interpretation:

6. To what extent is it important to a school district/state that an assessment of online

reading comprehension be easy to score?
Not at all important
Somewhat important
Very important
Essential

2
7. To what extent is it important to a school district/state that scoring be conducted
automatically?
Not at all important
Somewhat important
Very important
Essential

8. To what extent is it important to a school district/state that an assessment of online

reading comprehension be easy to interpret?
Not at all important
Somewhat important
Very important
Essential

Please explain your thinking in a single sentence about scoring/interpretation:

________________________________________________________________________________

The Extent To Which Meaningful And Relevant Information Is Obtained:

9. Of the following three formats, which provides information that would be most meaningful
and relevant for your district/state needs?
Multiple choice items (easiest to score, lower level of authenticity)
A simulated Internet (moderately hard to score, moderate level of authenticity)
The actual Internet with school filters in place (hardest to score, most authentic)

Please explain your thinking in a single sentence about the extent to which meaningful and
relevant information is obtained:

_________________________________________________________________________________

Type of Information Obtained:

For a school district/state, how important is the following information to know about
students’ ability to read and comprehend online information?

10. The speed with which students successfully read and solve informational problems online?
Not at all important
Somewhat important
Very important
Essential

3
11. The accuracy with which students successfully read and solve informational problems
online?
Not at all important
Somewhat important
Very important
Essential

12. The specific skills during online reading comprehension (locate, evaluate, synthesize, and
communicate) where individual students succeed or fail?
Not at all important
Somewhat important
Very important
Essential

13. The overall performance level of each student relative to other students?
Not at all important
Somewhat important
Very important
Essential

14. Videos of students’ actual online reading performance?

Not at all important
Somewhat important
Very important
Essential

Please explain your thinking in a single sentence about the type of information obtained:

_________________________________________________________________________________

Rank Ordering of Practicality Issues:

Please rank order the importance of the following issues of practicality in assessments. Use a
1 for the most important and a 5 for the least important.

Availability of the technology requirements

Ease of use

Administration time

Ease of scoring

Extent to which meaningful and relevant information is obtained

4
Additional Practicality Concerns:
What do you consider to be the most important aspects of practicality that a school district/
state take into consideration when making decisions about their assessments?

Please list and explain:

_________________________________________________________________________________

_________________________________________________________________________________

_________________________________________________________________________________

_________________________________________________________________________________

_________________________________________________________________________________

_________________________________________________________________________________