Northern Illinois University

Volume 3, Issue 2
December, 2004

The Nuts and Bolts Newsletter from

In this issue: SHARPEN YOUR PENCIL

Adaptive Assessment
Sharpen Your Pencil And so…you give your are too easy or too difficulty for the next
multiple choice tests on difficult. This computerized question; the next test
Call for Proposals: computer. Are there method of testing permits item is then randomly
Portfolio Conference methods and/or the collection of required selected from a pool of
and Capstone Projects technology that can assess feedback with fewer questions at a determined
Profiles in students’ strengths and questions. difficulty level depending
Assessment: Michael weakness on an item-by- Adaptive on whether or not the
Day item basis, and based on questioning is therefore a preceding question was
student responses, allow very efficient and effective answered correctly. For
Best Practice: Writing students to “test out” of means of knowledge- this purpose, several
Skills Rubric material where they based assessment. The pools of questions at
demonstrate proficiency? benefits of this approach various difficulty levels
Did You Know?
There are, and that include: (often called item banks)
Employer Evaluations
methodology is often • Time isn’t wasted on are set up and maintained.
Results
referred to as “computer inappropriate questions The random selection
Contributions adaptive assessment” or • Assessments are not process allows individuals
Solicited “adaptive testing.” burdened with to be assessed more than
Adaptive information that isn’t once in a content area and
FAQ: Assessment assessment/testing needed for a reliable receive different questions
Literature methodologies select measure of proficiency at the same level of
Another FAQ: questions at a specific • Results show areas difficulty each time they
Principles of level of difficulty based of strength and weakness take the test. This process
Assessment on the student’s previous clearly and accurately helps ensure the test
responses. Thus, the test During test result is a true measure of
Students’ Perceptions engine “adapts” the administration, the the individual’s
of Online Courses question selection process adaptive testing engine knowledge, and not a
according to the test- evaluates each response reflection of her or his
Kathleen Blake
Yancey Keynote takers’ abilities, and determines the ability to learn and study
Speaker eliminating questions that appropriate level of test questions.
Page 2

CALL FOR PROPOSALS

Portfolio Workshop and Capstone Projects
The Office of Assessment submitted through the Copies of the
Services (OAS) and the college office. The submission form(s) and
University Assessment completed packet 2004-2005 guidelines can
Panel (UAP) are pleased to (submission form, one be accessed at http://
offer resources to NIU electronic copy of the www.niu.edu/assessment/
faculty and staff to proposal, and 5 _activ/portdev.shtml and
support their efforts in paper copies at http://
using portfolios and of the www.niu.edu/
capstone courses in their proposal) assessment/
assessment programs. must be _activ/
Submission forms for submitted to capdev.shtml.
portfolio/capstone the Office of Please
proposals, as well as the Assessment contact Craig
current guidelines for Services by 1 Barnard,
submitting proposals, were February 2005 to be University
mailed to all faculty, eligible for consideration; Assessment Coordinator at
department chairs, and your college will notify you barnard@niu.edu or 753-
deans in early November of its deadline for the 7120 if you have questions
2004. Proposals should be receipt of proposals. about these initiatives.

PROFILES IN ASSESSMENT
Michael Day, First Year Composition
Michael Day, Associate Professor in
the Department of English, is
beginning his third year as the
Director of First Year Composition
at Northern Illinois University.
Michael indicates that his first
love is in teaching English, and he
has taught a number of courses
in technical writing,
Faster transmission
the course for new teaching
assistants in the department,
which incorporates learning to
use electronic media and the
Internet in English classes.
His most recent involvement
in assessment came through his
interest in and work with
electronic

electronic media, and portfolios in the

composition. He also has Department of
Better quality

a background in rhetoric, English.

and is currently teaching

Click either picture, above, to hear Michael Day’s assessment philosophy. Windows
Media Player is necessary to view these files. Download Windows Media Player for
free.
Page 3

BEST PRACTICE
English Department Writing Skills Rubric
Developed by the NIU Department of English, the following is an excellent example
of a clearly articulated scoring rubric. It has been used for a number of years for inter-
departmental assessments, and has also been used as the scoring rubric for all iterations
of the Office of Assessment Services’ assessment of junior-level writing skills. The
NIU Department of English offers more information on the rubric and related topics.

ENGLISH DEPARTMENT CRITERIA FOR JUNIOR-LEVEL WRITING

Descriptors Upper Half Descriptors Lower Half

4 5 6 3 2 1
Demonstrates and responds to and demonstrates Fails to understand and may lapse in and lacks
adequate the full range of exceptional insight fully the writer’s response to the commitment to
understanding of issues raised by the into the topic. task prompt the task or fails to
the writer’s task prompt address the
while persuading prompt.
reader of writers
commitment
Establishes an and establishes a and uses this voice Establishes an or fails to establish or uses an
appropriate writer’s strong sense of the in an authoritative inappropriate or a coherent writer’s inauthentic voice
presence writer’s voice or or innovative tenuous writer’s presence that jeopardizes
authenticity manner. presence credibility.

Demonstrates and communicates and is able to Demonstrates and fails to and does not
communicative proficiently with communicate unsatisfactory communicate this consider audience
awareness of an this audience complex ideas awareness of an to the audience at all.
educated audience effectively to this audience outside
outside the audience. the discipline
discipline
Clarifies major and arranges and may show Does not always and confuses and may persuade
aims, arranges material to create insight into make major aims readers about its readers that it has
material to support confidence in problematic or clear, arrange major aims or no major aims or
aims, and provides readers provocative material to support develops no major provides little or
enough material to aspects of the aims, or provide point adequately no relevant
satisfy expectations topic, or generate a enough material to material.
of readers unique stance or satisfy expectations
original taxonomy. of readers
Moves beyond and demonstrates and is able to Summarizes and is unaware of and is unable to
summary into complex critical theorize and material but lapses connections or examine material
analysis and engagement with conceptualize in critical analysis lacks critical coherently.
demonstrates material or abstract ideas or or is unable to engagement with
critical engagement formulates draw additional demonstrate the material
in the topic innovative implications. interrelatedness of
relationships ideas
between ideas
Controls sentence and shows mature and exhibits Loses control of or fails to and may be unable
level features of command of these mastery of these one or more acknowledge the to communicate
written language, features, features in an elements of written conventions of any meaning at all.
including grammar, particularly as especially effective language at the standard written
spelling, regards clarity and or innovative sentence level (such English, thereby
punctuation, and precision rhetorical style. as grammar, impeding the
usage spelling, communication
punctuation, or process
usage), but without
significantly
impeding
communication
Page 4

DID YOU KNOW?

Employer Evaluations Results
The Career Planning and
Placement Center has been
active in working with
employers who come to
campus to recruit NIU
students. The forms have
been distributed at all
campus based recruiting
functions; the information
gleaned from the
evaluations has been used
by the Center to improve
the quality of the services
that it offers.
General Education Strongly Agree
Skill Areas Surveyed Agree
The NIU students I am 20% 47%
interviewing can write
effectively for the
requirements of this job.
The NIU students I am 18% 69%
interviewing have the
analytical skills required
for this job.
The NIU students I am 25% 58%
interviewing have the
computer/technical skills
required for this job.
In 2003,
the Office
of
Assessment
Services
asked the
Center to
attach three
items to the
existing
Employer Evaluation. general education-related
These items requested areas. Areas surveyed
feedback regarding included effective writing
students’ skill levels in skills, analytical skills, and
computer/technical skills.
Disagree Strongly Don’t
Disagree Know
With a response rate of
47% (responding employer
32% n =148), this table shows
how NIU students were
rated.
The results of this
2% 10% survey will be shared with
NIU’s General Education
Committee, and serve as
one piece of feedback on
17% the success of the General
Education Program at
NIU. The survey is being
continued.

Contributions Solicited!
Contribute to Toolkit’s newest feature, “Sharpen your Pencil: Assessment Tips
from the Inside” or any of our other regular features. We’re looking to share the
wisdom of NIU faculty and staff, making the work of assessment more
productive. If you’d like material to be considered for inclusion in a future
edition of Toolkit, submit a Word document of no more than 300 words as an
email attachment to barnard@niu.edu.
Page 5
FAQ
Assessment Literature
“Where can I read more on how to
accomplish authentic assessment in my
department?” Try these resources:
Association of American Colleges and
Universities. 1992. Program Review and
Educational Quality in the Major: A Faculty
Handbook. Washington, D.C.: Association
of American Colleges and Universities
[32 pp.].
Banta, Trudy W., and others. 1996.
Assessment in Practice: Putting Principles to
Work on College Campuses. San Francisco:
Jossey-Bass [387 pp.]. The authors
identify and describe six steps that
characterize successful assessment
programs.
Ewell, Peter T. 1997. “Strengthening
Assessment for Academic Quality
Improvement.” In Planning and Management
for a Changing Environment, ed. M. W.
ANOTHER FAQ
Principles of Assessment
“Could you recommend a good source for
some principles of assessment?” Here are
some excerpts from 9 Principles of Good
Practice for Assessing Student Learning, posted
at the American Association of Higher
Education website:
1. Assessment is not an end in itself but a
vehicle for educational improvement.
2. Assessment is most effective when it
reflects...not only what students know, but
what they can do with what they know.
3. Assessment...entails comparing
educational performance with educational
purposes and expectations.
4. Assessment requires attention to
outcomes but also and equally to
the experiences that lead to those
outcomes.
Peterson, D. D. Dill, L. A. Mets, and
others. San Francisco: Jossey-Bass [21 pp.].
Gardiner, Lion F. 2000. “Assessment
and Evaluation in Higher Education:
Some Concepts and Principles.” The
National Academy Newsletter 1, no. 2. An
introduction to some of the important
concepts, principles, and methods of
effective assessment.
Palomba, Catherine A., and Trudy W.
Banta. 1999. Assessment Essentials: Planning,
Implementing, and Improving Assessment in
Higher Education. San Francisco: Jossey-
Bass [405 pp.].
Wergin, Jon F., and J. N. Swingen.
2000. Evaluating Academic Departments: Best
Practices, Institutional Implications. New
Pathways Working Paper Series.
Washington, D.C.: American Association
for Higher Education, Forum on Faculty
Roles and Rewards.
5. Assessment works best when it is
ongoing, not episodic. Assessment is a
process whose power is cumulative.
6. Assessment fosters wider improvement
when representatives from across the
educational community are involved.
7. Assessment makes a difference when it
[is] connected to issues or questions that
people really care about.
8. Assessment is most likely to lead to
improvement when it is part of a larger
set of conditions that promote change.
9. Through assessment, educators meet
responsibilities to students and to the
public. There is a compelling public stake
in education.
--Astin. Reproduction permission granted.
Page 6
Students’ Perceptions of Online Courses
As one avenue to measure
student perceptions and
satisfaction concerning
online courses, a pilot
project was
initiated
through the
auspices of
the Office of
Assessment
Services and
the Faculty
Development
and
Instructional Design
Center. The survey was
designed in-house and was
administered online to
students (n = 149) in both
the fall of 2003 and spring
of 2004 semesters.
Major findings of the
project include:
• 34 percent of students
had been enrolled in more
than one online course at
NIU
• 24 percent surveyed
had taken an online course
at another institution of
higher education
• 76 percent accessed
course materials from their
home base
Toolkit is brought to you by the Office of Assessment Services:
and George, Amy, and Lohita, the Assessment Research Assistants.
• Students were pleased A report containing
with the degree of detailed results of this
technology support they study has been mailed to
received from all sources key stakeholders across
• The campus, with an articulated
majority felt goal of obtaining direct
that their feedback, programmatic
effectiveness in feedback/use, and results
using required obtained from application
technology was of the contained data.
increased by For more information
the completion contact:
of their current Craig Barnard;
online course barnard@niu.edu
• 79 percent indicated Murali Krishnamurthi;
that the greatest benefit of mkrishna@niu.edu
online courses was “time Carol Scheidenhelm;
flexibility” cscheidenhelm@niu.edu
Kathleen Blake Yancey:
Portfolio Conference Keynote
Speaker
With a firm date of March 4, 2005, planning is
well underway to offer NIU’s first-ever, one-day,
multi-session portfolio conference! The keynote
speaker is to be Kathleen Blake Yancey, the R. Roy
Pearce Professor of Professional Communication at
Clemson University. Yancey has edited three books
on portfolios and is the author of Reflection in the
Writing Classroom (1998), a study of many kinds of
reflective activities that can be used with students.
Mark your calendar now for March 4, 2005,
and watch for a formal program announcement!
Craig Barnard, Assessment Coordinator
Donna Askins, Editor-in-Chief
Joyce Rossi, Assessment Secretary