USING PORTFOLIO ASSESSMENT FOR GENERAL EDUCATION IN AN

ADULT DEGREE PROGRAM

Sharon Drury, MS
Director of General Studies
Cynthia Tweedell, Ph.D
Coordinator of Assessment
Adult and Professional Studies
Indiana Wesleyan University
A special program for adult students is an important innovation, removing
barriers for a population in need of higher education. Yet for a higher education
community entrenched in traditions, innovative programs bring about skepticism.
For this reason, assessment in adult degree programs is of prime importance.
Innovators must demonstrate that student outcomes are comparable or exceed
those of traditional programs.
At Indiana Wesleyan University, a fast-growing adult program required
assessment measures that could inform changes and ultimately verify the validity
of the degree. In November, 1998, a full time Coordinator of Assessment was added
to assist faculty and administrators with assessment plans and data collection. The
Coordinator of Assessment works with all ten adult degree programs. Together
they developed program objectives and means to measure those objectives.
Attached is the Assessment Plan for General Education that was developed for the
Adult and Professional Studies Program by a team of faculty, the Director of
General Studies, and the Coordinator of Assessment (see Table 1). The objectives,
which have been in place for many years, are known as the "Ten Across Skills"
and permeate all Adult and Professional Studies undergraduate curricula. In the
second column of the attached Assessment Plan, the objectives are specifically tied
to the Indiana Wesleyan University student outcome expectations (World Changer
Outcomes). In this way it is clear that General Education in the Adult and
Professional Studies Division is part of the general mission of the University.
In the third column of the attached Assessment Plan are the means of measuring
the achievement of each objective. These include two direct measures: the
Academic Profile (a standardized test published by ETS), and specific writing
samples compiled in a student portfolio (called the Personal Learning Anthology).
Faculty and administrators set specific criteria by which to judge successful
achievement of each objective.
The student portfolio designed to assess General Education at Indiana Wesleyan, is
authentic assessment since it consists of the actual products of coursework which
are directly related to program objectives (Palomba and Banta, 1999, p. 116).
Many of these exhibits require students to apply skills in a real world setting so
they are particularly valuable in assessing learning outcomes.
The name "Personal Learning Anthology (PLA)" was chosen to avoid confusion
with the Prior Learning Assessment Portfolio or with portfolios of student work
done within some writing courses. Certainly the term "portfolio" has come to

mean many things: from exhibits to show potential employers, to collections of all
student work. The Personal Learning Anthology at Indiana Wesleyan is intended
to demonstrate actual learning in a variety of contexts. It is designed to show
growth in learning over a students academic career in the adult program.
Development Process
The process of designing a General Education portfolio began with the Curriculum
Council developing an ambitious four-month schedule for designing and
implementing a system to collect and assess student work. Beginning in November
1998, a Task Force of fourteen representative faculty met several times to not only
write appropriate learning objectives that were tied to university outcomes, but
also to design the components that would measure these objectives. The third step
was to set the criteria we as a faculty would consider proficiency and/or adequate
improvement throughout the program. Then, there needed to be a system for
collecting and scoring the students work. At that time, a faculty team would score
the assignments, discuss the results, and close the assessment loop by making
program improvements.
The Task Force started by gaining consensus on the wording of ten General
Education objectives and also agreed that a portfolio of General Education
exhibits would be designed as the primary assessment tool. A secondary assessment
tool is the Academic Profile, a nationally-normed test measuring general
knowledge areas of Humanities, Social Science, and Natural Science, as well as
proficiency levels in reading, writing, critical thinking, and math.
In order to design what should be included in a General Education portfolio, all
General Education faculty were polled for recommendations of the best
assignments that would assess student academic achievement appropriate to the
learning objectives in each course. The Task Force then took these
recommendations along with their own experiences in a variety of General
Education courses, and decided on the best courses and assignments to use in
assessing General Education learning.
Components of PLA
It was decided that the Personal Leaning Anthology would consist of exhibits from
15 General Education courses in their 62-hour program (See Table 2). At least one
assignment would become the exhibit that addressed each of the ten General
Education objectives. Some assignments measured more than one objective. Many
appropriate assignments already existed; others needed to be written into the
curriculum. Rubrics were written and piloted by the Faculty Task Force members
in order to standardize evaluation of the portfolio submissions. It was decided to
incorporate a midpoint and final review of the Personal Learning Anthologies.
An added challenge in an adult program is to demonstrate that students are
gaining skills that they did not have before entering the program. To do this, a
series of pre/post assessments became part of the portfolio requirements. One of
these was a writing sample students do in their first course, with the same in-class

writing sample being part of the curriculum in their last course. Other pre/post
assessments were in oral communication skills and teamwork.
The next task was to decide on how this would be implemented with faculty and
students. Since Indiana Wesleyans adult programs are a multi-site, accelerated
degree program, there is a Curriculum Council and appointed faculty content
experts who write the curriculum modules for each course. It was agreed that the
curriculum modules would be the primary way the university would continually
communicate the portfolio process to faculty and students. A schedule of revisions
was put in place, with the first group of students beginning their portfolios
February 24, 1999.
Faculty members were included in developing this assessment tool in a variety of
ways. At the beginning, faculty workshops were held to explain the importance of
assessment of student academic achievement. Faculty teams were included in the
development of the overall General Education assessment plan, and then a
representative team of faculty members dealt with implementation issues of the
portfolio system after the assessment plan was adopted.
Faculty members were trained on the use of the Personal Learning Anthologies in
the following ways:

Faculty who teach the first course received training on setting up the
portfolio. This included faculty workshops as well as newsletters and
memos at appropriate times in the development of the portfolio.
Faculty who taught at the midpoint and final reporting courses received
instructions on how to collect the PLAs, complete the summary chart, and
send in their reports. They did not grade the students work from past
classes; they merely counted to see that the exhibits were up-to-date in the
portfolio. A special page in their curriculum Faculty Guides delineated how
to do this. Large postage reply envelopes were provided for faculty to send
in their reports to the Assessment Coordinator. In addition, student success
at maintaining the PLA for the midpoint and final portfolio reviews was
included in the computation of the grade for these two courses.
All faculty who teach General Education courses were reminded in their
Faculty Guides to be sure that students put the proper assignment(s) in
their General Education portfolio.

Students were also introduced in their first course to the idea of maintaining a
portfolio during the course of their two years in the associate program. A notebook
and dividers with tabs was supplied to students in their first course, which is called
Principles and Practice of Lifelong Learning. A Table of Contents cover sheet
informed students as to which assignment was required as the exhibit for each
course. They also became aware of which learning objective was being assessed on
the scoring summary chart (see Table 2) in the curriculum modules for each
General Education course. For the students, this collection of a variety of
assignments throughout the program would build self-confidence and be ready to
show employers and family.

In all training and instructions about the Personal Learning Anthology, a

statement that copies of these portfolios will be reviewed by the Coordinator of
Assessment for program assessment purposes was included.
Assessment Day
The primary purpose of the annual departmental Assessment Day was to "assess
our assessment" to make needed improvements in the PLA process. Another
purpose of this Assessment Day was to make curricular improvements so program
objectives can be better achieved. Five of the ten program objectives were chosen
for evaluation. This time, faculty chose five writing assignments which could be
used to measure student progress on these objectives.
In preparation for the Assessment Day, the PLAs of two core groups were
collected. These were the first two groups to reach the midpoint of the PLA
compilation process. Copies were made of the assignments required for this
assessment and then the PLAs were mailed back to the groups facilitator for
distribution in class. The Coordinator of Assessment developed score sheets on
which faculty could record scores.
On May 19, 2000, six faculty met to score the papers. They were divided into two
groups of three. One group evaluated student progress on two objectives: critical
thinking and Christian worldview. The other group evaluated student progress on
information literacy, problem solving and ethical thought. They were instructed to
skim each paper, make no comments on the papers, and record a score on each
paper according to the rubric for that objective. To enhance reliability, the teams
spent about 20 minutes reading the first few papers together, discussing how the
rubric ought to be used. Then individuals scored papers on their own. After
scoring each paper they placed their initials in the top corner and passed the paper
on. Therefore all three faculty scored the same papers.
The experience was a generally cordial one. Faculty gained insights by seeing the
products of students in courses outside their areas. There was a break after two
hours where faculty had good conversation around a nice luncheon. They spent a
total of three hours reading papers.
The Coordinator of Assessment calculated a mean score for each paper. Pre-test
and later scores for a given objective were compared to document growth.
Closing the Loop
On July 5 the General Studies Curriculum Council met to discuss the results and
suggest improvements to the assessment process and to the curriculum. Among
suggestions for changes to the PLA process were:

Better training and communication for faculty to make sure they

understand the purpose and process of the PLA and are able to
communicate it to students.

Student self-scoring of the pre-test to reduce anxiety among students and

enable them to understand the rubric being used to evaluate them. Faculty
teams will still score selected pre-tests for assessment purposes.

Curricular improvements include:

More attention to critical thinking in faculty development.

Revision of the final course to include ethical thought.

In conclusion, we have found that the valuable information gained by assessing

actual products of coursework has far outweighed the extra effort it took to set up
the portfolio system. The total assessment plan has informed quality changes in the
General Education curriculum, and has ultimately verified the validity of the
degree. The Personal Learning Anthology has been a confidence builder for
students as they see their own progress and a valuable time of faculty discussion
when developing rubrics and scoring the exhibits. Portfolios continue to be a
requirement for adult students, and will be annually assessed as part of continuous
improvement at Indiana Wesleyan University.
References
Palomba, C.A. & Banta, T.W. (1999). Assessment essentials: Planning,
implementing and improving assessment in higher education. San Francisco: JosseyBass.
Table 1: APS GENERAL EDUCATION

Objective

strate an
nding of a
worldview

World Changer
Outcome
1a- Basics of the
Christian faith

Assessment Criteria & Procedures

Assessment Results

Use of the

Pre/post Ethics Writing Sample:

When a sample of 50 are reviewed
by faculty, 90% will meet or
exceed the criteria of a 10%
improvement in articulating a
Christian worldview as indicated
by a faculty-written scoring
rubric.

Ethics-BIL102

Ethics-ENG140
Opinion Writing
Samples:
Mean scores on Ethics
improved at 5.5% at
mid-point.

Assessment
FY 2000-200

3g- Agents of change

Pre/post Ethics Writing Sample:

When a sample of 50 are reviewed
by faculty, 90% will meet or
exceed the criteria of a 10%
improvement in ethical thought as
indicated by a faculty written
scoring rubric.

2e- Life-long
learning

Academic Profile: Institutional

scores will meet or exceed scores

Academic Profile:

MAT110 an
MAT103

3a- Commitment to
truth
3c- Human worth

Assessment
2001

Writing Samples:
Mean scores on
Christian world view
improved 76.76% at
mid-point.

3d- Stewardship

ethical thought
n in both a
nal and personal

1a- Basics of the

Christian faith
1d- Integration of
knowledge

values and skills

for lifelong

ehend complex
materials

from a national sample of other

comprehensive universities.
Personal Learning Anthology
(PLA) inclusions will exhibit
values and skills necessary for
lifelong learning.
1b- Liberal arts
foundation

ritical thinking
1b- Liberal arts
cerning ideas and foundation
nce.
2b- Critical thinking

p skill in problem
nd decision

1d- Integration of
knowledge
2a- Creativity
3d- Stewardship

IWU scores are

comparable to a
national sample.
Lowest performance
in math skills.

strengthened
103 required
ASGS degre

Due March, 2001

Academic Profile: College Reading Academic Profile:

scores will meet or exceed scores
from a national sample of
IWU scores are
comprehensive universities.
comparable to a
national sample.

Assessment
FY 2000-200

Academic Profile: Critical

thinking scores will meet or exceed
scores from a national sample of
comprehensive universities.
Pre/post Ethics Writing Sample:
When a sample of 50 are reviewed
by faculty, 90% will meet or
exceed the criteria of a 10%
improvement in critical thinking
as indicated by a faculty-written
scoring rubric.

Academic Profile:

Assessment
FY 2000-200

Pre/post Ethics Writing Sample:

When a sample of 50 are reviewed
by faculty, 90% will meet or
exceed the criteria of a 10%
improvement in problem solving
and decision making as indicated
by a faculty-written scoring
rubric.

Ethics-PHE 140
Writing Samples:

IWU scores are

comparable to a
national sample.
Ethics-HST 180
Writing Samples:
Mean scores on
critical thinking
improved 8.25% at
mid-point.
Assessment

FY 2000-200
Mean scores on
Problem Solving
improved 23.25% at
mid-point.

strate effective
ommunication

2c- Communication

Essay Samples: When a sample of

50 are reviewed by faculty, 90%
will have a score of 3 or more on
each of 6 traits on a standardized
writing rubric.
Academic Profile: College writing
scores will meet or exceed scores
from a national sample of
comprehensive universities.

Due March 2001

Assessment
FY 2000-200

Academic Profile:
IWU scores are
comparable to a
national sample.

strate effective
munication skills.

2c- Communication

Pre/post Oral Presentation: When

a sample of 50 are reviewed by
faculty, 90% will meet or exceed
the criteria of a 10% improvement
in oral communication skills as
indicated by a faculty-written
scoring rubric.

Due March 2001

Assessment
FY 2000-200

the ability to
ed information.

1b- Liberal Arts

2f- Leadership

Pre/post Writing Sample: When a

sample of 50 are reviewed by
faculty, 90% will meet or exceed
the criteria of a 10% improvement
in information literacy as
indicated by a faculty-written
scoring rubric.

Ethics-ECO205
Writing Samples:
Mean scores on
Information Literacy
improved 65.5% at
mid-point.

Assessment
FY 2000-200

nstrate the
work effectively

3h- Selfishness

Pre/post Group Process

Due March 2001
Assessment: When a sample of 50
assessments are reviewed by
faculty, 90% will meet or exceed
the criteria of a 10% improvement
in group process as indicated by a
Group Processes Assessment in the
middle and end of their program.

2a- Creativity
2f- Leadership
2g- Agents of change
3b- Inclusion
7/00 cbt

Personal Learning Anthology

A Portfolio showing my General Education Learning in the ASB Degree Program

Assessment
FY 2000-200

Student Name_____________________________________ Core Group_________

Copies of the Personal Learning Anthologies will be reviewed for program
assessment purposes.

Course #

Required PLA Exhibits

Ten Across Skill

Points
Possible Awarded

UNV111

Ethics Writing Sample

and Score Sheet

Christian View,
0-1
Ethical Thought,
Critical
Thinking,
Problem Solving,
Info Literacy

ENG140

Essays: Experience

Written
Communication

0-1

Persuasive
Opinion
ENG141

Written Report and Oral

Presentation Score Sheet

Written & Oral

Communication

0-1

PHE140

Individual Wellness
Project

Lifelong
Learning

0-1

BIL102

Reflection Paper

Christian World
View

0-1

COM115

Faith Integration Paper

Christian World
View

0-1

MAT110

Business Math Project

Problem Solving

0-1

HST180

Individual Research
Paper

Reading
Critical
Thinking

0-1

ECO205

Group Project

Team Building

0-1

PSY155

Philosophical Perspective Problem Solving

Group Process Assessment Team Building

0-1

INTERIM CHECK

Subtotal

0-10

ENG242

Final Critical Review

Critical
Thinking
Written Com.

0-1

MUS180

Definition of Aesthetics
Both Critiques

Lifelong
Learning
Critical
Thinking

0-1

PHL283

Analysis Paper

Reading
Oral Com.

0-1

Synthesis Pres. & Oral

Presentation Score Sheet
BIO203

Group Project with

Team Building
Group Process Assessment

BUS274

Ethics Writing Sample

and Score Sheet

FINAL CHECK
TOTAL

PERSONAL LEARNING

0-1

Christian View,
0-1
Ethical Thought,
Critical
Thinking,
Problem Solving,
Info Literacy
Subtotal
ANTHOLOGY

0-5
SCORE

This is to certify that __________________________ has received PLA evaluation

of
(check one) ? Satisfactory ? Unsatisfactory
(All exhibits completed to date) (Reported to Associate Dean)
Facilitator Signature__________________________________ Date _________

Copyright 2000, Adult Higher Education Alliance