Methods for Assessing

Student Learning
Outcomes
Beth Wuest

Interim Director, Academic Development and Assessment

Lisa Garza
Director, University Planning and Assessment
February 15, 2006

Workshop Goals
To become:

more aware of the importance of methods of assessment in relation

to student learning outcomes and program improvement

more knowledgeable about direct and indirect assessment methods

more competent at developing methods for assessing student

learning outcomes

more knowledgeable about using and adapting assessment

methods that are currently in practice

more adept at reviewing methods for assessing effectiveness and

efficiency

Overview

For evidence of success and program

improvement
All

programs are requested to have 5-8

learning outcomes with two assessment
methods for each outcome by March 31, 2006
An assessment report of these outcomes will
be due toward the end of the 2006-2007
academic year

Linkages to Other University

Assessment
Academic Program Review
College and University strategic planning
Program and University accreditations

Definitions

Outcomes
Desired

results expressed in general terms

Methods
Tools

or instruments used to gauge progress

toward achieving outcomes

Measures
Intended

performance targets expressed in

specific terms

Focus
At present we are focusing only on
outcomes and methods. Although
measures should be considered when
developing these, they will not be
specifically addressed until the first
assessment cycle (2006-2007).

Student Learning Outcomes

Describe specific behaviors that a student of

your program should demonstrate after
completing the program

Focus on the intended abilities, knowledge,

values, and attitudes of the student after
completion of the program

Key questions to consider

What is expected from a graduate of the program?

What is expected as the student progresses through the

program?

What does the student know? (cognitive)

What can the student do? (psychomotor)

What does the student care about? (affective)

Why are Student Learning Outcomes

So Important?

basis for program improvement

instruction, course design, curricular design

communicate instructional intent

increase awareness of learning (for students)

common language

advising materials

promotional materials

support accreditation

Methods of Assessing Learning

Outcomes

should provide an objective means of supporting the

outcomes, quality, efficiency or productivity of programs,
operations, activities or services

should indicate how you will assess each of your

outcomes

should indicate when you will assess each outcome

provide at least two ways to assess each outcome

Categories of Assessment Methods

student learning

direct assessments evaluate the competence of students

indirect assessments evaluate the perceived learning

student perception, employer perception

program or unit processes

direct assessments evaluate actual performance

customer satisfaction, error rates, time, cost, efficiency, productivity

indirect assessments evaluate the perceived performance

exam scores, rated portfolios

perceived timeliness, perceived capability

curriculum

methods used to check alignment of curriculum with outcomes

curriculum mapping

Examples of Direct Methods

Samples of individual student work

Pre-test and post-test evaluations
Standardized tests
Performance on licensure exams
Blind scored essay tests
Internal or external juried review of student work
Case study/problems
Capstone papers, projects or presentations
Project or course imbedded assessment
Documented observation and analysis of student
behavior/performance
Externally reviewed internship or practicum
Collections of work (portfolios) of individual students
Activity logs
Performances
Interviews (including videotaped)

Examples of Indirect Methods

Questionnaires and Surveys

Students
Graduating Seniors
Alumni
Employers

Syllabi and curriculum analysis

Transcript analysis

Describing Assessment Methods

What are you going to use?

Of and/or by whom?

point-of-service, capstone, throughout the year, end of program

For what purpose?

student, mentor, focus group, alumni

Context (e.g., where or when)?

presentation, assignment, test, survey, observation, performance

rating

desired learning outcome

example: Test the students at the end of the program for

their level of knowledge in XYZ

Creating Assessment Methods

What

Who

Where/When

Outcomes

Presentation

Student

Point-of-service

Learning

Assignment

Alumni

Capstone

Quality

Portfolio

Customer

Throughout the year

Timeliness

Test or exam

Instructor

End of year

Skills

Project

Mentor

End of program

Satisfaction

Performance

Focus group

In course

Preparation

Survey

Process

On the job

Efficiency

Direct measurement

Employer

Transcripts

Creating Assessment Methods

What

Who

Where/When

Outcomes

Presentation

Student

Point-of-service

Learning

Assignment

Alumni

Capstone

Quality

Portfolio

Customer

Throughout the year

Timeliness

Test or exam

Instructor

End of year

Skills

Project

Mentor

End of program

Satisfaction

Performance

Focus group

In course

Preparation

Survey

Process

On the job

Efficiency

Direct measurement

Employer

Transcripts

Locally Developed Surveys

institutional level
alumni

survey
academic advising
survey
image survey
student satisfaction
survey
Joe Meyer, Director,
Institutional Research

program or department
level
advisory

board surveys
employer surveys
customer surveys
program-specific surveys
graduating senior survey

Curriculum or Course-based

performance-based
capstone

courses
capstone projects
case studies
classroom assessment
course-embedded assignments
course-embedded exam questions
portfolios
reflective essays

Types of Examinations or Tests

standardized exams
national

test
state test

juried competitions
recitals
shows

or exhibitions

locally developed exams

pre-post

tests
course-embedded exam questions
comprehensive exam
qualifying exam

Assessment Matrix Can be Useful to

Link the Where with the
Outcomes
Learning
Outcome

Course
1234

Course
2345

Course
3456

Capstone

Introduced

Emphasized

Used

Assessed

Skills and
knowledge

Introduced

Used

Assessed

Communication
skills

Introduced

Application of
theory

Emphasized

Hints on Selecting Methods

match assessment method with learning outcome

Students completing BS in xyz will demonstrate competence in abc

principles comparable to graduates of other similar national programs

Student will be tested with a locally developed exam administered at the end
of the program

Students scores on the xyz principles on the xyz national examination

administered twice a year will be examined

the assessment results should be usable

Students completing BS in xyz will demonstrate competence in

conducting research

Seniors at the end of their capstone course develop a research design to

address the intended research question posed in a case study. A rubric will
be designed to assess the effectiveness of their ability to construct a
research design.

Graduating seniors will complete a senior research project. Completion of

the project will be recorded.
Not Useful

Hints on Selecting Methods

results should be easily interpreted and unambiguous

data should not be difficult to collect or access
information should be directly controllable by the unit or
program
identify multiple methods for assessing each outcome

direct and indirect methods

qualitative and quantitative
passive or active methods
within different courses
conducted by different groups

identify subcomponents where other methods may be

used that allow deeper analysis

Hints on Selecting Methods

use methods that can assess both the strengths and

weaknesses of your program

capstone or senior projects are ideal for student learning

outcomes assessment

when using surveys, target all stakeholders

build on existing data collection

accreditation criteria

program review

exercise

Selecting the Best Assessment

Methods

relationship to assessment provide you with the

information you need
reliability yields consistent responses over time
validity appropriate for what you want to measure
timeliness and cost preparation, response, and
analysis time; opportunity and tangible costs
motivation provides value to student, respondents are
motivated to participate
other

results easy to understand and interpret

changes in results can be attributed to changes in the program

After Identifying the Potential List of

Assessment Methods You Need to

select the best ones

consider possible performance targets for the future

try to identify at least two methods for assessing each outcome

balance between stretch targets versus achievable targets

Examples of methods

survey (using the Graduating Senior Survey) the students at the end of the
program as to their intention to continue their education in a graduate program
(indirect method)

students will rate their likelihood of attending a graduate program on a survey

(using the Graduating Senior Survey) that they will complete at the end of the
program

xyz graduates admission rate to xyz graduate program in the State of Texas will
be reviewed

After Identifying the Potential List of

Assessment Methods You Need to

develop assessment instruments

surveys
exams
assignments
scoring rubrics
portfolios

ideally you want them to be reliable, valid, and cheap

approaches
use external sources
seek help from internal sources (e.g., Academic Development and
Assessment Office)

do it yourself

the instrument may need to be modified based on assessment

results

Challenges and Pitfalls

one size does not fit all some methods work well for
one program but not others

do not try to do the perfect assessment all at once

take a continuous improvement approach

allow for ongoing feedback

match the assessment method to the outcome and not

vice-versa

Example

Outcome 1: Graduates will be satisfied that their

undergraduate degree has prepared them to succeed in
their professional career

xyz graduates will be surveyed in the annual alumni survey on their

preparedness to succeed in their career

95% of the xyz graduates surveyed in the annual alumni survey report that the xyz
program enabled them to be very prepared or extremely prepared to succeed in
their career (next phase)

on-site internship supervisors each semester will rate interns from the xyz
program on their skills necessary to succeed in the workplace

90% of on-site internship supervisors each semester rate interns from the xyz program
as having the skills necessary to succeed in their career (next phase)

students in their capstone course will be administered a locally

developed, standardized exam regarding career preparedness

95% of student in their capstone course are able to successful answer

90% of the questions regarding career preparedness on a locally
developed, standardized exam (next phase)

scores of graduates who have taken the state licensure exam in xyz
within one year of graduating from the xyz program will be evaluated

85% of the graduates are able to pass the state licensure exam in xyz
within one year of graduating from the xyz program (next phase)

senior portfolios will be examined annually using a locally devised rubric

to show evidence of preparedness for success in related professional
careers on three key measures: communication, leadership, and ethics

90% of senior portfolios examined annually using a locally devised rubric will
show evidence of preparedness for success in related professional careers
on three key measures: communication, leadership, and ethics (next phase)

students will be observed performing basic technical lab skills necessary

for successful employment in a senior laboratory course

90% of students will be able to perform basic technical lab skills necessary
for successful employment in a senior laboratory course (next phase)

Re-Cap of Process
Step 1: Define program mission
Step 2: Define program goals
Step 3: Define student learning
outcomes

Step 4: Inventory existing and

needed assessment methods
Step 5: Identify assessment methods for each
learning outcome

questions
and
comments