You are on page 1of 20

Portfolio Project

EDUC 765: Trends and Issues in Instructional Design


By: Lynda Rassbach

Submitted June 20, 2016

If you dont know where you are going, you might wind up someplace else.
~ Yogi Berra

PROJECT PROPOSAL
Project Title
FQAS Data and Evidence Analysis

Sponsoring Organization
Wisconsin Indianhead Technical College
Wisconsin Indianhead Technical College (WITC) is a technical college within the Wisconsin
Technical College System (WTCS). WITC offers career-focused associate degree programs,
technical diplomas, short-term certificates, customized training, and a wide array of courses for
personal or career enhancement. WITC has four campuses Ashland, New Richmond, Superior,
and Rice Lake two outreach centers at Hayward and Ladysmith, and a learning center located
in Spooner.

Project Description
In July, 2015, WTCS completed an analysis on the faculty certification process. The goal was to
update the process by developing a new, effective, flexible, and easy to understand process that
ensures that the WTCS hires high quality faculty who will then be supported with ongoing
professional development to support learner success. The analysis resulted in the introduction of
a new faculty quality assurance system (FQAS). New competencies were developed, and all
faculty in the WTCS are expected to meet these certification requirements. A new course, Data
and Evidence Analysis, was created to meet data analysis needs within the system. The required
competencies related to data analysis were identified for each faculty member resulting in three
course outcomes:
1. Explore a variety of data analysis tools
2. Analyze data from teaching and learning experiences
3. Use data to inform decision making about the teaching and learning process
This project will result in a training plan to support faculty as they demonstrate proficiency in
these course outcomes.

Aim
This course will provide a way for faculty at WITC to meet the FQAS certification requirement
of Data and Evidence Analysis by providing opportunities for faculty to demonstrate competence
in the following course outcomes:
1. Explore a variety of data analysis tools
2. Analyze data from teaching and learning experiences
3. Use data to inform decision making about the teaching and learning process

Target Audience
The target audience is all members of the WITC faculty full-time and part-time on all four
campuses Rice Lake, Superior, Ashland, New Richmond as well as the satellite locations
Spooner, Hayward and Ladysmith and online.

Delivery Options
Due to time and budget constraints, WITC has opted to offer Data and Evidence Analysis
training during in-service days planned for academic year 2016-2017; most full-time faculty will
be able to complete training during this time. In addition, the training will be developed in an
online format to meet the needs of full-time faculty who miss group training, part-time faculty
and all future new hires.

FRONT-END ANALYSIS: INSTRUCTIONAL NEED


Instructional Need
The Presidents Administrators Process Improvement committee of the WTCS was formed in July
2013 to identify strengths and opportunities within the existing Educational Personnel
Certification System (EPCS) and to make recommendations on improving the system. The
Process Improvement committee proposed the Wisconsin Administrative Rule which took effect
on January 1, 2016. All faculty hired on or after this date are governed by the new Faculty
Quality Assurance System (FQAS).
The Implementation Committee proposed a three phase plan to fully transition from the EPCS to
the FQAS. Each recommended phase has unique components to effectively transition to the new
plan.

Credit: Wisconsin Technical College System: Faculty Quality Assurance System Recommendations For Transition Plan

By state mandate, the following groups must complete the transition by June 20, 2022: (WTCS,
2016)
Transition Group 1: All Part-Time Provisional and all Part-Time Approval faculty. This
group includes part-time faculty that have not completed all of the previously designated
certification courses.
Transition Group 2: All Full-Time Provisional faculty. This includes full-time faculty
that have not completed all of the previously designated certification courses.
Transition Group 3: All 5 Year certificate holding faculty. This includes both part-time
and full-time faculty that have completed all of the previously designated certification
courses.
As part of the creation of the new FQAS system, all new faculty and all existing faculty at all
technical colleges within the state of Wisconsin are required to demonstrate competency in Data
and Evidence Analysis. An administrative decision was made by WITCs Board of Trustees that
all faculty would be required to complete Data and Evidence Analysis, including faculty with an
existing 5-year certification under the old system. The decision created a definite needs gap for
faculty at WITC. All faculty must demonstrate competency by completing the course. No one
will be grandfathered in; no one will receive credit through transcript review.
To bridge this gap and to facilitate the certification of faculty most efficiently and cost
effectively, WITC is offering Data and Evidence Analysis for all faculty during in-service days
planned into the academic calendar beginning Fall 2016.
Data and Evidence Analysis: Timeline
Module 1
Module 2
Module 3
Module 4
Module 5

Basic Statistics
Data Analysis Tools
Using Course Data to Inform Decision Making
Using Program Data to Inform Decision Making
Using College Data To Inform Decision Making

August 22, 2016


August 22, 2016
January 16-17, 2017
January 16-17, 2017
January 16-17, 2017

FRONT-END ANALYSIS: LEARNER CHARACTERISTICS


Learner Analysis
Primary Audience (Training is Mandatory)
All provisional faculty (both part-time and full-time) at all four WITC campuses,
including outreach centers and online
All 5-Year certificate holding faculty (both part-time and full-time) at all four WITC
campuses, including outreach centers and online
All future new hires (both part-time and full-time) at all four WITC campuses,
including outreach centers and online
Secondary Audience
None FQAS applies only to faculty
General Learner Characteristics
Demographics:
- Gender ratio is approximately equal male and female
- Range in age from 25 years to 65 years
Education:
- Technical diploma instructors minimally have an Associates degree and twoyears of work experience in their program area.
- Program instructors in the Associate degree programs minimally have a
Bachelors degree in their program area and two years of work experience work
experience outside of education in the program field
- General Studies instructors minimally have a Masters degree, with at least
eighteen hours in their content area and two years of work experience outside of
education.
- Adult Basic Education instructors minimally have a Bachelors degree with two
years of work experience outside of education.
Work Experience:
- Teaching experience ranges from 1 year to 40 years
- Work Experience outside of education is at least 2 years for all faculty
Personal and Social Characteristics
Motivation:
- Completing this professional development training is required by Wisconsin state
law for all faculty to maintain certification and, ultimately, keep their jobs.
- The option to this training is completing a 3-credit undergraduate Data and
Evidence analysis course through WTCS on the faculty members own time and
at their own expense.
- Failure to complete the professional development training will result in loss of
certification, and ultimately, loss of employment.

Attitude:
- All instructors who were fully certified under the old 5-year system are now
considered not certified which has contributed to feelings of being devalued.
- Most faculty perceive Data and Evidence Analysis as another hoop that they must
jump through in their already busy lives.
- Some faculty have expressed anxiety with the subject nature of the course
statistics and revealed concern that they cant do math.

Entry Characteristics
Prerequisite skills and knowledge:
- Level of proficiency in mathematics -- 10th grade
- Level of proficiency in reading 10th grade
- Proficient in using a calculator.
- Proficient in navigating in Blackboard Learn
- Proficient in Blackboard Learns grade book
- Able to complete an online quiz or survey
- Computer skills include the ability to download/save/upload files
- Access the web using a browser (Chrome, Internet Explorer, Firefox)
- Proficient in Microsoft Office products (Word, Excel)
Experience in data and evidence analysis
- Proficient at calculating mean, median and mode averages
- Can read a two dimensional chart or graph
- No formal statistics course required
Experience in classroom assessment
- Proficient in assessments such as quizzes, rubrics, exams, homework, projects
where evidence is gathered, assessed and a grade assigned
- Proficient using a grading system to assign a grade
Previous exposure to quality assurance processes
- Familiar with WTCS Quality Review Process (QRP) scope and purpose for
program review
- Familiar with Program Data Profiles (PDP) scope and purpose for program
review
- Familiar with Higher Learning Commissions (HLC) scope and purpose for
accreditation
- Familiar with Community College Survey of Student Engagement (CCSSE)
scope and purpose for college-college review

Contextual Analysis
Orienting Context
Goals:
- All faculty are required to complete Data and Analysis to reach WTCS
certification. Without certification, faculty cannot work in the technical college
system.

- Five-year certified faculty are being asked to complete this certification


requirement for the new FQAS system when they were fully certified under the
old system; loss of certification has contributed to feelings of being devalued.
- The course could be perceived as Lets just get it over with or Just another
thing I have to do which could hinder engagement in training.
Utility:
- Some faculty do not see usefulness in the new required course, because they are
not involved directly in program reviews or college accreditation; to these faculty,
this course is another hoop to jump through.
- Some faculty have expressed great interest in the new open data access policy that
is coinciding with the new course; they are relieved to finally understand what
data is available and interested in how to access relevant data for decision-making
purposes (i.e. program and course reviews.)
Accountability:
- All faculty are aware that the new FQAS system requires them to take Data and
Evidence Analysis to be certified to teach in their area.
- All faculty are aware that the alternative to completing the course through
professional development at WITC is that they complete the course on their own
time at their own expense.
- All faculty are aware that without certification, they will lose their employment at
WITC.

Instructional Context
Scheduling:
- Module 1 Data Analysis Tools emphasizing analysis tools (mean, median, mode,
standard deviation, normal distribution) will be offered during August 22 inservice by IPV.
- Modules 2, 3 & 4 will be offered during a two-day in-service in January 2017 on
the Rice Lake Campus in a break-out session format.
- All four modules will be offered online beginning January 2017 for faculty that
missed face-to-face training and new faculty hires.
Classroom Needs:
- One room setup for IPV to accommodate 50+ faculty on each campus for August
in-service
- Three rooms to able accommodate 75+ faculty on Rice Lake campus for January
in-service
Lighting:
- Each conference or classroom on each campus at WITC is well lit and lights can
be dimmed as necessary by the facilitator.
Sound:
- Wireless microphone/speaker system required for each in-service group due to
size of groups
Noise:

- The only activities on each campus on Teaching & Learning in-service days are
related to professional development; no construction or unusual activity is
expected.
Technology (August In-service Only):
- Computer lab required on each campus for 50+ faculty on each campus
- Faculty can bring their laptop to the training (can use regular classroom)
- IPV connection with rolling cart at each campus location
- Document camera on each campus
- Overhead projector required at each campus location
- Computer teaching station
- Power supply to accommodate 50+ computers (if dont use computer lab) at each
location
Technology (January In-service Only):
- Three computer labs that can accommodate 75+ faculty
- Faculty can bring their laptop to the training (can use regular classroom)
- Power supply to accommodate 75+ faculty (if computer lab not used)
- Wireless microphone/speaker system in each break-out session
Accommodations (January In-service Only):
- Hotel accommodations needed for 150+ faculty
- Breakfast (1 day)
- Lunch (2 days)
- Dinner (1 day)
Travel Expenses (January In-service Only):
- Transportation for 150+ faculty
- Mileage reimbursement

Technology Inventory
Blackboard Learn LMS
- Create a BB shell and have faculty self-enroll before August 2016 training
Microsoft Office Pro: Word, Excel
- All faculty have this software loaded on their laptops
Calculators
- Request faculty bring own calculator or use the calculator on their laptop
- Can provide calculators at each location for the August in-service
IPV connections required at each campus location for the August in-service
- Broadcast from the Rice Lake campus to Ashland, Superior, New Richmond,
Ladysmith and Hayward
Transfer Context
Transferability:
- Data analysis skills gained are transferable from course to course.
- Knowledge gained about data analysis within programs is transferable; all
programs have access to the same forms (QRP, TRA, PDP)

Opportunities to Use Knowledge:


- Faculty will see immediate applications of the data analysis in decision making
processes when they analyze classroom assessment results, complete their 3-year
program reviews, participate on the HLC accreditation committee and administer
technical skills attainment (TSA) assessments.
Support
- The Research & Data Coordinator is available through the Office of Institutional
Effectiveness to facilitate data storage and retrieval for all faculty
- A new mentor designation has been created -- Data and Evidence Analysis
mentor; three individuals have been selected to act as a resource to faculty with
questions relating to data analysis

INSTRUCTIONAL IMPACT BASED UPON LEARNER CHARACTERISTICS


Potential Audience Misconceptions

A fixed mindset instead of a growth mindset Some people have math brains and some
do not. I do not!
Data and Evidence Analysis is a statistics course I took statistics in college. I should
get credit through transcript review.
Does not apply to me -- Only faculty who have not completed their 5-year certification
have to complete Data and Evidence Analysis.
Perceived uselessness -- Heres another hoop I have to jump through to keep my job
thats not going to help me do my job.
Perceived uselessness Im never going to use this stuff!

Application of Adult Learning Theories


The learners within this training are well-educated adult learners who have a wealth of
experience in the area of data collection and a love for learning after all, they are teachers! All
faculty are experienced at assessing students and using the assessment data to make good
decisions regarding their students and programs as evidenced by high success rates in classrooms
and the longevity of programs at WITC. Many have also participated on program reviews and
HLC accreditation committees, so may be well versed in how data is collected at WITC and how
it is used in the decision making process. The training room(s) will be filled with resident
experts.
To acknowledge this experience and reassure faculty that they are valued, the course will be
designed with them in mind. Faculty at WITC will appreciate:
The Need to Know
- Course outcomes and learning objectives will be clearly stated within each module so
participants know what they will need to know and how they will be assessed on
those outcomes well in advance of training.
- Course content will emphasize the Whys? along with the Hows?
Adult Learner Experience
- Credit for prior experience will be explored for individual course competencies.

- Opportunities will be provided for each faculty to decide how they will show
proficiency for a course outcome submit a portfolio of data analysis tasks
completed with a reflection component, take a quiz, reflect on data analysis solutions
to case studies.
- Tapping into adult experiences through techniques that allow them to connect new
ideas with prior experiences (discussions, simulations, problem-solving activities, or
case methods) is beneficial.
Self-concept
- The instructor's role will be as a facilitator who presents the tasks and assists the
learners if mistakes are made or help is requested.
- The learner will be given pathways for completing course outcomes.
- Online resources, multimedia, case studies, group discussions, and lecture should be
used to connect with as many learning styles possible during training.
Collaborative Learning:
- Peer-to-peer instruction and resident experts should be utilized to take advantage of
the existing wealth of knowledge that already exists in the faculty.
- Learning strategies will capitalize on the diverse experiences in the group.
- Open-ended questions will be used to draw on the experiences of faculty within
collaborative groups.
Relevance and Readiness
- Teaching statistics for the sake of teaching statistics isnt going to fly.
- The learning objectives must be taught within a context that is relevant and
applicable to a typical faculty position at WITC.
Problem-centered Learners
- Data analysis case studies will be selected from WITCs historical self-studies,
program reviews, and HLC accreditations as examples of how data aided in the
decision-making process.
- Actual WITC data sets (QRP, PDP, CCSSE, TSA) will be used in break-out sessions
to support instruction.

Application of Motivational Theories


In the context of developing this data analysis course for adult learners, Kellers ARCS theory of
motivation fits well with Knowless five assumptions of andragogy. Kellers ARCS theory
suggests that there are four key elements in the learning process which can encourage and sustain
learners motivation. These four elements are attention, relevance, confidence and satisfaction.
Given that my project is designing a data analysis course for faculty members with many years
of teaching experience who work with data and make decisions on a daily basis, the motivational
theory fits nicely within the context of my instructional design project.
Attention
In order to get (and keep) each faculty members attention, course content will be taught in the
context of real world applications; faculty will work with the most current WTCS data sets that
are reported by the Office of Institutional Effectiveness. A variety of learning activities with
active participation (and a smattering of humor) will be used.

Relevance
Faculty at WITC are invested in their programs and looking for ways to efficiently complete
administrative tasks to get to the heart of teaching their students. Providing information that is
relevant to achieving this goal is key to a successful in-service. The program data profile (PDP)
will be the primary data source during the training, and the primary goal of training is teaching
faculty how to access and interpret the data that is essential in the Quality Program Review
(QRP) process. Faculty will be encouraged to build connections between the new information
presented and what they already know from previous experience.
Confidence
Statistics is a scary subject to most people; faculty will come to training with mixed feelings
about the subject matter and varied levels of confidence. For this reason, instructional materials
will be broken into smaller chunks followed by practice in small groups for peer-to-peer
feedback. Course outcomes and learning objectives will be provided in advance; faculty will be
given the opportunity to provide evidence that they have already met course outcomes.
Satisfaction
Skills and knowledge presented in this training are directly linked to continuous improvement
processes at WITC and the technical college system. Seeing the direct link to real world problem
solving will help to build a sense of satisfaction. Mastering the material in this in-service
training will increase efficiency on the job and ultimately, lead to more time with students. The
training is an add-on to faculty that were already certified; by providing this training during paid
in-service time, satisfaction is increased while stress is decreased.

Impact of a Diverse Audience on Instruction


Differences within the WITC faculty learning community include but are not limited to:
Age
Gender
Race
Educational background
Work experiences outside of education
Technical skill (i.e. architecture, telecommunications, welding, business)
Attitudes toward mathematics
Attitudes toward training
Learning disabilities
Physical disabilities
Learning styles
Technological and social media abilities
Computer literacy
Data and analysis experience (i.e. program reviews, HLC committees, no experience)

A universal design framework will be adopted to design instruction with these differences in
mind. Rather than designing the training for the average learner, training will be designed for the
potential learner. This potential learner will be characterized by a broad range of abilities and
disabilities, ages, skill levels, technological abilities, learning styles, and attitudes.
To meet the needs of this diverse learning community and create a positive, safe, welcoming
learning environment for all, this training will:
Offer pathways through the content which provide options, multiple modes of delivery
and varying forms of assessment
Design training materials with a constructivist point of view which allow students to
create their own meaning from within their current ability level
Provide all training materials in accessible formats (written, oral, visual, closed
captioning, where applicable)
Provide access to varied resources and ensure multiple perspectives by encouraging
learners to actively add to the resources
Vary learning materials and strategies to reflect learning styles
Create scaffolding and other support structures to ensure that learners entering the
training with low levels of ability are able to develop skills and confidence in data
analysis
Meet or exceed ADA compliance standards

GOAL ANALYSIS
Aim of Training: Analyze data and apply evidence-based decision-making techniques to
improve quality of instruction and learner success at WITC
Step 1 - Write down the goal.
Faculty at WITC select performance indicators for an outcome that is identified relative to a
classroom, an academic program or the college-at-large; they gather valid, reliable data,
analyze the data and apply evidence-based decision- making techniques to promote
continuous improvement in the teaching and learning process.
Step 2 - Write down everything someone would have to do for you to convince you that he
or she has achieved the goal.

Familiar with the continuous improvement process at WITC plan, do, review, act
Familiar with the continuous improvement process at WTCS
Explains evidence-based decision making
Uses the five-step problem solving process when confronted with a problem define
hypothesis, gather data, analyze data, draw conclusion, write action steps to address the
problem
Calculate measures of center (mean, median, mode)
Calculate measures of variation (range, standard deviation)

Calculate measures of relative position (percentile)


Calculate relative frequencies
Calculate rates (i.e. cost/FTE, retention, course completion)
Identify appropriate descriptive statistics to summarize a data set
Interpret statistical measurements
Construct a frequency table from a data set
Construct a histogram from a frequency table
Construct graphs from summarized data (bar, line, pie)
Construct a scatterplot of bi-variate data
Identify trends in data in table or graphical form
Identify the purpose of the Office of Institutional Effectiveness at WITC
Identify data available through the WTCS Quality Review Process (QRP)
Identify data available in COGNOS
Identify data available in program data profiles (PDP)
Identify scope and purpose of Noel-Levitz student satisfaction inventory (SSI)
Identify the scope and purpose of the Community College Survey of Student Engagement
(CCSSE)
Describe the process for requesting data from the Office of Institutional Effectiveness
Create a survey (Blackboard, Survey Monkey, Google docs) to gather data within the
classroom
Retrieve classroom performance data from Blackboards grade center
Identify sources of data within the classroom which can be used to analyze teaching and
learning (Blackboard, student evaluations, course evaluations, completion rates, attrition
rates, surveys)
Identify sources of data outside the classroom which can be used to analyze teaching and
learning (QRP, COGNOS, CCSSE, Employer Satisfaction Survey, Noel-Levitz SSI)
Analyze the available data relative to learner success
Use data to link achievement of learning outcomes to quality instructional practices and
to delivery mode
Identify the data available through the WTCS Quality Review Process (QRP) related to
program health
Identify the data available through college-wide assessments and surveys related to
program health
Identify the data available through COGNOS regarding individual learners
Where is the data at WITC stored? How is it accessed?
Who is in charge of WITCs data?
Select an appropriate report
Identify the source of data who collected it?
Determine how the data was collected where did it come from?
Identify possible problems within a data set missing data, small sample size, data
doesnt match the meaning of the characteristic being measured

Choose an appropriate metric for a characteristic of data


Use data to identify areas that need improvement
Compares summary data to benchmarks selected in advance
Select an appropriate data report
Recognize data trends

Step 3 Review the items in Step 2 and revise. Sort the list.
I sorted performances into four categories: summary statistics, identifying data analysis
tools at WITC, analyzing data, and using data to make decisions in the classroom, program
and college.
Summary Statistics
Calculate measures of center (mean, median, mode)
Calculate measures of variation (range, standard deviation)
Calculate measures of relative position (percentile)
Calculate relative frequencies
Calculate rates (i.e. cost/FTE, retention, course completion)
Identify appropriate descriptive statistics to summarize a data set
Interpret statistical measurements
Construct a frequency table from a data set
Construct a histogram from a frequency table
Construct graphs from summarized data (bar, line, pie)
Construct a scatterplot of bi-variate data
Identify trends in data in table or graphical form
Data Analysis Tools at WITC
Identify the purpose of the Office of Institutional Effectiveness at WITC
Identify data available through the WTCS Quality Review Process (QRP)
Identify data available in COGNOS
Identify data available in program data profiles (PDP)
Identify scope and purpose of Noel-Levitz student satisfaction inventory (SSI)
Identify the scope and purpose of the Community College Survey of Student Engagement
(CCSSE)
Describe the process for requesting data from the Office of Institutional Effectiveness
Create a survey (Blackboard, Survey Monkey, Google docs) to gather data within the
classroom
Retrieve classroom performance data from Blackboards grade center
Identify sources of data within the classroom which can be used to analyze teaching and
learning (Blackboard, student evaluations, course evaluations, completion rates, attrition
rates, surveys)

Identify sources of data outside the classroom which can be used to analyze teaching and
learning (QRP, COGNOS, CCSSE, Employer Satisfaction Survey, Noel-Levitz SSI)
Analyze the available data relative to learner success
Identify the data available through the WTCS Quality Review Process (QRP) related to
program health
Identify the data available through college-wide assessments and surveys related to
program health
Identify the data available through COGNOS regarding individual learners
Where is the data at WITC stored? How is it accessed?
Who is in charge of WITCs data?
Analyze Data from Teaching and Learning Experiences
Analyze the available data relative to learner success
Choose an appropriate metric performance indicator for a teaching and learning outcome
characteristic of data
Select an appropriate report data set from the Office of Institutional Effectiveness, if
possible
Create a process for gathering data with documentation, if a relevant data set is not
available through the Office of Institutional Effectiveness
Identify the source of data who collected it?
Analyze the reliability of the selected data
Determine how the data was collected where did it come from?
Analyze the validity of the selected data
Identify possible problems within a data set missing data, small sample size, data
doesnt match the meaning of the characteristic being measured
Choose appropriate summary statistics to describe the data collected
Document the analysis
Use Data to Inform Decisions and Identify Problems in the Teaching and Learning Process

Compare data from the classroom to national and college benchmarks


Use data to identify areas that need improvement by looking for changes in performance
indicators over time
Implement WITCs continuous improvement process of plan-do-review-act in the
classroom
Select an appropriate data report
Recognize data trends
Use data to assess instructional practices
Use data to assess learning outcomes by delivery mode (face-to-face, ITV and online) for
courses that are offered in more than one format
Use results of a statistical analysis to recommend action steps for improving the quality
of instruction

Step 4 Write a complete sentence to describe each of the items on your final list.
1. Summarize a given data set by calculating appropriate summary statistics, creating charts
and graphs, and identifying trends in the data.
2. Summarize the scope and purpose of data analysis tools at the college, program and
course levels that are available through WITCs Office of Institutional Effectiveness
3. Analyze data for a given performance indicator that has been identified from teaching
and learning experiences
4. Use classroom, program and college data to guide decision making and to identify
problems in the teaching and learning process as a participant in WITCs continuous
improvement process
Step 5 Combine the statements into a goal statement (or paragraph)
As part of the continuous improvement process at WITC, faculty gather valid, reliable
classroom, program or college data from documented, reputable sources, analyze the data
and use the data analysis to guide decision making relative to teaching and learning.

INSTRUCTIONAL OBJECTIVES
Project Goal
As part of the plan-do-check-act continuous improvement process at WITC, faculty gather
valid, reliable classroom, program or college data from documented, reputable sources,
analyze the data and use the data analysis to guide decision making relative to teaching and
learning outcomes.

Terminal Objectives and Enabling Objectives


1. Summarize a given data set by calculating appropriate summary statistics, creating
charts and graphs, and identifying trends in the data.
Domain: Cognitive
Level: Analyzing
Upon completion of this learning unit, you will be able to:
1.1. Calculate measures of center (mean, median, mode) using technology when provided
with a data set (Level: Applying)
1.2. Calculate measures of variation (range, standard deviation) using technology when
provided with a data set (Level: Applying)
1.3. Calculate measures of relative position (percentiles) using technology when provided
with a data set (Level: Applying)

1.4. Calculate relative frequencies identified in the WTCS Continuous Improvement


Indicator Library using technology when provided with a summarized data set (Level:
Applying)
1.5. Calculate rates identified in the WTCS Continuous Improvement Indicator Library
(i.e. cost/FTE, retention, course completion) using technology when provided with a
summarized data set (Level: Applying)
1.6. Identify appropriate statistical measurements to summarize a performance indicator
from a given data set (Level: Analyzing)
1.7. Interpret statistical metrics using the definitions found in the WTCS Continuous
Improvement Indicator Library with correct units of measure for a given performance
indicator (Level: Understanding)
1.8. Construct a relative frequency distribution for a performance indicator from a data set
(Level: Applying)
1.9. Construct a histogram using technology, including a title and labeled axes, from a
given frequency distribution (Level: Applying)
1.10. Identify the shape of a data sets distribution (normal) for a given frequency
distribution (Level: Analyzing)
1.11.
Construct an appropriate graph using technology, including a title and labeled
axes, to illustrate summarized data (bar, line, pie, time series) (Level: Applying)
1.12.
Identify trends in data presented in table or graphical form (Level: Analyzing)
2. Summarize the scope and purpose of data analysis tools at the college, program and
course levels that are available from WITCs Office of Institutional Effectiveness
Domain: Cognitive
Level: Understanding
Upon completion of this learning unit, you will be able to:
2.1. Summarize the scope and purpose of at least one source of college level data
available from the Office of Institutional Effectiveness tab on WITCs Connection
(Level: Understanding)
2.2. Summarize the scope and purpose of at least one source of program level data
available from the Office of Institutional Effectiveness tab on WITCs Connection
(Level: Understanding)
2.3. Interpret each of the performance indicators found in a given program data profile
(PDP) using the definitions provided in the PDP data dictionary (Level:
Understanding)
2.4. Summarize the scope and purpose of at least one source of course level data available
from the Office of Institutional Effectiveness tab on WITCs Connection (Level:
Understanding)
2.5. Summarize the scope and purpose of the Noel-Levitz student satisfaction inventory
(SSI) (Level: Understanding)
2.6. Summarize the scope and purpose of the Community College Survey of Student
Engagement (CCSSE) (Level: Understanding)

2.7. Summarize the scope and purpose of the National Community College Benchmark
Project (NCCBP) (Level: Understanding)
2.8. Explain the process for requesting specialized data from the Office of Institutional
Effectiveness (Level: Understanding)
3. Analyze data for a given performance indicator that has been identified relative to an
outcome from teaching and learning experiences
Domain: Cognitive

Level: Analyzing

Upon completion of this learning unit, you will be able to:


3.1. Design a process to gather data (i.e. student record review, summary reports from
Office of Institutional Effectiveness, formal survey, informal survey, focus group,
observation) for a given performance indicator (Level: Analyzing)
3.2. Analyze the reliability of the data gathered through the chosen data collection process
by checking whether the data is dependable, repeatable, trustworthy and consistent
(Level: Analyzing)
3.3. Analyze the validity of the data gathered through the chosen data collection process
by checking whether the data actually measures the performance indicator(s) selected
(Level: Analyzing)
3.4. Identify any problems or gaps that exist in a given data set (Level: Analyzing)
3.5. Summarize the data by completing statistical calculations, creating tables or building
charts or graphs that fit the intent of the performance indicator (Level: Applying)
4.

Use classroom, program and college data to inform decision making about the teaching
and learning process as part of WITCs plan-do-check-act continuous improvement
process
Domain: Cognitive

Level: Creating

Upon completion of this learning unit, you will be able to:


4.1. Explain the steps in WITCs continuous improvement model plan, do, check, act
in your own words as it relates to course and program health (Level: Understanding)
4.2. Select at least one performance indicator that can be used to measure a teaching and
learning outcome (Level: Applying)
4.3. Identify sources of internal data (i.e. Blackboard, grade book, student evaluations,
course evaluations, assignment completion rates, QRP, CCSSE, SSI) and external
data (i.e. CCSSE, SSI, NCCBP) which can be used to analyze a selected performance
indicator (Level: Applying)
4.4. Analyze the available data by calculating appropriate statistics, constructing tables of
summarized data or creating charts/graphs (Level: Analyzing)
4.5. Establish benchmarks for the given outcome, if appropriate (Level: Evaluating)

4.6. Use the data analysis to identify teaching and learning outcomes as met or not
met by comparing statistical measurements to established benchmarks (Level:
Evaluating)
4.7. Use data to identify outcomes that need improvement by looking for changes in
performance indicators over time (Level: Evaluating)
4.8. Use data to assess learning outcomes by delivery mode (face-to-face, ITV and online)
for courses that are offered in more than one format (Level: Evaluating)
4.9. Design (in writing) a plan for any outcome identified as needing improvement that
documents proposed changes in the teaching or learning process, outlines action steps
to be taken and provides a schedule for revisiting the outcome after changes have
been implemented (Level: Creating)

REFERENCES
Anderson, L.W. (Ed.), Krathwohl, D.R. (Ed.), Airasian, P.W., Cruikshank, K.A., Mayer, R.E.,
Pintrich, P.R., Raths, J., & Wittrock, M.C. (2001). A taxonomy for learning, teaching,
and assessing: A revision of Bloom's Taxonomy of Educational Objectives (Complete
edition). New York: Longman
Center for Mathematics and Science Education Research (2013). University of Wisconsin
Milwaukee: Milwaukee Mathematics Partnership. Blooms Revised Taxonomy
Mathematics. Retrieved from https://www4.uwm.edu/Org/mmp/ACM201213files/ACM-March15-BloomRevisedMath.pdf
Keller, J.M. (2013). ARCS Explained: What are the ARCS Categories? ARCSModel.com.
Retrieved from http://www.arcsmodel.com/#!arcs-categories/c1zqp .
Krathwohl, D. R. (2002). A Revision of Blooms Taxonomy. (PDF) in Theory into Practice. V
41. #4. Autumn, 2002. Ohio State University. Retrieved from
http://www.unco.edu/cetl/sir/stating_outcome/documents/Krathwohl.pdf
Pappas, C. (2013, May 9). The Adult Learning Theory - Andragogy - of Malcolm Knowles.
eLearning Industry. Retrieved from https://elearningindustry.com/the-adult-learningtheory-andragogy-of-malcolm-knowles.
Wilson, L. O. (2016). Anderson and Krathwohl Understanding the New Version of Blooms
Taxonomy. Retrieved from http://thesecondprinciple.com/teaching-essentials/beyondbloom-cognitive-taxonomy-revised/
Wisconsin Technical College System: Faculty Quality Assurance System. (2016).
[Implementation timeline for new FQAS system]. Recommendations for Transitions

Plan. Retrieved from http://mywtcs.wtcsystem.edu/wtcsinternal/cmspages/


getdocumentfile.aspx?nodeguid=2d0d37cb-3b3b-4596-83bc-4fb52a4b3038.
Wisconsin Technical College System: Data Systems/QRP (2016). WTCS Continuous
Improvement Data Library. Retrieved from http://mywtcs.wtcsystem.edu/data-systemsgrp/continuous-improvement-indicators .

You might also like