Assessment Handbook

DivisionofStudentAffairs BostonCollege
20102011

Originally compiled by Jennifer Lackie, M.Ed.10 Last updated January 2011 Assessment Planning Cycle and narrative adapted from materials by Gavin Henning, Dartmouth College

Index
AssessmentPlanningCycle. Step1:Describetheprogramandtheissueorquestion.. Step2:Developgoalstatement.. Step3:AligngoalstatementwiththeDivisionalLearningGoals. Step4:Identifyanddeveloplearningoutcomes.. Step5:Identifyanddevelopoperationaloutcomes... Step6:Developstrategiestoreachlearningandoperationaloutcomes.. Step7:Identifyassessmenttechniquesandconductassessment.. Part1:SelectingtheAssessmentTechnique. Part2:DevelopingtheAssessmentTool.. Part3:ConductAssessment/CollectData.. 5 6 7 8 9 13 14 15 15 18 19 20 21 22 23 25 26 27 28 29 35 36

Step8:Interpretassessmentdata.. Step9:Report/presentassessmentdataandfindings... Step10:Adjustprogrambasedonassessmentdata,asneeded. Appendices: DivisionalLearningGoals... DivisionalMissionandVisionStatement. QuickGuidetoDiscussingLearningOutcomes... StudentLearningOutcomesbyBloomsTaxonomy. BloomsTaxonomy.. FormativeAssessmentTechniques.. PreAssessmentConversationGuide.. StudentVoiceAssessmentProjectRequestForm.

This page intentionally left blank

Step 1:
Describe the issue or question you want to explore and an initiative, activity, or program that is effected by this question
Tobegintheassessmentplanningcycle,itisfirstnecessarytodeterminewhattoassess. Youmaywanttoassesstheoveralleffectivenessofaparticularprogramoraddressa certainquestionorissuethatexistswithinaservice/program.Therearesometimesmany issueswrappedupintooneForexample:Yourinitialquestionmaybe,whatisleadingto residencehallvandalism?Potentialissuesmightbealackofrespectfortheresidencehall,a lackofconnectiontothehallcommunity,orbehaviorsresultingfromhighriskalcoholuse. Onceyouhaveidentifiedtheissuesofconcern,youwillneedtodecidewhichspecific questionyoureallywanttofocusyourplanningandassessmenton. Then,youshoulddescribetheinitiative,activity,orprogramthataddressesthisproblemor issue.Afewsentencesummaryincludingwhoisinvolvedwiththeprogram(thetarget population)andageneraloverviewofwhattheprogramentailswillsuffice.

Example: TheLeadershipInstituteisaseriesofprograms,workshops,speakersand presentationsthroughouttheyearopentoallclubofficers.TheInstitute hopestoaddresstheissueofstudentleaderbehavioroncampus.Whilethe OfficeofCommunityDevelopmenthashighexpectationsforstudentleaders tobecampusleadersandrolemodelsforotherstudents,thestudentssome timesperceivetheirrolesaspositionalonlyandlimitedtothedescriptions outlinedintheclubororganizationsconstitution.

Step 2:
Develop your goal statement based on the issue/question
Onceyouhaveidentifiedthequestionorissueyouwanttofocuson,youneedtodevelopa broadgoalstatementforwhatyouwanttoaccomplish.Forexample,youmaywantto increasethefeelingofconnectiontotheresidencehallcommunityorbetterunderstand whetherXprogrammeetsitsmissionto. Example: ThegoalofthisprogramistochallengestudentsintheJesuittradition tobecomeincreasinglycomplexandincreasinglyprincipledintheir understandingofappropriatebehaviorforstudentleadersatour institution.

Step 3:
Align goal statement with the divisional learning goals
ItisassumedthatindividualdepartmentmissionsfitintothemissionoftheDivisionandthe strategicplan.Therefore,alignmentwiththemissionofyourdepartmentanddivisional learninggoalsensuresthattheDivisionsvariousprogramsandofficesareallmoving towardssimilaraims. Reviewyourdepartmentsmissionstatementandyearlygoals ReviewtheDivisionalLearningGoals(appendix2) Ifnecessary,reviewtheDivisionofStudentAffairsMissionStatement(appendix3) Finishthestatement:ThisgoalsupportsthedivisionallearninggoalsbyByfinishingthis statement,youwilldemonstratethealignment.Feelfreetowriteandexpanduponhow theprogram/servicefitsinwiththemissionasneeded. Itisalwaysimportanttoconsideryourstakeholdersduringthisstep.Thinkaboutwhoyour stakeholdersareandincludetheseindividuals/officesinyourplanning.Whoisaffectedby theprogramandwouldhavesomeinterestinit?Ensurethatasyoualignyourgoal statementwiththedepartmentalmissionandlearninggoals,youaretakinginto considerationtheinterestsandneedsofyourstakeholders Example: TheMissionStatementemphasizesthenurturanceofengagedlearn ers.Practiceinterviewsofferaunique,handsonlearningexperience thatfullyengagesstudentsinanintellectualandpersonalexercise,one thatincludesimmediatefeedbackonthequalityoftheirresponses. TheMissionStatementemphasizestheneedtofacilitatestudentlearn ingandformationintheirfullestsense(integratingintellectual,ethical, religiousandspiritual,andemotionalsocialdevelopment).Practicein terviewsofferstudentstheopportunitytoreflectonthetotalityoftheir experiencesatBostonCollegeandtolearntoarticulatetheseexperi encesintheserviceoftheircareergoals.

Program Level Department Level DivisionLevel UniversityLevel

Developedfrom:StudentVoice,DesigninganAssessmentProject,slide 15,11/4/2010

Step 4:
Identify and develop learning outcomes for a specific issue or question
InStep4,youwilldeveloplearningoutcomesforthespecificissueorquestionidentifiedin Steps13.ThisisacriticalstepintheAssessment Fig1:LearningOutcomesAssessment PlanningCycle,particularlyastheDivisionof StudentAffairsmovesforwardinitsStrategic LearningOutcomesAssessment Planningandbecomesincreasinglyfocusedon Outcomesassessmentisthemostvalidwayof studentlearning. demonstratingtheeffectivenessofservices, programs,andfacilities,especiallyindefendingand Thenextsectionwillprovidesomeuseful promotinghighereducation,andalsoinmeeting techniquesthatmightguideyourthinkingasyou accreditationstandards.Itisalsomostdifficult, considerlearningoutcomesforthespecificissueor complexandmisunderstoodofalltheassessment questionyouplantoassess. methodologies. (Upcraft&Schuh,1996) Brainstorm:Oneoftheeasiestwaystobegin writinglearningoutcomesistothinkaboutyour StudentVoice,HowtoWriteOutcomeStatements, program,activity,orserviceandbegin Slide11,10/22/09 brainstormingalistofallthethingsthatyouhope theparticipantswilllearnfromparticipatinginyourprogramorservice.Brainstormalist withoutworryingabouttheformatorqualityofyourwriting.Peopleoftenfinditmore effectivetobrainstormwithasmallgroupofcolleagues. Organize/Categorize:Onceyouhavecomeupwithafulllistofpotentiallearningbased outcomesforyourprogram,activity,orservice,reviewthelist.Removeanyduplicatesand sortandorganizetheremaininglearningoutcomesinmanageablegroups.Whenorganizing yourlist,itislikelythatyouwillfindmultipleoutcomesthatdealwiththesameaspectofa programorareverysimilarinotherways.Groupingandsortingyourideasmakesiteasier whenitcomestoactuallywritingtheoutcomes. SelectyourOutcomes:Wheneveryoubeginanewassessmentproject,itisrecommended thatyouhavethreetofivelearningoutcomes.Ifyouroutcomesaremorecomplex(e.g.: studentwillbeabletodemonstrateanincreaseinleadershipabilitiesasmeasuredbya rubric),youshouldhavefewer.Ifyouroutcomesaresimpler(e.g.:studentwillbeabletolist thenamesofthreeoffices),youcouldhavemorethanfivelearningoutcomes.Reviewthe organizedlistsyouhavebrainstormedanddeterminewhichlearningoutcomesyouwouldlike toassess.Youwillmostlikelyselectthelearningoutcomeswhicharemostcentraland importanttoyouractivity,programorservice.However,basedonyourownneedsandthose ofyourstakeholders,other,lesscriticallearningoutcomesmaybechosen.

10

Thenextstepsonlyutilizetheoutcomesyouhaveselected.Youwillnotneedtherest! Overtime,asyougatherassessmentdataandbecomemorecomfortablewiththeassessment process,youcanaddadditionaloutcomesasyouseefit.However,atthebeginning,startsmall. ChooseyourBloomverbs:Thekeytosuccessindesigninglearningoutcomesischoosingverbs whichreflectthelevelandnatureoflearningyouanticipatewillresultfromtheprogramor servicethatisbeingassessed.BestpracticesinstudentaffairssuggesttheuseofBlooms Taxonomytoselectverbsforlearningoutcomes. Bloomidentifiedsixlevelsoflearning:Remembering,Understanding,Applying,Analyzing, EvaluatingandCreating.Thelevelsbuildinincreasingorderofdifficultyfrombasic memorizationtohigherlevelsofcomplexandcriticalthinkingskills.AsisevidentinFig.2, movingupthepyramidreflectshigherorder Fig2:BloomsTaxonomy thinkingskills. by Usefullearningoutcomeverbsarecategorized theirlevelinBloomsTaxonomy.Alistof categorizedverbsisavailableinAppendix5. Whenchoosingaverbforthelearning outcome,itisimportanttoconsiderwhatyou intendtomeasure,asthatwillaffectthe BloomsTaxonomylevelfromwhichyouselect theverb.Forexample,ifyouwouldliketo measurewhetherastudentknowstheoffices on campusthatcanhelpthemwiththeirstudyhabits,youwoulduseaverbfromthe Rememberinglist.If,however,youwantedstudentstounderstandwhateachofthoseoffices did,youwouldselectaverbfromtheUnderstandinglist.AttheRememberinglevel,you areassessingwhetherstudentshavememorizedbasicinformation,attheUnderstanding level,youareassessingwhetherthestudentsunderstandtheinformationthatthey memorized. Includeindicatorofsuccessinoutcomes: Inorderfortheresultstobemeasurable,youneedtodeterminewhatexactlyindicatesthatan outcomewasachievedornot.Youdothisbyarticulatingexactlywhatdefinessuccess. ApooroutcomewouldbeAttheconclusionofXprogram,studentswillbeabletoidentify campusresourcesthatcansupporttheiracademicachievement.Thisisnotagood outcomebecausethereisnoindicationofwhatwemeanbyidentifycampusresources.If Iidentifyonecampusresource,didImeetthelearningoutcome? AbetteroutcomemightbeAttheconclusionofXprogram,studentswillbeableto identifyfivecampusresourcesthatcansupporttheiracademicachievement.Thisoutcome

11

ismoreappropriatebecauseitspecifiesexactlywhatindicatesthatthestudenthas achievedthelearningoutcome(theywillknowfivecampusresources). Theindicatorofsuccessiscalledthedegree.Degreecanbeexpressedinmanyways, dependingonthenatureofthelearningoutcome.Somecommonwaystoexpressdegree includeusingaspecificnumber(6,3),usingapercentageorspecificportionofthetotal(50%, 7outof10)orusingaqualifyingwordsuchassufficientlyoraccurately.Twoimportant thingstonote:First,multipledegreescanbeusedperoutcome(seebelow)andsecondly,the abovelistisnotexhaustivethereareadditionalwaystoindicatedegreebeyondthose specified. Formatting:Nowthatyouhavecreatedallofthecomponentsofalearningoutcome,youneed toputtogetherthepiecesandactuallywriteit!Therearetwocommonformatswhichareused towritelearningoutcomes:SWiBATandABCD.

SWiBAT:Oneformulathatissuccessfulindevelopinglearningoutcomesistobeginyour outcomesstatementwithStudentsWillBeAbleTo.Beginyouroutcomestatementwith thisandthenaddanactionverbandacondition.Forexample,studentswillbeableto successfullyfacilitateahallgovernmentmeetingasaresultofparticipatingintheresidence hallcounciltraining.MakesureyouincludeyourdegreewhenusingtheSWiBATformula!

ABCD:AnothertechniquethatcanbeusedtodeveloplearningoutcomesistheABCD format.UsingtheABCDformat,thelearningoutcomemusthaveaspecificidentified Audience,aparticularBehaviorwhichtheaudiencewillbeabletoknowordo(thisisthe verbyouselected),aConditionwhichwillleadtotheearningandaDegree,whichindicates theleveltowhichtheparticularbehaviorwillneedtobedemonstratedinordertoensure learningtookplace.AlthoughthistechniqueiscalledtheABCDformat,itisnotnecessary tohavethelearningoutcomefollowtheABCDformatinexactorderaslongasthe componentsareallthere.Onecould,forexample,writealearningoutcomeinBCDAor ADBCorder.Figure4,below,providesanexampleofhowalearningoutcomewouldlookif formattedinCABDorder. Fig4:ABCDStructureApplied Fig3:ABCDStructure
ABCDStructureofaLearningOutcome
Audience/Who Whodoestheoutcomepertainto? Behavior/What Whatdoyouexpecttheaudiencetoknow/beabletodo? Condition/How Underwhatconditionsorcircumstanceswillthelearningoccur? Degree/Howmuch Howmuchwillbeaccomplished,howwellwillthebehavior need tobeperformed,andtowhatlevel?
(Heinich,etal,1996)

Developing Learning Outcomes

Condition Audience
As a result of participating in the leadership workshop students

Behavior (verb in future tense)

Degree

will demonstrate three of the five leadership criteria

as stated in Kouzes and Posners The Leadership Challenge.

22

StudentVoice,HowtoWriteOutcomeStatements, Slide15,10/22/09

Henning,Wherearewegoing?,Slide22,10/22/09

12

Checkyourself:Wellwrittenoutcomeshavesomedefiningcharacteristics.SMARTisa mnemonicdevicethatcanserveasachecklist.SMARToutcomesare: Specific Measureable Aggressive,butattainable Resultsoriented Timebound Afteryouhavewrittenyourlearningoutcomes,verifythattheyareSMART.Ifyoudo,youwill bemoreeffectiveasyouplanandyourassessmentwillbeeasiertoimplement.

LearningOutcomeExamples: ResidentAssistantswillbeabletosuccessfullydemonstratethree techniquestheycanusetointerveneinadiscriminatoryjokeatthe endofRespectWeekend. AftertwomeetingswiththeUHSStaff,studentswithlatentTBwillbe abletoaccuratelydescribefourprosandfourconsofreceivingthe tuberculosistreatment.

13

Step5:
Identifyanddevelopoperationaloutcomes

Unlikelearningoutcomesthatmeasurewhatstudentsareabletodo,know,orvalue, operationaloutcomesareaggregateanddocumentprogresstowardstheoutcome. Operationaloutcomesmayincludethetotalnumberofprogramspresentedornumberof studentsserved.Forexample:30studentswillattendtheprogram;75%ofstudentswill completesomething,etc. Operationaloutcomestendtoberelatedtosatisfaction,quality,tracking,needsmet,and usage. Includespecificindicatorsofsuccesswhilewritingoperationaloutcomes.(Seestep4for moreinformationonindicatorsofsuccess).

Examples: TheCareerCenterwillrunpracticeinterviewsforatleast50 studentsduringthefallsemester. 75%ofthecustomerswhopurchaseticketsattheRobshamBox OfficewillindicatethattheyaresatisfiedorbetterwiththeBoxOf ficescustomerservice

14

Step 6:
Develop strategies to reach learning and operational outcomes
Aspreviouslymentioned,successfulassessmentisinextricablytiedtogoodplanning. Strategiesarethespecificstepsthatyouwilltakeorthethingsthatyouwilldotofoster eachofthelearningoutcomes. Alignstrategieswithoutcomesandgoals Onceyouhaveidentifiedyourgoalsandoutcomes(bothlearningandoperational), beginconsideringstrategiesthatwillfosterthosegoals.Forexample,ifyourgoalis forstudentstodevelopleadershipskillsandoneoutcomeforthatistheabilityto facilitateahallgovernmentmeeting,whatstrategieswouldhelpstudentsbeable tofacilitateameeting?Onepossibilitywouldbeforthemtoobserveother meetingsandidentifybestpracticesformeetingfacilitation. Thingstoconsiderwhendevelopingstrategies BestPractices:Thestrategiesusedshouldberootedintheories,conceptual frameworkandbestpractices.Identifyingtheories,conceptualframeworksand bestpracticesensuresthatyouarebeingintentionalinyourwork,andthatyour workisgroundedinresearchandevidence. Actualresources:Thestrategiesusedshouldberealisticwhenconsideringthe resourcesthatareavailabletoyou,includingtime,finances,humanresources, facilities,etc. Stakeholders:Thestakeholdersoftheactivityyouareassessingmayhave particularstrategieswhichtheyprefer.Considertheinterestsandneedsofyour stakeholderswhiledevelopingyourstrategies.Additionally,someofyourstrategies mayinvolveadditionalstakeholderswhoshouldbetakenintoaccount.For example,ifaparticulartrainingonselfimageinvolvestheDirectoroftheWomens ResourceCenter,itisimportanttoverifythattheDirectorisavailableandwillingto participate. Numberofstrategies:Therewilllikelybemultiplestrategiesforeachgoal. Example: LO:RAswillbeabletosuccessfullydemonstratethreetechniquesthey canusetointerveneinadiscriminatoryjokeattheendofRespectWeek end. Strategy1:Aspartoftheprogram,wewillrunaworkshopthatdelineates thestepbystepprocessforinterveningindiscriminatoryjokes. (Conceptual framework:ThisstepbystepprocessfollowstheBystander InterventionProgrammodel)

15

Step 7:
Identify assessment technique and conduct assessment
Duringthisstep,youwillidentityanddeveloptheassessmenttechniquesthatyouwilluse tomeasurethesuccessofachievingtheoutcomesandgoal(s)foryourprogram.The programyouhavechosentoassessshoulddrivetheassessmenttechniqueyouselect,not theotherwayaround.Duringthisstepyouwillselecttheassessmenttechniquewhich youwillusetoconductyourassessment,createtheassessmenttool,andconductthe assessmentusingthetoolsyouhavedeveloped. Part1:SelectingtheAssessmentTechnique Whenselectinganassessmenttechnique,itisimportanttokeepthefollowing considerationsinmind: Directvs.IndirectAssessment:Assessmentcanbedirect,indirect,orboth.Whendirect assessmentisused,studentswillbeaskedtodemonstrateexpectedoutcomes.Direct assessmentisevidenceoriented.Whenindirectassessmentisused,studentswillbe askedtoreporthowtheyperceivemasteryofparticularconceptsoroutcomes. Indirectassessmentisperceptionoriented. Itisimportanttoconsiderwhattypeofassessment(directorindirect)youwantto conductbeforeyouselectyourtechnique,asthetechniquemayvarybasedonthe typeofassessmentyouchoose.Pleasenotethatalthoughindirectmethodsmay sometimesbeusefulandappropriategiventhelearningoutcomeyouintendto measure,ingeneralitispreferabletousedirectassessmenttechniques.

Example: IndirectAssessment:Studentvisitshealthservicesandmedicalprofes sionalaskshowthestudentisfeeling.(Studentisbeingaskedtoreport ontheirperceptionsoftheirhealth.) DirectAssessment:Studentvisitshealthservicesandmedicalprofessional takesthestudentstemperature,listenstothestudentslungs,etc.(The medicalprofessionalisobservingthestudentshealth.) or IndirectAssessment:Atapreliminaryinterview,careercounselorasksa studenthowskilledtheyareininterviewing.(Studentisbeingaskedto reporttheirperceptionsofaskill.) DirectAssessment:Careercounselorobservesastudentdoingapractice interviewandratesthestudentsinterviewingskills.(Counselorisobserv ingthestudentsskills)

16

Quantitativevs.QualitativeData:Beforeselectinganassessmenttechnique,itisalso importanttoconsiderwhattypeofdatayouwouldliketogather.Quantitativedata focusesonnumbers,whilequalitativedatafocusesontext,narrativeandstudent stories.Therearebenefitsandchallengestoeach.Whilequantitativedataiseasierto reportandanalyze,qualitativedatacanbemuchmoreindepthandrobust.Itisalso simplertogeneralizequantitativedatatoalargerpopulationandcanbeeasier(in time,resources)toconduct.Ontheotherhand,duetoitsrobustnature,qualitative datamayallowforagreaterunderstandingofstudentlearninganddevelopment. Howwillyouusethedata?:Priortoconductinganassessment,thinkaboutwhowill usethedataandhow.Yourstakeholders(andthekindofdatatheyliketosee)and howyouplantousethedatawillinfluenceyourtechnique. Availableresources:Evaluateavailableresourcesbeforeselectinganassessment technique.Someassessmenttechniquesrequiregreatertime,financialresources,etc. Culturalsensitivity:Certaintechniquesmaybemoreinvasivethanothers.Some students,forexample,mayfeeluncomfortabletalkingaboutparticulartopicsina groupsetting(likeafocusgroup).Keepthisinmindwhenselectinganddesigningthe assessmenttechnique.

AssessmentTechniques
Therearemanydifferentassessmenttechniqueswhichcanbeusedinanygivensituation.This sectionoutlinesseveralofthemajortechniques,butitshouldnotbeconsideredanexhaustive list.Appendix6alsolistsanumberofassessmenttechniqueswhichcanprovideaquickpulse onhowaparticularprogramorserviceisgoing. Tracking/ExistingData Thistechniqueisthesimplest,butalsomayreveallessinformationthansomeofthe otherassessmenttechniques. Trackingincludesgraduationandretentionrates,housingoccupancy,income/ expenditures,staffing,numberofprograms/workshops,enrollmentdata,SATscores, participationinprograms,attendanceatevents,usageofservices,numberofphone calls,etc. Mostofficesalreadytrackcertainpiecesofinformationlookatwhatyourofficeis alreadydoingandusethatdataaspartofyourassessment. Otherofficesmayalsobetrackingcertaininformation.InstitutionalResearch,for example,hasawealthofinformation.Beforebeginninganassessmentproject,itis alwaysusefultounderstandwhatinformationotherofficeshavethatcaninformyour ownassessment. Trackingisoneofthesimplesttechniquestouse.Verylittleexpertiseindatacollection oranalysisisneededandminimaltimeisrequired.

17

Surveys Therearetwomajortypesofsurveys:nationalsurveys(thinkNSSEandCIRP)andlocal surveyswhicharedevelopedbyyou,youroffice,ortheinstitution. Surveysaregoodforselfreportsaboutperceptions(howsatisfiedwereyou?), behaviors(howmanytimesdidyoudoX?)andobservations(doyouthinkthatcampus isawelcomingplace?).Surveysarealsousefulfortestsorquizzestomeasurewhat studentslearnedandforpre/posttests. Althoughsurveyscanbeusefulforgroupsofanysize,theyaremostappropriatefor largegroups.Moreinformationmaybegleanedfromsmallergroupsusingother assessmenttechniques. Onemajorprobleminhighereducationissurveyfatigue.Therefore,itisalwaysagood ideatoreviewotherassessmenttechniquesanddetermineifanyofthemare appropriate.Althoughitmaybenaturaltosendoutasurvey,theremaybemore usefultechniqueswhichcanprovidethesame,orbetter,results. Youmustconsiderthatwhenstudentsareselfreportinginformationitshouldbe takenwithagrainofsalt.Researchshowsthatstudentsaremorelikelytoreportdoing sociallyacceptablebehaviors(studying),andlesslikelydoingsociallyinappropriate behaviors(takingillegaldrugs). Uponrequest,StudentVoicewillconsultwithyouonsurveysyouhavedeveloped. FocusGroups Focusgroupsareusefulwhenyourgoalistounderstandtheperceptions,beliefsor opinionsofagroupofstudents(orotheraudience).Theycanbeusedasadirector indirectmethodofassessmentandcangatheralotofdatainashortamountoftime. However,becausetheresultsfromafocusgroupcannotbegeneralizedtoalarger population,theyarelessusefulincreatingstatisticalreports. Focusgroupsarenotexpensive,butdorequireadegreeofskillinfacilitation, particularlyifthegroupdynamicsoftheparticipantsarechallenging.Evenwithstrong facilitation,focusgroupdiscussionsareunpredictableandmayveerinunanticipated directionsmakingtheselesscontrolledthanothertechniques. Focusgroupscanbeconductedwith610people.Inordertovalidatefindings,the samequestionsshouldbeusedwithmultiplefocusgroupsandwithdifferent individualsinvolvedineach.Asaresult,focusgroupstakemoretimeandplanning thanotherassessmenttechniques. Finally,sincefocusgroupscollectqualitativedata,theyrequiremoretimetoanalyze theresultsthanfortechniqueswhichcollectquantitativedata.

18

Rubrics Arubricisascoringtoolusedtoevaluatestudentperformanceorwork.Itisauseful assessmenttechnique,particularlyatthehigherlevelsofBloomsTaxonomy. Rubricsincludedescriptionsofwhatsomethinglookslikeatdifferentlevelsofmastery, andthusprovideconsistencybetweendifferentraters. Rubricsallowassessorstogatherrichdata,facilitateprovidingfeedbacktostudentson theirperformance,andallowforadirectmeasureoflearning. Datacanbecollected,andanalyzedbyarubricthroughobservation(i.e.,roleplay, training,groupwork)orthroughartifactanalysis(i.e.,journal,resumeorreflection paper). Therearemanydifferentkindsofrubrics.Oneofthebenefitsofrubricsisthattheyare highlyflexible. StudentVoicewillassistyouwiththedevelopmentofyourrubric,butitisalwaysbest practicetohavemultipleindividualsreviewandtestyourrubricbeforeyouuseit. Passingarubricthroughseveraltestspriortousewillconfirmthatyourscalesare appropriate,thedimensionswillgiveyoutheinformationyouneed,andthe descriptionswithineachdimensionareclear. Ensureinterraterreliability:Beforeyouusearubric,itisimportanttotrainthe individualswhowillbeconductingtheassessmentwithyourrubric.Youwanttoverify thattheyknowhowtherubricworksandensurethatalltheassessorsunderstandthe rubricinthesameway.Youmayhavetheassessorsdoapracticerunononeindividual andverifythattheyratedthestudentsimilarly.Keepinmindthattheassessorsdonot havetomatchupexactlywiththeirratings,buttheyshouldbeinthesameballpark.If majordiscrepanciesexist,itiscriticalthattherubricbeadjustedtoclearupany confusion. Part2:DevelopingtheAssessmentTool AnumberofresourcesareavailableonBCsassessmentwebsitetohelpdevelopvarious assessmenttools.StudentVoicealsohasanumberofwebinarstohelpasyoubegin developingyourassessmenttool. VPSAAssessmentWebsite:http://www.bc.edu/offices/vpsa/Assessment.html StudentVoiceWebsite:www.studentvoice.com Makesurethattheassessmenttechniqueyouchosetargetstheoutcomesyouhopeto achieve.Itiseasytogetboggeddownwithsatisfactionassessment,soensuringyour questionsarefocusedonwhatyoureallywanttoknowiscritical. Finally,considerwhatkindofinformationiscriticaltoyourstakeholders.Thinkabout whetheryourstakeholdersaremoreinterestedinquantitativedataorqualitativedata.If yoursupervisorneedstoknowaboutX,makesureyouincludethatintheassessment!Be surethatyourassessmenttechniquewillallowyoutoreportbasedontheseneeds.

19

Part3:Conductassessment/collectdata Usingthetechniqueyouidentifiedinthisstep,conductyourassessment.StudentVoice, willbeaninvaluableresourceinhelpingyouconductyourassessmentandcollectthe data. Timing:Whencollectingdata,considerhowtheacademiccalendarmightaffectresponse rates.Studentstendtobemuchbusiertowardtheendofeachsemesterandasaresult maynotbeaswillingtocompletevoluntaryassessments. Rememberthatstudentsaremuchmorewillingtoparticipateinassessmentswhichthey feelhassomesaliencetothem.Assessmentswhichstudentsperceivetobeirrelevantwill generallyhavealowerresponserate.Therefore,highlightingthesalienceofan assessmentandhowthedatawillbeusedisimportantinordertoincreaseresponserates.

20

Step 8:
Interpret Assessment Data
Makesenseofthedata Inthisstep,youmustinterpretthedataandmakesenseofitbyansweringquestionssuch as: Basedonthedata,didwereachouroutcomes?Ifnot,towhatextentdidwereach ouroutcomes? Basedonthedata,arethereadjustmentsweshouldconsiderfornexttime? Itisoftenbesttoanalyzeyourassessmentwithasmallgroupofcolleagues.Everyone shouldlookatthedataseparately,andthencometogethertosharefindings.Younever knowwhatyoumightcomeupwiththisway! Consideralternativehypotheses Makesuretoconsideralternativehypotheseswhenevaluatingthedata.Perhapstheissue youchosetofocusonisalcoholuseandyouhavejustimplementedanewprogram.You reviewyourdataandyoufindoutthatthereisadecreaseinusageacrosscampusafter thefirstyearofimplementationoftheprogram.Canyouattributethedecreasetoyour program?Maybe,maybenot.Youmaywanttochecktoseeiftherewereanypolicyor enforcementchangesthatmayhavealsocontributedtolowerusagerate. Waystoevaluate Lookforpatternsandgapsinyourdata,includingdifferencesbetweensubgroups(males vs.females,participantsvs.nonparticipants,sophomoresvs.juniors).

UseStudentVoice StudentVoicehasgreattoolstohelpyouanalyzeyourassessmentdata.Noneedto rememberstatisticsfromyearsago,justuseStudentVoicetohelpyoumakesenseofyour data.

21

Step 9:
Report/present assessment data and findings
AstheDivisionmovesforwardinitsstrategicplan,itisimportantthatassessmentresultsand findingsaresharedwithcolleaguesandotherstakeholdersintheDivisionwhomightbenefit fromyourfindings. Identifyyouraudience Whoneedstoseethereport?Theanswerdependsuponwhatthereportisabout andtheinterestedstakeholders.Thinkaboutwhosequestionsyouaddressedand thosewhosequestionsyoudidnot. Thinkaboutwhowouldbeaffectedbytheimplicationsofthedata,bigpictureand smallpicture,andsendsomeversionofthereporttothoseindividualsandoffices. Thinkaboutyourcontentandformatasitrelatestoyouraudience Differentpeoplemayalsowant/needslightlydifferentinformation,soitis importanttotailoryourreporttoeachperson.Youwilllikelyhavemultiplereports foranyoneassessment.Whenwritingeachreport,considerwhatinformationthe recipientwillfindmostusefuldonotoverwhelmyourstakeholdersbysendingall ofyourfindingstoeveryone! Examplesofwaystopresentyourcontentbasedonaudience: o Aonepageexecutivesummaryforbusyadministrators o Apresentationforhealthservicesstaffsothattheycandiscusstheimplications o Afullreportonthewebtodisseminatetothecampuscommunity o Apressreleasetogettheinformationtothepublic Timing:decidewhenthereportwillbedisseminated Yourtimingisjustasimportantashowtheinformationwillbedistributed.For example,itmaybeveryineffectivetodisseminatetheinformationatthebeginning orendofasemesterorduringabreak.Ifapplicable,alsoconsiderthebudgetcycle andhowyourreportmightimpactresourceallocation. Considerwhatyoucandotokeepthereportfrombeingshelved Itisimportanttoconsiderhowyoucangetpeopletoreadthisreport.Thismay relatebacktoyourassessmenttechniqueandpresentationformat.Forexample,if theVPisanumberspersonbutyourdirectorlikestoreadaboutstudentstories, youmaydecidethatyouneedbothquantitativeandqualitativeresultsto effectivelydemonstratethedata. Youmayalsodecidetogetothersinvolvedinthebeginning.Sometimesan assessmentwillbecriticizedonitsmethodbecauseoneormoreconstituenciesdid notexpecttheresultsortheywerenotpreparedfortheimplications.Forexample, ifimplicationscouldbeincreasedresidencehallfeesitmightbehelpfultotalkwith thestudentsenatepriortotheassessment.Youwouldwantthemtobe supportiveoftheassessmentsothattheyaremorelikelytoaccepttheresults.

22

Step 10:
Make adjustments based on assessment data
Nowthatyouhavethedata,youcanreviewyourfindingsandidentifyparticularaspectsof theprogram(oreventheprogramasawhole)thatmayneedtobeadjusted.Youmayfind thatyouroutcomesarenotpossibletoreachorthataparticularstrategyisnotworking. Makingchangesisnotalwaysnecessarysometimesyouwillfindyourprogram/service wasnotonlyhighlyeffectiveinitsintendedoutcomes,butstudentswerelearningeven morethanyouthought!Ifthisisthecase,youmayneedtoaddsomeoutcomestoyour list. TherearefourpossibleactionsthatyoucantakeinStep10: Keepcurrentprogramasitis Tweakprogram Replacewithabrandnewprogram Cutprogram Sometimesyouwillrealizethatyouneedtoassessyourprogramdifferently.Onceany adjustmentsaremade,theassessmentcyclewillbeginagainforanotherroundbutwitha slightlydifferenttwist! Assessmentshouldneverstopsoyoumustidentifywhattoassessnext.Asstudents constantlychange,ourprogramsandserviceswillneedtobetweakedandadjustedto continuetobemosteffective.Thinkaboutyournextcycleofassessment.Youmayrealize awaytoimproveyourassessmenteffortsoryoumayjustwanttoreassesswitha differentsetofstudentsandseeifyougetthesameresults. o Forexample,ifyoupreviouslyassessedresidencehallcounciltraining: Nowyouknowthatstudentsincreasedtheirmeetingfacilitationskills,so youmaywanttounderstandhowthishappened.Whatwasmostuseful? Alternately,youmaywanttoassessdifferenttypesofleadershipskillsaside frommeetingfacilitation.

23

Appendix 1: Divisional Learning Goals

Thefivedivisionallearningdomainsare: Intellectual ReligiousandSpiritual Ethical Emotionalsocial InterculturalandGlobal(InclusiveCommunity) Learninggoalstatements: 1. Studentswilldevelopgreatercognitivecomplexityandmaturity(Intellectual) 2. Studentswillunderstand,andmorefullyintegrate,ethicsandvaluesintotheirlives (Ethical) 3. Studentswillengagemorefullywiththeirownfaithorbeliefsanddevelopagreater awarenessofthereligiousandspiritualidentitiesofothers(ReligiousandSpiritual) 4. Studentswilldevelopgreateremotionalandsocialmaturity(Emotionalsocial) Studentswillunderstandtheirownculturaltraditionanddevelopgreaterculturaland globalcompetency(InterculturalandGlobal) Examplesofoutcomesforeachgoalstatement INTELLECTUAL:Studentswilldevelopgreatercognitivecomplexityandmaturity. Examplesofnextleveloutcomesinclude,butarenotlimitedto: StudentswhoparticipateinlearningexperiencesprovidedbytheDivisionofStudentAffairswill: Demonstrateanalytical,reflective,andcreativethinking Constructnewknowledgeandintegratethisknowledgeintoestablishedideasandex periences Thinkcriticallyandproactivelywithregardtolongtermcareergoals Demonstratetheabilitytolistentoothersandbeopentonewperspectives Integrateclassroomlearningwithexperientialandpersonallearning ETHICAL:Studentswillunderstand,andmorefullyintegrate,ethicsandvaluesintotheirlives. Examplesofnextleveloutcomesinclude,butarenotlimitedto: StudentswhoparticipateinlearningexperiencesprovidedbytheDivisionofStudentAffairswill: Beabletoclarifyandarticulatepersonal,work,andlifestylevaluesandbeliefs Developanincreasedunderstandingofsocialjusticeissues Demonstratetheabilitytochallengeunfair,uncivil,orunjustbehaviorofothers Actincongruencewithpersonalvaluesandbeliefs

24

SPIRITUALANDRELIGIOUS:Studentswillengagemorefullywiththeirownfaithorbeliefsand developagreaterawarenessofthereligiousandspiritualidentitiesofothers. Examplesofnextleveloutcomesinclude,butarenotlimitedto: StudentswhoparticipateinlearningexperiencesprovidedbytheDivisionofStudentAffairswill: Articulateandexploreissuesofpurpose,meaning,andfaith DemonstrateanunderstandingoftheJesuittraditionandhowitrelatestoreligious andspiritualidentity Compareandcontrastvariousbeliefsystems Seekoutinvolvementwithindividualsandgroupsfromdifferentspiritualandreli gioustraditions Questionactsofincongruencewithreligiousandspiritualbeliefs EMOTIONALSOCIAL:Studentswilldevelopgreateremotionalandsocialmaturity. Examplesofnextleveloutcomesinclude,butarenotlimitedto: StudentswhoparticipateinlearningexperiencesprovidedbytheDivisionofStudentAffairswill: Demonstraterealisticselfappraisal,selfunderstanding,andselfrespect Giveevidenceofunderstandingandaffirmingtheimportanceofcivicengagement andparticipation Describetheirpersonalstrengthsinrelationtocareeroptions Demonstratetheabilitytotaketheperspectiveofothers Seekoutopportunitiestoparticipateinservice/volunteeropportunitiesthatare characterizedbyreciprocity Demonstrateanabilitytomanagetheirownbehavioralandemotionalhealth Takeresponsibilityfortheirindividualchoicesandactions INTERCULTURALANDGLOBAL:Studentswillunderstandtheirownculturaltraditionandde velopgreaterculturalandglobalcompetency. Examplesofnextleveloutcomesinclude,butarenotlimitedto: StudentswhoparticipateinlearningexperiencesprovidedbytheDivisionofStudentAffairswill: Criticallyreflectupontheirownculturaltraditionsandvalues Demonstrateanunderstandingofdifferentculturaltraditionsandvalues Seekoutinvolvementwithindividualsandgroupsfromdifferentculturaltraditions Giveevidenceofculturalcompetency

25

Appendix 2: Divisional Mission and Vision Statement

MissionStatement ThemissionoftheDivisionofStudentAffairsistofacilitatestudentlearningandformationin theirfullestsense(integratingintellectual,ethical,religiousandspiritual,andemotionalsocial development),promoteaninclusivecommunityofengagedlearners,andadvancetheJesuit, CatholicheritageandvaluesofBostonCollege. VisionStatement TheDivisionofStudentAffairswillbeafullpartnerintheprocessoflearningandformationat BostonCollege.Byprovidingadiversityofexperientiallearningopportunities,buildingcom munitiesofrespectandsupport,encouragingandsupportinglivesofintegrity,offeringser vicesthatremovebarrierstolearning,promotingstudentsreadinesstolearn,andcollaborat ingwiththeDivisionsofAcademicAffairsandMissionandMinistry,StudentAffairswillhelp studentsengagefullywiththeUniversityseducationalopportunitiesandprogressintheir journeyofpersonalformation. StudentAffairsenhancestheJesuit,Catholicmissionoftheuniversitythroughitsprograms, interactionswithstudents,andmentoring.TheDivisiondoesthisbypromotingalifecommit tedtoserviceandsocialjustice,sponsoringdialogueonimportantstudentissues,supporting religiousandspiritualdevelopment,promotingmoraldevelopmentandbehaviorgroundedin distinctprinciples,andsupportingtheintegrationoffaithwithjustice.Ultimately,theDivision desirestheactualizationoftheuniversitysmissioninthelivesandhabitsofstudents. StudentAffairsstrivesforexcellenceineveryaspectofitsorganization,operations,andpro gramandservicedesignanddelivery;theDivisioniscommittedtoevidencebasedpractice andtheuseofassessmentdatatostrengthenprogramsandservices.Throughreflective,stra tegicplanning,carefulallocationofresources,andcomprehensivestaffdevelopment,theDivi sionworkstoprovidelearningexperiences,programs,andservicesthatproducesoundstu dentoutcomes. Inallofitswork,StudentAffairsseekstoknowandunderstandstudents,comprehendand addresstheirneeds,andrepresentandadvocatefortheirneeds;itintendstoworkwithstu dentstocreateastrong,healthylearningenvironmentandaninclusive,participatorycampus culture.OnespecialfocusoftheDivisionsworkisonrecognizingandaddressingtheneedsof underrepresentedstudents. StudentAffairsintendstocreateandsustainacaringcommunitywithintheDivision;torecog nizethecontributionsandachievementsofallstaffandleadership;andtoencourageanap propriateandsustainableapproachtoselfcareforallmembersoftheDivision.

26

Appendix 3: Quick Guide to Discussing Learning Outcomes

Afteryouhavewrittenyourlearningoutcomes,lookthemoverandaskyourselfthefollow questions: Didyouidentifytheprogram,serviceoractivitythatyouplantoassess?Didyouidentify, ifnecessary,thespecificaspectoftheprogram,serviceoractivity,thatwillleadto studentlearning?(ThisiscalledtheCondition) Didyouidentifythegroupyouwanttoassess?Perhapsthismightbeallstudents,ora specificgroupofthem.(ThisiscalledtheAudience) Didyouidentifywhatyouraudienceshouldbeabletoknow,do,orvalueafter participatinginthecondition?Whenyouwroteitinthelearningoutcome,didyou includeaBloomverb?(ThisiscalledtheBehavior) Didyouidentifyhowmany,much,ortowhatdegreeyouexpectyouraudiencetodo thebehavior?(ThisiscalledtheDegree) *Note:Theaudience,behavior,conditionanddegreecanbewritteninthe learningoutcomeinanyorder!

CheckYourself!
IsthelearningoutcomeSMART? Specific? Measurable? Aggressivebutattainable? Resultsoriented? Timebound? Doesthelearningoutcomefollowthe3 Ms? Manageable? Measurable? Meaningful? Answerkey:
TheresponsestoSMART,the3MsandQ1andQ2 shouldbeyes,allotherresponsesshouldbeno. Responseswhichdeviatefromtheseindicatethatitis probablynecessarytoeditthelearningoutcome. 1. Doyouhavemorethanonelearningout

comeperstatement?(Hint:ifyouhavethe wordand,youmight!) 2. Doyouuseverbslikelearn, appreciate,valueordevelop? 3. Doesitfocusonoutcomes,notprocesses? 4. Isthebehaviorthelearningsomething theytheaudiencewasunabletodobefore exposuretoaprogram? 5. Areyourlearningoutcomeswordyand complex?(Hint:ifyouhavetoreadthe learningoutcomemorethanoncetoun derstandit,itprobablyis!)

27

Appendix 4: SLO Verbs by Blooms Taxonomy Level

RelatedBehaviorsandVerbs Arrange,cite,count,define,describe,draw, identify,label,list,match,memorize,name. outline,point(to),quote,recall,recite,rec ognize,record,relate,repeatreproduce, select,state,tabulate,tell,trace,underline write Understandingsome Accountfor,annotate,calculate,change, thingthathasbeencom classify,compute,convert,defend,distin guish,discuss,estimate,explain,group,gen municatedwithoutnec essarilyrelatingittoany eralize,giveexamples,infer,interpret,para thingelse phrase,predict,reiterate,reword,rewrite, review,summarize,translate Usingageneralconcept Add,apply,adopt,calculate,change,clas tosolveproblemsina sify,collect,complete,compute,conduct, particularsituation;using construct,demonstrate,discover,divide, learnedmaterialinnew examine,execute,extend,illustrate,imple andconcretesituations ment,interview,graph,makeuseof,ma nipulate,modify,operate,perform,pre pare,produce,react,relate,roleplay, show,solve,translate,use breakingsomething analyze,arrange,breakdown,catalogue, downintoitsparts;may classify,compare,contrast,diagram,experi focusonidentificationof ment,differentiate,discriminate,dissect, partsoranalysisofrela distinguish,identify,illustrate,infer,meas tionshipsbetweenparts, ure,outline,pointout,relate,select,sepa orrecognitionoforgan rate,sort,subdivide,test izationalprinciples judgingthevalueofma accept,appraise,assess,arbitrate,argue, terialormethodsasthey award,choose,compare,conclude,con mightbeappliedina trast,criticize,defend,describe,discrimi particularsituation;judg nate,evaluate,explain,grade,interpret, ingwiththeuseofdefi judge,justify,order,presentacasefor,pri nitecriteria oritize,rank,rate,recommend,referee,re ject,review,select,support,test creatingsomethingnew Assemble,blend,build,change,categorize, byputtingpartsofdiffer combine,compile,compose,conceive,cre entideastogetherto ate,design,develop,devise,drive,establish, makeawhole. formulate,generate,group,hypothesize, integrate,modify,order,organize,plan,pre dict,prescribe,produce,propose,rear range,reconstruct,reorder,revise,specify, tell,transform.write Definition Recallingorremember ingsomethingwithout necessarilyunderstand ing,usingorchangingit

Category Remembering

Understanding

Applying

Analyzing

Evaluating

Creating/ Generating

Jennifer Lackie Adapted from: http://www.k-state.edu/assessment/slo/blooms.htm, University of Georgia

28

Appendix 5: Blooms Taxonomy

Evaluation Appraise Assess Choose Compare Estimate Evaluate Measure Score Justify Select Predict

Synthesis Arrange Assemble Collect Compose Create Design Formulate Organize Plan Propose

Application Demonstrate Employ Illustrate Schedule Sketch Use Apply Interpret

Comprehension Describe Discuss Explain Locate Recognize Report Translate Classify Indicate Review

Analysis Compare Appraise Calculate Debate Diagram Differentiate Relate Solve Test Classify Experiment Criticize Contrast

Knowledge Articulate Define List Label Locate Identify State Reproduce Arrange Order Recall Relate

29

Appendix 6: Formative Assessment

Althoughassessmenthastraditionallybeenconductedtowardstheendofaprogram,assess mentcanbeusedatothertimesaswell.Formativeassessment,orassessmentalongtheway, canhelpyoubetterunderstandwhetheryourgoalsarebeingmetasyouprogressthrougha programortheyear.Thistoolwillhelpyouidentifysuccessesandchallengesandallowyouto makeadjustmentsasneeded.Ifstudentsarenotunderstandingaconcept,itisbettertoreal izethisduringtheyearratherthanwishingthatyouhadknownthiswasaproblemearlier. Thisguideidentifiesseveralformativeassessmenttechniques,whichvaryinlength,amountof preprequired,andtheamountofinformationyoucangleanfromtheresults.Notallofthe techniqueswillbesuitableforeveryprogram,buteachhasitsstrengthsandcanbeusedin studentaffairsprogrammingindifferentways.Afterpickingatechnique,itmaybemodified tobestmeetyourneeds. Assessmenttechniquesshouldbechosenbaseduponthespecificcontentorlearningwhichis tobemeasured.Indecidingwhethertouseaparticulartechnique,considerwhichhasthepo tentialtogiveyouusefulinformation.Howdoyouthinkthefeedbackyouwillreceivewillbe helpfultoyou?Ingeneral,donotaskformoreinformationthatyouneedorinformationyou arenotwillingtouse.Althoughformativeassessmenttechniquesaregenerallyquick,itisal waysappropriatetoaskacolleaguetoreviewthetechniqueinadvancetoensureitwillcollect thedesiredinformation. Afteryoureviewthefeedbackyoureceivefromtheseassessments,itisimportanttoshare theresultswithyourstudentsandidentifyhowyouplantoutilizethoseresults. Pleasenotethatforthesakeofefficiencyandspace,eachoftheseinstructionshavebeen writtenusingthetermprogram.However,theyalsoapplytoworkshops,lectureseries,for mativediscussions,interventions,classes,etc.

30

FormativeAssessmentTechniques
(byskillareaassessed) PriorKnowledge,RecallandUnderstanding BackgroundProbe MisconceptionPreconceptionCheck OneminutePaper MuddiestPoint SynthesisandCreativeThinking OneSentenceSummary ProblemSolving Problem/IssueRecognition WhatsthePrinciple? DocumentedProblemSolutions ApplicationandPerformance ApplicationCards DirectedParaphrasing StudentGeneratedTestQuestions StudentAwarenessoftheirAttitudesandValues SecretPolling EverydayEthicalDilemmas LearnerReactionstoTeaching Chainnotes

TheTechniques
(inalphabeticalorder) Applicationcards Purpose:Toverifythatstudentsunderstoodthematerialandcanapplyittoareallifesitua tion. Todo:Afterteachingaboutanimportanttheory,principle,orprocedure,askstudentstowrite downatleastonerealworldapplicationforwhattheyhavejustlearnedtodeterminehow welltheycantransfertheirlearning.Quicklyreadthroughtheapplicationsandcategorize them.Considerpickingoutabroadrangeofexamplesandpresentthemtothegrouptoen couragepeertopeerlearning. BackgroundProbe Purpose:Tocollectspecificandusefulfeedbackonstudents'priorlearningonatopic.Canalso beusedtogetabetterunderstandingofwhystudentsareparticipatinginaparticularpro gram. Todo:Inordertogetageneralideaofwherestudentsarecomingfrom,havethemlistclasses theyhavetakenonaspecifictopic,trainingstheyhavebeento,and/orexperiencestheyhave haddoingX,YorZ.Thisisaneffectivewayofquicklygettingapulseonwherepeoplearein theirexperiencessothatyoucantailormaterialappropriately. ChainNotes Purpose:Togetonthespotfeedbackwhileprogramisinprogress.Usefulinlargegroups whenyouwanttogetaquickreadonwhatpeoplearethinkinginthemiddleoftheprogram. Todo:Atsomepointaftertheprogramhasstarted,passaroundanenvelopeonwhichyou (administrator/staffmember)havewrittenonequestionabouttheprogram.Thestudents haveallbeengivenindexcardsbeforehand.Whentheenvelopereachesastudent,heorshe spendslessthanaminutewritingaresponsetothequestion,thendropsthecardintheenve lopeandpassesiton.Thisassessmenttechniqueresultsinarich,compositerecordofeach individualstudentsreactionstotheprograminaction.

31

Directedparaphrasing Purpose:Toensurestudentsunderstoodaconcept,process,idea,etc. Todo:Askstudentstowritealaymans"translation"ofsomethingtheyhavejustlearned gearedtoaspecifiedindividualoraudiencetoassesstheirabilitytocomprehendandtrans ferconcepts.Categorizestudentresponsesaccordingtocharacteristicsyoufeelareimportant. Analyzetheresponsesbothwithinandacrosscategories,notingwaysyoucouldaddressstu dentneeds. DocumentedProblemSolutions Purpose:Tounderstandwhatthestudentwasthinkingandwhatstepsheorshetooktosolve aproblem.Thisisalsousefulinhelpingstudentsclarifytheirthinkingandbecomemoredelib erateintheirproblemsolving. Todo:Developclearcutinstructionsfortheclassandrecognizethatthestudentswillneed moretimeforthistaskthanwasrequiredbyyourselforyourcolleagues.Preparethestudents forthisactivityandassurethemthatthisisnotagradedexercise.Emphasizethatyouareas interestedinhowtheyapproachaproblemasinobtainingacorrectanswerandthatthisac tivitywillhelpthem(andyou)analyzetheirproblemsolvingskills. MisconceptionPreconceptionCheck Purpose:Toassessstudentsmisconceptionsorpreconceptionsonaspecifictopicatthebegin ningofaprogam.Itcanalsobeusedattheendofaprogramtoseeifmisconceptionsare clearinguporgrowing,whetherpreconceptionsarebeingreshapedandtoseeifstudentsare improvingintheirabilitiestoassessrelevantinformationandfilteroutinaccuracies. Todo:Askstudentsaquestion(orafewquestions)beforeandafteraparticularprogram.This canbedoneinpopcornstyle,aformal(ungraded)quiz,groupwork,etc. MuddiestPoint Purpose:Tounderstandwhatstudentsfoundmostconfusing.Thisexerciseasksstudentsto reflectupontheprogram,todecidewhatwasunclear,andtoselfassesstheirlearning. Todo:Atendofprogram,askstudentstoidentifyand/ordescribethemuddiestpointofa workshop(orsegmentoftheworkshop)youareinterestedin.Passoutslipsofpaperorindex cardsforstudentstowriteon.Collecttheresponsesasorbeforestudentsleave.Respondto thestudents'feedbackduringthenextclassmeetingorassoonaspossibleafterward.Youcan specificallyaskstudents"Whatpointremainsleastcleartoyou?"or"Whatimportantques tiondoyouhavethatremainsunanswered?" Oneminutepaper Purpose:Togetinformationfromstudentsattheconclusionofaprogram.Dependingonthe question(s)youask,thisrequiresstudentstorecallandunderstandthematerial,evaluate themselvesonwhattheyrecallandselfassesstheirlearningandunderstanding. Todo:Towardstheendofaworkshop/program/class,askstudentstoansweronahalfsheet ofpaperaquickquestionthatwillhelpyougaugehowmuchandwhatthestudentslearned. Examplesinclude:"Whatisthemostimportantpointyoulearnedtoday?",Whatwasmost helpfulinyourlearningaboutX?orWhatareyoumostand/orleastsatisfiedwith?.

32

Onesentencesummary Purpose:Toseethemainideasthatstudentstookfromtheprogram. Todo:Askstudentstosummarizethemainideasthey'vetakenawayfromalecture, discussion,orassignedreading.Studentssummarizeknowledgeofatopicbycon structingasinglesentencethatanswersthequestions"Whodoeswhattowhom, when,where,how,andwhy?"Thepurposeistorequirestudentstoselectonlythede finingfeaturesofanidea.Evaluatethequalityofeachsummaryquicklyandholistically. Notewhetherstudentshaveidentifiedtheessentialconceptsoftheclasstopicand theirinterrelationships.Shareyourobservationswithyourstudents. PrePostTest Purpose:Toassessstudentlearningbeforeandafteraprograminordertodemon strateprogression. Todo:Havestudentscompleteaminitestordoaparticularexercisebeforeaprogram begins.Havestudentsrepeatthetaskattheendoftheprogram.Analyzethepretest andposttesttobetterunderstandthegrowthinlearningthestudentsexperiencedas aresultoftheprogram. Problem/IssueRecognition Purpose:Toassesswhetherstudentsareabletoidentifytheprobleminasituation. Todo:Presentacommonproblem(orseveralexamplesofdifferentproblems)tostu dentsandaskthemtoidentifythebasictypeofproblemrepresentedbyeachsample. Theycandosobyashowofhands,groupworkandchoosingthebestanswer,writing themdown,etc.Thisisparticularlyhelpfulwhenteachingabouthelpingskills. SecretPolling Purpose:Topollstudentswhoseemhesitanttorespondtoquestionspubliclyorwhen youwantresponsesthatarenotswayedbytheinfluenceofotherstudents Todo:Askamultiplechoicequestionandassignanumber(15)toeachresponse. Havestudentsholdupthenumberoffingersthatcorrelateswiththeirresponseto theirchestsothatonlythepresentercanseetheirresponse.Samplequestionsinclude IfXhappened,whatwouldofficeproceduresrequirethatyoudo?Putonefingerupif youthinkY,2fingersifyouthinkZ.Ormoreserioustopicslikehowmanyofyou haveexperiencedgenderdiscriminationwithinthepastyear?1ifyouhave,2ifyou havenot SelfConfidenceWorksheet Purpose:Togaugeastudentscomfortlevelonaparticulartopic. Todo:Thiscanbeadministeredeitheratthebeginningofaparticularsessionor throughoutthesession(s)tomeasureprogress.Inadvance,designasurveywherestu dentscanratehowcomfortabletheyfeelinanyparticularareas(forexample,public speakingskills,challengingafriendonhis/herinappropriateuseofalcohol,bookinga roomthroughEBMS,etc).Studentscanthenratetheirconfidenceonascaleofnone,

33

low,medium,orhigh.Youcanthenreviewtheselfevaluationsandtailorstrategies accordingly. StudentGeneratedHandbook Purpose:Toseeifstudentsunderstoodaconceptwellenoughtoteachittoothers. Todo:Havestudentswriteahandbookormanual,ofanylength3stepsto30pages regardingacertaintaskthatisgearedtowardsotherstudentswhowouldbelearning thematerial.Thismaybeparticularlyhelpfulwhentrainingstudentworkersandhav ingthemwritedowntheinstructionsforwelcomingindividualswhoentertheoffice orrespondingtofrontdeskinquiries.Reviewthehandbooks/guidesforconcepts thatthestudentsgraspedormayhavemissed.Givegroupfeedbacktostudentsand reviewanytopicsorareaswhichprovedtobeparticularlychallenging. StudentGeneratedTestQuestions Purpose:Toseeifstudentsunderstoodmaterialenoughtobeabletoworkwithit,ap plyit,etc. Todo:Allowstudentstowritetestquestionsandmodelanswersforspecifiedtopics,in aformatconsistentwithatypicalcoursesexams.Thiswillgivestudentstheopportu nitytoevaluatetheworkshop,activity,orprogramtopics,reflectonwhattheyunder stand,andwhataregoodtestitems.Makearoughtallyofthequestionsyourstudents proposeandthetopicsthattheycover.Evaluatethequestionsandusethegoodones aspromptsfordiscussion. WhatsthePrinciple? Purpose:Toassesswhetherstudentscanidentifytheprinciple(s)onewoulduseto solveaproblem. Todo:ThistechniquefollowstheProblemRecognitiontechniqueinthatonceatypeof problemiscorrectlyidentified,studentsmustidentifywhatprinciples,techniques,or standardsintheclassmustbeappliedtosolveorresolvetheproblem.

34

35

Appendix 7: Pre-Assessment Conversation Guide

Thisconversationguidecanbeastartingpointforassessingprogramsandservicesin yourdepartment.Itmaybehelpfultotalkthroughthesequestionsinastaffmeeting, withyoursupervisor,orwithcolleagueortwoasyoubegintoplanforassessingyour program. GUIDINGQUESTIONS Whatistheprogramorserviceyouwanttoassess? Whatisitthatyouwanttoknowaboutthisprogram?Whatisthespecificissueor potentialproblem? Howisthisprogramalignedwithyourdepartmentsgoals? Whatdoyouwantstudentstolearn,know,orbeabletodoafterparticipatinginthis programorinitiative?Thesewillbecomethebasisforyourlearningoutcomes,for whichyouwillmakelearningoutcomestatements. Doyouhaveanyoperationaloutcomesinmind,likeattendanceexpectations,reduc tioninX,etc.? Whoneedstobeinvolved,orknowabout,thisassessment?Howmightyouinclude themintheplanning? Whattechniqueswillyouusetoconductyourassessment?Doyouneedfurther trainingorinformationinordertodothis?Consulttheworkshopscheduleoronline webinarscheduleonStudentVoice(www.studentvoice.com) Whendoyouwanttoconductyourassessment?Checkwithyourdirectortosee whatelsemightbegoingoninthedepartmentoracrossthedivisionwhichmight influenceyourtiming.

36

Appendix 8
DIVISIONOFSTUDENTAFFAIRS

STUDENTVOICE ASSESSMENTPROJECTREQUEST FORM

TitleofProject:

NameofProjectsponsor: Department:

Todaysdate: Proposeddateofassessment launch: Pleasedescribewhyyouwanttoundertakethisassessmentbyrespondingtothefollowingitems:

1.Whatisthepurposeofyourassessment? 2.Whatistheprogramorservicebeingassessed? 3.Whatarethespecificdepartmentorprogramgoalstowhichthisisrelated? 4.UnderwhichoftheDivisionalLearningDomain(s)doesthisassessmentfall?(Intellectual, ReligiousandSpiritual,Ethical,EmotionalSocial,orInterculturalandGlobal)Note:notallprojects willfallunderaspecificlearningdomain,particularlyifyouareassessinganoperationaloutcome. 5.Whatexistingdatahavebeenconsidered(e.g.,datafromInstitutionalResearch,Student Services,otherStudentAffairsdepartments)? 6.Whatisyourtargetsample? 7.Whatassessmenttechniqueswillyouuse?(e.g.survey,focusgroup,rubric,interview,mobile datacollection) 8Willtheassessmenttoolidentifyrespondentsorwillrespondentsremainanonymous? 9Howwillyouusethecollecteddata?

10.Withwhomandhowdoyouplantosharethecollecteddata?
11.Isthisaonetimeassessmentorwillitberepeated?Ifrepeated,pleaseindicatetheproposed schedule.

37

INSTRUCTIONSFORCOMPLETINGANDUPLOADINGPROJECTREQUESTFORM Theprojectrequestformismeanttobeastartingpointforyourproject.Itisimportantto bethoroughandthoughtfulincompletingalloftheitems,yetyoushouldalsobebrief.It shouldtakelessthan30minutestocompletetherequestform.Thisformwillbeusedin yourinitialconsultationwiththeStudentVoiceconsultantandwillbereviewedandap provedbyKatieODairandJessicaGreenefromInstitutionalResearch. 1. Filloutallitemsontheformandsaveittoyourdesktoportotheserver.Youmaycon sultwithyourassessmentcommitteeliaisonorKatieODairatanytimeforassistance. LogintoStudentVoice:www.studentvoice.com. 1. GotothemyStudentVoicetab 2. Ontheleftside,clickrequestaproject 3. Filloutallrelevantprojectinformation.Foranewproject,clicknewproject 4. IfyouareusingaPDA,checkallandlistthemobiledevicenumbers(Katiewillhave thesewhenyoucheckit/themout)inthecommentbox.ThiswilltellStudentVoicetosinc theprojecttothePDA. 5. Underemailinformationclickunsureifyouwouldliketotalkaboutadministeringthe assessmentwiththeSVconsultant. 6. Underadditionalassistance,clickyesifyouwanthelpfromaconsultantonthedesign, phrasing,andadministrationoftheassessment. 7. ClickNext 8. UnderprojectnoteswriteinanythingyoudlikeyourSVconsultantorKatieorJessto knowaboutspecifictothisproject.Clicksubmit. 9. Youwillbetakentoyourprojectdashboard.UnderProjectfilesuploadyourProject RequestFormbybrowsingyourfilesanduploadingit.Ifyouhaveadraftsurveyorques tionstosubmit,makesureyouuploadthoseaswell.Makesureyoupressuploadafter youbrowse!Youwillknowitwassuccessfulwhenyouseeyouruploadeddocumentsap pear. 10. Oncecompleted,KatieODairandJessicaGreenewillreviewandapproveeachrequest andyourStudentVoiceconsultantwillcontactyou.Thiswillbedonewithinoneweekof uploadingyourform. 11. Pleasenote,someprojectsmayrequiretheadditionalstepofpreparingdocumentsfor theInstitutionalReviewBoardprocess.KatieandJessicawillassistwiththisstepifrequired. 12. TheStudentVoiceconsultantwillthenworkwithyoutocompleteyourassessment

38