Nereh 2002 | Votre 89 | Nanbee 8 Redesigning Professional Orvelopment Pages 45-51 Does It Make a Difference? Evaluating Professional Development Thomas R. Guskey Using five critical levels of evaluation, you can improve your ‘schoo!'s professional development program. But be sure to start with the desired result improved student outcomes. Educators hav lng considered professional developmentio be their ght ‘something hey desenw as decicald and hardworking indiwduals. But legisators and policymakers nave recentybegun © question fat ight As education budgets grow toh, thylook atwhal schools spend on professionel davelopmontand wantto know, Does the investment yolé tangible payofs or could tat money'be spentin beter way? Such questions make aflecive evaluation of professional cewiopment programs more important than ever. ‘Teatonally educators haven paid much atenton oerluating ei professional development efrs "Many consider evaluation a cosy ime-consuming process tat ders tlenton rom more imporant lactvtes such as planing, implomentaton, and follow-up. Others fel theylack the sil and eperise to become moive in igoreus evaluations as a result, they ether neglect eveluaton issues completsiyor leave them 0 -ovalvain experts" {Good evaluations dorthave tobe complicated. They simply require though planning, te abiyo ask ‘900d questons, and ¢ basic understancing ofhow ind valid answers. What's more, teycan prove ‘meaninglul information that you can use to make thought, responsible decisions about professional ‘development processes and ofc What Is Evaluation? In simplesttoms evaluation sre systematic inves tgation of merit or wort (Jint Commitee on ‘Standards for Educational Esaluaon, 1964 . 2). Systomaticimplos a focused, thoughtful, and Intentional process. We conduct evaluations fr cea reasons and wih exctntentInvesigation refers to he colacion and anal of perSrentiniormaton rough appropsate methods and eehniques. Ment ‘rwodh denoles appraise! and judgment We use evaluations to detemine he valve of something © help anewer euch questions as, le thi program or acivtyachiewng is itanded resus? Is tbete han ‘what was done inthe pas? I itbetr tan another, competing ach? Isitworh the costs? ‘Some educators undersians the Impanance of evaluation for evert-dhven professional development ‘actives, such as workshops and seminars, but orgtthe wide enge afless formal, ongoing job+ ‘embedded professional development acties study groups, acon research, collaborate planning, ‘curiulum development, structured bsenations, per coaching, mentoring, and so. Butregardless of storm, professional development should te a purpesetu endeavor. Trough evaluton, you can ‘dolerine whethor hese actus are achieving thei purposes, Critical Levels of Professional Development Evaluation Efeciv professional development valuations require te collacion and analysis of the five oral evs ‘of information shown In Figute 1 (Guskey, 20008), Win each suezaeding lve, he process ofgahering ‘evaluation information gets abitmore complax And because each lavel builds on hase that come betore, success atone lve is usualynecessary or success at higher eels Fur 1. Fv Levek of Professonal evelopment Evaluation ‘valuation | What How Wit Whatis Measured or | How Wit Love! ‘Questions Are | Information Be | Assessed” Information BeAdsrosses? | Gathored? Uso? 1 Did heyikeit? [Questionnaires [tical satstacion win | Toimprove Participants | was yeicime | 20ministeed at | re experience program desion| Reactions | ver ene [he end ofthe and delivery ‘seesion Did ihe material make sense? a itbe seul? Was te leacer knowledgeable and help? Were he rotreshments fresh ana tasty? Was he room ‘height tomporatre? Were the chairs comfortable? 2 Did partcpants | Paperand- [Now knowledge and | Toimprove Participants | accuirote | porci shilsofpartcipants | program Learning |iniondod instruments conto format, krowiedge and | atone and hla? Simla corganizason Demonstrations: Paricpant refectons (ora! ance writen) Paricipant portoios 3 Was Distictand — [Teorganizatons | To dooument Organization | implementation | schootrecords | advocacy suppor, | and improve support & faciocatd, — Twi rom eecommadation, | organizaton range |racitaied and | rtren facilitator, and support supperas? | ee recognition pte Was be a change forts suppor pubic fend over? | Structures stom {erews with were | parkcipants and bs aad is trictor school us séminiswators eficenty? Partepant Wore suicent | Pomercogeind and sharaa? atoas Impacton ne orgrizaion? Diditaoctne organizations lima and procedures? a. id partdpants [Questionnaires | Degre and quatiyet | To doouner Participants |ofecivelyapoy | csacaag fimlementaton | andimprov the Useoftew [renew |Sucued implementation Kowiedge | knowiesge ana | eriows otprogram sedate |oxea | tora ana cont superisors Paripant ratectons (rl andl witon) Parspant portios Diet obsenatons Video or ausio tapes S-Student | Whatwas he | Studoniocorts [Sudontieaming | Tofoase ana Learning |impacton ., [oweomes prove outcomes |tidenia? | S001 Toco aspocs of ‘cuvstonaies | * Cogntive Dis itaec (Performance | Pare” student | Stuctros 8 design perormance or | itrvewrs wih ‘Achievement | implementation, fscvevement | students, + aectve | niblowse parent, (Atttudes a | Todomonstaie Disitintuence fee rs.andor | pispostions) | fe overall scminseatrs impactot Physical or + Peychomator | mmectot ‘emotional welt | Paricipant (Sielis & poteodealh being? portoios Behaviors) | dewlonmen bro secents more confdent asleamer? Ws stent stendance Improving? re dropoutsecroasing? Level 1: Partcpants Reactons The frst lel of evaluation looks at paricpants'reactons tothe professional development experiance, Tis isthe most common for of professional development evaluations, and the easiestIype of Information to gather an analy ‘A Level 1, ou address questions focusing on whether or na parisipants ked ihe experience. Did they feel ther ime was wall spent? Did te material make sansa them? Were he actives well planned ‘and meaninglul? Was the leader knowledgeable and helpful? Did te parscipans nd te information usotut? Important questions fr professional development workshops and éominar also inctude, Was the cofae hotand ready on me? Was the room atthe ight temperature? Were the cel comfonabie? To some, ‘questions such as hese may seem sil and inconsequential. But experianced protssionel developers know tne importance of atonding to these basic human needs. Informaton on pariipants reactions ls generally gathered tough questonnalres handed outat the nd ‘ofasession or actly, Thase questonnalres ypcaly include a combination ofretng-scale ems and ‘open-ended response questions tha allow paripan i make personal comments. Because of he ‘general nature of his information, many orgarizaions use ie same questionnaire oral heir professional eselopmentactttes ‘Some educators refer to these measures of partcipans'reactons as happiness quotens sisting that hey reveal ony he entrainment value ofan acy, nots quali wors ut measuring Participant’ infal sastacion wit the experience can help you improve the design and dalveryof programs or actives invalid ways. Level 2: Partkpants Learning In aditon king their professional development experience, we alse hope tat participants arn someting tom I Level 2 focuses on measuring he knowledge and skis hat perepants gained Depending on tie goals of he program orci. is can involve anying fom a pencl-and-paper ‘assessment (Can paricipants describe the erica tributes of mastaryleeming and ghe examples of how these mightbe applied in ypcal tassroom sitaions?) oa simulaon or ful-scale kil ‘demonsvation (Presented wih variety of dlasstoam confels, can partepants diagnose each situation land ten prasrive and car out fair and workable Solon?) You can also use oral personal refectons or portoios tat paricpants assemble to document theirlearing. ‘though you can usualy gather Level 2 evaluation informason atthe completion ofa professional ‘dovelopment acy. requires more than a standardized form. Measures must show atainmontof speci earning goals. This means tetinaieators of suecesstul leaning need to be outined Betore scttes begin, You can use this information as a basa forimprowng he conten, format. anc ‘organization ofthe rogram or actives. Level: Organization Support and Change ‘A Level 3, the focus shits othe organization, Lack f organizaion suppor and change can sabotage any professional development fot, even when all the incu aspects of professional development are ‘done ght ‘Suppose, for oample, het several secondary school educators parcipais ina protssional “development program on cooperate leaning, Theygan a thorough understanding ofthe theory and ‘develop a vretyo assroom aces based on cooperative learning principles. Following hel taining, theytrtoimplement hese actties in schools where studens are graded-on the cue" according to theirelatve standing among dassmates and great imporanceisstached to selecing the cass valedictorian. Organizaton poles and practices such as these mako learning highly competive and will thwart most valiant ers to have students cooperate and help one another leam (Guskey, 20006). The lack of posite result inthis case doesn retect por inning or nadequate eating, butrather