Evaluation Report on School-Wide Positive Behavior Intervention and Supports (SWPBS)

EdTech 505 Final Evaluation Project & Report

Vertoria Lagroon EdTech 505-4172 December 8, 2010

TABLE OF CONTENTS
Learning Reflection ..................................................................................................................................................4 Executive Summary .................................................................................................................................................5 Purposes of the Evaluation ...................................................................................................................................6 Evaluation Purpose .............................................................................................................................................6 Central Questions.................................................................................................................................................6 Impact of the Results ..........................................................................................................................................7 Background Information .......................................................................................................................................8 Program Origin .....................................................................................................................................................8 Program Goals and Standards ........................................................................................................................8 Prior Programs .....................................................................................................................................................9 Program Stakeholders .................................................................................................................................... 10 Program Characteristics ................................................................................................................................ 11 Description of Evaluation Design .................................................................................................................... 14 Results ........................................................................................................................................................................ 17 Office Discipline Referrals by Quarter ..................................................................................................... 17 Elementary School ....................................................................................................................................... 17 Middle School ................................................................................................................................................ 18 High School ..................................................................................................................................................... 19 Relationship Between Rewards and Referrals ..................................................................................... 20 SWPBS Team Survey Results ....................................................................................................................... 21 SWPBS Stakeholder Attitudes ..................................................................................................................... 22 Discussion of the Results .................................................................................................................................... 23 Frequency of Desired and Undesired Behaviors ................................................................................. 23 SWPBS Implementation across the District........................................................................................... 23 Correlation between Positive Reinforcements and Office Discipline Referrals ..................... 24 Attitudes of SWPBS Stakeholders in McCormick County School District .................................. 24 Impact of SWPBS on Stakeholders ............................................................................................................ 24 Overall Effectiveness of SWPBS .................................................................................................................. 24 Conclusions & Recommendations .................................................................................................................. 25 Immediate Results ............................................................................................................................................ 25 Long-Range Planning ...................................................................................................................................... 25 Evaluation Insights.......................................................................................................................................... 25 References ................................................................................................................................................................ 27 2

Appendices ............................................................................................................................................................... 28 Appendix A: Student Survey......................................................................................................................... 28 Appendix B: SWPBS Team Member Survey ........................................................................................... 30 Appendix C: PBIS School Faculty, Staff, and Administration Survey ........................................... 34

LEARNING REFLECTION
As I reflect on the learning that has taken place over the course of this entire semester and while completing this intricate evaluation project, one word comes to mind. That word is: Wow! I am literally amazed at the amount of knowledge, skill, and experience I have attained as a result of engaging in various evaluation readings, assignments, and projects. I was made aware of how significant and crucial evaluation is to the success of any project, program, or product. After discovering that evaluation was more than a tier in Blooms Taxonomy or the nerve-racking experience of a principal sitting in on your class I learned several reasons for evaluation. Evaluation can be used for a variety of reasons, such as quality comparison, innovation, and improvement. Comparative evaluations are used to determine the best or most effective program from several options. Evaluations are also used when implementing a new program. When a program needs to be improved or altered, evaluations are done as well. My final evaluation project had the purpose of evaluating a newly implemented program. Specifically during the project, I learned some of the more difficult lessons about evaluations. Evaluation is not a small, meaningless process. Evaluation requires much thought and effort. I plan to use all that I have learned about evaluation in both my career as an educator and as a technology enthusiast. In the field of education and technology, the best, most innovative method, program, or machine is announced almost monthly. With many fads to follow, being a good evaluator is critical in order to prevent wasted time and effort on fruitless trends. The skills that I have acquired this semester will be beneficial t o me when determining which strategies and instructional practices to employ. As a good evaluator, I can determine the effectiveness and impact of new educational technologies on student learning. After performing mini evaluations, I will be able to determine and use only the best in my classroom for my students.

EXECUTIVE SUMMARY
The evaluation reported in this document sought to evaluate a newly implemented program at McCormick County School District. The evaluated program was school wide Positive Behavior Intervention and Supports (SWPBS). SWPBS was established to teach behavioral expectations and reward students for following them, rather than waiting for misbehavior to occur before responding. The purpose of school-wide PBIS is to establish a climate in which appropriate behavior is the norm (PBIS.Org, 2010). The purpose of the evaluation was to ultimately determine if SWPBS is successfully achieving published outcomes and objectives. SWPBS advertises the following outcomes (PBIS.Org, 2010): Decrease in office discipline referrals Increase in instructional time Decrease in administrator time spent on discipline issues Efficient and effective use of scare resources Increase in perceived school safety Sustainability through team approach The evaluation was performed using a goals-based model, of which the above indicated objectives were evaluated on achievement and rate of success, effectiveness. Data was collected from each of the three schools across the district to determine if grade-level of students is a factor in SWPBS success. The conclusion of the evaluation resulted in the inability to determine the effectiveness of SWPBS in McCormick County School District.

PURPOSES OF THE EVALUATION

EVALUATION PURPOSE The purpose of this evaluation was to provide an in depth look at the overall effectiveness of the School-Wide Positive Behavior Intervention and Supports program (SWPBS) currently being implemented in McCormick County School District. The evaluation goal was to outline the possible impact the program may have on all stakeholders, which includes but is not limited to: school administration, all faculty and staff, and community members. Student input and attitudes were collected as valuable information as well. Data was collected and analyzed in order to determine whether and to what degree the objectives of SWPBS are being achieved. The evaluation accomplished the first three levels of evaluation defined by Kirkpatrick (1994). The evaluation sought out to discover the impressions of the program by the participants, the effectiveness of the program, and the impact the program had on the participants. Ultimately, the purpose of the evaluation is to determine if the program, SWPBS, is successfully reducing undesired behaviors. CENTRAL QUESTIONS The evaluation sought to answer several key or central questions. Is negative, undesired behavior decreasing? Is positive, desired behavior increasing? The two questions above are important because the answers determine whether or not SWPBS have successfully met their objective of making negative behavior less frequent. Is SWPBS being implemented identically across the district? This question was asked to ensure that SWPBS was implemented as directed. Without identical implementation methods, varying results would be expected across the district.

Is there a correlation between the number of positive reinforcements given by a faculty or staff member and the number of discipline referrals that faculty member completes? This question seeks to determine if SWPBS is responsible for the outcome of less frequent undesired behavior versus other conditions. A negative correlation would mean that as the number of positive reinforcements increased, the amount of discipline referrals decreased. It could then be implied, that SWPBS was successful. What are the attitudes held by the stakeholders about SWPBS? How are stakeholders impacted by SWPBS? The above questions are for gathering information and were answered through surveys. The answers to these questions also help to determine if personal opinions and attitudes towards SWPBS affect its success, either positively or negatively. How effective is SWPBS? The question goes beyond SWPBS successfully accomplishing objectives and goes into the rate in which the objectives are achieved. This question was asked to address how fast SWPBS can turn around negative student behavior. IMPACT OF THE RESULTS The results of this evaluation would greatly impact school faculty and staff, including principals and assistant principals. All faculty and administration are directly impacted by the results, as certain results can yield buy-in among these individuals. Students are indirectly impacted. The results of this evaluation could prompt new decisions in discipline policy, the way in which SWPBS is implemented, or possibly the continuation of SWPBS among other decisions that could be made at the district level. Such changes trickle down to students and inadvertently to the community. Ultimately, the manner in which students are impacted, determines the effectiveness of SWPBS, as the program seeks to reduce undesired behavior among students. Students are the target population and the over-arching goal is to see long-term and sustained changes in negative behavior.

BACKGROUND INFORMATI ON
PROGRAM ORIGIN SWPBS began in 1998 as a special component of Positive Behavior Intervention and Supports (PBIS). The program came into existence in response to one size fits all approaches to school-wide discipline, which has focused mainly on reacting to specific student misbehavior by implementing punishmentbased strategies including reprimands, loss of privileges, office referrals, suspensions, and expulsions (PBIS.Org, 2010). The rationale for SWPBS is to provide positive behavior support for all students by making targeted, negative behaviors less frequent and desired behavior more functional. Positive behavior support originated from behaviorally-based systems and seeks to enhance and design effective learning environments. Research has shown that inconsistent punishment, without other positive strategies, is ineffective. SWPBS includes proactive strategies to support appropriate student behaviors by teaching positive social behaviors. Introducing, modeling, and reinforcing positive social behavior is an important step of a student's educational experience (PBIS.Org, 2010). SWPBS was established to teach behavioral expectations and reward students for following them, rather than waiting for misbehavior to occur before responding. The purpose of school-wide PBIS is to establish a climate in which appropriate behavior is the norm (PBIS.Org, 2010). PROGRAM GOALS AND STANDARDS SWPBS is a program that provides continuous positive behavior support for all students within a school and is implemented in numerous areas of the school, including the classroom and nonclassroom settings such as hallways, buses, and restrooms. The fundamental idea behind SWPBS is teaching behavioral expectations just as core curriculum subjects are taught.

FIGURE 1- THE FOLLOWING DIAGRAM ILLUSTRATES THE MULTI-LEVEL APPROACH OFFERED TO ALL STUDENTS IN THE SCHOOL. THESE GROUP DEPICTIONS REPRESENT SYSTEMS OF SUPPORT NOT CHILDREN: RETRIEVED FROM HTTP://WWW.PBIS.ORG/SCHOOL/DEFAULT.ASPX

PRIOR PROGRAMS After an extensive internet search, prior or current programs similar to SWPBS were not found. However, there was an early version of SWPBS known as Effective Behavior Support (EBS). EBS sought to address concerns about school climate and problem behavior and officially began in 1994, (Sugai, 2007). EBS was founded under the conceptual foundations of behaviorism and applied behavior analysis (APA). The program was developed by the Institute on Violence and Destructive Behavior at the University of Oregon. EBS was a school wide behavioral support program that was designed to prevent disruptive behavior by all students, including those that exhibit chronic behavior problems (Center for Effective Collaboration and Practice CECP, 2001). The implementation and set-up of EBS is very similar to that of SWPBS. Outcomes of EBS were as follows: Faculty support the EBS program because they feel it empowers them to create a structured environment where learning takes place without the interruptions caused by behavior problems. Teachers are also pleased with the program since it appears to have been
9

successful, even with children with the most problematic behaviors. In most EBS schools, the impact has been extremely significant in changing student behavior. According to recent research, one school that implemented EBS reports a decrease in the number of discipline referrals by an average of 42 percent during the programs first year of implementation, and another school projects a decrease in office referrals from approximately 7,000 to less than 2,000 after four years of implementation (a reduction of about 71 percent) (Center for Effective Collaboration and Practice CECP, 2001). The school district provided character education to all students prior to the implementation of SWPBS. The entire district would focus on a specific character word each month. Instruction would be provided on the be particular characteristic in various ways at the different school levels. A traditional discipline policy was also in place. The discipline policy provided consistent punishments for undesired behaviors district wide. PROGRAM STAKEHOLDERS The design, development, and implementation of the general SWPBS program included the involvement of eight universities and four educational agencies. Stakeholders include: University of Oregon University of Connecticut University of South Florida University of Missouri University of Kansas University of Florida University of North Carolina University of Kentucky Illinois State Board of Education Mountain Plains Regional Resource Center Sheppard-Pratt Health Systems The May Institute US Department of Education, Office of Special Education: TA Center on PBIS

The stakeholders in regard to the implementation of SWPBS in McCormick County School District include district administration, school administration, all school faculty and staff, and involved community agencies and members. School faculty and staff include all school employees and volunteers, ranging from teachers to custodians and bus drivers. Guidance counselors, secretaries, and receptionists are included as well. Students are also a vital component of the program and their feedback and attitudes will be collected and analyzed.
10

PROGRAM CHARACTERISTICS There are standard procedures to follow when implementing SWPBS as outlined by the program creators. The order of tasks below is presented exactly as SWPBS suggests. Typically, a team of approximately ten representative members of the school will attend a two or three day training provided by skilled trainers. This team will be comprised of administrators, classified, and regular and special education teachers. The school will focus on three to five behavioral expectations that are positively stated and easy to remember. In other words, rather than telling students what not to do, the school will focus on the preferred behaviors. After the SWPBS team determines the 3-5 behavioral expectations that suit the needs of their school, they will take this information back to the staff to ensure at least 80% of the staff buy into the chosen expectations. Consistency from class to class and adult to adult is very important for successful implementation of SWPBS. The team will then create a matrix of what the behavioral expectations look like, sound like, and feel like in all the non-classroom areas. This matrix will have approximately three positively stated examples for each area. See Figure 2.

FIGURE 2 SAMPLE MATRIX OF POSITIVE BEHAVIORAL EXPECTATIONS. RETRIEVED FROM HTTP://WWW.PBIS.ORG/SCHOOL/SWPBS_FOR_BEGINNERS.ASPX

The SWPBS team would take the matrix back to the whole staff to ensure 80% buy-in from the entire staff on what expectations are taught in each area. Another primary activity for the SWPBS team is determining how the behavioral expectations and routines will be taught in and around the school. Many schools
11

choose to use several days at the beginning of each year to take the students around the school to stations, where the skills are taught in setting specific locations. For example, a bus may be brought to the school and the children will practice lining up, entering the bus, sitting on the bus, and exiting the bus using hula hoops to denote proper body space distance in lining up to enter the bus. The next activity the SWPBS team will begin is the fine tuning of the office discipline referral form. The team will decide "What behaviors are an instant trip to the office and what behaviors are taken care of in the classroom." It is very important that every staff member is consistent. Another activity for the SWPBS team is to determine a "gotcha" program. The gotchas are a system for labeling appropriate behavior. Students are rewarded gotchas when they are demonstrating behavior outlined in the positive behavior matrix. Examples of gotchas are provided below.

FIGURE 3 EXAMPLE OF A GOTCHA SWPBS REWARD TOKEN. RETRIEVED FROM HTTP://WWW.PBIS.ORG/COMMON/CMS/DOCUMENTS/STUDENT/GOTCHA%20RESOURCES/CHAMP ION%20CHIP%20PBS.PDF

12

FIGURE 4 - EXAMPLE OF A GOTCHA SWPBS REWARD TOKEN. RETRIEVED FROM NORTH POINT H.S. PBIS FOR STUDENTS.PPT

13

DESCRIPTION OF EVALUATION DESIGN

The design and methodology of this evaluation followed a goal-based model. Data was collected and analyzed in a number of ways to determine whether SWPBS successfully accomplished all of its objectives within the McCormick County School district. SWPBS advertises the following outcomes (PBIS.Org, 2010): Decrease in office discipline referrals Increase in instructional time Decrease in administrator time spent on discipline issues Efficient and effective use of scare resources Increase in perceived school safety Sustainability through team approach All of the outcomes were tested by collecting both quantitative and qualitative data. The outcome of a decrease in office discipline referrals was tested by acquiring records of the number of office discipline referrals that have been completed per teacher for the high, middle, and elementary school. The number of rewards that were issued per teacher was also collected. The data was charted to determine a correlation between the number of rewards issued by a teacher and the number of referrals completed by a teacher. The total number of office discipline referrals was compared to the number of discipline referrals completed in past years for each school for specific time periods. This was to judge if the office discipline referrals have decreased in response to the implementation of SWPBS. This data also addresses the issue of effectiveness of the program. From the charts, one can determine how fast change in behavior is occurring. The remaining outcomes, excluding sustainability through team approach, were tested through a series of surveys. Surveys were given to administrators, teachers, and students questioning the relevant outcomes of SWPBS in comparison to past years. Students were questioned about the amount of time spent on instruction in the classroom and their perception of school safety. Administrators were questioned about the amount of time spent on discipline issues. For the student survey see Appendix A. The evaluation will also address the central questions posed earlier in the report in the purpose section. Is negative, undesired behavior decreasing? Is positive, desired behavior increasing? The evaluation will answer these questions by collecting pertinent data from the elementary, middle, and high school that make up the district. Each school will report the number of discipline referrals they have each quarter.
14

The office discipline referrals will be categorized into the following sections of problem behavior:
Minor- Warning Tardy Forgery/ Theft Dress code violation Use/ Possession of tobacco Use/ Possession of alcohol Use/ Possession of combustibles Bomb threat/ False alarm Arson Use/ Possession of weapons Other behavior Unknown behavior Use/Possession of drugs Minor - Property misuse Minor - Other Minor - Dress code violation Minor - Technology violation Minor - Tardy Technology violation Inappropriate display of affection Inappropriate location/ Out of bounds area Minor Unknown Gang affiliation display Skipping Truancy Fighting Property damage/ Vandalism Vandalism Lying/ Cheating Minor Disruption Minor - Inappropriate language Minor - Physical contact/ Physical aggression Physical aggression Skip class/ Truancy Harassment/ Bullying Minor - Defiance/ Disrespect/ Non-compliance Abusive language/ inappropriate language/ profanity Disruption Defiance/ Disrespect/ Insubordination/ Non-compliance

This data will be cross-referenced with data from the first semester of the last school year before implementation of SWPBS. The purpose of this is to determine if undesired behaviors are decreasing over time with SWPBS usage and the relative speed of success.
15

Is SWPBS being implemented identically across the district? A portion of this question will be answered from the data discussed above. However, this will also be gauged by responses to surveys of the SWPBS team members. The SWPBS team members were selected faculty that participated in the initial training and planning for SWPBS implementation. Is there a correlation between the number of positive reinforcements given by a faculty or staff member and the number of discipline referrals that faculty member completes? This question will be answered in conjunction with the office discipline referral data collection along with data reported by school on the number of positive reinforcement tokens given out by teachers. The number of tokens given and the number of office discipline referrals completed will be compared by teacher at the three different schools. A correlation will be determined. What are the attitudes held by the stakeholders about SWPBS? How are stakeholders impacted by SWPBS? The answer to these questions will be gauged through survey results. Every teachers first period class will also issue an online survey to their students for valuable input. How effective is SWPBS? This question will be answered by analyzing all collected data and using the analysis to determine if SWPBS achieved all outcomes at McCormick County School District.

16

RESULTS
OFFICE DISCIPLINE REFERRALS BY QUARTER ELEMENTARY SCHOOL

80 70 60 50 40 30 20 10 0 2009 - 2010 55 57

74

1st Quarter 25 2nd Quarter

2010 - 2011

FIGURE 5 - RESULTS FOR TOTAL OFFICE DISCIPLINE REFERRALS FOR THE FIRST SEMESTER FOR THE ELEMENTARY SCHOOL

Elementary Behavior Infractions, 1st Quarter 2010 - 2011 School Year

0 21 12 5 12 7 Fighting Lying/ Cheating Minor - Inappropriate language Abusive language/ inappropriate language/ profanity Disruption Defiance/ Disrespect/ Insubordination/ Non-compliance

FIGURE 6 - BREAKDOWN OF OFFICE DISCIPLINE REFERRALS FOR THE ELEMENTARY SCHOOL, 1ST QUARTER OF THE 2010 -2011 SCHOOL YEAR

17

Elementary Behavior Infractions, 2nd Quarter 2010 - 2011 School Year

4 4 Abusive language/ inappropriate language/ profanity

Disruption
17 Fighting

FIGURE 7 - BREAKDOWN OF OFFICE DISCIPLINE REFERRALS FOR THE ELEMENTARY SCHOOL, 2ND QUARTER OF THE 2010 -2011 SCHOOL YEAR

MIDDLE SCHOOL

89 90 79 80 70 60 50 40 30 20 10 0 2009 - 2010 2010 - 2011 1st Quarter 2nd Quarter 67 73

FIGURE 8 - RESULTS FOR THE OFFICE DISCIPLINE REFERRALS FOR THE FIRST SEMESTER FOR THE MIDDLE SCHOOL

18

HIGH SCHOOL

134 140

120
100 80 60 40 20 0 84

112 99 1st Quarter 2nd Quarter

2009 - 2010

2010 - 2011

FIGURE 9 - RESULTS FOR THE OFFICE DISCIPLINE REFERRALS FOR THE FIRST SEMESTER FOR THE HIGH SCHOOL

High School Behavior Infractions, 1st Quarter 2010 - 2011 School Year
Tardy Forgery/ Theft 4% 17% 17% 17% 2% 4% 13% 3% Dress code violation Use/ Possession of tobacco Use/Possession of drugs Gang affiliation display Skipping Fighting Property damage/ Vandalism Technology violation Inappropriate location/ Out of bounds area Abusive language/ inappropriate language/ profanity Disruption Defiance/ Disrespect/ Insubordination/ Non-compliance
FIGURE 10 - BREAKDOWN OF OFFICE DISCIPLINE REFERRALS FOR THE HIGH SCHOOL, 1ST QUARTER OF THE 2010 -2011 SCHOOL YEAR

2% 2% 4% 6% 5%

4%

19

High School Behavior Infractions, 2nd Quarter 2010 - 2011 School Year
Tardy Forgery/ Theft Dress code violation 15% 21% 4% 3% 1% 18% 3% 5% 3% 2% Use/ Possession of tobacco Use/Possession of drugs Gang affiliation display Skipping Fighting Property damage/ Vandalism Technology violation Inappropriate location/ Out of bounds area Abusive language/ inappropriate language/ profanity

15%

2%
3%

5%

Disruption
Defiance/ Disrespect/ Insubordination/ Non-compliance
FIGURE 11 - BREAKDOWN OF OFFICE DISCIPLINE REFERRALS FOR THE HIGH SCHOOL, 2ND QUARTER OF THE 2010 -2011 SCHOOL YEAR

RELATIONSHIP BETWEEN REWARDS AND REFERRALS

160 Total Number of Office Discipline Referrals for the First Semester 140 120 100 80 60 40 20 0 0 500 1000 1500 2000 Elementary Middle High

Total Number of Rewards Redeemed the First Semester

FIGURE 12 - LINE GRAPH REFLECTING NUMBER OF REWARDS VERSUS NUMBER OF REFERRALS FOR ALL THREE SCHOOLS

20

SWPBS TEAM SURVEY RESULTS

120% 100% 80% 98% 95%

100%

100%

100%

100%

100%

60% 56%
40% 20% 2% 3% 0% 2% 44%

67%

73% 63%

33%

37% 27%

Yes

No

Other Response

FIGURE 13 - SWPBS TEAM SURVEY RESULTS ON THE IMPLEMENTATION OF SWPBS

21

SWPBS STAKEHOLDER ATTITUDES

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

Faculty
Staff Administration Students

22

DISCUSSION OF THE RESULTS

FREQUENCY OF DESIRED AND UNDESIRED BEHAVIORS As outlined in the purpose section, this evaluation sought to determine if SWPBS was causing a decrease in undesired behaviors in students and an increase in desired behaviors. The total number of office referrals was collected for a semester or two quarters for each school and compared the total number of office discipline referrals from the previous school year for the same time period. As shown in the results section, the elementary school initially saw an increase in the amount of office discipline referrals. The 2009 2010 school year reported fifty-five office discipline referrals in the first quarter and the 2010 2011 school year reported seventy-four. However, the second quarter showed a significant drop. Last years report was fifty-seven referrals for the second quarter, while this year there was a fifty-six percent decrease in the amount of office discipline referrals. With this short evaluation period, it appears that over time undesired behaviors have decreased while desired behaviors have increased at the elementary level. The middle school and high school both show a trend of an increase in office discipline referrals from 1st quarter to 2nd quarter of the 09-10 school year and 10-11 school year as well. From the data it is apparent that undesired behaviors are not decreasing at the middle and high school level. SWPBS IMPLEMENTATION ACROSS THE DISTRICT Data for this component of the evaluation was collected through survey results of surveys completed by the SWPBS team members from each school. Questions addressed the implementation of SWPBS at their various schools. Results were compared with the guidelines laid out by SWPBS. See Appendix B for the survey. As seen in the chart in the results section, the survey results were very scattered. Across the district, the SWPBS team was not on one accord concerning the implementation of SWPBS. Ninety-five percent of survey takers felt somewhat familiar with the standard procedures for implementing SWPBS. Five of the eleven steps of the implementation process were completed with one hundred percent or close to one hundred percent participation from the SWPBS team. The remaining six steps were either not completed by the SWPBS team or the majority of the SWPBS team felt that they were not completed. The creators of SWPBS state that the success of SWPBS is conditional, based upon strongly recommended steps of implementation. Without the achievement of those conditions, an accurate evaluation of the objectives of SWPBS cannot rightly be performed.

23

CORRELATION BETWEEN POSITIVE REINFORCEMENTS AND OFFICE DISCIPLINE REFERRALS Data collection and analysis show that over the course of the first semester there is a negative correlation between the number of positive reinforcements redeemed and the amount of office discipline referrals completed for the elementary and middle school. Therefore, at these schools, as the number of positive reinforcements redeemed increased the amount of office discipline referrals decreased. The evaluation results illustrate a positive correlation between the number of positive reinforcements redeemed and the amount of office discipline referrals. ATTITUDES OF SWPBS STAKEHOLDERS IN MCCORMICK COUNTY SCHOOL DISTRICT Results of the surveys show that attitudes towards SWPBS by faculty, staff, and administrators across the district are split. See the survey, Appendix C. The results of the survey are organized here. IMPACT OF SWPBS ON STAKEHOLDERS The answer for this central question is provided with the results of the surveys taken by SWPBS stakeholders and students within the district. Seventy-nine percent of faculty that participated in the survey felt that there were more disruptions in class this year as opposed to last school year. However, sixtythree percent of the faculty and ninety-two percent of the administration felt that there was an increase in teaching and learning. OVERALL EFFECTIVENESS OF SWPBS The effectiveness of SWPBS can only be judged by measuring the achievement of the published outcome of SWPBS, provided implementation requirements are met. Since the results of this evaluation report reflect insufficient implementation of the SWPBS program, the effectiveness cannot accurately be determined.

24

CONCLUSIONS & RECOMMENDATIONS

IMMEDIATE RESULTS SWPBS teams members hold a meeting to discuss the results of the evaluation. SWPBS team should review and revise the agreed upon characteristics of major and minor behavior infractions for each school: elementary, middle, and high. SWPBS team should agree on what behaviors constitute an office discipline referrals and what behaviors constitute reward tokens. The agreed upon method of teaching expected behaviors for each school level, should be explained in detail to the remaining faculty and staff by the SWPBS team. Faculty and staff should be encouraged to give more tokens.

LONG-RANGE PLANNING The results of this initial and short-term evaluation project will help in the longrange planning of discipline policy implementation and maintenance in McCormick County School District. The evaluation reflects that SWPBS should be considered over several years in the future in order to accurately determine success or failure. However, over a short time-span increasing rewards given led to less discipline referrals. The district can plan on positively reinforcing behavior expectations in years to come. EVALUATION INSIGHTS If given the opportunity to perform the evaluation project over completely, there are some things that I would have done different. In terms of data collection, I would have liked to require all faculty and staff to keep a running tally of the number of tokens they individually give out and the number of office discipline referrals they individually complete. This kind of data would have made the results more accurate, as the results I used were totals for the each school and compared against rewards that were redeemed. There are incentives in place that may entice students to keep their rewards to redeem for more attractive rewards later in the year. It would have been valuable to have a valid comparison between the attitude towards SWPBS held by a teacher and the
25

number of behavior problems that teacher has encountered since the implementation of SWPBS. The original plan for the evaluation was also to determine the success of SWPBS in meeting the published outcomes. This did not go as planned due to the required implementation conditions not being met by the district.

26

REFERENCES
(2010). Retrieved November 3, 2010, from PBIS.Org: http://www.pbis.org/school/default.aspx Center for Effective Collaboration and Practice CECP. (2001). Success Stories. Retrieved November 24, 2010, from Center for Effective Collaboration and Practice: http://cecp.air.org/resources/success/ebs.asp Kirkpatrick, D. L. (1994). Evaluating Training Programs. San Francisco: Berett-Koehler. Sugai, G. (2007, November 1). Presentations. Retrieved November 24, 2010, from PBIS.org: http://www.pbis.org/common/pbisresources/presentations/gs19982008swpbsperspectiv e.pdf

27

APPENDICES
APPENDIX A: STUDENT SURVEY Survey accessible at this link: http://goo.gl/4xKL5 PBIS Student Survey This survey is for information only and to find out the opinion of the student body concerning PBIS at your school. The purpose is to find out student feelings towards the new way discipline is being handled and the rewards and incentives that are provided. All answers are confidential. Please be honest and answer every question. 1. How would you rate yourself as a student in terms of behavior? Behavior Do you make good or bad decisions at school?
1 Bad 2 3 4 5 Good

2. How would you rate yourself as a student in terms of academics? Academics Do you make good or bad grades?
1 Bad 2 3 4 5 Good

3. Do you like receiving tokens for being a good student? Do you like being rewarded for obeying the rules? Yes No 4. Do you like receiving many chances to behave better before being punished? Do you like having another chance when you make a mistake? Yes No

28

5. In comparison to last school year, are students getting in more trouble or less trouble this school year? More Less Same 6. Do you feel safer at school this year in comparison to last school year? Yes No Same 7. In comparison to last school year, is more or less time spent on teaching and learning in class this year? Are you learning more or less this school year? More Less Same 8. In comparison to last school year, are there more or less distractions in the classroom? More Less Same 9. Were you taught the school expectations at the beginning of the school year? Did your teacher teach you the school rules? Yes No Cant remember 10. In your opinion, is the overall behavior of the students getting better or worse? Better Worse Same 11. In a few sentences, write your true opinions of the new discipline policy. Do you agree or disagree with it? Do you feel it should be changed? Why or why not?

29

APPENDIX B: SWPBS TEAM MEMBER SURVEY Survey accessible at this link: http://goo.gl/1qtSO SWPBS Team Member Survey This survey is for faculty that participated in the professional development training for Positive Behavior Intervention and Supports (PBIS) in the summer of 2010. Results are anonymous and are being collected solely for the purpose of information in an evaluation project. Please be honest and answer all questions.

* Required

1. Are you familiar with the standard procedures for implementing PBIS school wide, which is also known as SWPBS? *

Yes No Somewhat

2. Rate your familiarity with the standard procedures for implementing SWPBS. * 1 Very unfamiliar 2 3 4 5 6 7 8 9 10 Very familiar

3. As a SWPBS team member did you attend a two or three day training as a representative chosen from the school district? *

Yes No

30

4. Did you, along with other SWPBS team members, determine 3 to 5 behavior expectations that suited the needs of your respective school to be used in a behavioral expectations matrix? *

Yes No

5. Did the SWPBS team create a behavior expectations matrix of what the behavioral expectations look like, sound like, and feel like in all classroom and non-classroom areas? *

Yes No

6. Did the SWPBS team share the matrix with faculty and staff to ensure 80% buy-in from the entire staff prior to implementation? *

Yes No

7. In your opinion, what percentage of the staff bought into SWPBS at the time of implementation? * What percentage of the staff was in favor of SWPBS at the time of implementation?

8. Did the SWPBS team determine how the behavioral expectations and routines would be taught in and around the school? *

Yes No

9. Was the determined method for teaching the behavioral expectations explained to all faculty? *

Yes No

31

10. Rate how well the method of teaching the behavioral expectations were explained to faculty that were not members of the SWPBS team. * 1 Very poor 2 3 4 5 6 7 8 9 10 Very well

11. Was time set aside to teach behavior expectations at the specific locations to the students? *

Yes No Don't know

12. Did the SWPBS team revise the office discipline referral form? *

Yes No

13. Did the SWPBS team determine which behaviors were major and minor? * Major behaviors result in an immediate trip to the office.

Yes No

14. Were the characteristics of major and minor misbehavior consistently shared with the entire staff? *

Yes No

32

15. Did the SWPBS team develop a rewards system? *

Yes No

16. How would you rate the implementation of the "gotcha" program? * "Gotcha" program is the rewards system. 1 Poorly implemented 2 3 4 5 6 7 8 9 10 Perfectly implemented

17. What school did you represent? *

High Middle Elementary

33

APPENDIX C: PBIS SCHOOL FACULTY, STAFF, AND ADMINISTRATION SURVEY The survey is accessible at this link: http://goo.gl/V4XbO PBIS School Faculty, Staff, and Administration Survey This survey is for information only and to find out the opinion of PBIS stakeholders in your school district. The purpose is to find out feelings towards the new way discipline is being handled and the rewards and incentives that are provided. All answers are confidential. Please be honest and answer every question.

* Required

1. What job category best applies to you? *

Faculty - Teacher, Guidance, etc Staff - Custodial, Transportation, Food/Nutrition, etc Administration

2. In your opinion, is it a good idea to reward students for obeying school behavior expectations? *

Yes No

3. How many chances should a student get to exhibit expected behaviors? * How many times should a student be allowed to misbehave before being punished? 1 2 3 4 5 6 7 8 9 10

34

4. In comparison to last school year, are students getting in more trouble or less trouble this school year? *

More Less Same

5. In your opinion, does the school campus feel safer this year in comparison to last school year? *

Yes No Same

6. In comparison to last school year, is more or less time spent on teaching and learning in the classroom this year? *

More Less Same

7. In comparison to last school year, are there more or less disruptions in the classroom? *

More Less Same

8. In your opinion, is the overall behavior of the students getting better or worse? *

Better Worse Same

35

9. In a few sentences, write your true opinions of the new discipline policy. Do you agree or disagree with it? Do you feel it should be changed? Why or why not? *

36