PBIS - Positive Behavior Interventions and Supports

Program Evaluation Report

Group 4: Latasha McCord Kecia McGouirk Kristy Slaughter

FRIT 8435 Spring 2013 Georgia Southern University May 6, 2013

Group 4 - Page 2

Table of Contents

I. II.

III.

IV. V.

VI.

VII.

Executive Summary......................................................................................... Introduction to the report............................................................................... a Purpose of the evaluation......................................................................... b Audiences for the evaluation report......................................................... c Limitations of the evaluation..................................................................... d Overview of report contents...................................................................... Focus of the evaluation.................................................................................... a Description of the evaluation object.......................................................... b Evaluative questions used to focus the study............................................ c Information needed to complete the evaluation...................................... Brief overview of evaluation plan and procedures.......................................... Presentation of evaluation results................................................................... a Summary of evaluation findings................................................................ b Interpretation of evaluation findings......................................................... Conclusions and recommendations................................................................. a Criteria and standards used to judge evaluation object............................ b Judgments about evaluation object (strengths and weaknesses)............. c Recommendations..................................................................................... Appendices...................................................................................................... a Description of evaluation plan/design, instruments, and data analysis and interpretation..................................................................................... b Literature Review......................................................................................

3 4 4 4 4 5 5 5 7 7 8 8 8 9 13 13 14 14 15 15 18

Group 4 - Page 3

Executive Summary
The information in this evaluation report combined a literature review (www.pbis.org) with a case study of a central Georgia high school to determine the effectiveness of a schoolwide PBIS program in a public high school setting. It was found that the school adequately implemented the program and was able to decrease problem behaviors as well as increase academic achievement. However, this particular school does show a need for some minor adjustments in order to maintain the success of the program. The PBIS program implemented at the district level of ninth through twelfth grades in this report will demonstrate areas of improvement in students tardies, attendance, disciplinary behavior, and academic achievement. The study found that tardies and disciplinary referrals decreased somewhat during the year of PBIS implementation when compared to the same time periods the year before implementation. Absences also decreased overall in the second and third quarters of implementation. Furthermore, the number of first-time participants increased nearly 25 percent after the first quarter incentive was awarded, demonstrating that students who had not received a reward the first time improved their behavior and were able to participate in the second quarter reward. It is worth noting that the overall number of participants did not increase significantly in spite of the increase of first-time participants. For this reason, it is the recommendation of the evaluators that further study be conducted on what incentives may be motivational for high school students. The study should include a survey of student interests.

Group 4 - Page 4

Introduction
The purpose for this evaluation is to investigate the Positive Behavior Interventions & Supports (PBIS) program in a central Georgia high school to determine if the program is serving its intended purpose: to have a positive influence on student behaviors including being on time to school and to class, not receiving any disciplinary referrals, improving attendance, and passing all of their classes. The audience for this evaluation includes administration, teachers, students, parents and the Board of Education. These individuals will assist with the results of student attendance, grades, behavior from disciplinary reports, date of incentives, pros/cons of incentives and the outcome from both disciplinary reports and student participation utilizing SWIS (School Wide Information System) and PowerSchool. The utilization of such data will help answer stakeholders questions such as how well the program is functioning and what changes may be needed in order to improve it. The limitations of this evaluation include the short timeframe available for the study and the availability of access to peer-reviewed texts by experts in the field. There is an abundance of general research about positive behavioral interventions and supports, but there is less research on specific studies regarding the effectiveness of the program on academics and behavior. An additional limitation of the study was smaller amounts of literature on schools and districts implementing PBIS programs. This program evaluation is limited to one school system in central Georgia. Its results are intended to demonstrate the effectiveness in this one setting. The current report also reviews only available quantitative data. For a more thorough analysis,

Group 4 - Page 5

further qualitative data should be collected including surveys of all stakeholders. Students should be surveyed to determine if they feel the incentives that are offered are indeed motivating them to modify their behavior. Teachers, parents, and other stakeholders should be surveyed to see if they feel there has been a change in attitudes or behaviors since the inception of the program. This report contains a thorough description of the object of the evaluation, a description of the evaluation plan and procedures, a summary of the findings along with detailed charts, interpretations of the data, and the recommendations of the evaluators. Included in the Appendices are the charts containing the complete evaluation plan, as well as some additional diagrams that may help in the understanding of the PBIS program.

Focus of the Evaluation

Teachers and administrators have been utilizing the most effective ways to interact with students, support their learning, and help them become lifelong learners for years. Unfortunately, there is not a quick fix for every challenge that arises in schools across the country. However, certain practices have been shown to be more effective when used correctly. Positive Behavioral Interventions and Supports, or PBIS, (www.pbis.org) defines, teaches, and supports appropriate student behaviors to create a positive learning environment where problem behaviors are decreased and academic performance is increased (Appendix B). The object of this evaluation is the Positive Behavior Interventions & Supports (PBIS) program in a high school environment to determine if the program does indeed have a positive

Group 4 - Page 6

influence on student behaviors. PBIS is a data-driven, research-based intervention with the goal of improving behavioral and academic outcomes. PBIS comes directly from the 1997 reauthorization of the Individuals with Disabilities Education Act (IDEA). PBIS is used interchangeably with SWPBS, which is short for School-wide Positive Behavior Supports. (Appendix C). PBIS is an implementation framework that is designed to enhance academic and social behavior outcomes for all students by emphasizing the use of data for informing decisions about the selection, implementation, and progress monitoring of evidence-based behavioral practices (Appendix D). PBIS includes allowing choice and decisions about selfcontrol, personal goal setting, and additional independent choices. Encouraging and allowing self-determination and personal goals for all individuals in the school pushes students to become their own advocates and provides further reason for expanding their abilities and confidence. With the idea of making individual decisions, a student helps plan his or her own behavior goals that can be reached successfully and celebrated among peers and staff members through school-wide incentives. Several school systems around the state of Georgia have begun implementing PBIS programs into their school day. Students are rewarded for desirable behaviors such as getting to school on time, not having any tardies to class, receiving good grades on Report Cards, and not receiving any disciplinary referrals. For this report, the focus will be on using the quantitative data available from one school system in central Georgia. The school district is a rural school system located in the heart of Georgia near both Macon and Atlanta. The system includes six schools-- three elementary, two middle, one high school, and one alternative school-- and serves just over 4,000 students. The high school that is the focus of this evaluation serves 1086 students. The population is 68%

Group 4 - Page 7

Caucasian, 27% African-American, and 5% other ethnicities. Fifty-six percent of the students are classified as economically disadvantaged. The questions being evaluated include:

Do PBIS (Positive Behavior Interventions & Supports) programs help schools to increase desired behaviors including being on time for school and class, not receiving disciplinary referrals, not having excessive absences, and improving academic grades?

Is participation in the PBIS program growing, or are the same students receiving rewards each time? In order to evaluate the program, data concerning participation in the PBIS program was

collected from the high school in the district. The data includes the numbers of students who were eligible to participate in each of the three quarterly rewards during the 2012-2013 school year to determine if the program has grown in the course of the year. Numbers of students participating for the first time each quarter were also analyzed to determine whether the same students were being recognized each quarter or if more students were qualifying for rewards. For comparison, data was also collected and analyzed from the current year and the previous school year concerning students tardies to class and to school, absences, disciplinary referrals, and classes passed. Data was collected from the School Wide Information System (SWIS) and from PowerSchool records.

Group 4 - Page 8

Overview of Evaluation Plan and Procedures

To assess the impact of the PBIS Program in a high school setting, information detailing students tardies, absences, disciplinary referrals, and passing grades was collected through emails and system data analysis. Since the program has not undergone a formal evaluation in this particular school district, much of the assessment of the effectiveness of PBIS was circumstantial and based upon literature research and from the previous years data prior to PBIS going into effect. Time constraints and cooperation with administration played a factor in this evaluation. Due to anonymity reasons, administrators asked that we not copy and paste actual SWIS (School Wide Information System) and PowerSchool data in our report. Instead, we created our own bar charts based on the numbers received from administration. Therefore, quantitative data relating to PBIS was discreetly obtained from administration and examined through the generated reports from SWIS on students disciplinary referrals and generated reports from PowerSchool on students tardies, absences, and grades. In all, the quantitative data associated with PBIS was thoroughly assessed in this program evaluation.

Presentation of Evaluation Results

The results of this evaluation demonstrate that there was only a slight increase from the first quarter to the second quarter (from 596 to 619 students) in the total number of students receiving awards for having no tardies to class or to school and having no disciplinary referrals (see Figure 1 below). This represents less than a five percent increase. Furthermore, there was

Group 4 - Page 9

no increase in the total number from the second quarter to the third quarter. The number of students meeting the second tier goal, which also included having no absences for the quarter, actually decreased by nearly 30 percent each quarter. The numbers of students meeting the third tier goal, which includes all of the criteria for the first two, but also includes passing all of their courses, also showed a significant decline each quarter.

2012-2013 School Year

Number of students meeting Goal #1: Zero tardies to class Zero tardies to school Zero disciplinary referrals for the quarter Number of students meeting Goal #2: Met criteria for Goal #1 and Zero absences Number of students meeting Goal #3: Met criteria for Goal #2 and Passing grades in all classes Figure 1

1st Quarter

2nd Quarter

3rd Quarter

596

619

618

235

153

116

217

141

102

If the data in Figure 1 were all that was available, then it would be a logical assumption that PBIS was ineffective. However, upon further evaluation, several important factors can be observed. For example, 152 of the 619 participants in the second quarter were being rewarded for the first time. This indicates that there was nearly a 25 percent increase in first-time participants from the first quarter. An additional 66 students were first-time participants for the rewards in the third quarter. This substantial increase in new participants indicates that the PBIS rewards may have been a motivating factor for students to improve specific aspects of their behavior.

Group 4 - Page 10

Data analysis concerning specific target behaviors from the year prior to implementation of PBIS also supports the idea that some modest improvements may have occurred in behavior patterns during the implementation year. The figures below illustrate a slight improvement in almost every area each quarter compared to the previous year. Tardies to school, tardies to class, disciplinary referrals, and absences were consistently reduced, with the exception of the first quarter of implementation when student absences increased slightly (Figure 5). However, this irregularity may have occurred because students were not yet fully aware of the PBIS system and the rewards that were to come. One final chart demonstrates that the number of students passing all of their classes increased slightly over the previous year as well.

Number of Tardies to School per Day on Average

Tardies to High School Grades 9-12

50 40 30 20 10 0 1st Quarter 2nd Quarter 3rd Quarter 2011-2012 2012-2013 42 38 35 33

27 24

Figure 2

Group 4 - Page 11

Tardies to Class
Number of Tardies to Class per Day on Average

Grades 9-12
20 15 10 5 0 1st Quarter 2nd Quarter 3rd Quarter 2012-2013 2011-2012 12 10 18 15

14

12

Figure 3

Disciplinary Referrals
Grades 9-12
1,000 900 800 700 600 500 400 300 200 100

Number of Referrals

746 540 458

686

717

630

1st Quarter
Figure 4

2nd Quarter

3rd Quarter 2011-2012 2012-2013

Group 4 - Page 12

Absences Grades 9-12

2,000 Number of Absences 1723 1682 1,500 1438 1470 1789 1758

1,000 1st Quarter 2nd Quarter 3rd Quarter 2011-2012 2012-2013

Figure 5

Passing All Classes Grades 9-12

Number of Students Passing all Classes 1,000 900 800 700 600 500 1st Quarter
Figure 6

874 893 715 722

821 836

2nd Quarter 3rd Quarter 2011-2012 2012-2013

Group 4 - Page 13

Conclusions and Recommendations

The quantity of tardies, referrals, absences, and failing grades that occurred in the 20112012 school year compared to the 2012-2013 with PBIS in place determine that PBIS programs do seem to assist with the increase of desired behaviors. The preferred behaviors included being on time for school and for class, not receiving disciplinary referrals, not having excessive absences, and improving student grades. Additionally, the specific level of performance expected in order to show the criteria met are at least a 10 percent decline in tardies to class and to school, disciplinary referrals, absences, and failing grades. Though not every criteria met the 10 percent mark every quarter, the evaluators feel that the improvements are significant enough to continue the program for another year. Furthermore, in evaluating the growth of the participation and verifying if the same students are in receipt of rewards each time, the number of students awarded with PBIS incentives during the school year should definitely be considered. In order for this criteria to be fully successful, we felt there should be at least a 10 percent increase in the number of students eligible for rewards during the school year. This goal was greatly exceeded for first-time participants throughout the course of the year. Based upon the evaluation of program data relating to students problem behaviors and academic achievement of the current PBIS program, this evaluation study concludes the following: Program Strengths
Shows sufficient improvements in targeted areas

Group 4 - Page 14

Increases problem awareness Places a greater focus on students Places more emphasis on preventions Removes potential triggers of bully behavior Teaches effective, efficient, and relevant alternative social skills Establishes a consistent and fair reward system

Program Weaknesses

The principles and concepts of behavior management associated with PBIS can be complex and difficult to implement on a school-wide basis

Secondary and tertiary level interventions may need improvement Universal implementation of meaningful incentives for students who display desired behaviors

Time and labor intensive May be costly depending on the rewards offered

Recommendations It is the recommendation of this evaluation study that the implementation of the PBIS program justifies that the school district continue to do what is already showing some improvement. Throughout the current assessment it was deemed vital to pay close attention to the smallest changes that might produce larger effects, in particular with regard to the targeted aspects of student behavior. In addition, it was noted that it is highly important to avoid defining a large number of goals when considering implementing the PBIS Program. The current

Group 4 - Page 15

expectations concerning tardies, discipline, absences and grades are sufficient. Furthermore, other recommendations include collecting and utilizing additional qualitative data such as surveys of stakeholders for decision-making, and adapting any initiative which involves families, students, faculty, and fiscal structure. This thorough evaluation demonstrated that a new program should not be added without thorough research which includes defining the expectations of the program. As with any new program, the establishment of policy clarity should be considered before investing in any implementation.

Appendices Appendix A
Below is an evaluation plan of the PBIS program (Positive Behavior Interventions & Supports). PBIS is a data-driven, research-based intervention with the goal of improving student behavioral and academic outcomes. The school system under evaluation started using PBIS in the 2012-2013 school year. The evaluation compares current and previous school years PBIS results with grades 9-12 and to determine whether or not the program seems to be effective.
Planning the Evaluation Reporting:

Evaluation Questions

Audience for the Report

Report Content

Report Format

Reporting Schedule

Context for Presenting Report PowerPoint presentation to administration, faculty and the Board of

1. Do PBIS (Positive Behavior Interventions & Supports) programs help

Administration Teachers Parents Students Board of

Results of student attendance, grades, and behavior from disciplinary

Presentation to Faculty, Student/Parent Letter, Mass Email, Posted on

Quarterly

Group 4 - Page 16

schools to increase desired behaviors including being on time for school and class, not receiving disciplinary referrals, not having excessive absences, and improving academic grades? 2. Is participation in PBIS program growing, or are the same students receiving the rewards each time?

Education

reports using SWIS (School Wide Information System) and PowerSchool

District Website & Presentation to the Board of Education

Education. Onepage letter to students and parents, which entails the findings.

Administration Teachers Parents Students Board of Education

Date of incentives, pros/cons of incentives and the results of student participation using SWIS (School Wide Information System)

Presentation to Faculty, Student/Parent Letter, Mass Email, Posted on District Website & Presentation to the Board of Education

Quarterly

PowerPoint presentation to administration, faculty and the Board of Education. Onepage letter to students and parents, which entails the findings.

Summarizing the Evaluation Plan:

Evaluatio n Questions

Informati on Required

Design

Informati on Source

Method for Collecting Informati on

Sampling

Informati on Collectio n Procedur es Obtaining quarterly reports of SWIS data on student disciplinary referrals. arranged and conducted with Assistant Principal (AP), IC, and teachers.

Schedule

Analysis Procedure s

1. Do PBIS (Positive Behavior Interventio ns & Supports) programs help schools to increase desired behaviors including being on time for school and class, not receiving disciplinary referrals,

Number of disciplinary referrals issued each quarter for current school year and previous school year. Notice of decrease in student behavior issues and increase in students grades by teachers.

Interrupted Time Series

Administra tor, Instruction al Coach (IC), Teachers, and SWIS Reports

SWIS Data Reports

District High school students

Quarterly SWIS reports & in April

Descriptive stats from SWIS data reports. Summarize Integrate all results to describe PBIS outcomes.

Group 4 - Page 17

not having excessive absences, and improving academic grades? 2. Is participatio n in PBIS program growing, or are the same students receiving the rewards each time? Numbers of students participatin g in rewards each quarter Interrupted Time Series SWIS Reports SWIS Reports District High school students Obtaining quarterly reports of SWIS data Quarterly SWIS reports Descriptive stats from SWIS data reports

Reporting Procedures:

Evaluation Questions 1. Do PBIS (Positive Behavior Interventions & Supports) programs help schools to increase desired behaviors including being on time for school and class, not receiving disciplinary referrals, not having excessive absences, and improving academic grades? 2. Is participation in PBIS program growing, or are the same students receiving the rewards each time?

Interpretation Procedures Is there at least a 10% decrease in students tardies to class, tardies to school, disciplinary referrals, absences, and failing grades? If not, what solutions are recommended?

Audience(s)

Content

Format

Schedule

Administration Teachers Parents Students Board of Education

Results of student attendance, grades, and behavior from disciplinary reports using SWIS (School Wide Information System) and PowerSchool. Utilizing data will help answer questions: What is the program doing well? What changes are needed? Date of incentives, pros/cons of incentives and the results of student participation using SWIS (School Wide Information System). Utilizing data will help answer questions: What is the program doing well? What changes are needed?

Presentation to Faculty, Student/Parent Letter, Mass Email, Posted on District Website, & Presentation to the Board of Education

At the end of each quarter- October, January, March, and May

Is there at least a 10% increase in the number of students eligible for rewards during the school year? If not, what solutions are recommended?

Administration Teachers Parents Students Board of Education

Presentation to Faculty, Student/Parent Letter, Mass Email, Posted on District Website, & Presentation to the Board of Education

At the end of each quarter- October, January, March, and May

Group 4 - Page 18

Appendix B

http://www.pbis.org/common/cms/media/4Elements.jpg

Appendix C

Group 4 - Page 19 http://www.pbis.org/common/cms/media/ContinuumTriangle.jpg

Appendix D

http://www.cber.uconn.edu/wp-content/uploads/2011/11/PBIS-keynote-overview-1hr-Nov-9-2011POST.pdf