Teacher Effectiveness Measure (TEM): Summative Process

By now, all Memphis City Schools educators are familiar with the Teacher Effectiveness Measure (TEM) as our districts new, multi-dimensional teacher evaluation system. This document is intended to prepare you for the upcoming summative evaluation process during which all teachers will receive data and ratings for each TEM component as well as an overall TEM score. Details are provided on 1) the different TEM components each educator group will use for the summative score; 2) how each component will be scored in the TEM profile; and 3) the timeline for the summative evaluation process. If you have additional questions, please visit the TEI website (www.mcstei.com) or contact the Department of Teacher Talent & Effectiveness (DTTE) (tei@mcsk12.net; 416-0135)

TEM Components by Educator Group

Educator Group refers to which group a teacher has been assigned to for his or her observation rubric. These groups include General Education, Exceptional Children (Self-Contained or Inclusion), Library/Media Specialists, Guidance Counselors, Psych/Social Workers, and Instructional Facilitators. Although the majority of teachers will use the same TEM components, these components vary for some educator groups. These distinctions are provided in detail below: General Education Teachers in Tested Subjects TEM Component Student Growth Student Achievement Observations Stakeholder Perceptions Teacher Knowledge Measurement Used Individual Value-Added (TVAAS) Data Menu of Options Provided by Tennessee Dept. of Education TEM General Education Rubric Spring 2012 Tripod Student Survey Results Menu of Options Provided by MCS Weighting 35% (State Mandated) 15% (State Mandated) 40% 5% 5%

General Education Teachers in Non-Tested Subjects TEM Component Measurement Used PreK 3: School-Level Literacy or Math TVAAS 9 12: School-Level Literacy or Math TVAAS CTE: School-Level Literacy or Math TVAAS World Language: School-Level Literacy TVAAS ELL: School-Level Literacy TVAAS P.E./Health: School-Level Overall TVAAS Comp Tech: School-Level Overall TVAAS Fine Arts: School-Level Overall TVAAS Menu of Options Provided by Tennessee Dept. of Education TEM General Education Rubric Spring 2012 Tripod Student Survey Results Menu of Options Provided by MCS Weighting

Student Growth

35% (State Mandated)

Student Achievement Observations Stakeholder Perceptions Teacher Knowledge

15% (State Mandated) 40% 5% 5%

Teacher Effectiveness Measure (TEM): Summative Process

Teachers of Exceptional Children1 TEM Component Student Growth Student Achievement Observations Stakeholder Perceptions Teacher Knowledge Measurement Used School-Level Overall TVAAS Menu of Options Provided by Tennessee Dept. of Education TEM Exceptional Children Rubric (Inclusion or Self-Contained) Not Applicable Menu of Options Provided by MCS Weighting 35% (State Mandated) 15% (State Mandated) 45% 0% 5%

Library/Media Specialists TEM Component Student Growth Student Achievement Observations Stakeholder Perceptions Teacher Knowledge Measurement Used School-Level Literacy TVAAS Menu of Options Provided by Tennessee Dept. of Education TEM Library/Media Specialists Rubric Not Applicable Not Applicable Weighting 35% (State Mandated) 15% (State Mandated) 50% 0% 0%

Guidance Counselors TEM Component Student Growth Student Achievement Observations Stakeholder Perceptions Teacher Knowledge Measurement Used School-Level Overall TVAAS Menu of Options Provided by Tennessee Dept. of Education TEM Guidance Counselor Rubric Not Applicable Not Applicable Weighting 35% (State Mandated) 15% (State Mandated) 50% 0% 0%

Psych/Social Workers TEM Component Student Growth Student Achievement Observations Stakeholder Perceptions Teacher Knowledge
1

Measurement Used School- or District-Level Overall TVAAS Menu of Options Provided by Tennessee Dept. of Education TEM Psych/Social Workers Rubric Not Applicable Not Applicable

Weighting 35% (State Mandated) 15% (State Mandated) 50% 0% 0%

CLUE teachers are to use the General Education TEM classification instead of Exceptional Children educator group.

Teacher Effectiveness Measure (TEM): Summative Process

Instructional Facilitators TEM Component Student Growth Student Achievement Observations Stakeholder Perceptions Teacher Knowledge Measurement Used School-Level Literacy, Math or Overall TVAAS Menu of Options Provided by Tennessee Dept. of Education TEM Psych/Social Workers Rubric Not Applicable Not Applicable Weighting 35% (State Mandated) 15% (State Mandated) 50% 0% 0%

Scoring the TEM

As with TVAAS and TEM observations, each component of a teachers TEM profile will receive a score on a scale of 1 5. Those scores will then be weighted2 and added together as demonstrated in the table below: Range of TEM Component Scores (Minimum Maximum) 1-5 1-5 1.00 5.00
(average rounded to the nearest .01)

Weighting x 35 x 15 x 40 x5 x5

Student Growth (TVAAS) Student Achievement Observations Stakeholder Perceptions Teacher Knowledge

Weighted Score (Min Max) 35 175 15 75 40 200 5 25 5 25

15 15

TOTAL

100 - 500

Once the total weighted score is calculated, a TEM teacher effectiveness rating is assigned based on the ranges provided below: Effectiveness Rating TEM 1 TEM 2 TEM 3 TEM 4 TEM 5

TEM Score Range 100 - 200 200 274.99 275 349.99 350 424.99 425 500

Performing Significantly Below Expectations Performing Below Expectations Meeting Expectations Performing Above Expectations Performing Significantly Above Expectations

This table reflects the TEM profile of a teacher with all five possible TEM components. The weighted calculations will vary somewhat for educator groups with fewer than five TEM components.

Teacher Effectiveness Measure (TEM): Summative Process

Scoring Student Growth (TVAAS)
What is TVAAS (Student Growth)? TVAAS is a measure of growth in performance from one year to the next on state-mandated tests. The TVAAS system calculates growth from tests covered under the Tennessee Comprehensive Assessment Program (TCAP) in reading/language arts, math, science, and social studies and other core subjects. The general expectation for TVAAS is that a years worth of instruction should result in a years worth of student growth. Is TVAAS the same as Adequate Yearly Progress (AYP)? No. AYP is a required measurement under the No Child Left Behind (NCLB) Act. In short, AYP refers to whether or not a student, class, school, district, or state met a pre-set target of proficiency on a criterionreferenced test (CRT). TVAAS, by contrast, is a growth measure. It is based on the gains students make (or dont make) compared to their test outcomes in previous years. Which teachers have individual-level TVAAS results? Teachers whose students are assessed at the end of the year by a state-mandated test have TVAAS results. At the present time, that means core subject teachers in grades 3 8 and teachers in Algebra I, Algebra II, Biology I, English I, English II, and U.S. History have TVAAS results. In 201112, English III will be a tested subject that counts for TVAAS. What if I dont have individual-level TVAAS? According to state law, teachers in non-tested subjects must use school-level TVAAS data for 35% of their evaluation (TEM) score until the state approves a system for measuring value-add in non-tested subjects. In 2012, we are expecting that the work of several state-level committees will provide a more direct measurement of the impact of teachers in non-tested subjects. For the first time this year, a school-level composite TVAAS score will be calculated. This new statistic will combine TVAAS results from all the teachers in a school and report that as a single number: the TVAAS Composite. How will Student Growth (35% of TEM) be scored? As with all TEM components, TVAAS will be scored on a scale of one through five, and these scores will be provided by the state of Tennessee. All teachers will use their most stable TVAAS data from the 2010-11 school year. That means that those who have taught a tested subject for two years will receive a two year composite. Those who have taught a tested subject for three or more years will receive a three-year composite. If you have taught a subject Those teachers and schools whose students demonstrate less than a year of growth will either receive a score of 1 (Significantly Below Expectations) or 2 (Below Expectations). Teachers and schools with students demonstrating one year of growth will receive a score of 3 (Meeting Expectations). Those with Students demonstrating more than one year of growth will receive a score of 4 (Above Expectations) or 5 (Significantly Above Expectations). When will TVAAS scores be available? MCS will use the TVAAS data that is currently available to calculate individual teachers TEM scores. These TVAAS data became available to principals and teachers in tested subjects in November 2011.

Teacher Effectiveness Measure (TEM): Summative Process

Scoring Student Achievement
How will Student Achievement (15% of TEM) be scored? 15% of the TEM evaluation score must include a measure of student achievement outcomes based on a menu of options that the state of Tennessee has deemed appropriate for a teachers educator group as outlined in the table below. As with all TEM components, each these achievement outcomes will be scored on a scale of one through five, and these scores will be provided by Memphis City Schools. The options3 identified by the state are:
Graduation rate / CTE concentrator graduation rate School-wide TVAAS or individual TVAAS for teachers in top 3 quintiles Postsecondary matriculation/persistence / placement ACT suite of assessments/SAT suite of assessments State assessments (discipline-specific/TCAP) Completion/success in advanced coursework, including dual credit and dual enrollment National/State-used off the shelf assessments 9th grade promotion to the 10th grade/ 9th grade retention rate

Teachers with TVAAS (4-8) Teachers with TVAAS (9-12) PK-3 Fine Arts Middle/High School nonassessed courses World Languages Computer Technology PE and Health Educators Instructional Facilitators Library Media Specialists ELL Exceptional Children CTE Psych/Social Workers

X X X X X X X X X X X X X X

X X X X X X X X X X X X X X

X X

X X X X X X

AP/IB/NIC suites of assessments

X X X

X X X X X X X X X X X

X X X X X X X X X X X

X X X X

X X X X X X X

X X X X X X X X X X X

X X X X X X X X X X X

X X X X X X X X X X X

X X X

X X X X

National/State off the shelf tests include: PreK-12 diagnostic or achievement/attainment assessments (e.g. SAT 10, Dibels, DRA, Kindergarten-readiness, end of course, etc.) DOE will develop standard criteria for approval of tests submitted by LEAs. AP/IB/NIC suites of assessments include: Courses designed for Advanced Placement (AP), International Baccalaureate (IB), National Industry Certification (NIC) assessments Participation in advanced coursework includes: School level calculated secondary rates (e.g. Honors, AP, IB, NIC, college/high school dual enrollment and dual credit) according to SBE uniform grading policy.

Teacher Effectiveness Measure (TEM): Summative Process

What is the selection process for the Student Achievement measure? In Spring 2012, teachers will be able to select an appropriate Student Achievement measure using their Randa Tower software accounts. By state regulation, principals must agree with teachers selections. If a teacher and principal cannot reach consensus, the principal shall choose the Student Achievement measure. MCS will provide principals with all available Student Achievement data and corresponding TEM scores from the 2010-11 academic year prior to the selection deadline to assist teachers and principals in this process. When will Student Achievement scores be available? As with Student Growth, MCS will use the Student Achievement data that is currently available to calculate individual teachers TEM scores. MCS will provide principals with all available school-level Student Achievement data and corresponding TEM scores from the 2010-11 academic year in Spring 2012 to assist teachers and principals in the selection process.

Scoring Observations
How will Observations be scored? As with all TEM components, observation ratings are scored on a scale of one to five. MCS will average all of the indicators that have been rated across all of a teachers observations to generate a composite observation score. Please note that any unrated indicators will not count towards a teachers final composite score. MCS will only average those indicators that have been rated a 1 5. Unlike the other TEM components, teachers observation composite averages will be rounded to the nearest 0.01 instead of the nearest whole number. Teachers in the General Education educator group will only receive scores for the indicators in the Teach and Cultivate a Learning Environment domains, whereas other educator groups will receive scores for the domains in their specialized rubrics.4 What are the requirements for a complete observation score? According to state regulations, teachers with Professional licensure should be observed a minimum of 4 times each academic year for a combined total of no fewer than 60 minutes. Teachers with Apprentice licensure should be observed a minimum of 6 times for a combined total of no fewer than 90 minutes. Half of a teachers observations should be announced and half of the observations should be unannounced. All teachers should receive a post conference following each observation. Teachers should also have the following documentation, located in the Randa Tower software: Self-Assessment Completed by the teacher during the first observation cycle Professional Growth & Support Plan (PGSP) Completed jointly by the teacher and a school administrator during the first observation cycle Observation Scores Completed by the observer after each observation Post-Observation Conference Record Submitted by the observer after incorporating the teachers reflections and any adjustments to the PGSP following an observation post conference

Librarians are exempt from the Plan domain.

Teacher Effectiveness Measure (TEM): Summative Process

When will the composite TEM observation score be available? Administrators have been instructed to complete all observations by March 30th. Individual observation scores have been made available to teachers on a continual basis following each observation. MCS will provide teachers and principals with the overall observation composite score along with all TEM data by April 30th, 2012.

Scoring Stakeholder Perceptions

How will Stakeholder Perceptions be scored? For 2011-12, Stakeholder Perceptions will be measured using the Spring 2012 results from the Tripod Student Survey, which will be administered in early March. The survey asks students to assess observable teaching practices in their classrooms according to the Seven Cs: Caring about students (encouragement and support) o Ex: My teacher in this class makes me feel that s/he really cares about me Controlling behavior (press for cooperation and peer support) o Ex: Our class stays busy and doesnt waste time Clarifying lessons (success seems feasible) o Ex: My teacher explains difficult things clearly Challenging lessons (press for effort, perseverance and rigor) o Ex: My teacher wants me to explain my answers why I think what I think Captivating students (learning seems interesting and relevant) o Ex: My teacher makes learning enjoyable Conferring with students (students sense their ideas are respected) o Ex: My teacher wants us to share our thoughts Consolidating knowledge (ideas get connected and integrated) o Ex: My teacher takes the time to summarize what we learn each day MCS will then compare individual teachers Tripod results to the average Tripod results for all teachers in similar content areas and grade levels and create an index score of 1 5, like with TVAAS scores. An example is provided in the table below. Teaching Practice Caring Controlling Clarifying Challenging Captivating Conferring Consolidating Overall Individual Teachers Score 70 54 43 55 62 45 57 56 Avg. for MCS Peer Teachers 55 52 40 51 48 41 54 49

Index: Level 4

Teacher Effectiveness Measure (TEM): Summative Process

What if I dont have students that will take the Tripod survey this year? Teachers in the Exceptional Children, Library/Media Specialist, Guidance Counselor, Psych/Social Worker and Instructional Facilitator educator groups will not receive Tripod survey results in 2011-12. These groups will not have Stakeholder Perceptions as part of their TEM profiles, and the adjusted TEM component weightings are reflected on pages 2 3 of this document. When will the Stakeholder Perceptions score be available? MCS will provide teachers and principals with Tripod results and the index score along with all TEM data by April 30th, 2012.

Scoring Teacher Knowledge

How will Teacher Knowledge be scored? As with Student Achievement, teachers will select from a menu of options for the Teacher Knowledge component. Please note that this menu of options will be revisited on an annual basis and potentially revised as additional measures become available. The menu of options for 2011-12 is detailed in the table below.

Option
Content-Specific Professional Development

Description
For 2011-12, this option entails the successful completion of 10 hours of the Common Core online modules already required as part of teachers 57 PD hours. The deadline for completion is March 1st, 2012. Parts 1 & 2: AVATAR Course # 10490.12500 Part 3: AVATAR Course # 11225.13658 Part 4: AVATAR Course # 11226.13965

Scoring Method
Teachers who successfully complete all four Common Core courses by March 1st, will receive a 5. Those who do not complete all four courses by March 1st will receive a 1.

Portfolio of Work

Teachers may submit a portfolio of work that will be reviewed by a committee of peers in their content area that have been identified by the district. These committees have developed the rubrics used to score portfolios for each content area. The rubrics are available on the TEI website under the Content Knowledge section of the TEM page: (www.mcstei.com/tem/content-knowledge) Portfolios must be submitted no later than March 16th by mail or in person to: Office of Teacher Effectiveness Measurement 2597 Avery Avenue Room 260 Memphis, TN 38112

Teachers will receive a score of 1 5 on their portfolios based on the rubric for their content area.

Teacher Effectiveness Measure (TEM): Summative Process

Observation by a Content Specialist Teachers may request an observation from a content specialist in their subject area to count towards both Teacher Knowledge and Observation TEM components. It should be noted that this option is limited by Content Specialists availability and case load capacity. If one of a teachers 4-6 required classroom observations was conducted by a content specialist, the observations scores will satisfy the knowledge requirement. If a teacher has not been observed by a content specialist, an additional classroom observation using the TEM rubric will be conducted to satisfy this requirement. These observations will be conducted during the week of April 2nd-April 5th and April 9th-April 13th. All rated indicators of the Content Specialists observation will be averaged and rounded to the nearest hundredth place, as will the composite Observation score.

What is the selection process for Teacher Knowledge measure? In early February, teachers will receive an email with a Survey Monkey link to select their Teacher Knowledge component. Unlike the Student Achievement component, teachers do not need approval from their principals to select an option. Teachers who select a Content Specialist observation as their top choice will receive a follow-up email from a district administrator confirming whether or not a Specialist will be available to assist with this component. If capacity for observations becomes limited, teachers may be asked to select another option. What if my educator group does not have a Teacher Knowledge component for the TEM? Teachers in the Library/Media Specialist, Guidance Counselor, Psych/Social Worker and Instructional Facilitator educator groups are exempt from the Teacher Knowledge components of the TEM in 2011-12. The adjusted TEM component weightings for these groups are reflected on pages 2 3 of this document. When will Teacher Knowledge scores be available? Although the timeline may vary by the option selected, MCS will provide teachers and principals with Teacher Knowledge scores along with all TEM data by April 30th, 2012.

The Summative TEM Process and Timeline

What is the decision-making process for teacher reelections based on TEM scores? By April 30, 2012, you and your principal will receive your TEM profile with scores for each component and composite TEM scores of 1 - 5. Student Growth and Achievement data for the current year will not be available, so teachers will receive scores for these components based on existing 2010-11 data where available. Scores for the other TEM components (Observations, Stakeholder Perceptions and Teacher Content Knowledge) will be provided for the current year by April 30th. Once teachers and principals receive these TEM Profiles, schoollevel administrators will schedule a summative evaluation meeting with each teacher to review his or her TEM data and notify teachers of reelection recommendations.

Teacher Effectiveness Measure (TEM): Summative Process

Guidance on Proper Use of the TEM for Reelection Recommendations The TEM is a management tool providing school leaders the ability to readily identify appropriate support, development and recognition opportunities for teachers aligned to their individual performance data. Tennessee law states that a district may also use teacher performance data as grounds for a teachers dismissal if he or she is performing Below Expectations or Significantly Below Expectations. For MCS, this means that teachers scoring a TEM 1 or TEM 2 may be considered for non-reelection. Teachers scoring a TEM 3 or higher will not be eligible for dismissal based on their TEM evaluation scores because they are meeting performance expectations. However, a teachers TEM score alone does not determine whether or not he or she will be reelected. If a teacher scores a TEM 1 or 2, school leaders must ultimately make recommendations based on whether or not a teachers performance or potential to improve will lead to positive student outcomes. Further, please note that a teachers TEM score is not the only factor influencing reelection recommendations. As with prior school years, any teacher may be considered for non-reelection for criteria other than, or, in combination with TEM scores including 1) incompetence, 2) inefficiency, 3) neglect of duty, 4) unprofessional or improper conduct, and 5) insubordination. As with previous years, teachers who are recommended for non-reelection based on TEM scores or for any other reason will have their files reviewed by district personnel before a final determination is made. Teachers who are reelected in May will remain with the district for the 2012-13 school year. As with previous years, teachers who are recommended for non-reelection based on TEM scores or for any other reason will have their files reviewed by district personnel before a final determination is made. Teachers with tenure are guaranteed due process. If teachers are recommended for non-reelection for reasons other than or in addition to their TEM scores, Human Resources will make a final determination as with prior years. A flow chart of this process is provided below:

10

Teacher Effectiveness Measure (TEM): Summative Process

A timeline for this summative TEM process is provided below:

March March 5 - 9 March 30 April 30 May 15 June 15

Deadline for teachers and principals to select teachers Student Achievement option in Randa Tower software. Details forthcoming. Tripod student survey is administered for Stakeholder Perceptions TEM component. Deadline for completing all Observations. All teachers receive TEM Profiles. Individual summative meetings with schoollevel administrators begin. Deadline for principals to complete summative TEM meetings and submit all teacher reelection recommendations. Current-year Student Growth and Achievement data is available from the state. TEM Profiles are updated.

Please note that teachers must file any grievances based on evaluations between April 1st and May 14th. Teachers may only grieve errors related to the evaluation process and the accuracy of the evaluation data assigned to them. To learn more about the grievance process or to get forms to file a grievance, please contact the Memphis Education Association (MEA), 901-454-0966.

11