Dynamic Learning Maps (DLM)

Standard Setting and Performance

Levels

Presented at the November 2018 SBE Meeting

Title 14, Regulation 101, Section 5
5.0 Levels of Performance
• 5.1 There shall be four levels of student performance
relative to the Content Standards on the State’s General and
Alternate Assessments administered to students in grades 3
through 8 and 11 in English Language Arts/Literacy and
mathematics; to students in grades 5, 8, and 10 in Science;
and to students in grades 4 and 7 in Social Studies, and high
school students taking U.S. History (EOC).
• The cut points for Performance Levels 1 through 4 shall
be determined by the Department with the consent of the
State Board of Education, using advice from a standard
setting body. The standard setting body shall utilize a
proven method for setting standards on test instruments
that utilizes student assessment data in making the
recommendation.

2
 Advanced Organizer
 Review the following:
– Structure of Dynamic Learning Maps (DLM)
– Dynamic Learning Map’s 4 Levels of
Performance
– Background, methodology and procedures of
DLM Standard Setting
– Final DLM Cut Points
– Delaware Impact Data
 Questions
 Consent of the DLM performance levels and
methodology requested from State Board of
Education

3
Delaware’s Alternate Assessment
• Delaware, along with other states, were advised by USED
that Alternate Assessments with grade band standards would
no longer pass Peer Review.
• Delaware joined the Dynamic Learning Maps (DLM)
consortium of which their ELA, Mathematics and Science
assessments were aligned to Delaware’s standards.
• DLM’s assessments are adaptive; Delaware administers the
year-end model.
• Last fall, over 400 teachers and administrators participated in
training on the standards, instruction and the new
assessment.
• Delaware’s Alternate Assessment is designed to measure
what students with the most significant cognitive disabilities
know and are able to do.

4
Structure of Dynamic Learning Maps
• Essential Elements (EE)
• Linkage Levels (Targets)
• Nodes in the learning maps represent:
– Knowledge
– Skills
– Understanding
– Foundational Skills
• Includes multiple and alternate pathways by
which students may demonstrate their
understanding of content knowledge and
skills

5
 DLM Standard Setting

• Standard setting was conducted in 2015

for English Language Arts and
Mathematics and for Science in 2016
• Three cut points are set to classify student
performance into four levels
• The policy performance level descriptors
were established by state partners in 2014
and used in standard setting

6
 DLM Policy Performance Levels
DLM Performance Performance Level Descriptors
Levels
Emerging (E) The student demonstrates emerging understanding of
and ability to apply content knowledge and skills
represented by the Essential Elements.
Approaching (AP) Student’s understanding of and ability to apply targeted
content knowledge and skills represented by the
Essential Elements is approaching the target.
Target (T) The student’s understanding of and ability to apply
content knowledge and skills represented by the
Essential Elements is at target.
Advanced (AD) The student demonstrates advanced understanding of
and ability to apply content knowledge and skills
represented by the Essential Elements.
DE Performance Levels
PL1: A student’s performance in the
tested domain shows minimal
understanding and evidence of an
inability to apply the fundamental skills
and knowledge articulated in the DE
Content Standards.
PL2: A student’s performance in the
tested domain shows a partial or
incomplete understanding of the
fundamental skills and knowledge
articulated in the DE Content Standards.
PL3: A student’s performance in the
tested domain indicates an
understanding of the fundamental skills
and knowledge articulated in the DE
Content Standards.
PL4: A student’s performance in the
tested domain goes well beyond the
fundamental skills and knowledge
required for students to be at adequate
understanding.
7
Performance Level Descriptors (PLDs)

• The by-grade PLDs were developed

during standard setting
• Panelists drafted lists of knowledge and
skills and determined specific expectations
at each performance level
• DLM content team used the lists to
develop the grade-specific descriptions for
each level
• DLM state partners reviewed and provided
feedback on the draft PLDs
8
 Standard Setting Panels
• 54 panelists participated in standard setting for
ELA & Math, among them
– 74% were classroom teachers
– Average years of teaching experience:
• ELA = 16.6 years
• Math = 20.6 years
• Students with cognitive disabilities = 15.7 years
• 32 panelist participated in standard setting for
Science, among them
– 72% were classroom teachers
– Average years of teaching experience:
• Science = 13.2 years
• Students with cognitive disabilities = 14.3 years

9
 Methodology

• The Condensed Mastery Profile method

was used in standard setting
– This method is based on holistic standard-
setting approach, such as body of work for
direct writing and profile approach for
performance-based assessments
– In the process, panels reviewed ordered
student profiles of skill mastery to set cut
points based on the total number of skills
mastered.

10
Procedures for Standard Setting
1. Determine mastery and develop profiles
a. Determine linkage level mastery
b. Develop profiles of EE/linkage level mastery
2. Profile selection
3. Panel profile ratings
a. Range finding
b. Pinpointing
4. Statistical analysis of panelist ratings
5. Review of impact data
6. Adjust cut points for cross-grade
consistency

11
 Final Cut Points for ELA
Grade Emerging/ Approaching/ Target/ Total Number
Approaching Target Advanced Linkage Levels
3
36 50 71 80
4
38 57 75 85
5
35 53 76 85
6
27 46 65 80
7
27 52 73 90
8
23 48 72 85
9
20 48 68 85
10
17 47 72 85
11
18 47 70 85

12
 Final Cut Points for Math
Grade Emerging/ Approaching/ Target/ Total Number
Approaching Target Advanced Linkage Levels

3 12 21 37 55

4 20 30 56 80

5 15 32 48 75

6 13 28 38 55

7 19 37 53 70

8 17 40 53 70

9 10 21 33 40

10 8 21 36 45

11 8 18 38 45

13
 Final Cut Points for Science
Emerging/ Approaching/ Target/ Total Number
Grade Linkage Levels
Approaching Target Advanced

4 9 15 21 27

5 10 17 25 27

6 9 15 21 27

8 10 16 23 27

9-12 8 16 23 27

Biology 9 15 22 30

14
Alternate Assessment
ELA, Mathematics & Science
Grades 3 to 8 and 10

15
Alternate Assessment
ELA Proficiency 2018

16
Alternate Assessment
Math Proficiency 2018

17
 Alternate Assessment
Science Proficiency 2018

18
Questions

19