26/02/2019 Understanding Review Response Types - Eli Review

Eli Review Learning Resources

Understanding Review Response Types
This document offers an in-depth breakdown of the different response types instructors can add to a Review Task. There are a couple of important points to keep in
mind when thinking about review task response types:

Every review must have at least one response type.

There is no limit on the number of response types a review can have.

To learn more about review tasks in general, consult either the Instructor User Guide or the Student User Guide for more details.

Trait Identification
Rating Scales
Likert Scales
Contextual Comments
Final Comment

Trait Identification
In a trait identification response, the prompt names a trait that you wish reviewers to identify in their peers’ work. You might, for example, create a prompt that says:
“The article summary begins with a full citation of the article.” Reviewers respond by checking a box to indicate if the trait is present or absent in the draft they are
reviewing.

Trait identification items offer writers formative feedback the helps them prioritize next steps for revision. The absence of required elements lets writers and
instructors know what they might work on in the next draft. It also helps to indicate where they are meeting expectations when important traits are present.

Trait identification elements are also aggregated for the whole class, which helps a teacher to see what kinds of issues to prioritize in discussion with the larger
group. If your review results page shows that 95% of your students have a full citation in their article summary, for instance, you know you can likely move on to
work on something that a smaller percentage of students have mastered or prioritized in their drafts.

When creating a review task, adding a Trait Identification set will prompt instructors to create a prompt and add traits to the set. Traits can easily be
edited or rearranged.

When students respond to the review, they'll see the list of traits and can check each one that they observe as they read.

The instructor's review report will show responses to the trait set at the class level. The review report for a student will show that student's trait
identification score relative to the rest of the class.

https://elireview.com/learn/tutorials/instructors/review-response-types/ 1/8
26/02/2019 Understanding Review Response Types - Eli Review

Rating Scales
Scaled response items are questions that the teacher asks the reviewer to respond to on a scale. The prompt may frame the scale for students right in the question:
“On a scale of 1 to 5 with 1 being lowest and 5 highest, how confident are you that the article citation is in correct MLA format?”

Scaled response items are the most versatile, and they can be used to provide feedback ranging from formal to informal at just about any stage in the process. A
teacher asking students to write limericks to practice meter and rhyme-scheme for instance might frame a number of scaled items to vote for a fun round of “Best in
Class Awards…” in a number of humorous categories: best rhyme, best meter, funniest, etc. Or, you could use scaled items to replicate criteria found on standardized
tests such as the ACT Writing Exam in order to do test prep.

Rating scales are aggregated for the whole group so the teacher can see class averages and compare the results of individual students with those of the whole group.
Eli identifies “peer exemplars” by showing the instructor those students whose work scores highest on scaled items. This provides you with an instant set of possible
models and peer scaffolding opportunities. You can ask the large group to look at an example ranked highly, for instance, and discuss the choices the writer made
that have caused it to be a strong draft in a particular area. Best of all, these are examples nominated by students. So even when the teacher disagrees, revealing the
trend creates a “teachable moment.”

Keep in mind about scales:

You can create ratings with anywhere from 2 to 26 stars.

You can add as many rating scales to a review as you need.
Each question to which you want a distinct answer should be created as a new rating scale.
You can enable the “Ask students to explain their response” and have students send a message to the writers they’re reviewing about the response and how
they might address any issues they found.

When adding a rating scale to a review, instructors are asked to create a prompt for the scale, to select the number of stars on the scale, and have the
option to ask reviewers to explain their rating.

When students respond to a review with a rating scale, they select the rating they feel appropriate and, if required, are prompted to explain that rating.

In the review report, the class average is displayed with the prompt and, if looking at an individual student, their rating is displayed along with
explanations from reviewers.

Likert Scales
A likert scale functions much like a rating scale but with a few important distinctions. In this case, it’s a scale in which case the prompt is typically framed as a
declarative statement to which the respondent is asked to select one from a set of pre-defined choices (strongly agree, agree, etc). Options presented to respondents
(called “likert items”) are usually balanced on either side of a neutral item to give the set a balanced spectrum.

Keep in mind about likert scales:

You can add as many likert scales to a review as you need.

Each question to which you want a distinct answer should be created as a _separate_ likert scale.
Each scale can have as many items as necessary.
Items need not be on a scale – you are free to customize the response items however you need.
You can enable the “Ask students to explain their response” and have students send a message to the writers they’re reviewing about the response and how
they might address any issues they found.

https://elireview.com/learn/tutorials/instructors/review-response-types/ 2/8
26/02/2019 Understanding Review Response Types - Eli Review

As with rating scales, when adding a likert scale, instructors provide a prompt and can ask reviewers to explain their response, but the responses are
from a list of choices.

When reviewers respond to a review with a likert scale, they choose one of the options on the scale and are prompted to explain their response.

The review report includes a table with a breakdown of class responses to the likert scale along with explanations provided by reviewers.

Contextual Comments
Comments are open-ended text responses from reviewers, similar to what a reviewer might write in the margin or at the end of a draft. Comments can be attached to
specific passages in a text for assignments that have been typed or pasted into the Eli editor, or they can be made as global suggestions to the writer. Instructors can
prompt reviewers to offer specific types of feedback with a comment response if they like. For instance, a teacher might say “please offer comments on claims that
could use more evidence to strengthen the writers’ overall argument.”

Comments are often the most valuable bits of information for the writer, because they can offer concrete revision suggestions as well as confirmation that they are on
the right track. With Eli, all comments from all reviewers are collected in one place, making it easy for a writer to strategize a revision plan.

Instructors can view comments from reviewers as they come in during a review. In a face-to-face classroom, this is very helpful when you are trying to monitor
whether reviewers are attending to the appropriate criteria at the appropriate stage of the writing process. If an instructor sees early on that reviewers are focusing on
grammar and mechanics too early in the process, for instance, she can intervene and redirect their focus to higher-order concerns such as development and
arrangement.

Instructors can also endorse a comment by clicking a button. This allows the writer to see that the instructor thinks a particular comment is valuable, and it also
allows the reviewer to see that she has made what the instructor feels is a helpful response.

Keep in mind about contextual comments:

Unlike trait identification sets, rating and likert scales, you simply enable contextual comments by checking a box, and you only do this one time.
Reviewers can make as many comments as they wish. Their comments can be edited and deleted before being submitted to their reviewees.
Contextual comments can be tied to passages of texts:
if the texts being reviewed are writing tasks that were “Composed in Eli.”
students must highlight the passage before clicking the “Add Comment” button
after the review, writers can see the context of the comment by clicking the “View Comment in Context” link
Comments work best when they are short and concise. Students usually need to be instructed to review this way – the like to write lengthy comments.
Writers can rate the comments the get individually, and you can endorse comments individually too. These ratings help identify the best reviewers in your
class. They usually need to be encouraged to do this as well.

https://elireview.com/learn/tutorials/instructors/review-response-types/ 3/8
26/02/2019 Understanding Review Response Types - Eli Review

When enabling contextual comments, instructors have the option to add a prompt to guide reviewers on requirements for comments.

While responding, reviewers are shown the prompt with the option to add as many comments as they like. Students can highlight passages of text and
have comments anchored to them.

In the review report, instructors can see all feedback given to - or give by - a reviewer, including how the recipient rated it and whether or not it was
added to a revision plan.

Final Comment

https://elireview.com/learn/tutorials/instructors/review-response-types/ 4/8
26/02/2019 Understanding Review Response Types - Eli Review
Final comments function much like Contextual Comments, but in this case students may only leave a single comment. For assignments that include more than one
product – for example, a resume and a cover letter may be turned in for review at the same time – a Final Comment option gives students the ability to make a global
comment about both documents.

This is a simple text box for responding to all of the products in a holistic way. Teachers can use this response, too, to address broader issues of purpose and
audience. For instance, a prompt for a final comment to a resume and cover letter might ask “what are the most interesting qualifications an employer might choose
to highlight or ask questions about in an interview with the writer?”

Keep in mind about Final Comments:

Like Contextual Comments, Final Comments are a single feature that is enabled by checking the appropriate box.
You can add a prompt to the final comment that will guide how students respond.
Final Comments cannot be tied to a passage of text, unlike Contextual Comments.
Final Comments will be differentiated from Contextual Comments in the review report with a distinctive label.
You many endorse final comments just like Contextual Comments.

Enabling the final comment response type looks similar to contextual comments, since both ask for only a prompt, but reviewers will see the difference.

When reviewers are prompted for a final comment, they see only one box in which to compose a response.

In review reports, final comments look and act much like contextual comments (ratings, revision plans, endorsements) but are denoted by "Final
Comment".

https://elireview.com/learn/tutorials/instructors/review-response-types/ 5/8
26/02/2019 Understanding Review Response Types - Eli Review

https://elireview.com/learn/tutorials/instructors/review-response-types/ 6/8
26/02/2019 Understanding Review Response Types - Eli Review

https://elireview.com/learn/tutorials/instructors/review-response-types/ 7/8
26/02/2019 Understanding Review Response Types - Eli Review

https://elireview.com/learn/tutorials/instructors/review-response-types/ 8/8