EDET 755: Evaluations Group 1: Ron McGinty, Janessa Senn, Barbara Solomon, Nathan Winters Introduction: The following

site evaluations represent a collaborative effort examining two online courses available for review through the MIT Open Courseware website. While these two course are unique based on their respective disciplines, both courses serve to provide insight into the importance of creating and supporting effective online instruction. The date of these courses is significant as they reveal the evolution of pedagogical practices based on educational theory. While critiquing each course, we tried to assess the strengths and weaknesses of the delivery system realizing the human element often determines success within the educational arena.

Evaluation 1 Site: MIT Open Courseware Course Title: War & American Society Evaluator(s): Ron Course URL: Date of Course: McGinty, Nathan http://ocw.mit.edu/courses/history/21h- Fall 2002 Winters 223-war-american-society-fall2002/index.htm 1. Course Overview and Introduction Strengths: Weaknesses: The course overview provides a brief The syllabus provides a structure for the introduction and explains the course course; however, it does not provide a requirements. Students are able to access clearly stated purpose. This course fails to a calendar of events and find information take advantage of the enhance about supplemental readings communication features available through easily. Reading materials are available online instructional platforms or online through the class website or by Management Learning Systems (MLS). The handout provided in class. Assignments professor does not provide a selfare accessible through the class website in introduction, nor does he require students the form of PDF files. to introduce themselves to the class online. The introduction does not provide a list of minimum technical skills needed for success in the course and netiquette is not mentioned in the course documents. 2. Learning Objectives Strengths: Weaknesses: Instructional topics/units are provided with Learning objectives are not clearly stated a systematic timeline. Students are able to in the course documents. Without clearly prepare for class discussions and readings defined objectives, it is reasonable to

based on the class calendar. Students should be able to examine how Americans have told stories of modern war in history, literature, and pop culture.

assume the learning objectives not measureable. Learning objectives are not written from the students perspective. Learning objectives are not clearly designed and instructions are not provided to students on how to meet the learning objectives. 3. Assessment and Measurement Strengths: Weaknesses: This course requires class participation, The professor does not provide systematic four papers and a presentation. Class scoring rubrics for the papers or the participation accounts for 15% of the presentations. The professor does grade. The first paper serves as a nonprovide the learner with guidelines for the graded formative assessment providing presentation. Class participation requires feedback to the learner. The remaining in class attendance and does make use of three papers are summative in nature and a Learning Management System. Scoring account for a 75% of the students guidelines are not available for grade. The second paper is a reflective determining an A, B, C, etc. exercise based on course activities and resources. The third paper requires the learner to analyze a historical figure and the fourth paper is a reflective exercise analyzing the Vietnam Era. Finally, students are expected to facilitate a class discussion for the final assessment worth 10% of their grade. Each assignment is sequences, varied, and integral to the learning process. Assessments are easily accessible online. 4. Instructional Materials Strengths: Weaknesses: All readings are available online and The purpose of instructional materials is through the university library. Films are unclear. Direction for the use of materials reserved through the Film Office of the for learning activities should be enhanced. Comparative Media Studies Program and local video stores. The syllabus provides a systematic break down of each instructional unit with specific readings and films. Proper citations of each reading and film as well as additional resources are available, thus extending the learning process. Materials are comprehensive representing a variety of time frames demonstrating the importance of pop

culture. 5. Learner Interaction and Engagement Strengths: Weaknesses: Students are expected to participate in a Learner engagement can be increased variety of situations. First, students are through peer collaboration available expected to take an active role in class through a Learner Management System discussion. Second, student presentations (LMS). Instructional feedback is limited, as involve student-centered learning and it is not fully defined in the syllabus. require a high level of student engagement. Student interaction occurs in several forms student to text, student to film, student to student, and student to teacher. 6. Course Navigation and Technology Strengths: Weaknesses: The course navigation is easy to The course lacks a variety of resources understand. There is a static course menu available to online courses. For instance, bar at the left of each page, which acts as a the site does not take full advantage of guide for the student. The course site has audio, video, and graphics. The majority of a clean feel to it. There seems to be little resources available are simply text files visual clutter that could potentially saved as the Adobe PDF format, books, increase cognitive load. Many site and videos from the library. The course operations occur through the use of the lacks collaborative electronic resources JavaScript language. such as discussion threads, message boards, emails, etc. 7. Learner Support Strengths: Weaknesses: The site offers a downloadable version of It is not clear how to contact the the materials for offline access. The site instructor independently of the site clearly identifies a contact portal from the administrator. There are no email top toolbar for questions and concerns. addresses or phone numbers listed for The assignments have clear instructions on support. The contact us portal appears the expectations and methods for to be the only method of support completion. 8. Accessibility Strengths: Weaknesses: A student using a screen reader should The top horizontal menu bar may cause have few problems navigating the actual some issue with screen readers, as there is course content. There is no flash content no Skip Navigation option. within the site, which would cause any issues. There is Alt. text present for all images. 9. Course Administration

Strengths: As with our student view, we have limited insight to course administration. Due to the cleanliness of the MIT Open Course Ware site, the administration functionality would seem streamlined and easy to accomplish.

Weaknesses: Based on the student view, course administration is basic due to the basic functionality of the site. There seem to be no plug-ins or add-ons to be configured for this particular LMS.

War & American Society Summary Strengths: The scope and sequence of the course is logical, and assignments are spaced throughout the semester adequately. Assignments are varied and require a high level of engagement on the part of the students. The formal assessment tasks involve both formative and summative assessments. Moreover, students are provided information concerning how their grade will be calculated; however, additional rubrics would be helpful. Navigation of the course site is relatively seamless and easy to understand. The course menu bar on the left is consistently present throughout the course and provides a linkage to various sections in the course. The design of the site is mostly clutter-free which allows a user to have minimal distractions. The technologies used allow a diverse audience to participate. There is seemingly little in the way for the navigation of virtually any browser or platform, aside from the presence of JavaScript. In terms of user support, the course offers a downloadable hard-copy of the course, which can be accessed offline. There is a contact portal clearly identified which allows a user to contact support technicians. In addition, this course is accessible to most persons of disabilities who require assistive technologies. There are two lectures per week of ninety minutes. A student must actively participate in discussion in order to fulfill all requirements for the course. This seems like a fair amount of time given for this type of course. Weaknesses: As the Internet is a vast resource, technology and multimedia resources could be better used for this course. There is a lack of resources aside from the text and lecture format. The course represents first generation online instruction common in the late 1990s as students are primarily provided the text and assignments online. As defined in chapter ten of our text, fully integrated online courses should be open, distributed, dynamic, globally accessible, filtered, interactive, and archival in nature (p. 248). This is not evident in the current course design. Furthermore, this course relies primarily on

teacher-to-student interaction common in traditional classroom environments. Studentto-student interaction could be improved upon by capitalizing on the functions provided through Learning Management Sites such as discussion boards, chat rooms, and web conferencing. Taking full advantage of mixed modes would optimize learning for this course. Reflection: The online course has very basic functionality and lacks meaningful contact information for the instructor. A student has no easy way to contact the instructor from the Internet. In addition, there is no Skip Navigation functionality to the site, which could be seen as a nuisance by those using screen readers to access the course. This course represents the initial transition from the traditional classroom to online instruction. Overtime, the instructor needs to adjust class assignments to reflect the student-centered approach to learning and increase the practicality of the online environment. Finally, the instructor needs to state with clarity the learning objectives for the benefit of the students and the alignment of the curriculum.

Evaluation 2 Site: MIT Open Courseware Course Title: The Film Experience Evaluator(s): Course URL: Date of Course: Janessa Senn, http://ocw.mit.edu/courses/literature/21l- Fall 2012 Barbara Solomon 011-the-film-experience-fall2012/index.htm 1. Course Overview and Introduction Strengths: Weaknesses: The course description provides students Though there is a clear navigation on with the purpose of the course and the where to find information, the syllabus syllabus describes the structure of the seems to be the only defining instructions course. Clear expectations for available. The forms of communication participation in discussions are stated in are not evident or easily located; there is the syllabus under various task no contact information for the professor requirements. or to find assistance. Neither course/institutional policies are enforced, nor is there a link to current policies. Prerequisite knowledge in the discipline and required competencies are not stated. The minimum technical skills expected of the student are implied/assumed; nothing clearly defines what a student needs to be able to do or of what programs he will need to have knowledge. There is no self-

introduction of the instructor, and students are not asked to introduce themselves. 2. Learning Objectives Strengths: Weaknesses: Though the learning objectives are not The learning objectives are not clearly clearly defined, they are made measurable stated from the perspective of a student. through the required participation on multiple activities throughout the course. The learning objectives are consistent with the course-level objectives that are described in the primary goals of the syllabus. The student instructions on how to meet the learning objective are adequate, but I think they need to be more detailed. For the level of the course, the learning objectives are appropriately designed. 3. Assessment and Measurement Strengths: Weaknesses: Assessments incorporated within this There are some specific and descriptive course measure the learning objectives. criteria provided for the evaluation of They support the course activities and students work and participation; a resources. The course grading policy is detailed rubric would be effective. Due to clearly explained in the syllabus. The the spacing of assignments, it seems that assessment instruments selected are students have unstructured opportunities sequenced, varied, and appropriate to the for self-reflection. student work being assessed. There are writing assessments, as well as formal and informal assessments dispersed throughout the duration of the course. When comparing the suggested writing topics to the course schedule, the assignments include application-based assignments; students take what they learn from the lecture and apply their knowledge to a chosen writing topic. 4. Instructional Materials Strengths: Weaknesses: The instructional materials contribute to The instructional materials are not current; the achievement of the stated course and however, they are appropriate and the learning objectives of the unit of study. necessary for the content of the course, Both the purpose of instructional materials which analyzes the progression of film.

and how they are to be used for learning activities are clearly explained. All resources and materials used in the course are appropriately cited; this course mostly deals with movies and the cited information is located in Films and Readings. Even though this course focuses around directors (there are many), the required and supplemental films present a variety of perspectives on the course content. Furthermore, there is a clear explanation of required and optional materials. The instructional materials are easily accessible and easy to use for the student; everything is a click away. 5. Learner Interaction and Engagement Strengths: Weaknesses: The earning activities promote the It is unclear whether the learning activities achievement of the stated learning provide opportunities for interaction that objectives. Students engage with course support active learning; based on analysis content in a variety of ways: watching of the tasks; it appears that interaction is films, viewing and participating in lectures, limited. The instructor does not provide a reading selected readings, writing about plan for classroom response time and them, and participating in other feedback on assignments. The assessments. This variety reflects the requirements for student interaction are multiple learning styles of the students. not clearly articulated because there is no expected student interaction. In addition to this, there are no opportunities or tools provided to encourage student-student collaboration. Interaction is not essential in this course. If anything, faculty-tostudent interaction is necessary for the submission and feedback of assignments. Other than this, interaction is not encouraged. 6. Course Navigation and Technology Strengths: Weaknesses: Navigation throughout the online The course technology does not components of the course is logical, incorporate tools for the promotion of consistent, and efficient. Access to course active discussion, reflection and idea content functions as expected. Assigned sharing between and among students. readings, video clips and film screenings support the learning objectives. The course takes excellent advantage of public

domain media resources and utilizes varied technologies for the playback of video clips. Students can readily access the technologies required in the course. 7. Learner Support Strengths: Weaknesses: The MIT Open Courseware site offers The course instructions do not articulate general help related to downloading and or link to a clear description of the using course materials via a Technology technical support offered; nor do they FAQ link. The course home page provides indicate the institutions accessibility a general contact link that provides an policies and services. Further, course online form request for technical help. instructions do not explain how or if academic support services assist students who require help. Nor do the course instructions detail how students can access the services. 8. Accessibility Strengths: Weaknesses: The course design facilitates readability The course does not demonstrate a and minimizes distractions. White serves commitment to accessibility for all as the background color with contrasting students. The course employs only black font. The active menu choice and minimal and incomplete alternatives for hyperlinks are colored red for increased audio/visual lecture content. The course emphasis and to aid in navigation. Tables does not employ accessible technologies with shaded rows enhance the nor does it provide guidance on how to organization and readability of the text obtain the use of assistive heavy pages. technologies. Graphics are un-captioned. Video snips/films are not closedcaptioned. The course does not incorporate the principles of Universal Design for Learning (UDL) and is not consistent with Web Content Accessibility Guidelines (WCAG). 9. Course Administration Strengths: Weaknesses: The courses topic format design spans an The course design does not appear to accelerated thirteen-week format. The include a grade book. It is unclear how amount of time students will spend students will gain access to their earned viewing lectures, watching films and grades for completed assignments. writing essays is reasonable and accessible for a part-time student. The three required writing assignments and one oral presentation are reasonable for the

courses 13 week accelerated timeframe. The course grading structure includes appropriate weights for activity categories. The Film Experience Summary Strengths: Organization and expectation are strengths of this course. The instructor provides a detailed schedule of assignments and activities within a given time frame. Furthermore, the instructor includes specifications about each of the assignments and maintains high expectations that are in correlation to the learning objectives through a variety of assessments. The instructional materials are appropriate and comprehensive to achieve stated course objectives and learning outcomes. Navigation of the course site is logical and consistent for all content areas. Links to external resources such as reading materials and online media clips are active and current. Many technologies and media formats are compatible with the course curriculum. The course site is constructed with simplicity and readability in mind. A technology FAQ is available via the course Help link as well as an online form to request more specific help. Course requirements are varied and appropriate for the accelerated pace of this course. Weaknesses: Overall, the design and introduction of this course is fairly vague. This course provides little to no insight into the design of the course in terms of available instructions, contact information, institution policies, required knowledge and skills. There is a strong sense of anonymity because the teacher never formally introduces himself and students are not expected to interact with each other. This course is structured around learner-centered curriculum. The lack of interaction may lead to a lack of student motivation in this course. The course is devoid of tools designed to support active collaboration, the sharing of ideas or submission of questions. It is unclear how students will gauge their progression in the course because a grade book tool is not apparent. Neither technical specifications nor guidelines supporting the access of media files are addressed on the course site. This is a surprising oversight as the vast majority of the required content is video based. In addition, the course does not demonstrate a commitment to accessibility for all students. Information related to obtaining assistive technologies is absent.

Reflection: The instructors strong organization of lessons and inclusion of detailed assignments are the definite strengths of this course. Unfortunately, the design of the course, though online, is more closely aligned to independent learning than a fully integrated online course. Upon close evaluation of the course design, essential components appear to be missing from the learner environment. Most obvious is the absence of collaboration and discussion tools. The syllabus is detailed in respect to class lectures and video viewing requirements, however lacks any pretense of encouraging student-student discourse. This course has the potential to become a model for effective online learning. The materials (films and video clips) are stable, the content has a global appeal, and the technology necessary for review of the content is practical. The application of online course standards and alignment to accepted online instruction models would re-vitalize this outdated course.