Professional Documents
Culture Documents
Introduction
This evaluation plan describes the procedures that will be used to evaluate
Workstation is a virtual machine software suite for x86 or x86-64 computers and it allows
one physical machine to run multiple operating systems simultaneously. Currently, the
VMware Workstation to provide its computer science students remote access to multiple
virtual operating environments for distance learning. The document begins with
evaluation and the audiences with whom the evaluation results will be shared. Next, the
intended decisions based on the evaluation results and questions to be addressed during
the evaluation process are described. In addition, the proposed methods and
instrumentation, the selected sample and inherent limitations, logistics and timeline are
explained including who, where, when, and how will be doing what. The final section in
This evaluation plan has been prepared by Lifang Chang, a graduate student in the
program of Teaching, learning and Technology at Lehigh University. The plan has been
informed by discussions with the client, Dr. Jorge Diaz-Herrera, Dean of the College of
Computing and Information sciences, the supervising instructor, Dr. Tonya Bevedare
Background
1
New York’s Rochester Institute of Technology (RIT), with an enrollment of more
than 15,000 students, is known for its professional, science and technology programs.
six departments in the Golisano College of Computing and Information Sciences at RIT.
The department of NSSA has five computing labs that serve nearly 400 students, many of
whom are working and can’t be on campus every day. The IT (Information Technology)
administrators and managers chose VMware virtualization software to provide a way for
students to remotely access the multiple operating environments they normally work with
in the lab.
Windows and Linux) and configurations to multiple students with a single server. Using
available virtual machine rather than having to be physically present in the lab to log on
to individual systems.
As shown in the following screen capture of the system interface, VMM can run
2
(P1)
(P2)
3
(P3)
and also can support a virtual machine running Windows XP on a Windows Vista host
(P4).
(P4)
The VMW software suite is distributed by electric download from the VMware
Purposes
4
The purpose of this evaluation is to collect data which will provide decision
makers at the College of Computing and Information Science in RIT with the timely,
effectiveness from the target audience of students using the virtualization software.
of VMware Workstation initiative into the overall teaching and learning environment,
with a goal of providing information that can be used to improve the future
Audiences
The primary audiences for the evaluation will be the dean council including the
college dean, Dr. Jorge Diaz-Herrera and other 16 council members. Important secondary
audiences include supporting faculties and staff of the VMware Workstation operating
The evaluation will be used to inform two major decisions to be made by the dean
council and supporting faculties and staff. Also, most of these decisions will be made
regardless of the quantity and quality of information available to the decision makers.
5
2. What techniques and strategies should be used for effectively integrating
utilization?
6. What techniques and strategies should be used for integrating the VMware
College?
In order to have the information needed to guide the types of decisions noted
• Does the student think that the general skills learned with the VMware
career plans?
6
B. What are reactions of the faulty and computing labs’ staff to the VMware
Workstation system?
C. How can the department of NSSA best share its experience with VMware
D. What would the NSSA department like to work with the other VMware
virtualization technologies?
Methods
Couple of descriptive and interpretive methodologies will be used to assist the evaluators
in gathering data to answer the questions listed above as well as other questions that will
surely revealed during the evaluation time. The data collection methods include:
1. Interviews with faculties who have integrated VMware Workstation with their
curriculums.
5. Expert review of the VMware Workstation software from three types of experts
The primary source of data will be collected through interviews with faculties and
technology staff in the department. The evaluator will go to the computing lab making
7
virtualization deployment environment, networking security, students’ academic
performance and open-ended questions (see Appendix A). Information will also be
gathered via a questionnaire what will be given to students (see Appendix B). The
One final method of gathering information will be expert review. This evaluation
requires three different types of experts in content, usability, and distance learning area.
Ideally, an expert who can cover three different areas at the same time is preferred.
The relationships among these five methods and the evaluation questions
identified in the previous section are described in the evaluation matrix that appears
below.
Method What are What are How can the What would the
student reactions of department best NSSA
reactions to the faulty and share its experience department like
using VMM? computing with other to work with the
labs’ staff to departments? other VMware
the VMW? technologies?
Faculty
Interview X X
Staff
Interview X X X
Lab
Observation X X
Student
Questionnaire X X
Expert
Review X X
Sample
Data will be gathered from several sources during the evaluation. The participants
• 25 – 30 students who has used VMware software from the NSSA department.
The participants in the evaluation include both undergraduate and graduate students, and
8
also represent a variety of age groups. The students are randomly selected with female
and male.
Information Technology and Distance learning. Accordingly, they will focus on the
• At least 3 faculties who has used VMware software with their curriculums.
Instrumentation
will complete after they have used the VMware virtualization software. The questionnaire
will be administered during the last ten minutes of one class period. Moreover, the user
• A protocol for the User Interviews that will be conducted with selected faculties
• A protocol for the User Interviews that will be conducted with selected
computing lab staff who have supported the VMware Workstation operating environment.
• A description of the in-lab observation that will record student and instructor
9
• Brief resumes of six experts from whom a panel of three will be selected to
All instruments are subject to major revisions, and should only be considered as
rough drafts of the actual instruments to be developed and refined during the real
evaluation.
Limitations
limitations on the interpretation and utility of this evaluation. First, many of the
participants in this evaluation study are students with a wide range of technology skills.
Different levels of technology skills bring up different sets of needs and expectations
with respect to the technology itself and the context of the operating systems. Therefore,
the results from this evaluation will be affected by the background of the participating
students. On the other hand, a relatively small sample of students is also a limitation.
Students may have varying levels of interest in the topic of networking security,
interests and future job plans. It can be anticipated that randomly selected students will
Second, only one virtualization solution will be evaluated. The results cannot be
installs “Remote Laboratory Emulation Systems” on the computing lab. The evaluation
results may not be appropriate for the future ones that will be developed.
Third, the data for this evaluation will be collected over a brief period of time
during the Summer Semester of 2008. Given the brevity of this evaluation, it will be
especially difficult to find out how effective the software are in influencing transfer to
10
teaching and technology practices in the colleges. Consequently, longer term evaluations
Fourth, the evaluator, Lifang Chang, is a novice undertaking this evaluation for
the first time. Although she will be supervised by an experienced evaluator- Dr. Tonya
Bevedare Amankwatia, it is still a student project and the results should be viewed from
that perspective.
Couple of people will participate in this evaluation, including faculty and graduate
students from Lehigh University. Their roles and responsibilities are displayed in the
following table. Roles and responsibilities for the Rochester Institute of Technology
The majority of the evaluation work will take place at Rochester Institute of
Technology. The College of computing and Information Science has agreed to sponsor
11
the evaluation project at Lehigh University. Space and equipment for the IT graduate
This evaluation plan will be revised based on an earlier draft prepared in April
2008. The final evaluation plan will be reviewed, revised, and approved by May 2008.
The initial data collection phase of the evaluation itself will commence June, 2008. The
first interim report will be delivered by July 1, 2008. The final report of this evaluation
Budget
Some parts of the budget are hypothetical since resources are not available to pay
for this evaluation. The detailed proposed budget is showed in the following table.
Item 4/15/08~7/10/08
Duplication
12
Total Duplication $150
Misc.
Administration
Participation
Incentives $400
Evaluator Travel $200
Total Misc.
Administration $600
Total
Evaluation Budget $4202.5
13
Appendix A
Note: This protocol is represented in this proposal as an example of the type of tool we
might use in the VMware Workstation evaluation project.
2. What types of assignments and lab activities do you use VMware for (e.g.,
creating virtual local networks, programming exercises, etc.):
a. on campus?
b. off campus?
c. Is it necessary to use VMware system for these activities or is the use of virtual
system optional?
4. How could you provide help when students work on VMware system?
a. on campus?
b. off campus?
… … …
[Continued…]
14
Appendix B
Note: This survey is represented in this proposal as an example of the type of tool we
might use in the VMware Workstation evaluation project.
1. Participate description:
a. Undergraduate student
b. MS student
c. PhD. student
d. Female
e. Male
… ….
[Continued….]
15