VMmark Bench Marking Guide 2.1-20110310

VMware VMmark Benchmarking Guide
VMmark 2.1 Revised March 10, 2011
EN-000006-03
You can find the most up-to-date technical documentation on the VMware Web site at: http://www.vmware.com/support/ The VMware Web site also provides the latest product updates. If you have comments about this documentation, submit your feedback to: docfeedback@vmware.com
Copyright 2007-2011 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at http://www.vmware.com/go/patents. VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies. Revision: 20110310
VMware, Inc. 3401 Hillview Ave. Palo Alto, CA 94304 www.vmware.com
VMware, Inc.
Contents
Overview
13
Why Virtualization? 13 Why a Multi-Host Virtual Machine Benchmark? 13 Intended Audience 14 Legal Notice 14
Preparation Checklists and Worksheets

Infrastructure Preparation 15 Virtual Machine Preparation 15 Standby Virtual Machine 15 Mail Server Virtual Machine 15 Olio Database Virtual Machine 16 Olio Web Virtual Machine 16 DVD Store 2 Database Virtual Machine DVD Store 2 Web Virtual Machine 16 Deploy Template Virtual Machine 16
15
16
What is the VMmark Benchmark? 19

Overview of VMmark 2.1 Benchmark 19 VMmark Benchmark Workloads 21 Mail Server Virtual Machine 21 Standby Virtual Machine 22 Web 2.0 Simulation 22 E-Commerce Simulation 22 Virtual Machine Cloning and Deployment 23 Dynamic Virtual Machine Relocation 23 Dynamic Storage Relocation 24 Automated Load Balancing 24 VMmark Client Systems 25 VMmark Harness 26 Contents of VMmark Kit 27 VMmark 2.1 Benchmark Scoring Methodology 28 Example One: Single-Tile Benchmark Scoring 29 Example Two: Multiple-Tile Benchmark Scoring 30 Reference Scores 32 VMmark Version Notes 33 VMmark 2.0 Vs. VMmark 2.1 33 VMmark 1.x Vs. VMmark 2.0 33 VMmark 1.1 Vs. VMmark 1.1.1 33 VMmark 1.0 Vs. VMmark 1.1 33 VMmark Run and Reporting Rules 33
VMmark Benchmark Requirements 35

VMmark Version and Settings Requirements 35 VMmark Hardware Requirements 35 Server Hardware Requirements 35 Server Storage Requirements 35
VMware, Inc.
Server Memory Requirements 35 Other Server Requirements 36 Client Hardware Requirements 36 Hardware Requirements for Physical Clients 36 Hardware Requirements for Virtual Clients 36 vCenter Hardware Requirements 37 Network Requirements 37 Network Hardware 37 Network Topology 37 VMmark Software Requirements 39 Workload Software Requirements 39 Client Software Requirements 39 Software Versions 40 Software License Summary 40 Free Software 40 Evaluation Software 40 Purchased Software 40
Preparing the Infrastructure for VMmark 2.1 Benchmark Tests
43
Create the Virtualization Infrastructure 43 Install vSphere vCenter 43 Install VMware ESX 43 Configure vCenter 44 Configure Time Synchronization on the ESX Hosts and the vCenter Server 44 Configure Time Synchronization for ESX Hosts 44 Configure Time Synchronization for the vCenter Server 44 Prepare the vSphere vCenter Server for the Deploy Infrastructure Operations 46 Create the VMmark Tiles 47 Create the Mail Server Virtual Machines 49 Create the Standby Virtual Machines 51 Create the Olio Database Virtual Machines 53 Create the Olio Web Virtual Machines 55 Create the DVD Store 2 Database Virtual Machines 57 Create the DVD Store 2 Web Virtual Machines 59
Configuring the Client Systems
61
Prepare the Client Machine 61 Prepare a Client Physical Machine 61 Prepare a Client Virtual Machine 61 Install Microsoft Windows Server on the Client Systems 63 Install Microsoft Windows Server on a Physical System 63 Install Microsoft Windows Server on a Virtual System 63 Install the VMware Tools Package in Microsoft Windows 64 Enable Time Synchronization in VMware Tools 65 Install Windows Updates 65 Configure Microsoft Windows on the Client Systems 66 Install Windows Server Components 66 Disable Data Execution Prevention (DEP) on the Client 67 Install or Update Windows PowerShell and vSphere PowerCLI 67 For Windows Server 2003 67 For Windows Server 2008 68 Install the General Software 69 Install and Configure the STAF Package 69 Install the STAF Package 69 Configure the Windows Server 2008 Firewall to Allow STAFProc.exe to Pass Through 69
4 VMware, Inc.
Contents
Configure STAF to Start Automatically 69 Create a Customized STAF.cfg File 71 Install the STAX Execution Engine 71 Install the STAF EVENT Service 71 Install a Java JRE 72 Install the Cygwin Environment 73 Install and Configure the VMmark Harness 75 Install the VMmark Harness Package 75 Configure the VMmark Harness 75 Edit the cygserver.conf File 79 Edit Registry Settings in the Client System 79 Automatically Apply the Registry Settings 79 Manually Apply the Registry Settings 80 Update the hosts File 81 Obtain Exchange Server 2007 and LoadGen 81 Configure Windows to Not Use a Proxy Server 82 Create Additional Client Systems 83 Copying the Disk Image 83 Cloning a Virtual Machine 83 Configuring Additional Clients Manually 83 Install and Configure the Workload Client Software 84 Install the Standby Client Software 84 Install the Olio Client Software 84 Install the DVD Store 2 Client Software 84 Install and Configure the Mail Server Client 84 Set the Clients Preferred Domain Name Server 84 Make the Client System a Domain Member 85 Install Exchange Server 2007 Tools 86 Install and Configure LoadGen on the Client 87 Backup the Microsoft Exchange Server 2007 Database 88
Running the VMmark Benchmark
91
Tile Placement 91 Running the VMmark Harness 91 Running Multiple-Tile Benchmarks 93 Configuring the VMmark Harness for Multiple-Tile Benchmarks 93 Starting Multiple-Tile Benchmarks 93 VMmark Results Files 94 Creating and Submitting a VMmark Full Disclosure Report 95 Configure the Prime Client to Log Into Virtualization Hosts Without a Password 95 Enable Reporting in the VMMARK2.CONFIG File 96 Create a Name-Mapping File 96 Generate an HTML-Formatted Table of Results and Ratios 97 Edit the disclosure.HTML File 97 Submit the Full Disclosure Report for Review 97 Running VMmark without STAXMonitor 98
Creating the Workload Virtual Machines 99

Create and Configure the Standby Virtual Machines 100 Create the Standby Virtual Machine 100 Install Microsoft Windows Server 2003 Enterprise Edition 101 Install the VMware Tools Package in Microsoft Windows 102 Install Windows Updates 103 Install and Configure the STAF Package 103 Install the STAF Package 103
VMware, Inc. 5
Create a Customized STAF.cfg File 104 Configure STAF to Start Automatically 104 Update the HOSTS File 104 Create and Configure the Mail Server Virtual Machines 105 Create the Mail Server Virtual Machine 105 Install Microsoft Windows Server 2008 Enterprise Edition 106 Install the VMware Tools Package in Microsoft Windows 107 Install Windows Updates 108 Create the Database and Backup Drives 108 Create a Second Virtual Disk 108 Create and Format the Database Partition 108 Create and Format the Backup Partition 109 Install and Configure the STAF Package 109 Install the STAF Package 109 Create a Customized STAF.cfg File 110 Configure the Windows Server 2008 Firewall to Allow STAFProc.exe to Pass Through 110 Update the hosts File 110 Install Windows Server Components and Windows Support Tools 110 Install Windows PowerShell 110 Install Windows Server Components 111 Configure Internet Explorer to Not Use a Proxy Server 111 Create and Configure the SLES Virtual Machines 112 Create the Virtual Machine 112 Install and Configure SUSE Linux Enterprise Server 11 113 Install the VMware Tools Package in the Linux Virtual Machine 114 Install and Configure the STAF Package 115 Install the STAF Package 115 Create a Customized STAF.cfg File 116 Set Open File Limits 116 Disable Persistent Network Adapter Name Rules 116 Configure the Linux System to Automatically Start STAFProc 116 Update the hosts File 116
Installing Applications in the Workload Virtual Machines

Prepare the Mail Server Virtual Machines 119 Clone the Mail Server Virtual Machines 119 Make the Mail Server Virtual Machine a Domain Controller 120 Install Microsoft Exchange Server 2007 121 Configure Microsoft Exchange Server 121 Update Exchange Server 2007 122 Increase the Transport Database Cache Size 122 Configure STAF to Start Automatically 123 Prepare the Standby Virtual Machines 124 Clone the Standby Virtual Machine 124 Prepare the Template for the Deploy Operation 124 Prepare the Olio Database Virtual Machines 125 Customize the SLES 11 Virtual Machine Template 125 Clone the SLES11 Template Virtual Machine 125 Customize the Virtual Machine for Olio Database 125 Create a Second Virtual Disk 125 Install the Software for the Olio Database Virtual Machines 126 Partition, Format, and Mount the Olio Database Disk 126 Install MySQL 126 Install Tomcat and Java 127
119
VMware, Inc.
Contents
Copy Files 127 Configure the Olio Database 128 Clone the Olio Database Virtual Machine 128 Prepare the Olio Web Virtual Machines 129 Customize the SLES 11 Virtual Machine Template 129 Clone the SLES11 Template Virtual Machine 129 Customize the Virtual Machine for Olio Web 129 Create a Second Virtual Disk 129 Install the Software for the Olio Web Virtual Machines 130 Partition, Format, and Mount the Olio Filestore Disk 130 Tune TCP/IP 130 Edit the /etc/HOSTNAME and /etc/hosts Files 130 Install Memcached 131 Copy the Fileloader 131 Install a 64-bit Java JRE 131 Install tc Server Application Server 132 Install Spring Framework 132 Install EclipseLink 132 Install MySQL Connector/J 133 Install JSTL 133 Configure tc Server 133 Clone the Olio Web Virtual Machine 134 Prepare the DVD Store 2 Database Virtual Machines 135 Customize the SLES 11 Virtual Machine Template 135 Clone the SLES11 Template Virtual Machine 135 Customize the Virtual Machine for DVD Store 2 Database 135 Create a Second Virtual Disk 135 Create a Third (Temporary) Virtual Disk 136 Install the Software for the DVD Store 2 Database Virtual Machines 136 Configure the Network and Set the Hostname 136 Partition, Format, and Mount the DVD Store 2 Database Disk 136 Partition, Format, and Mount the DVD Store 2 Temporary Disk 137 Install MySQL 137 Prepare the DVD Store 2 Database 138 Clone the DVD Store 2 Database Virtual Machine 139 Prepare the DVD Store 2 Web Virtual Machines 140 Customize the SLES 11 Virtual Machine Template 140 Clone the SLES11 Template Virtual Machine 140 Customize the Virtual Machine for DVD Store 2 Web 140 Install the Software for the DVD Store 2 Web Virtual Machines 140 Configure the Network 140 Install Apache and PHP 140 Untar, Edit, and Move the DVD Store 2 Files 141 Copy and Clone the DVD Store 2 Web Virtual Machine 141
Optional Configurations and Settings
143
Running a Subset of the Workloads (a Partial Tile) 143 Optional Windows System Settings 144 Disable the Manage Your Server Window 144 Set the System to Autologon 144 Disable the <Ctrl> <Alt> <Del> Prompt 144 Disable Password Protection on Resume 144 Disable Internet Explorer Enhanced Security 145 Disable the Shutdown Event Tracker 145
VMware, Inc. 7
Using a PVSCSI Virtual Device Driver 146 Configuring Network Settings in SLES 11 147
Troubleshooting
149
Workload Troubleshooting 149 Mailserver Troubleshooting 149 Error Validating State of Active Directory 149 Microsoft Exchange Load Generator -- Error 1722 149 Correcting Mail Server Drive Letters and Names 149 LoadGen Log File Errors 150 Mailserver Virtual Machines Produce No Results 150 No Exchange Window Visible 151 LoadGen Cant Connect to Active Directory 151 LoadGen Crashes If Exchange Server Transport Service Stops 151 Deleting the Microsoft Exchange Database 152 DVD Store 2 Troubleshooting 153 (504) Gateway Timeout Error 153 Miscellaneous Troubleshooting 154 Out of Bounds Error by Harness During Time Sync 154 Popup: Space Allocated for Logging PowerShell Events is Nearly Full 154 Infrastructure Operations Troubleshooting 154 STAF Complains About Trust Level 155 STAF is Unable to Connect to a Particular Server 155 The VMmark Harness Complains That Clients Are Out of Sync 155 VMware vSphere PowerCLI Snap-In Loads Slowly 155 PowerCLI Snap-In Takes About 20 Seconds to Load 155 PowerCLI Snap-In Takes About Two Minutes to Load 156 Error: File C:\VMmark2\tools\VMmarkPScli.ps1 cannot be loaded. 156 Networking Issues 157 Setting the Number of Ports on a Virtual Switch in ESX 157 Setting the Network Name 157 Setting Network to Connect at Power-on 157 Performing a Short Run 158 Testing a Single Failing Client-Workload Pair 159 The VMmark2ConfigChecker Script 160 Running the VMmark2ConfigChecker Script 160 The VMmark2ConfigChecker script is supplied as part of the VMmark kit (in the C:\VMmark2\tools directory). The script can be run in two modes: on a running ESX/ESXi host or on previously-collected reporter results. Both methods are described below. 160 Running the VMmark2ConfigChecker Script on an ESX/ESXi Host 160 Running the VMmark2ConfigChecker Script on Previously-Collected Results 160 Customizing the VMmark2ConfigChecker Script 161 Recovering Data from Abnormal Runs 162 Recovering Data From An Aborted Run 162 Recovering Data if the Harness Hangs 162 Manually Running the vSphere-get-config.sh Script 163
VMware, Inc.
Figures
Figure 1. Example Organization of a Virtual Environment 13 Figure 1-1. Conceptual Example of a Single Server Running One VMmark Tile (Eight Virtual Machines) 20 Figure 2-1. Sample VMmark Benchmark Network Configuration (Not Shown: vCenter Server, VMmark 2.1 Multi-Host Setup, Dedicated vMotion Network) 38 Figure 3-1. Overview of Virtual Machine Creation for Mail Server Workloads 50 Figure 3-2. Overview of Virtual Machine Creation for Standby Workloads 52 Figure 3-3. Overview of Virtual Machine Creation for Olio Database 54 Figure 3-4. Overview of Virtual Machine Creation for Olio Web 56 Figure 3-5. Overview of Virtual Machine Creation for DVD Store 2 Database 58 Figure 3-6. Overview of Virtual Machine Creation for DVD Store 2 Web 60
VMware, Inc.
10
VMware, Inc.
Tables
Table 1. Microsoft Windows Virtual Machine Worksheet 18 Table 2. Linux Virtual Machine Worksheet 18 Table 1-1. VMmark 2.1 Workload Summary 21 Table 1-2. VMmark Client Summary 25 Table 1-3. Individual VMmark Workload Metrics 28 Table 1-4. Example Reference System Workload Scores (Artificial Data) 29 Table 1-5. Single-Tile Example Test System Workload Scores (Artificial Data) 30 Table 1-6. Multiple-Tile Example Test System Workload Scores (Artificial Data) 31 Table 1-7. Reference System Workload Scores 32 Table 3-1. Registry Entries to Change 45 Table 3-2. Registry Entries to Change 45 Table 4-1. Registry Entry to Modify 80 Table 4-2. Registry Entries to Add 80 Table 4-3. Registry Entry to Modify 81 Table D-1. Mail Server Virtual Machine Disk Partitions 150
VMware, Inc.
11
12
VMware, Inc.
Overview
This document describes a virtual machine benchmarking utility called VMmark, it provides instructions for configuring a system for VMmark testing, it details the steps required to perform such a test, and it discusses the interpretation of the data acquired.
Why Virtualization?
Trends in computer hardware have led to the proliferation of powerful yet relatively inexpensive multiprocessor servers. In many cases, applications do not fully utilize these systems. As recent industry developments such as multi-core processors become commonplace, the degree of underutilization should increase. These realities have led to renewed interest in virtual machines for server consolidation. Virtual machine environments provide a software layer that enables users to create multiple independent virtual machines on the same physical server, as shown in Figure 1. Figure 1. Example Organization of a Virtual Environment
Virtual Machine
Virtual Machine
Virtualization Layer Physical Hardware
By running multiple virtual machines simultaneously, a physical server can be driven to much higher utilizations, albeit with some virtualization overhead. Although the underlying physical resources are shared, each virtual machine is fully isolated from other virtual machines and executes a separate operating system instance and separate application software.
Why a Multi-Host Virtual Machine Benchmark?

It is crucial for users to have meaningful and precise metrics in order to effectively compare the suitability and performance of different platforms for virtual environments. Traditional single-workload performance and scalability benchmarks for non-virtualized environments, such as those available through SPEC (see http://www.spec.org/) or TPC (see http://www.tpc.org), were developed with neither virtual machines nor
VMware, Inc.
13
server consolidation in mind. These metrics focus on achieving maximum system performance for a single workload by driving at least one of the underlying hardware resources to saturation. These types of benchmarks do not provide sufficient insight into the scalability of virtual environments supporting multiple simultaneous workloads. More recently, multi-workload server consolidation benchmarks, such as VMmark 1.1 and SPECvirt_sc2010, have been released. These benchmarks measure single-host performance in virtualized environments. However the rapid pace of innovation has quickly transformed typical server usage by enabling easier virtualization of bursty and heavy workloads, dynamic virtual machine relocation (vMotion), dynamic datastore relocation (storage vMotion), and the automation of many provisioning and administrative tasks across large-scale multi-host environments. In this paradigm, a significant portion of the stress on CPU, network, disk, and memory subsystems is generated by the underlying infrastructure operations. Load balancing across multiple hosts can also greatly affect application performance. Any relevant benchmarking methodology must still focus on user-centric application performance while accounting for the effects of this infrastructure activity on overall platform performance.
Intended Audience
This document is written for users with a relatively advanced understanding of system administration. However, familiarity with virtualization software and with benchmarking methodology is not assumed.
Legal Notice
This documentation contains information including but not limited to the installation and operation of the Software. Modifications, additions, deletions or other updates (Modifications) to the information may be incorporated in future releases. VMware, Inc., its affiliates or subsidiaries (VMware) are not responsible for any Modifications made to the published version of this documentation unless performed by VMware. All information is provided as is and is believed to be accurate at the time of publication. VMware shall not be liable for any damages arising out of or in connection with the information and recommended actions provided herein (if any), including direct, indirect, consequential damages, loss of business profits or special damages, even if VMware has been advised of the possibility of such damages.
14
VMware, Inc.
Many of the operations required to configure a VMmark Benchmark test system can be performed in parallel to save time (for example, while the operating system is being installed in one virtual machine, you can be installing applications in another). These checklists and worksheets are designed to help track which tasks have been completed and which remain to be done. The checklists and worksheets are highly abbreviated and are intended for use in conjunction with the detailed material elsewhere in the book. NOTE When this document is viewed electronically, clicking on the cross references (for example, on page 100) brings you to the cross-reference target.
Infrastructure Preparation
Install VMware vCenter Server (on page 43). Install VMware ESX/ESXi (on page 43). Configure vCenter (on page 44). Prepare the vCenter server (on page 46).
Virtual Machine Preparation

This section summarizes the preparation of the virtual machines used as VMmark 2.1 workloads. NOTE It can be useful to create one complete tile (i.e., tile 0) and make sure its working smoothly, then make backups of this working configuration before cloning the workload virtual machines to create the additional tiles. More information about this methodology can be found in Performing a Short Run on page 158.
Standby Virtual Machine

Create the standby virtual machine (on page 100). Install 32-bit Microsoft Windows Server 2003 (on page 101). Install VMware Tools (on page 102). Install and configure the STAF package (on page 103). Create clones of the standby virtual machine for additional tiles (on page 124).
Mail Server Virtual Machine

Create the mail server virtual machine (on page 105). Install 64-bit Microsoft Windows Server 2008 (on page 106). Create the database and backup drives (on page 108). Install VMware Tools (on page 107).
VMware, Inc. 15
Install and configure the STAF package (on page 109). Install Windows Server Components and Windows Support Tools (on page 110). Create clones of the mail server virtual machine (on page 119). Make the mail server virtual machine a domain controller (on page 120). Install Microsoft Exchange Server (on page 121). Configure Microsoft Exchange Server (on page 121). Update Microsoft Exchange Server (on page 122). Increase the transport database cache size (on page 122). Configure STAF to start automatically (on page 123). Repeat the previous five steps for each additional mail server virtual machine, starting with making the mail server virtual machine a domain controller (on page 120).
Olio Database Virtual Machine

Download and uncompress the Olio database template virtual machine (Step 1 through Step 3 on page 53). (Alternatively: Create the virtual machine from scratch (on page 112) and configure it (on page 125).) Add the virtual machine to your vSphere inventory (Step 4 and Step 5 on page 53). Create clones of the Olio database virtual machine for additional tiles (on page 128).
Olio Web Virtual Machine

Download and uncompress the Olio web template virtual machine (Step 1 through Step 3 on page 55). (Alternatively: Create the virtual machine from scratch (on page 112) and configure it (on page 129).) Add the virtual machine to your vSphere inventory (Step 4 and Step 5 on page 55). Create clones of the Olio web virtual machine for additional tiles (on page 134).
DVD Store 2 Database Virtual Machine

Download and uncompress the DVD Store 2 database template virtual machine (Step 1 through Step 3 on page 57). (Alternatively: Create the virtual machine from scratch (on page 112) and configure it (on page 135).) Add the virtual machine to your vSphere inventory (Step 4 and Step 5 on page 57). Create the 20GB temporary disk (on page 136). Prepare the DVD Store 2 database (on page 138). Create clones of the DVD Store 2 database virtual machine for additional tiles (on page 139).
DVD Store 2 Web Virtual Machine

Download and uncompress the DVD Store 2 web template virtual machine (Step 1 through Step 3 on page 59). (Alternatively: Create the virtual machine from scratch (on page 112) and configure it (on page 140).) Add the virtual machine to your vSphere inventory (Step 4 and Step 5 on page 59). Create clones of the DVD Store 2 database virtual machine for additional tiles (on page 141).
Deploy Template Virtual Machine

Clone the standby virtual machine as a deploy template (Step 1 in Prepare the Template for the Deploy Operation on page 124). Install an outdated copy of VMware Tools (beginning with Step 2 in Prepare the Template for the Deploy Operation on page 124).
16
VMware, Inc.
Client Preparation
This section summarizes the preparation of the VMmark 2.1 clients. Prepare the client machine (on page 61). Install Microsoft Windows Server (on page 63). Install Windows Server components (on page 66). Disable data execution prevention (on page 67). Install or update Windows PowerShell and vSphere PowerCLI (on page 67). Install and configure the STAF package (on page 69). Install the STAX execution engine (on page 71). Install the STAF EVENT service (on page 71). Install a Java JRE (on page 72). Install the Cygwin environment (on page 73). Install the VMmark Harness (on page 75). Configure the VMmark Harness (on page 75). Edit the cygserver.conf file (on page 79). Edit registry settings (on page 79). Update the HOSTS file (on page 81). Obtain Exchange Server 2003 and LoadGen (on page 81). Configure Windows to not use a proxy server (on page 82). Create any additional client systems you will need (on page 83). (Note that this must be done before the mail server client is installed and configured.) Set the clients preferred domain name server (on page 84). Make the client system a domain member (on page 85). Install Exchange Server 2007 Tools (on page 86). Install and configure LoadGen (on page 87). Create a backup of the mail server database (on page 88).
VMware, Inc.
17
Virtual Machine Worksheets

These worksheets provide a place to record information about the virtual machines used in a VMmark Benchmarking setup. It may be helpful to photocopy this page and fill out one copy for each tile being prepared. NOTE Virtual Machine Name should be in the format of WorkloadNameN where N is the tile number. For example, the Mail server in the first tile, Tile0, should be named Mailserver0. Table 1. Microsoft Windows Virtual Machine Worksheet
Virtual Machine Name MailserverN Function Mail server (64-bit Windows Server 2008) Configuration 4 vCPUs 8GB RAM 32GB boot disk 40GB data disk 1 vCPU 512MB RAM 4GB boot disk Hostname IP Address
StandbyN
Standby system (32-bit Windows Server 2003)
Table 2. Linux Virtual Machine Worksheet

Virtual Machine Name OlioDBN Function Olio Database (64-bit SLES 11) Configuration 2 vCPU 2GB RAM 10GB boot disk 4GB data disk 4 vCPUs 6GB RAM 10GB boot disk 70GB data disk 4 vCPUs 4GB RAM 10GB boot disk 35GB data disk 2 vCPUs 2GB RAM 10GB boot disk 2 vCPUs 2GB RAM 10GB boot disk 2 vCPUs 2GB RAM 10GB boot disk Hostname IP Address
OlioWebN
Olio Web (64-bit SLES 11)
DS2DBN
DVD Store 2 Database (64-bit SLES 11)
DS2WebAN
DVD Store 2 Web (64-bit SLES 11) DVD Store 2 Web (64-bit SLES 11) DVD Store 2 Web (64-bit SLES 11)
DS2WebBN
DS2WebCN
18
VMware, Inc.
What is the VMmark Benchmark?
This chapter provides an overview of the VMmark Benchmark and describes how it works. The chapter consists of the following sections: Overview of VMmark 2.1 Benchmark on page 19 VMmark Benchmark Workloads on page 21 VMmark Client Systems on page 25 VMmark Harness on page 26 Contents of VMmark Kit on page 27 VMmark 2.1 Benchmark Scoring Methodology on page 28 Reference Scores on page 32 VMmark Version Notes on page 33 VMmark Run and Reporting Rules on page 33
Overview of VMmark 2.1 Benchmark

State-of-the-art server consolidation typically collects several diverse workloads onto a virtualization platforma collection of physical servers accessing shared storage and network resources. Dynamic load balancing techniques ensure that all system resources such as CPU, network, and disk are more efficiently utilized. In fact, virtual environments tend to function more smoothly when resource demands are balanced across multiple resources. The unit of work for a benchmark of virtualized consolidation environments can be naturally defined as a collection of virtual machines executing a set of diverse workloads. The VMmark 2.1 Benchmark follows the convention of VMmark 1.x and refers to this unit of work as a tile. The total number of tiles that a multi-host platform can accommodate gives a coarse-grain measure of that platform's consolidation capacity. This concept is similar to some server benchmarks, such as TPC-C, that scale the workload in a step-wise fashion to increase the system load. Tiles are relatively heavyweight objects that cannot by themselves capture small variations in platform performance. To address this, both the number of tiles and the performance of each individual workload determine the overall benchmark score. Each workload within a tile is constrained to execute at less than full utilization of its virtual machine. However, the performance of each workload can vary to a degree with the speed and capabilities of the underlying platform. For instance, the addition of a fast disk array might result in disk-centric workloads producing a more favorable score. These variations can capture system improvements that do not warrant the addition of another tile. However the workload throttling forces the use of additional tiles for large jumps in platform performance.
VMware, Inc.
19
When a tile is added the performance of workloads in existing tiles might decrease. If the system has not been overcommitted, however, and the minimum quality-of-service metrics are met, the aggregate score, including the new tile, should increase. The result is a flexible benchmark metric that provides a measure of the total number of workloads that can be supported by a particular multi-host platform as well as the overall performance level within the virtual machines. VMmark 2.1 generates a realistic measure of platform performance by incorporating a variety of platform-level workloads in addition to traditional application-level workloads. Live migration both of virtual machines and of their underlying disk files has quickly become a powerful and commonplace tool in virtualized datacenters. Likewise, the ability to quickly clone and deploy virtual machines onto available resources has transformed traditional server provisioning. Each of these operations place non-trivial resource demands on the underlying computational infrastructure and must be included in order to accurately characterize platform performance. Virtual machine migration, clone and deploy, and storage migration operations are repeatedly performed on a set of the workload virtual machines to simulate the additional resource demands typical in production datacenters. Additionally, automated load balancing is enabled to ensure application-level workloads are relocated to satisfy their resource needs as the computational loads vary among the individual hosts over time. VMware VMmark is designed to benchmark the performance of the virtualization software and hardware and is not designed as a benchmark of any other software component. Figure 1-1. Conceptual Example of a Single Server Running One VMmark Tile (Eight Virtual Machines)
ESX Server One VMmark tile (eight virtual machines)
20
VMware, Inc.
Chapter 1 What is the VMmark Benchmark?
VMmark Benchmark Workloads

A meaningful tiled consolidation benchmark is based on a set of relevant datacenter workloads. A survey of datacenter applications led to the inclusion of the following workloads: Mail server Standby system Web 2.0 load simulation E-Commerce simulation Rather than develop workloads from scratch, existing workloads or benchmarks were used wherever possible. This reduces implementation effort and provides a well-understood foundation upon which to build. The workloads chosen for use in VMmark are representative of popular applications commonly run by VMware customers. However, the inclusion of any particular application does not imply that the vendor claims support for running their application in virtual machines. These four workloads are summarized in Table 1-1. Table 1-1. VMmark 2.1 Workload Summary
Workload Name Mail server Standby system Web 2.0 E-Commerce Application Exchange 2007 None Olio DVD Store 2 Virtual Machine Platform Windows 2008 64-bit, 4 vCPU, 8GB RAM, 32GB boot & 40GB data disks Windows 2003 32-bit, 1 vCPU, 512MB RAM, 4GB disk Back end: SLES 11 64-bit, 2 vCPU, 2GB RAM, 10GB boot & 4GB data disks Front end: SLES 11 64-bit, 4 vCPU, 6GB RAM, 10GB boot & 70GB data disks Back end: SLES 11 64-bit, 4 vCPU, 4GB RAM, 10GB boot & 35GB data disks Front end (3): SLES 11 64-bit, 2 vCPU, 2GB RAM, 10GB disk
The design of some of the workloads did not completely align with the design goals for VMmark 2.1. Some modifications to the workloads were thus required in order to make them suitable for a multi-workload data center benchmark. Additionally, individual loads were tuned to generate load levels that were representative of workloads in a consolidation environment. VMmark leverages components of these workload virtual machines to run some common virtualization procedures, which include: Dynamic storage relocation Dynamic virtual machine relocation Virtual machine cloning and deployment Automated load balancing The following sections discuss each workload and any necessary modifications.
Mail Server Virtual Machine

Most businesses today provide employees with email as a means for communicating. For this reason, mail servers are important workloads in modern datacenters, not just for their often large resource requirements, but also for their strict response-time requirements. The most common mail server deployed among VMware customers is Microsoft Exchange. We therefore chose to use Microsoft Exchange 2007 to represent the mail server workload in VMmark. Microsoft provides a well-known load generation utility called LoadGen that simulates users of the Exchange mail server. For consistency with VMmark design we made a few changes in how we implement this utility. In its default configuration LoadGen requires too much disk space for our purposes, so we reduced the initial mailbox sizes of the simulated users.
VMware, Inc.
21
We fix the load by limiting our configuration to 1000 heavy profile users with a more limited mailbox size compared to the default to fix the load. This does not cause the Exchange virtual machine to be bottlenecked on any resource. VMmark design also requires a periodic rate metric for each workload. Rather than using the number of users as a metric, as is typical, we periodically measure the number of transactions executed by the mail server and use that as our metric. The Microsoft Exchange Server 2007 workload is run in 64-bit Microsoft Windows Server 2008 Enterprise Edition on a virtual machine with four virtual CPUs and 8GB of memory.
Standby Virtual Machine

Many computing environments contain standby systems ready for new workloads, or for workloads with bursty behavior. These extremely lightly loaded (essentially idle) systems are attractive targets when consolidating servers. Even when idle, however, these systems still place resource demands on the virtualization layer and can impact the performance of other virtual machines. For this reason, one standby virtual machine is included in the benchmark tile. The standby virtual machine does not produce a metric that affects the benchmark score. However, it is required to answer a periodic heartbeat for the VMmark test to be considered valid. The standby virtual machine runs 32-bit Microsoft Windows Server 2003 Enterprise Edition and has one virtual CPU and 512MB of memory.
Web 2.0 Simulation

Web 2.0 applications such as social networking have become ubiquitous in recent years. These applications tend to be built from a wide range of open-source tools and applications running in a distributed fashion. The Olio workload (http://incubator.apache.org/olio/) mimics a Web 2.0 application focused on social networking and events. Olio has been implemented in a variety of technologies, including PHP, Java EE, and Ruby on Rails. Olio defines an example of a Web 2.0 application supporting the following common operations: HomePage, Login, TagSearch, EventDetail, PersonDetail, AddPerson, and AddEvent. VMmark 2.1 includes a two-tier Java-based implementation of the Olio workload. Olio was originally designed to use the Faban benchmark driver framework from Sun Microsystems. The Ruby on Rails implementation of Olio was modified to use the Rain benchmark framework from the RAD Lab at the University of California, Berkeley (http://radlab.cs.berkeley.edu). We subsequently adapted the Java implementation of Olio to the Rain framework for use within VMmark 2.1. We also replaced the default Glassfish application server with the Tomcat-based tc Server. The Rain framework and tc Server application server provided greater flexibility and easier maintenance when adapting the Olio workload for use within VMmark 2.1. The load is fixed at 400 concurrent users. We periodically measure the throughtput and quality of service for each operation. We use the total of completed operations per minute as the primary metric in VMmark 2.1. Minimum quality-of-service metrics must also be met. The Olio workload used in each VMmark 2.1 tile utilizes two virtual machines, each running 64-bit Novell SLES 11. The OlioWeb tier runs the tc Server Java EE container application on a virtual machine with four virtual CPUs and 6GB of memory. The Olio database tier runs the MySQL database on a virtual machine with two virtual CPUs and 2GB of memory.
E-Commerce Simulation
Databases running transactional workloads support a wide array of applications, typically as part of a multi-tier architecture. Databases tend to be resource intensive and exercise most server and infrastructure components. In many cases, database systems also face strict response-time demands. Transaction processing often exhibits bursty behavior, resulting in widely varying resource demands over time. The ability of the underlying platform to support usage spikes is critical to maintaining acceptable performance.
22
VMware, Inc.
DVD Store Version 2 (DS2) is a complete online e-commerce test application with a back-end database component, a Web application layer, and driver programs. (See http://www.delltechcenter.com/page/DVD+Store for additional information.) The DS2 driver simulates users logging into a Web server and browsing a catalog of products using basic queries. Users may select items for purchase then proceed to check out or continue shopping. Each Web server communicates with a database server that maintains user account and inventory data. The DS2 workload used in VMmark 2.1 utilizes four virtual machines in each tilethree Web servers and one database server. One Web server delivers a constant load to the database throughout each benchmark interval. The two remaining Web servers deliver periodic load to the database during the benchmark interval to create a bursty overall load profile and varying resource demands. For VMmark 2.1, each Web server is driven by 10 driver threads when active. The performance metric for this workload is the total number of transactions per minute. Minimum quality-of-service metrics must also be met. The DS2 workload used in each VMmark 2.1 tile utilizes four virtual machines, each running 64-bit Novell SLES 11. The three virtual machines in the DS2 Web tier (DS2WebA, DS2WebB, and DS2WebC) each run the Apache 2.2 Web server on virtual machines with two virtual CPUs and 2GB of memory. The DS2 database tier runs the MySQL database on a virtual machine with four virtual CPUs and 4GB of memory.
Virtual Machine Cloning and Deployment

Creating a new virtual machine and installing a guest operating system and applications can be time consuming. Using virtual machine cloning technology, administrators can make many copies of a virtual machine using a single installation and configuration process. Cloning, configuration, and deployment operations create bursty loads on platform resources, particularly the storage subsystem as the virtual machine files are copied. VMmark 2.1 implements a workflow involving the cloning of a virtual machine, the subsequent upgrade of the new virtual machine's paravirtualized drivers and tools, followed by the deployment of a functional virtual server. This infrastructure workload acts on a copy of the standby server virtual machine containing an outdated copy of the VMware Tools suite. VMmark 2.1 clones this template, customizes the operating system, upgrades VMware Tools on the newly-created virtual machine, verifies successful deployment, then destroys the new virtual machine. The benchmark then waits five minutes and repeats this process, continuing for the duration of the benchmark period. The number of concurrent clone and deploy operations increases with the number of tiles and the number of hosts in the benchmark cluster. The performance metric used is the number of clone and deploy operations per hour.
Dynamic Virtual Machine Relocation

Live migration technology such as VMware vMotion leverages the complete virtualization of servers, storage, and networking to move an entire running virtual machine seamlessly from one server to another. During a vMotion operation, the active memory and precise execution state of a virtual machine is rapidly transmitted over a high speed network from one physical server to another and access to the virtual machines disk storage is instantly switched to the new physical host. This transition can result in bursty loads on platform resources, particularly the networking subsystem. VMmark 2.1 mimics the manual relocation of a virtual machine, which can be a common task performed by an administrator. This infrastructure workload acts on the Olio database virtual machines. An Olio database virtual machine is selected in a round-robin fashion from among all the tiles, a destination host is selected at random from among all hosts in the benchmark cluster (other than the virtual machines current location), and the virtual machine is moved to the destination host. VMmark 2.1 then waits three minutes and repeats this process, continuing for the duration of the benchmark period. The number of concurrent relocation operations increases with the number of tiles and the number of hosts in the benchmark cluster. The performance metric used is the number of relocations per hour.
VMware, Inc.
23
Dynamic Storage Relocation

Live migration of virtual machine disk files across or within storage arrays enables enormous flexibility for storage maintenance, upgrades, and load balancing. Storage relocations can create bursty loads on platform resources, particularly the storage subsystem. VMmark 2.1 implements a scenario involving the relocation of a virtual machine's disk files to a maintenance partition followed by a return trip of the files to their original location. This round-trip approach models an administrator temporarily evacuating a disk partition and subsequently returning the system to its initial state after completing maintenance on the storage system. This infrastructure workload acts on the standby server virtual machines. A standby server virtual machine is selected in a round-robin fashion from among all the tiles, the virtual machines files are moved to a maintenance partition, then the files are moved back to their original location. VMmark 2.1 then waits five minutes and repeats this process, continuing for the duration of the benchmark period. The number of concurrent storage relocation operations increases with the number of tiles and the number of hosts in the benchmark cluster. The performance metric used is the number of relocations per hour.
Automated Load Balancing

Automatically balancing resource demands among multiple physical servers using technology such as VMware's Distributed Resource Scheduler (DRS) has become a fundamental part of modern virtualized datacenters. Intelligently allocating and balancing resources allows the underlying platform to respond effectively to bursty load conditions even when utilizations are high. VMmark 2.1 requires that DRS be enabled and running at the aggressive level (or higher) to insure that rebalancing occurs in a timely manner when utilizations are high. This should improve overall performance by addressing load imbalances occurring during the benchmark interval.
24
VMware, Inc.
VMmark Client Systems

Each tile requires a client system. These systems run software that provides loads for most of the workloads. The prime client also runs the VMmark Harness software that can start, stop, and measure the performance of the workloads. Table 1-2 provides a summary of the client software associated with each workload. Table 1-2. VMmark Client Summary
Workload Name Mail server Standby system Web 2.0 Simulation E-Commerce Simulation Client Software LoadGen IdleVMTest Olio client DVD Store 2 client
In addition to the software listed above, all client systems must also be running the STAF software package, a Java JDK, Cygwin, and Microsoft Exchange management tools. The prime client must also be running the STAX software package and up-to-date versions of PowerShell and the VMware vSphere PowerCLI.
VMware, Inc.
25
VMmark Harness
The VMmark Harness is a utility run on the prime client system that can start and stop the applications running on the workload virtual machines and can report the results of a test run. The VMmark Harness is based on the open-source Software Testing Automation Framework (STAF, see http://staf.sourceforge.net/index.php) and its companion execution engine, STAX. These tools support the development and running of distributed coordinated tests across heterogeneous machines and operating systems. The VMmark Harness consists of several STAX XML modules, the VMMARK2.CONFIG file, and several workload-specific configuration files. The main STAX module, vmmark2_main.xml, processes the VMMARK2.CONFIG file to configure the test to be run. Each workload has its own <workload>_functions.xml module that contains the workload-specific code needed to initialize the test, run the test, and collect the results. The VMMARK2.CONFIG file defines the actual test, identifying all the clients and server virtual machines involved in the test, the number of tiles to be run, and the workloads within each tile. After the VMMARK2.CONFIG file has been processed, the VMmark Harness performs pre-run system and timing validation and initiates the setup phase for the VMmark infrastructure operations and for each workload in each tile. After the setup has completed, the VMmark Harness simultaneously initiates the individual workloads in all the tiles. When the workload runs have completed, the harness again validates the timing, then collects the results into a results directory.
26
VMware, Inc.
Contents of VMmark Kit

VMmark is distributed as a compressed file named VMmark2-version-date.zip (where version is the version of VMmark included in the kit and date is the date of release). When extracted, the contents can be found in the following directories: docs This directory contains documentation files, including the VMmark EULA and the VMmark Run and Reporting Rules. NOTE The VMware VMmark Benchmarking Guide (this book) is now available as a separate download at http://www.vmware.com/products/vmmark/ (the same place from which the VMmark kit can be downloaded). vclient0 This directory contains some of the client-side elements and templates required for each workload (additional software must be acquired and installed separately). VMmark2 This directory contains the STAX driver code, tools used by the harness to generate infrastructure operations, .jar files used by the harness, a results directory where timestamped results files are saved, the script used to collect host system information for reporting, and tools for generating VMmark metrics and ratios in both text and HTML formats. vserver This directory contains elements that need to be installed on the workload-specific virtual machines after the installation of each virtual machines application or benchmark software (some of which must be acquired and installed separately).
VMware, Inc.
27
VMmark 2.1 Benchmark Scoring Methodology

A VMmark 2.1 score aggregates the throughput metrics of all application and infrastructure workloads to create a single overall benchmark score that can be used to quickly compare different platform configurations. Every workload must also pass its minimum quality-of-service requirements for the benchmark result to be considered compliant. The scoring methodology is conceptually straightforward despite the underlying complexity and large scope of the benchmark. After a VMmark Benchmark test run completes, each individual application and infrastructure workload reports its relevant performance metric. These performance metrics are shown in Table 1-3. Table 1-3. Individual VMmark Workload Metrics
Workload Name Mail server Olio server Standby server DS2WebA DS2WebB DS2WebC Clone and deploy vMotion Storage DRS vMotion Application(s) Microsoft Exchange tc Server, MySQL None Apache, MySQL Apache, MySQL Apache, MySQL None None None N/A Metric Actions/minute Operations/minute None Transactions/minute Transactions/minute Transactions/minute Clones/hour VM migrations/hour VM migrations/hour None
NOTE The standby server workload does not produce a metric that affects the benchmark score. However the standby server is required to answer to a periodic heartbeat request in order for the VMmark test to be considered valid. Likewise, DRS must be enabled to allow the platform to automatically balance resources during the benchmark run. DRS does not produce a metric but will affect the performance of other workloads by managing the overall resource allocations to improve performance and provide stability. The DVD Store implementation in VMmark 2.1 is a multi-tier workload containing three webserver virtual machines with varying load patterns accessing a single database virtual machine. For scoring purposes, each webserver generates a separate results file to be processed independently from the others. This approach provides a clearer understanding of platform behavior. These metrics are collected at frequent intervals during the course of the run. The standard VMmark 2.1 workload is designed to run for at least 3 hours with workload metrics reported every 60 seconds. This means that rather than having a single number upon completion of a test run, the user will have a series of numbers for each of the workloads. However, each workload score is defined as a single number: the average of a consecutive subset of the series of datapoints for that workload. Normalization allows the integration of the different component metrics into an overall score. A reference system is commonly used for normalization when computing benchmark scores. For example, SPEC CPU2000 (see http://www.spec.org/cpu2000/) takes this approach, using a Sun Ultra5_10 with a 300Mhz processor as its reference platform. VMmark 2.1 measures consolidation workloads within virtual environments and therefore requires a reference platform capable of successfully running a single tile. The steady state for the benchmark is defined as the middle two hours of the three-hour run. The first and last half hours are the ramp-up and ramp-down times, respectively. The steady state is further divided into three 40-minute sections. For each of the 40-minute sections we compute the overall result for the platform and select the median score of the three as the reported score.
28
VMware, Inc.
After a valid run, the metrics of the application workloads within each tile are computed and aggregated into a score for that tile. This aggregation is performed by first normalizing the different performance metrics (such as Actions/minute and operations/minute) with respect to a reference platform. Then a geometric mean of the normalized scores is computed as the final score for the tile. The resulting per-tile scores are then summed to create the application-workload portion of the final metric. The metrics for the infrastructure workloads are aggregated separately using the same mathematical technique of normalization with respect to a reference platform followed by the computation of the geometric mean. Unlike the application workloads, the infrastructure workloads are not scaled explicitly by the user. Consequently, the infrastructure workloads are compiled as a single group and no multi-tile sums are required. The final benchmark score is then computed as a weighted average of the application-workload component and the infrastructure-workload component. VMmark 2.1 gives weights of 80% to the application-workload component and 20% to the infrastructure-workload component. These weights were chosen to reflect the relative contribution of infrastructure and application workloads to overall resource demands. The benchmark helps measure the virtualization overheads of the individual workloads as well as the scalability of the entire system. Therefore results for multi-tile runs are reported as the aggregate score for all tiles, the individual scores for each of the tiles, and the scores for the workloads within the tiles as well as the individual scores for each infrastructure workload. If any of the workloads within any tile fails to run, produces errors during a run, or fails its minimum quality-of-service requirement, that entire VMmark run is considered to be invalid. This applies to programs running on both the servers and the client systems. Also, the configuration of the workloads, the versions of the benchmarks, operating systems, tools, and all other software used must conform to the specifications in the VMmark documentation. To illustrate the scoring methodology, consider the following two examples. The first example demonstrates how to compute the score for a single tile benchmark run, while the second example demonstrates how to compute the scores for a multiple-tile benchmark run. For these examples, assume the reference system had the scores shown in Table 1-4. Table 1-4. Example Reference System Workload Scores (Artificial Data)
Workload Name Mail server Standby Olio DS2WebA DS2WebB DS2WebC Clone and deploy vMotion Storage vMotion DRS Score 1000 actions/minute None 1000 operations/minute 1000 Transactions/minute 1000 Transactions/minute 1000 Transactions/minute 10 clones/hour 10 VM migrations/hour 10 VM migrations/hour None
Example One: Single-Tile Benchmark Scoring

Suppose you perform a single-tile VMmark run and, according to the methodology described earlier, you remove the first and last thirty minutes, calculate results for each of the three 40-minute sections, and select the median of the three results to obtain the scores shown in Table 1-5.
VMware, Inc.
29
Table 1-5. Single-Tile Example Test System Workload Scores (Artificial Data)
Workload Name Mail server Standby Olio DS2WebA DS2WebB DS2WebC Clone and deploy vMotion Storage vMotion DRS Score 950 actions/minute None 940 operations/minute 1020 Transactions/minute 980 Transactions/minute 970 Transactions/minute 9 clones/hour 11 VM migrations/hour 9 VM migrations/hour None
To compute the score for this tile, you first compute each workload's normalized scores by dividing the score for each workload by the reference score for that workload: Mail server: Standby: Olio DS2WebA DS2WebB DS2WebC Clone and deploy vMotion Storage vMotion DRS 950/1000 n/a 940/1000 1020/1000 980/1000 970/1000 9/10 11/10 9/10 n/a = 0.95 n/a = 0.94 = 1.02 = 0.98 = 0.97 = 0.90 = 1.10 = 0.90 n/a
You would then combine the normalized scores for the application workloads using a geometric mean: (0.95 * 0.94 * 1.02 * 0.98 * 0.97)^(0.2) = 0.97
Next, you would combine the normalized scores for the infrastructure workloads using a geometric mean: (1.10 * 0.90 * 0.90)^(0.333333) = 0.96
Finally, you would combine the geometric means with a weighted average (80% for application workloads, 20% for infrastructure workloads): (0.8 * 0.97) + (0.2 * 0.96) = 0.97
The score for this tile is 0.97. For reporting, include this score as well as the individual scores for all of the workloads (both raw and normalized).
Example Two: Multiple-Tile Benchmark Scoring

This example demonstrates the scoring for multiple tiles. Suppose you perform a VMmark run containing four tiles and obtain the scores shown in Table 1-6.
30
VMware, Inc.
Table 1-6. Multiple-Tile Example Test System Workload Scores (Artificial Data)
Workload Name Mail server Standby Olio DS2WebA DS2WebB DS2WebC Clone and deploy vMotion Storage vMotion DRS Tile 1 Score 1100 n/a 940 950 940 930 Tile 2 Score 1050 n/a 950 1000 1010 1020 21 22 11 n/a Tile 3 Score 990 n/a 890 990 1020 960 Tile 4 Score 1080 n/a 940 970 970 1050
The scores for the application workload tiles would be: Normalized scores: 0.97 1.01 0.97 1.00
The overall application component to the score would be the sum of these scores: 0.97 + 1.01 + 0.97 + 1.00 = 3.95
The score for the infrastructure workloads would be: Normalized score: 1.72
The overall score for the system would be the weighted average of these scores: (0.8 * 3.95) + (0.2 * 1.72) = 3.50
Along with the overall score, a valid test report must include both the individual tile scores and the workload scores (both raw and normalized).
VMware, Inc.
31
Reference Scores
A pair of identical HP ProLiant DL380 servers running VMware ESX Server 4.0 U1, build 208167 were used in the reference platform. Each server contained two 3.16 GHz quad-core Intel Xeon X5460 CPUs, and was configured with 32GB of memory. Storage was provided by an EMC Clariion CX3-20 disk array connected via a 4Gb/s fiber channel link and containing eight 15,000 RPM disks configured in RAID0. The servers were connected to the disk array using a Brocade DS-5000 fiber switch. Each server used a single 1Gigabit Ethernet link for vMotion. The systems were managed by a Virtual Center 4.0 U1 system. The load-generating client was an HP ProLiant DL360 G5 with one 2.66 GHz quad-core Intel Xeon X5355 CPU and 8GB of memory running 32-bit Microsoft Windows Server 2003 R2 with Service Pack 2. The client and the reference system were connected through a single 1 Gigabit Ethernet link. Network connectivity was provided by an HP ProCurve 5406zl switch. The reference scores obtained on this system are shown in Table 1-7. Table 1-7. Reference System Workload Scores
Workload Name Standby server Mail server Olio DS2WebA DS2WebB DS2WebC Clone and deploy vMotion Storage vMotion Score None 330.25 send mail actions/minute 4641.43 operations/minute 2199.18 operations/minute 1518.55 operations/minute 1058.05 operations/minute 4.0 DeployedVMs/Hour 16.0 vMotionedVMs/Hour 9.0 Storage vMotionedVMs/Hour
32
VMware, Inc.
VMmark Version Notes

VMmark 2.0 Vs. VMmark 2.1
VMmark 2.1 adds support for client systems running certain versions of Windows Server 2008 (in addition to Windows Server 2003, which was supported in VMmark 2.0) as well as support for virtualized clients (subject to certain conditions). The benchmark harness now scales the storage relocation workload in the same fashion as the other infrastructure workloads. The VMmark Benchmarking Guide is updated to reflect these changes. All published VMmark 2.0 benchmark scores are considered directly comparable to VMmark 2.1 scores.
VMmark 1.x Vs. VMmark 2.0

While VMmark 1.x was designed as a single-system consolidation benchmark consisting of six isolated single-tier workloads, VMmark 2.0 was designed as a multi-host benchmark reflecting typical, modern-day usage of virtualized infrastructure. VMmark 2.0 consisted of two single-tier application workloads, two multi-tier application workloads, and four infrastructure-level workloads. The workloads and load levels of VMmark 2.0 were completely changed from VMmark 1.x and the benchmark scores were in no way considered comparable.
VMmark 1.1 Vs. VMmark 1.1.1

VMmark 1.1.1 was a minor maintenance release. It included updates to the VMmark Benchmarking Guide, updated reporting script and disclosure template, and new versions of the Run and Reporting Rules and Review Panel Guidelines and Procedures. VMmark 1.1.1 scores were considered directly comparable to VMmark 1.0 and 1.1 benchmark scores.
VMmark 1.0 Vs. VMmark 1.1

In order to reflect the increasing use of 64-bit operating systems in data centers, VMmark 1.1 used 64-bit operating systems and applications in three of the workload virtual machines (Java server, Web server, and Database server). The workloads and load levels were unchanged from VMmark 1.0, however, and the VMmark 1.0 and 1.1 benchmark scores were thus considered directly comparable.
VMmark Run and Reporting Rules

In addition to the guidelines presented above, specific run and reporting rules must be followed for a VMmark test to be considered valid, and for the results of that test to be publishable. These run and reporting rules are included in the \docs directory of the VMmark kit. NOTE You should also check the VMmark download page for any later release of the run and reporting rules. Visit: http://www.vmware.com/products/vmmark and select the Download button.
VMware, Inc.
33
34
VMware, Inc.
VMmark Benchmark Requirements
This chapter describes the hardware and software required in order to perform VMmark benchmark testing. It consists of the following sections: VMmark Version and Settings Requirements on page 35 VMmark Hardware Requirements on page 35 VMmark Software Requirements on page 39 Software License Summary on page 40
VMmark Version and Settings Requirements

Both vCenter and ESX versions must be currently available or planned to be generally available within 90 days. The DRS aggressiveness level must be 3 or higher. No DRS rules are allowed. Using resource pools is not allowed.
VMmark Hardware Requirements

This section describes the hardware required in order to run VMmark.
Server Hardware Requirements

The server hardware must be supported by the hypervisor being used and must be configured to meet or exceed the minimum CPU, minimum memory, and other basic requirements for that product and version.
Server Storage Requirements

Configure the servers with enough datastore capacity to hold all the virtual machines required for the VMmark Benchmark runs. This is approximately 300GB per tile (plus an additional 20GB during the initial configuration). This capacity can be in one or multiple datastore partitions and can be any hardware type supported by the hypervisor for datastore use, including SCSI, SAN, iSCSI, and NFS. NOTE In order to support the dynamic storage relocation operations performed by VMmark 2.x, the benchmark requires that all ESX hosts used in a test have access to the same shared storage.
Server Memory Requirements

The VMmark Benchmark uses approximately 27GB of physical RAM per tile. The test might not run smoothly with less than this amount of memory.
VMware, Inc.
35
Other Server Requirements

In order to support the vMotion and DRS infrastructure operations included in VMmark 2.x, the benchmark requires a minimum of two ESX hosts and requires that all ESX hosts used in a test be vMotion compatible.
Client Hardware Requirements

Each VMmark tile requires its own client machine. These client machines must meet or exceed the specifications detailed in the sections below.
Hardware Requirements for Physical Clients

When client functions are run directly on a physical system (physical client), the system must meet or exceed the following requirements: Four CPU cores 4GB of RAM 15GB of available disk space (for Windows Server 2003); 32GB of available disk space (for Windows Server 2008) Physical client systems should not be resource constrained and should not be running services or applications unrelated to the benchmark. One of the client machines (the prime client) must be a physical system.
Hardware Requirements for Virtual Clients

Other than the prime client (which must be a physical system), client systems can be either physical systems or virtual machines. When client functions are run in a virtual machines (i.e., virtual clients) the conditions detailed in the following sections must be met. For clarification of the details addressed below, or if you have questions regarding anything not addressed, contact VMware at vmmark-info@vmware.com. This specification should not be considered exhaustive. These conditions are intended to ensure that virtual client systems introduce no resource constraints that might affect the benchmark results. If benchmark output files, including the results of time synchronization checking performed by the VMmark harness, suggest that client resource constraints affected a benchmark result, that result could be deemed non-compliant, despite meeting the conditions detailed below. Virtualization Platform for Virtual Clients The hosts must be running VMware ESX/ESXi 4.1 or later. The CPUs in the servers hosting the virtual clients must not be overcommitted. NOTE This means the sum of the vCPUs configured for all the virtual machines running on a host must not exceed the number of logical CPUs that host contains. The memory in the servers hosting the virtual clients must not be overcommitted. NOTE This means the sum of the memory allocated to all the virtual machines running on a host must not exceed the amount of physical memory that host contains. The hosts physical processors must be equivalent to or faster than: Intel: Xeon 5400-series (Harpertown) AMD: Opteron 24xx- or 84xx-series (Istanbul) NOTE Each hyper-threaded core can be considered two CPUs when calculating the level of CPU commitment.
36
VMware, Inc.
Chapter 2 VMmark Benchmark Requirements
The physical storage used for the virtual machines should be typical of a modern datacenter and should offer performance sufficient to meet the client virtual machines resource requirements without introducing delays that might affect the benchmark results. The physical network infrastructure used for virtual machine traffic should be typical of a modern datacenter and should offer performance sufficient to meet the client virtual machines resource requirements without introducing delays that might affect the benchmark results. All virtual clients must have identical resource, scheduling, and tuning configurations. Virtual Hardware for Virtual Clients When client functions are run in a virtual client, each virtual system must meet or exceed the following requirements: Virtual hardware version 7 or later. Four virtual CPUs 4GB of RAM 15GB of available disk space (for Windows Server 2003); 32GB of available disk space (for Windows Server 2008) One virtual network adapter per virtual client. This can be whatever virtual device type desired. Virtual client systems should not be resource constrained and should not be running services or applications unrelated to the benchmark.
vCenter Hardware Requirements

The vCenter Server can be run on either a physical system or a virtual machine. In either case the underlying system (whether physical or virtual) must be supported by the vCenter version being used and must be configured to meet or exceed the minimum CPU, minimum memory, and other basic requirements for that product and version. NOTE If the vCenter Server is run on a virtual machine, that virtual machine must not be on a host that is part of the cluster under test.
Network Requirements
VMmark tests require a dedicated private gigabit (or faster) network between the server system and the client machine (or machines) and a separate dedicated vMotion network for infrastructure operations. Some configurations might need multiple network links for optimal performance.
Network Hardware
All components of this network (network interface cards, cables, and switches) should be at least gigabit rated, and all should be set to at least gigabit speeds. This is to ensure that the system performance is not limited by network bandwidth constraints, thus artificially skewing the benchmark results.
Network Topology
Connect the client system (or systems) to the server system through switches that are dedicated to the benchmark tests and are not shared with any other systems. Additionally, though the VMmark test systems may be connected to a company-wide network during setup, neither the client system (or systems) nor the server systems should have a network connection to any other network, whether an internal company network or the external Internet, during the VMmark tests. Use of a completely private network facilitates the configuration of the domain controller which is part of each tile and ensures that extraneous network traffic does not impact the VMmark tests. Configure the servers, each workload virtual machine, and each client system with a static IP address. Figure 2-1 illustrates a sample network configuration.
VMware, Inc. 37
Figure 2-1. Sample VMmark Benchmark Network Configuration (Not Shown: vCenter Server, VMmark 2.1 Multi-Host Setup, Dedicated vMotion Network)
Prime Client
Private Gigabit Ethernet switch
One client per tile Client
Client
Server
Private Fibre Channel switch Optional SAN, iSCSI, or NFS (SAN shown)
SAN
38
VMware, Inc.
VMmark Software Requirements

This section describes the software required in order to run VMmark.
Workload Software Requirements

Each VMmark tile consists of eight virtual machines: A Microsoft Windows Server 2008 Enterprise Edition 64-bit virtual machine runs the following workload: Mail server: Microsoft Exchange 2007 A Microsoft Windows Server 2003 Enterprise Edition 32-bit virtual machine runs the following workload: Standby system: Powered on, but running no application software Six SLES 11 64-bit Linux virtual machines run the following workloads: Olio database Olio web DVD Store 2 database Three DVD Store 2 web systems
Client Software Requirements

Each VMmark tile requires a client machine. These client machines run Microsoft Windows Server 2003 Enterprise Edition (32-bit) or Microsoft Windows Server 2008 Enterprise Edition (64-bit) with the following applications: NOTE The prime client must be a physical system. The other client systems can be either physical or virtual. VMmark harness STAF framework and STAX execution engine LoadGen Microsoft Outlook 2007 (standalone or included in Microsoft Office 2007) Microsoft Exchange 2007 management tools Cygwin A Java JDK Rain workload toolkit
VMware, Inc.
39
Software Versions
The VMmark benchmark is designed to work with specific versions of the operating systems, workload software, and other software packages. Where it is relevant, specific versions are called out for each such software package, often with the notation or later (for example, STAF version 3.4.1 or later). When this notation is included, minor revisions later than the one specified can be used with VMmark, but major revisions can not. With a version number of the form X.Y (or X.Y.Z), a change in the value of X represents a major revision, while a change in the value of Y, or Z, or both Y and Z, represents a minor revision. Similarly, the addition of a service pack is considered a minor revision. In the case of operating systems, for both workload and client systems, the service pack or release can be later than specified in this guide, but the base version (for example, Windows Server 2008), the edition (for example, Enterprise Edition), and the bit architecture (32-bit or 64-bit) should not be changed. NOTE To avoid communication problems, we recommend that all clients and all workload virtual machines be running the same version of STAF.
Software License Summary

Some of the software required to run VMmark 2.1 is free, some is available in no-cost evaluation versions, and some requires paid licenses. The following list summarizes these three categories. NOTE Throughout this document we provide URLs from which third-party software packages can be downloaded. While we believe these URLs to be accurate at the time of publication, we can not guarantee their continued accuracy. The inclusion of these URLs is intended as a convenience to the user; in some cases it might be necessary to download the required software from alternate locations.
Free Software
You can download the following free software packages from the Web: Cygwin environment STAF/STAX software Olio DVD Store 2 Java SDK VMware VMmark Harness (available only as part of the free VMmark download)
Evaluation Software
You can download evaluation versions of the following software packages from the Web: VMware vSphere (which includes vCenter Server and ESX) tc Server SLES 11 Linux (64-bit) Microsoft Outlook 2007 (standalone or included in Microsoft Office 2007)
Purchased Software
You must purchase licenses for the following software packages if you do not already own them:
40
VMware, Inc.
Microsoft Windows Server 2003 Enterprise Edition, 32-bit One or two copies per tile (one for the standby virtual machine and (optionally) one for that tiles client system). NOTE Any Service Pack 1 or later release of Windows Server 2003 Enterprise Edition, 32-bit may be used for the standby virtual machine, and any Service Pack 2 or later release of Windows Server 2003, 32-bit Enterprise Edition may be used for the client system. The instructions in this guide, however, are based on the use of Service Pack 2. The installation and configuration of other releases and service packs may vary slightly from these instructions. Microsoft Windows Server 2008 Enterprise Edition, 64-bit One or two copies per tile (one for the Mail Server and (optionally) one for that tiles client system). NOTE Though Windows Server 2008 Enterprise Edition, 64-bit with or without a service pack, and any release, may be used, the installation instructions in this guide are based on the use of Windows Server 2008 Enterprise Edition with Service Pack 2. The installation and configuration of other releases and service packs might vary slightly from these instructions. NOTE Each tile needs one client, and that client must be running either Microsoft Windows Server 2003 Enterprise Edition, 32-bit or Microsoft Windows Server 2008 Enterprise Edition, 64-bit. Thus in total each tile requires one Windows Server 2003 license, one Windows Server 2008 license, and one additional license of either type. Microsoft Exchange Server 2007 Enterprise Edition This is available as part of the Microsoft Developers Network (MSDN) Universal subscription. NOTE Because some of these packages may be available only in physical form (that is, CDs), order them early in the planning process to allow time for shipping.
VMware, Inc.
41
42
VMware, Inc.
Preparing the Infrastructure for VMmark 2.1 Benchmark Tests
This chapter describes the steps required in order to prepare the infrastructure for VMmark 2.1 tests. It consists of the following sections: Create the Virtualization Infrastructure on page 43 Create the VMmark Tiles on page 47
Create the Virtualization Infrastructure

The following subsections describe the steps needed to prepare the virtualization infrastructure for VMmark 2.1 tests.
Install vSphere vCenter

Install a version of VMware vSphere vCenter compatible with VMmark 2.1. VMmark 2.1 is designed to run with vCenter 4.0 or later. The installed version of vCenter must either be a released, generally available version, or must meet the pre-release requirements detailed in the VMmark Run and Reporting Rules. Make sure that all relevant updates and patches are installed. NOTE vCenter Server 4.1 requires a 64-bit Windows operating system. NOTE vCenter 4.0 cant manage ESX 4.1 hosts. Thus if any of your hosts will be running ESX 4.1, you will need to install vCenter 4.1. NOTE All ESX hosts should be able to reach the same shared storage in order to enable vMotion and Storage vMotion infrastructure operations.
Install VMware ESX

Install a hypervisor (i.e., VMware ESX, VMware ESXi, or a combination) according to VMware instructions on all servers to be used in the test. The hypervisor must either be a released, generally available version, or must meet the pre-release requirements detailed in the VMmark Run and Reporting Rules. Make sure that all relevant updates and patches are installed. Make sure that hardware on which you are installing the hypervisor meets the requirements outlined in Server Hardware Requirements on page 35, including the datastore capacity and vMotion compatibility.
VMware, Inc.
43
Configure vCenter
1 Using the vSphere Client, create a new datacenter on your vSphere server (File > New > Datacenter). NOTE If theres already a datacenter on your vSphere server, use that one instead. 2 Create a cluster under the datacenter as follows: a b c Right-click on the new datacenter and select New Cluster.... At the Cluster Features window, enter a name for the cluster (recording the name for later use), click the box next to Turn on VMware DRS, then click Next. At the VMware DRS window, under Fully automated, move the Migration threshold: slider to the 4th option (from left to right), closer to the Aggressive label, then click Next. When the slider is in the correct place, the text below the slider should read Apply priority 1, priority 2, priority 3, and priority 4 recommendations. vCenter will apply recommendations that promise even a moderate improvement to the clusters load balance. At the Power Management window, leave Power Management at its default setting of Off, then click Next. At the VMware EVC window, choose the VMware EVC setting that best fits your expected hosts in the cluster, then click Next. At the Virtual Machine Swapfile Location window, choose the Swapfile Policy for Virtual Machines that best fits your benchmarking environment, then click Next. At the Ready to Complete window, click Finish.
d e f g 3 4 5
Add hosts to your cluster by right clicking on the cluster name and selecting Add Host.... In order to enable vMotion and Storage vMotion infrastructure operations, make sure that all the ESX hosts in your cluster can reach the same shared storage. Make sure that all hosts can vMotion to all other hosts within the cluster.
Configure Time Synchronization on the ESX Hosts and the vCenter Server
The time on the ESX hosts and on the vCenter Server need to be synchronized to a common NTP server. The following sections describe how to accomplish this.
Configure Time Synchronization for ESX Hosts

In order to maintain the timing of all systems and virtual machines under test we recommend that all hosts use NTP. This can be done via the vSphere Client. Follow these links for additional detail: http://kb.vmware.com/kb/1003063 http://kb.vmware.com/kb/1003736 http://www.vmware.com/support/pubs/
Configure Time Synchronization for the vCenter Server

To configure your vCenter Server Windows system to synchronize with an external NTP server, follow these steps: 1 2 Open the Windows Registry editor (Start Menu > Run > Regedit). Navigate to and expand the following registry folder: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\W32Time
44
VMware, Inc.
Chapter 3 Preparing the Infrastructure for VMmark 2.1 Benchmark Tests
Referring to Table 3-1, modify each of the registry entries as follows: a b c d e f

Path
Navigate to the entry specified in the path column of Table 3-1. Right-click the entry and select Modify. If the Type in Table 3-1 is REG_DWORD, select the Decimal radio button. Enter the new data from Table 3-1. Click OK. Repeat for each entry in Table 3-1.
Type Data NTP 5 1
Table 3-1. Registry Entries to Change
Parameters\Type Config\AnnounceFlags TimeProviders\NtpServer\Enabled 4 Modify the NTP server name: a b c d
REG_SZ REG_DWORD REG_DWORD
Still within W32Time, navigate to Parameters. Right click on NtpServer. Select Modify. Enter the DNS name or IP address of your NTP server. If you enter a DNS name (rather than an IP address) you must append ,0x1 to the end. NOTE Use the same NTP server to synchronize your vCenter Server system and all your hosts. (Remember which NTP server youre using, as some configurations will also use it for their prime client, as described in Install Microsoft Windows Server on a Physical System on page 63.)
Referring to Table 3-2, modify each of the registry entries as follows: a b c d e f

Path
Navigate to the entry specified in the path column of Table 3-2. Right-click the entry and select Modify. Select the Decimal radio button. Enter the new data from Table 3-2. Click OK. Repeat for each new entry in Table 3-2.
Type Data 900 3600 3600
Table 3-2. Registry Entries to Change
TimeProviders\NtpClient\SpecialPollInterval Config\MaxPosPhaseCorrection Config\MaxNegPhaseCorrection 6 7 Close the registry editor.
REG_DWORD REG_DWORD REG_DWORD
Restart the time service by entering the following at a Command Prompt: net stop w32time && net start w32time
VMware, Inc.
45
Prepare the vSphere vCenter Server for the Deploy Infrastructure Operations
One of the operations in a VMmark 2 test is to deploy a server and upgrade its VMware Tools installation. In order to allow this, follow these steps: 1 Create a standby system virtual machine from an existing virtual machine with outdated VMware tools. VMware Tools must be installed but must be older than the oldest host in your cluster. For example, if you have a mix of ESX 4.0 U1 and ESX 4.0 U2 hosts in your cluster, you should install the tools from the ESX 4.0 initial release. See Prepare the Template for the Deploy Operation on page 124. Install the sysprep files needed for the Standby virtual machine on the system hosting the vSphere vCenter Server. Instructions to do this can be found in vSphere Basic System Administration, available on the VMware website. Using the vSphere vCenter Server's Customization Specifications Manager (View > Management > Customization Specification Manager), create a new OS customization specification that corresponds to your benchmarking environment. Take special care in filling out the Computer Name, Windows License, Password, Time Zone, and Networking options. This specification must allow a newly deployed virtual machine to be able to be customized and then pinged by the prime client, not conflicting with any other virtual machine names or IP addresses. Edit the newly-created customization specification: a b 5 6 At Administrator Password screen, select Automatically log on as the Administrator, then set Number of times to logon automatically: 100 times. At the Run Once menu, add the following command: secedit /configure /db c:\SignedOff.sdb /cfg c:\SignedOff.ini
Save the customization specification. If your cluster contains four or more hosts and you will be running four or more tiles, VMmark will attempt additional simultaneous Clone and deploy operations. The number of operations is half of the smaller of either the number of hosts or the number of tiles, rounded down to an integer. To prepare your environment, select the previously created OS customization specification and Duplicate it, making enough copies for all clone and deploy operations your environment will need. Once done, edit these copies to change the VMnames and IP addresses (noting them for later use, when you edit the VMMARK2.CONFIG file in Configure the VMmark Harness on page 75).
Move the ESX servers you will be testing into this cluster. NOTE Only ESX servers in this cluster will be used for the VMmark 2.1 test. Servers can be moved into and out of the cluster as needed for testing or they can be placed into maintenance mode to remove them testing.
46
VMware, Inc.
Create the VMmark Tiles

Each VMmark tile consists of eight workload virtual machines: Mail server virtual machine (running Exchange) Standby virtual machine (an idle machine) Olio database virtual machine (running the Olio database application) Olio web virtual machine (running the Olio web driver) DVD Store 2 database virtual machine (running the DS2 back end database component) DVD Store 2 web A virtual machine (running the DS2 driver) DVD Store 2 web B virtual machine (running the DS2 driver) DVD Store 2 web C virtual machine (running the DS2 driver) The following sections describe the creation of these virtual machines. The initial mail server and standby virtual machines, which run Windows, must be built from scratch. There are template virtual machines, running SLES 11, available for the Olio workload (both database and web tiers) and the DVD Store workload (both database and Web tiers). You may use these templates to avoid having to build the virtual machines from scratch. NOTE These sections describe procedures using the VMware vCenter Client connected to an ESX host. Other interfaces and other hypervisors may be different. NOTE It can be useful to create one complete tile (that is, Tile0) and make sure it is working smoothly, then make backups of this working configuration before cloning the workload virtual machines to create the additional tiles. More information about this methodology can be found in Performing a Short Run on page 158.
VMware, Inc.
47
NOTE For full harness functionality, we highly recommended that you adhere to the following naming conventions when creating the tiles and virtual machines: Tiles should be numbered, starting from zero. Example: Tile0, Tile1, Tile2 Virtual machines should be named StandbyN, MailserverN, OlioDBN, OlioWebN, DS2DBN, DS2WebAN, DS2WebBN, DS2WebCN, with N replaced by the appropriate tile number. For Example, for Tile0: Standby0 Mailserver0 OlioDB0 OlioWeb0 DS2DB0 DS2WebA0 DS2WebB0 DS2WebC0 And for Tile 1: Standby1 Mailserver1 OlioDB1 OlioWeb1 DS2DB1 DS2WebA1 DS2WebB1 DS2WebC1 If these naming rules are not adhered to, several areas of functionality during the benchmark run will be hampered. These are: automatic tile to host placement before run, virtual machine resources review (vCPU and memory) before run, and virtual machine powered on review before run.
48
VMware, Inc.
Create the Mail Server Virtual Machines

This section describes the creation of the mail server virtual machines. The initial mail server virtual machine must be build from scratch; there is no template virtual machine available. See Figure 3-1 for an overview of the process. To create the mail server virtual machines 1 Follow the instructions in Create and Configure the Mail Server Virtual Machines on page 105 to create a virtual machine for the mail server, install Windows, create the database and backup drives, install VMware Tools, install and configure STAF, and install Windows Server components and support tools. Follow the instructions in Clone the Mail Server Virtual Machines on page 119, and make as many clones of the mail server virtual machine template as you will need (one for each tile you expect to run).
NOTE Unlike the other virtual machines, the mail server requires custom settings for each tile, because it acts as the domain controller for all the virtual machines in its tile. After the machine is fully configured, changing these settings is time consuming. It is most efficient to partially configure a virtual machine as a template, clone that virtual machine, and complete the configuration within each clone. 3 4 5 6 7 8 Follow the instructions in Make the Mail Server Virtual Machine a Domain Controller on page 120, and promote each of the Exchange virtual machines to be a domain controller. Follow the instructions in Install Microsoft Exchange Server 2007 on page 121, and install the software on each of the Exchange virtual machines. Follow the instructions in Configure Microsoft Exchange Server on page 121, and configure each of the Exchange virtual machines. Follow the instructions in Update Exchange Server 2007 on page 122, and update each of the Exchange virtual machines. Follow the instructions in Increase the Transport Database Cache Size on page 122, and increase the cache size on each of the Exchange virtual machines. Follow the instructions in Configure STAF to Start Automatically on page 123, and configure each of the Exchange virtual machines.
VMware, Inc.
49
Figure 3-1. Overview of Virtual Machine Creation for Mail Server Workloads
Create Mail Server VM Template MailServerTemplate Install MS Windows Install VMware Tools
Clone for Each Tile
mailserver0 Configure
...
Install MS Exchange
Install MS Exchange
Install MS Exchange
50
VMware, Inc.
Create the Standby Virtual Machines

This section describes the creation of the standby virtual machines. The initial standby virtual machine must be build from scratch; there is no template virtual machine available. See Figure 3-2 for an overview of the process. To create the standby virtual machines 1 Follow the instructions in Create and Configure the Standby Virtual Machines on page 100, and create a virtual machine for the standby system using the following specifications: Name: standby0 Virtual processors: 1 Virtual memory: 512MB Disk Size: 4GB 2 3 Follow the instructions in Prepare the Standby Virtual Machines on page 124, and create the standby virtual machine for tile 0. Follow the instructions in Clone the Standby Virtual Machine on page 124, and make as many clones of the standby virtual machine as you will need (one for each additional tile you expect to run).
VMware, Inc.
51
Figure 3-2. Overview of Virtual Machine Creation for Standby Workloads
Create First Standby VM (standby0)
Install MS Windows Install VMware Tools
Configure
Clone for Additional Tiles
standby1
standby2
standby3
...
52
VMware, Inc.
Create the Olio Database Virtual Machines

This section describes the creation of the Olio database virtual machines using the pre-built template virtual machine available from VMware. See Figure 3-3 for an overview of the process. Alternatively, you may build this virtual machine from scratch, as described in Appendix A, Create and Configure the SLES Virtual Machines on page 112. To create the Olio database virtual machines 1 Using a Windows or Linux system, download from VMware the pre-built Olio database template virtual machine .tgz file. NOTE If you dont wish to use this pre-built virtual disk file, you can also configure the virtual machine from scratch, as described in Create and Configure the SLES Virtual Machines on page 112 and Prepare the Olio Database Virtual Machines on page 125. You would then skip the rest of this section. 2 Copy the template virtual machine .tgz file to a datastore accessible from the ESX/ESXi hosts in your cluster. NOTE This can be done using WinSCP or Cygwin from Windows, or SCP or an NFS mount from Linux. NOTE To enable remote tech support mode in ESXi, allowing SCP connections, see VMware Knowledge Base article #1017910 (http://kb.vmware.com/kb/1017910). 3 Uncompress the template virtual machine .tgz file by entering the following from an ESX/ESXi command line: cd /vmfs/volumes/<datastore> tar xvzf <template.tgz> (where <datastore> is the datastore and directory where you placed the virtual disk template file and <template.tgz> is the name of the virtual disk template file). Within the vSphere Client Inventory pane, select the desired ESX/ESXi host, select the Summary tab, in the Resources pane right-click the datastore where you placed the virtual machine files, and select Browse Datastore.... Navigate to the directory where you placed the virtual machine files. Right click on the .vmx file, select Add to Inventory, and follow the instructions to add the virtual machine to the vCenter inventory. Clone the pre-built virtual machine, as described in Clone the Olio Database Virtual Machine on page 128, and make as many clones of the Olio database virtual machine as you will need (one for each additional tile you expect to run).
5 6
VMware, Inc.
53
Figure 3-3. Overview of Virtual Machine Creation for Olio Database
Download Pre-Built Olio Database VM
Clone Pre-Built VM as OlioDB0
OlioDB1
OlioDB2
OlioDB3
...
54
VMware, Inc.
Create the Olio Web Virtual Machines

This section describes the creation of the Olio web virtual machines using the pre-built template VM available from VMware. See Figure 3-4 for an overview of the process. Alternatively, you may build this virtual machine from scratch, as described in Appendix A, Create and Configure the SLES Virtual Machines on page 112. 1 Using a Windows or Linux system, download from VMware the pre-built Olio web template virtual machine .tgz file. NOTE If you dont wish to use this pre-built virtual disk file, you can also configure the virtual machine from scratch, as described in Create and Configure the SLES Virtual Machines on page 112 and Prepare the Olio Web Virtual Machines on page 129. You would then skip the rest of this section. 2 Copy the template virtual machine .tgz file to a datastore accessible from the ESX/ESXi hosts in your cluster. NOTE This can be done using WinSCP or Cygwin from Windows, or SCP or an NFS mount from Linux. NOTE To enable remote tech support mode in ESXi, allowing SCP connections, see VMware Knowledge Base article #1017910 (http://kb.vmware.com/kb/1017910). 3 Uncompress the template virtual machine .tgz file by entering the following from an ESX/ESXi command line: cd /vmfs/volumes/<datastore> tar xvzf <template.tgz> (where <datastore> is the datastore and directory where you placed the virtual disk template file and <template.tgz> is the name of the virtual disk template file). Within the vSphere Client Inventory pane, select the desired ESX/ESXi host, select the Summary tab, in the Resources pane right-click the datastore where you placed the virtual machine files, and select Browse Datastore.... Navigate to the directory where you placed the virtual machine files. Right click on the .vmx file, select Add to Inventory, and follow the instructions to add the virtual machine to the vCenter inventory. Clone the pre-built virtual machine, as described in Clone the Olio Web Virtual Machine on page 134, and make as many clones of the Olio web virtual machine as you will need (one for each additional tile you expect to run).
5 6
VMware, Inc.
55
Figure 3-4. Overview of Virtual Machine Creation for Olio Web
Download Pre-Built Olio Web VM
Install Java, tc Server, etc.
Clone Pre-Built VM as OlioWeb0
OlioWeb1
OlioWeb2
OlioWeb3
...
56
VMware, Inc.
Create the DVD Store 2 Database Virtual Machines

This section describes the creation of the DVD Store 2 database virtual machines using the pre-built template VM available from VMware. See Figure 3-5 for an overview of the process. Alternatively, you may build this virtual machine from scratch, as described in Appendix A, Create and Configure the SLES Virtual Machines on page 112. To create the DVD Store 2 database virtual machines 1 Using a Windows or Linux system, download from VMware the pre-built DVD Store 2 database template virtual machine .tgz file. NOTE If you dont wish to use this pre-built virtual disk file, you can also configure the virtual machine from scratch, as described in Create and Configure the SLES Virtual Machines on page 112 and Prepare the DVD Store 2 Database Virtual Machines on page 135. You would then skip the rest of this section. 2 Copy the template virtual machine .tgz file to a datastore accessible from the ESX/ESXi hosts in your cluster. NOTE This can be done using WinSCP or Cygwin from Windows, or SCP or an NFS mount from Linux. NOTE To enable remote tech support mode in ESXi, allowing SCP connections, see VMware Knowledge Base article #1017910 (http://kb.vmware.com/kb/1017910). 3 Uncompress the template virtual machine .tgz file by entering the following from an ESX/ESXi command line: cd /vmfs/volumes/<datastore> tar xvzf <template.tgz> (where <datastore> is the datastore and directory where you placed the virtual disk template file and <template.tgz> is the name of the virtual disk template file). Within the vSphere Client Inventory pane, select the desired ESX/ESXi host, select the Summary tab, in the Resources pane right-click the datastore where you placed the virtual machine files, and select Browse Datastore.... Navigate to the directory where you placed the virtual machine files. Right click on the .vmx file, select Add to Inventory, and follow the instructions to add the virtual machine to the vCenter inventory. Prepare the DVD Store 2 database, as described in Prepare the DVD Store 2 Database on page 138. Steps 1-3 of that procedure have already been completed in the template virtual machine. Clone the pre-built virtual machine, as described in Clone the DVD Store 2 Database Virtual Machine on page 139, and make as many clones of the DVD Store 2 database virtual machine as you will need (one for each additional tile you expect to run).
5 6 7
VMware, Inc.
57
Figure 3-5. Overview of Virtual Machine Creation for DVD Store 2 Database
Download Pre-Built DVD Store 2 Database VM
Create Temporary 20GB Disk
Prepare Database
Clone Pre-Built VM as DS2DB0
DS2DB1
DS2DB2
DS2DB3
...
58
VMware, Inc.
Create the DVD Store 2 Web Virtual Machines

This section describes the creation of the DVD Store 2 web virtual machines using the pre-built template VM available from VMware. See Figure 3-5 for an overview of the process. Alternatively, you may build this virtual machine from scratch, as described in Appendix A, Create and Configure the SLES Virtual Machines on page 112. To create the DVD Store 2 web virtual machines 1 Using a Windows or Linux system, download from VMware the pre-built DVD Store 2 web template virtual machine .tgz file. NOTE If you dont wish to use this pre-built virtual disk file, you can also configure the virtual machine from scratch, as described in Create and Configure the SLES Virtual Machines on page 112 and Prepare the DVD Store 2 Web Virtual Machines on page 140. You would then skip the rest of this section. 2 Copy the template virtual machine .tgz file to a datastore accessible from the ESX/ESXi hosts in your cluster. NOTE This can be done using WinSCP or Cygwin from Windows, or SCP or an NFS mount from Linux. NOTE To enable remote tech support mode in ESXi, allowing SCP connections, see VMware Knowledge Base article #1017910 (http://kb.vmware.com/kb/1017910). 3 Uncompress the template virtual machine .tgz file by entering the following from an ESX/ESXi command line: cd /vmfs/volumes/<datastore> tar xvzf <template.tgz> (where <datastore> is the datastore and directory where you placed the virtual disk template file and <template.tgz> is the name of the virtual disk template file). Within the vSphere Client Inventory pane, select the desired ESX/ESXi host, select the Summary tab, in the Resources pane right-click the datastore where you placed the virtual machine files, and select Browse Datastore.... Navigate to the directory where you placed the virtual machine files. Right click on the .vmx file, select Add to Inventory, and follow the instructions to add the virtual machine to the vCenter inventory. Follow the instructions in Copy and Clone the DVD Store 2 Web Virtual Machine on page 141 to clone the pre-built virtual machine (naming the clone DS2WebA0) and configure its network and hostname settings, then to make two more copies (naming them DS2WebB0 and DS2WebC0) and configure their network and hostname settings, then make as many clones of the DVD Store 2 web virtual machines as you will need (three for each additional tile you expect to run), and configure their network and hostname settings.
5 6
VMware, Inc.
59
Figure 3-6. Overview of Virtual Machine Creation for DVD Store 2 Web
Download Pre-Built DVD Store 2 Web VM
Clone Pre-Built VM three times as DS2WebA0, DS2WebB0, & DS2WebC0
Clone for Additional Tiles (three copies for each tile)
DS2WebA1 DS2WebB1 DS2WebC1
...
60
VMware, Inc.
Configuring the Client Systems
This chapter describes the configuration required for the client machines, one of which is needed for each tile in the VMmark Benchmark test. This includes the operating system, the client software for workload virtual machines, and, on the prime client system, the VMmark Harness that controls the workloads. The chapter consists of the following sections: Prepare the Client Machine on page 61. Install Microsoft Windows Server on the Client Systems on page 63. Configure Microsoft Windows on the Client Systems on page 66. Install and Configure the VMmark Harness on page 75. Install the General Software on page 69. Create Additional Client Systems on page 83. Install and Configure the Workload Client Software on page 84.
Prepare the Client Machine

As of VMmark 2.1, client systems can be either physical machines or virtual machines, subject to the limitations described in Client Hardware Requirements on page 36. Proceed to the appropriate section below. NOTE The prime client must be a physical machine.
Prepare a Client Physical Machine

If you will be creating a client system on a physical machine, make sure the system meets the requirements detailed in Hardware Requirements for Physical Clients on page 36. Once the physical machine is ready, proceed to install Microsoft Windows Server, as described in Install Microsoft Windows Server on the Client Systems on page 63.
Prepare a Client Virtual Machine

This section describes the creation of a client virtual machine. You can either clone the template you created for the mailserver workload virtual machine or create new virtual machine from scratch.
VMware, Inc.
61
To clone the mailserver template workload virtual machine: 1 Clone the mailserver0 template you created in Create and Configure the Mail Server Virtual Machines on page 105, naming the clone client-template. NOTE The client virtual machines must not be run on a host that is part of the cluster under test. The prime client must be a physical machine. 2 3 4 Change the virtual machine memory size to 4GB. Remove the second virtual disk (used in the mailserver for the database and backup drives). Skip ahead to Install a Java JRE on page 72 and follow the instructions from that point forward.
To create a client virtual machine from scratch, follow these steps: Using the New Virtual Machine wizard in the VMware vCenter Client (Inventory > File > New > Virtual Machine), configure a new virtual machine with the following settings: 1 2 3 4 5 When you are asked to Select the configuration for the virtual machine, select the Custom radio button, then click Next. When you are asked to Specify a Name and Location for this Virtual Machine, enter client-template, then click Next. When you are asked to Select a Datastore..., select your desired datastore, then click Next. When you are asked for a Virtual Machine Version, select Virtual Machine Version: 7, then click Next. When you are asked to Specify the guest operating system..., select the Microsoft Windows radio button and in the Version: window choose Microsoft Windows Server 2008, Enterprise Edition (64-bit) or Microsoft Windows Server 2003, Enterprise Edition (32-bit), as appropriate, then click Next. When you are asked to specify the Number of virtual processors, select 4, then click Next. When you are asked the Memory Configuration, select 4GB, then click Next. When you are asked to Create Network Connections, set the number of NICs to 1, choose the network to use, choose the Adapter type, make sure Connect at Power On is checked, then click Next. NOTE If you will be using the high-performance VMXNET3 adapter, choosing it at this point will save a few steps later. If you do so, however, you might not have network access from the virtual machine until you install VMware Tools, which includes the custom VMXNET3 driver. 9 10 11 When asked Which SCSI controller type would you like to use?, choose either the LSI Logic Parallel or LSI Logic SAS, then click Next. When asked to Select the type of disk to use, select Create a new virtual disk, then click Next. When asked to Specify the virtual disk size and provisioning policy, enter 32GB, leave the Disk Provisioning options unchecked, select a location (either Store with the virtual machine or Specify a datastore:), then click Next. In the Advanced Options window, leave the options unchanged: Virtual Device Node: SCSI (0:0) Mode: Independent: leave unselected Then click Next. 13 At the Ready to Complete window, click Finish to create the virtual machine.
6 7 8
12
Once the virtual machine is ready, proceed to install Microsoft Windows Server, as described in Install Microsoft Windows Server on the Client Systems on page 63.
62
VMware, Inc.
Chapter 4 Configuring the Client Systems
Install Microsoft Windows Server on the Client Systems

This section describes the installation of Microsoft Windows Server on the client systems. NOTE The client systems should be named clientN.your.company.com, where N is the tile number that this client machine will be controlling, and your.company.com is your companys domain name. The first client would be client0.your.company.com, the second would be client1.your.company.com, and so on. NOTE The Windows Server 2003 instructions in this Chapter are based on the use of Windows Server 2003 Enterprise Edition with Service Pack 2. The installation and configuration of other releases and service packs vary slightly from these instructions. For Windows Server 2003 clients you can start with other Windows Server 2003 releases or service packs, but you must use 32-bit Enterprise Edition and you must upgrade each client system to at least Service Pack 2 before running the benchmark. The Windows Server 2008 instructions in this Chapter are based on the use of Windows Server 2008 Enterprise Edition with Service Pack 2. The installation and configuration of other releases and service packs might vary slightly from these instructions. For Windows Server 2008 clients you can use other Windows Server 2008 releases or service packs, but you must use 64-bit Enterprise Edition. Proceed to the appropriate section below: Install Microsoft Windows Server on a Physical System on page 63. Install Microsoft Windows Server on a Virtual System on page 63.
Install Microsoft Windows Server on a Physical System

Install Microsoft Windows Server 2003 Enterprise Edition (32-bit; with at least Service Pack 2) or Microsoft Windows Server 2008 Enterprise Edition (64-bit) on each client system, perform any configuration steps necessary for network access in your environment, and install all available updates, both high-priority and optional (but dont turn on automatic updating). NOTE Physical clients will typically obtain time synchronization from their mailserver. However, if your setup separates the prime client and client0 functions onto two servers, you will need to configure the prime client to use the same NTP time source as the vCenter server and ESX hosts, as described in Configure Time Synchronization for the vCenter Server on page 44. Once the installation is complete, proceed to Configure Microsoft Windows on the Client Systems on page 66.
Install Microsoft Windows Server on a Virtual System

Install Microsoft Windows Server 2003 Enterprise Edition (32-bit; with at least Service Pack 2) or Microsoft Windows Server 2008 Enterprise Edition (64-bit) on the virtual machine. The simplest way to install an operating system in a virtual machine is to use an ISO image (or images) of the installation disc (or discs). 1 2 3 4 5 6 Create an ISO image of the operating system installation disc(s). Do this with any third-party CD ripping software tool. Copy the ISO image file(s) to a file system accessible from the system running the VMware vCenter Client. Select the correct virtual machine (created in Prepare a Client Virtual Machine on page 61). From the Summary tab in the vCenter Client, under Commands, select Power On. Select the Console tab in the vCenter Client. You will see the virtual machine try to boot, but fail. Click the Connect/disconnect the CD/DVD... button in the menu bar (it looks like a CD with a wrench in front of it), select CD/DVD Drive 1, then select Connect to ISO image... (either on local disk or on a datastore). Browse to the ISO image of the first operating system installation disc, select it, and click Open.
VMware, Inc.
63
8 9
Still under the Console tab in the vCenter Client, click within the center of the console window (you may see no change, or your cursor may disappear). Restart the virtual machine by pressing <Ctrl><Alt><Insert> (this gets passed to the virtual machine as <Ctrl><Alt><Delete>, and causes it to reboot). The virtual machine will boot from the ISO image. NOTE If you have problems getting the system to boot from the ISO image, you can press <Esc> as the virtual machine is booting. This brings up a boot menu, from which you can select CD-ROM Drive.
10 11 12 13
Follow the Microsoft Windows installation prompts until the installation is complete. When the installation is complete and the system reboots, provide a new Administrator password, then follow the subsequent Windows prompts, accepting the defaults. After the Windows Desktop appears, an Initial Configuration Tasks wizard will come up. Make any configuration changes required, then click Close. Adjust the screen resolution if desired (Start Menu > Control Panel > Personalization > Display Settings).
Install the VMware Tools Package in Microsoft Windows

Follow these steps to install the VMware Tools package in the Microsoft Windows virtual machine: NOTE Among the drivers included in the VMware Tools package is a high-performance VMXNET3 network driver. To use this driver, it might be necessary to manually select it instead of the automatically-installed driver. Note that if you change network drivers after an operating system is installed, you might need to reconfigure any networking settings, for example, proxy server or IP address. For VMware ESX (as opposed to ESXi, which is addressed in the next section): 1 2 3 4 5 6 7 8 Power-on the virtual machine. In the vCenter Client, right-click on the virtual machine and select Guest > Install/Upgrade VMware Tools, then click OK. If an AutoPlay window appears in the guest operating system, click Run setup.exe Published by VMware, Inc. When a Welcome to the installation wizard for VMware Tools window appears, click Next. When the Setup Type window appears, choose Typical, and click Next. When the Ready to Install the Program window appears, click Install. When the Installation Wizard Completed window appears, click Finish. At the VMware Tools window, click Yes to allow the system to be restarted.
For VMware ESXi: 1 2 Power-on the virtual machine. From a web browser on the vCenter Server system, navigate to http://packages.vmware.com/tools/esx, select the ESX version and host operating system you're running, and download its .iso file, saving it to a file system accessible from the system running the VMware vCenter Client. Click the Connect/disconnect the CD/DVD... button in the menu bar (it looks like a CD with a wrench in front of it), select CD/DVD Drive 1, then select Connect to ISO image... (either on local disk, or on a datastore). Browse to the VMware Tools .iso file you just downloaded, select it, and click Open. When a Welcome to the installation wizard for VMware Tools window appears, this time inside the virtual machine console (it may take a few minutes), click Next.
4 5
64
VMware, Inc.
6 7 8 9
When the Setup Type window appears, choose Typical, and click Next. When the Ready to Install the Program window appears, click Install. When the Installation Wizard Completed window appears, click Finish. At the VMware Tools window, click Yes to allow the system to be restarted.
Enable Time Synchronization in VMware Tools

1 2 3 4 After the system has restarted, right-click the VMware icon in the system tray on the task bar and choose Open VMware Tools. Select the Options tab. Check the box next to Time synchronization between the virtual machine and the host operating system. Click OK to close the dialog box.
Install Windows Updates

For Windows Server 2003: Install all available windows updates, both high-priority and optional, by following these steps: 1 2 3 4 5 Run Windows Update (Start Menu > Windows Update). At the Microsoft Windows Update page, click Custom. Install any new Windows Update software, then install all available updates, both high-priority and optional. When the updates are complete, restart the system by clicking Restart now. After the system restarts, check again for updates. If there are more available, install them, restart the system, and repeat until no more updates appear.
Turn off automatic updates by following these steps: 1 2 3 4 Open the Control Panel (Start Menu > Settings > Control Panel). At the Control Panel, double-click Automatic Updates. At the Automatic Updates window, make sure the radio button for Turn off Automatic Updates is selected, then click OK. Close the Control Panel.
For Windows Server 2008: Install all available windows updates, both high-priority and optional, but dont turn on automatic updating, by following these steps: 1 2 3 4 5 6 7 Run Windows Update (Start Menu > All Programs > Windows Update). In the Windows Update window, click Change Settings. Under Important Updates select Never check for updates (not recommended), then click OK. Click the Check for updates button. Install any new Windows Update software, then install all available updates. When the updates are complete, restart the system by clicking Restart now. After the system restarts, check again for updates. If there are more available, install them, restart the system, and repeat until no more updates appear.
VMware, Inc.
65
Configure Microsoft Windows on the Client Systems

Once Microsoft Windows is installed in the client system according to the instructions in Install Microsoft Windows Server on the Client Systems on page 63, youll need to configure it, as described in this section.
Install Windows Server Components

Follow the instructions in the appropriate section below (for either Windows Server 2003 or Windows Server 2008) to install various Windows Server components. For Windows Server 2003: 1 2 3 4 5 6 7 8 9 10 Open the Start menu, then select Settings > Control Panel > Add or Remove Programs. In the left pane, select Add/Remove Windows Components. In the Components list click on Application Server, causing the component to be highlighted but not resulting in a checkmark in the box. Click the Details button. In the Subcomponents of Application Server list click on Internet Information Services (IIS), causing the component to be highlighted but not resulting in a checkmark in the box. Click the Details button. In the Subcomponents of Internet Information Services (IIS) list click the Common Files check box, resulting in a checkmark in the box. Click OK to return to Application Server window, then OK again to return to the Windows Component Wizard window. Click Next and complete the installation. At this point you might be prompted for an installation disc.
For Windows Server 2008: 1 2 3 4 5 6 7 8 9 Open the Start menu and select Server Manager. In the left pane of the Server Manager window, click Roles. In the right pane of the Server Manager window, click Add Roles. At the Before You Begin window, click Next. At the Select Server Roles window, add a checkmark next to Web Server (IIS). When the Add features required for Web Server (IIS)? window appears, click the Add Required Features button. Back at the Select Server Roles window, click Next. At the Web Server (IIS) window, click Next. At the Select Role Services window add checkmarks to the following role services: Basic Authentication Windows Authentication Digest Authentication Dynamic Content Compression IIS 6 Metabase Compatibility IIS 6 Management Console Still in the Select Role Services window, click Next. At the Confirm Installation Selections window, click Install. At the Installation Results window, wait for the installation to complete, then click Close. Close the Server Manager window.
VMware, Inc.
10 11 12 13
66
Disable Data Execution Prevention (DEP) on the Client

In order to avoid LoadGen crashes on the Windows Server 2003 client systems, disable the Data Execution Prevention (DEP) feature in the Microsoft Windows boot.ini file. NOTE This section is not required for Windows Server 2008 clients. More information about this feature may be found at: http://msdn.microsoft.com/en-us/library/ff557134.aspx To disable DEP 1 Open the file C:\boot.ini for editing a b c d 2 Open the System Properties utility (Start Menu > Control Panel > System). Click the Advanced tab. In the Startup and Recovery section, click the Settings button. In the System startup section, click the Edit button. The boot.ini file will open in Notepad.
Change the value of the /noexecute option to alwaysoff:

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows Server 2003, Enterprise" /noexecute=alwaysoff /fastdetect
3 4 5 6
Save the boot.ini file (File > Save) and close Notepad. Click OK to close the Startup and Recovery window. Click OK to close the System Properties utility. Reboot the client system.
Install or Update Windows PowerShell and vSphere PowerCLI

For Windows Server 2003
The prime client needs PowerShell version 2 and vSphere PowerCLI. It doesnt hurt, however, to have them on the other client systems as well. Install Windows PowerShell Version 2 1 Download the Windows Management Framework Core package update from: http://www.microsoft.com/downloads/details.aspx?FamilyId=f002462b-c8f2-417a-92a3-287f5f81407e&di splaylang=en Install the update (WindowsServer2003-KB968930-x86-ENG.exe) on the client systems.
Install vSphere PowerCLI 1 2 Download the latest version of VMware vSphere PowerCLI from: http://www.vmware.com/support/developer/PowerCLI/ Install the package on the client system. a b Accept all defaults, including VIX installation. After vSphere PowerCLI finishes installing, wait for the VIX installation to finish before continuing.
VMware, Inc.
67
For Windows Server 2008

Install Windows PowerShell 1 2 3 4 5 6 7 8 9 10 11 Open the Start menu and select Server Manager. In the left pane of the Server Manager window, click Features. In the right pane of the Server Manager window, click Add Features. In the Select Features window, add a checkmark next to Windows PowerShell, then click Next. In the Confirm Installation Selections window, click Install. At the Installation Results window click Close. Close the Server Manager window. Run Windows Update (Start Menu > All Programs > Windows Update). Click the Check for updates button. Install all available updates (including optional updates). When the updates are complete, restart the system.
Install vSphere PowerCLI 1 2 Download the latest version of VMware vSphere PowerCLI from: http://www.vmware.com/support/developer/PowerCLI/ Install the package on the client system. a b Accept all defaults, including VIX installation. After vSphere PowerCLI finishes installing, wait for the VIX installation to finish before continuing.
68
VMware, Inc.
Install the General Software

Each client system requires a number of support packages in order to interoperate with the VMmark Harness and with the workloads. The sections below describe the installation of these packages.
Install and Configure the STAF Package

Install and configure the STAF package in the client system, as described in the following sections. NOTE To avoid communication problems, we recommend that all clients and all workload virtual machines be running the same version of STAF.
Install the STAF Package

Follow these steps to install the STAF package in the client system: 1 2 Go to: http://staf.sourceforge.net/getcurrent.php Download the executable (*.exe) version 3.4.1 or later STAF package with a bundled JVM. For 32-bit operating systems, this will be: Windows 2000, XP, 2003, Vista, WinSrv2008, Windows 7 (32-bit) and the filename will be similar to: STAF344-setup-win32.exe (The exact name of the file might be different, depending on the current version of the package.) For 64-bit operating systems, this will be: Windows Server 2003/2008/2008R2, Vista, Windows 7 (AMD64, aka x64) and the filename will be similar to: STAF344-setup-winamd64.exe (Again, the exact name of the file might be different, depending on the current version of the package.) 3 Run the executable setup file, following the prompts to install the software, and leaving all options at their defaults.
Configure the Windows Server 2008 Firewall to Allow STAFProc.exe to Pass Through
Windows Server 2008 includes a firewall. Follow these steps to configure the Windows firewall to allow STAF to pass through: 1 2 3 4 5 6 7 8 Open the Start menu then select Control Panel. In the Control Panel window, double-click Windows Firewall. In the Windows Firewall window, click Allow a program through Windows Firewall. In the Windows Firewall Settings window click the Add Program... button. In the Add a Program window, click Browse. In the Browse window, navigate to C:\staf\bin\STAFProc.exe, then click the Open button. Back at the Add a Program window, click OK. Back at the Windows Firewall window click OK.
Configure STAF to Start Automatically

By default STAF starts when a user first logs in. Follow these steps to configure STAF to start when the system boots: For Windows Server 2003: 1 2
VMware, Inc.
Open the Start menu, and choose Control Panel > Scheduled Tasks > Add Scheduled Task. When the Scheduled Task Wizard appears, click Next.
69
Scroll to Start STAF 3.4.4, select it, and click Next. (The exact entry will vary depending on which version of STAF you have installed. You might have to browse to C:\STAF\startSTAFProc.bat.) Select the When my computer starts radio button, and click Next. NOTE For the prime client, select the When I log on radio button instead of the When my computer starts radio button.
5 6 7 8 9 10 11
Enter (or confirm) the Administrator user name, enter password, and click Next. Remove <computername> if it appears as a prefix. Check the Open Advanced properties for this task when I click Finish box. Click Finish. When the Start STAF properties window opens, select the Settings tab. If the box labeled Stop the task if it runs for [72] hours is checked, uncheck it. Click to OK to close the properties window. Restart windows.
For Windows Server 2008: 1 2 3 4 5 6 Open the Start menu then select Control Panel. In the Control Panel window, double-click Administrative Tools. In the Administrative Tools window, double-click Task Scheduler. In the Task Scheduler window, in the right-most pane (Actions), click Create Basic Task. In the Create Basic Task Wizard window, in the Name: filed, enter STAF, then click Next. In the Task Trigger window, select When the computer starts, then click Next. NOTE For the prime client, select the When I log on radio button instead of the When the computer starts radio button. 7 8 9 10 11 12 13 14 15 16 17 18 In the Action window, select Start a program, then click Next. In the Start a Program window, click Browse, go to C:\STAF\, select the startSTAFProc Windows Batch File (not the startSTAFProc Icon), then click Open. Back at the Start a Program window, click Next. At the Summary window, add a check mark next to Open the Properties dialog for this task when I click Finish, then click Finish. At the STAF Properties (Local Computer) window, under the General tab, select the Run whether user is logged on or not radio button. Click the Change User or Group... button. At the Select User or Group window, enter the Administrator username and click OK. Back at the STAF Properties (Local Computer) window, select the Settings tab and remove the check mark next to Stop the task if it runs longer than:, then click OK. At the Task Scheduler window, enter the administrator password, then click OK. Close the Task Scheduler window. Close the Administrative Tools window. Restart windows.
70
VMware, Inc.
Create a Customized STAF.cfg File

1 Using WordPad (Start Menu > All Programs > Accessories > WordPad), open C:\STAF\bin\STAF.cfg. (To see this file, you have to change Files of type to All Documents (*.*) within WordPad.) NOTE If you have customized a STAF.cfg file for another client, starting with that one can save some of the steps below. If, however, you use the STAF.cfg from Client 0 for another client, comment out the service stax and service event entries (be sure to get all the lines, including those that are continued with a backslash) 2 In the trust level block add the networking mask for the systems to be trusted and set them to trust level 5. For example: trust machine client0 level 5 Alternatively, to trust all machines in the 192.168 network, add the line: trust machine 192.168.*.* level 5 If this system is a prime client add the following entries: service stax library JSTAF execute C:\STAF\services\stax\STAX.jar OPTION \ J2=-Xverify:none parms "numthreads 100 logtcelapsedtime enabled logtcstartstop \ enabled" service event library JSTAF execute C:\STAF\services\stax\STAFEvent.jar 4 5 Save the edited file as C:\STAF\bin\STAF.cfg and close WordPad. It will save as STAF.cfg.txt, despite showing as STAF.cfg. Configure Windows to show file extensions even for known file types: a b c d 6 Open the Start menu then select Control Panel. In the Control Panel window, double-click Folder Options. In the Folder Options window, under the View tab, uncheck the box next to Hide extensions for known file types, then click OK. Close the Control Panel window.
Rename the file C:\STAF\bin\STAF.cfg.txt as C:\STAF\bin\STAF.cfg.
Install the STAX Execution Engine

NOTE STAX is only required on the prime client. Its presence on the other clients, however, does not cause problems. Follow these steps to install and configure the STAX execution engine in the client system: 1 2 3 Download version 3.4.1 or later of the STAX package for Windows from: http://staf.sourceforge.net/getstax.php Create a services directory under the STAF directory. Uncompress the zip file and place the stax directory it contains in C:\STAF\services.
Install the STAF EVENT Service

Follow these steps to install and configure the STAF EVENT service in the client system: 1 2 Download version 3.1.4 or later of the EVENT package from: http://staf.sourceforge.net/getservices.php Uncompress the zip file, placing the event directory it contains into the C:\STAF\services directory. You should end up with:
VMware, Inc.
71
C:\STAF\services\event\event.pdf C:\STAF\services\event\History C:\STAF\services\event\License.htm C:\STAF\services\event\STAFEvent.jar 3 Restart the client system.
Install a Java JRE

Install a Java JRE. You can use any vendors JRE, as long its a released version (i.e., not a beta), is publicly available, and is a default build type. Details of this JRE must be included in the disclosure file for a result to be compliant. NOTE If you used the mailserver template as a starting point for a client virtual machine you might need to revisit your network settings. Part of the preparation of that template is to configure Internet Explorer to not use a proxy server. Make the following adjustments when installing the JRE: 1 Install the package (for example, jre-windows-6u23-x64) on the client system, accepting all defaults except the installation directory, which should be changed from the default (often C:\Program Files\Java) to C:\Java. Set the JAVA_HOME environment variable (which you might have to create) to correspond to your java installation path (which might vary, depending on which java version you installed). For example: JAVA_HOME = "c:\Java\jre6\" For Windows Server 2003: a b c d Open the Start menu, select Settings, then select Control Panel. In the Control Panel window, open System. In the System Properties window, select the Advanced tab, then click the Environment Variables button. Under System variables, look for JAVA_HOME. If the variable is not yet defined, click the New button, add the variable and set it to correspond to your Java installation path. If the variable is already defined, click the Edit button and set it to correspond to your Java installation path. For Windows Server 2008: a b c d e Open the Start menu then select Control Panel. In the Control Panel window, open System. In the System window, select the Advanced System Setting. In the System Properties window, click the Environment Variables button. Under System variables, look for JAVA_HOME. If the variable is not yet defined, click the New button, add the variable and set it to correspond to your Java installation path. If the variable is already defined, click the Edit button and set it to correspond to your Java installation path.
72
VMware, Inc.
Install the Cygwin Environment

Follow these steps to download and install the Cygwin environment: 1 2 Download the Cygwin setup utility from: http://www.cygwin.com/setup.exe If this is your first client system save time by first downloading the files without installing: a b c d e f Download the Cygwin setup utility, setup.exe, saving it to the Desktop. Run the Cygwin setup utility, setup.exe. At the Cygwin Net Release Setup Program window, click Next. At the Choose A Download Source window, select Download Without Installing and click Next. Follow the remaining prompts to choose a local package directory, connection type, and download site. At the top of the list in the Select Packages window, next to the word All, click once on the word Default. The status of all the packages will change from Default to Install. Click Next. NOTE This download may take a long time. h i j 3 4 5 6 If you see a Resolving Dependencies window, accept the additional packages required and click Next. At the Installation Complete window, click Finish. Make the Cygwin download directory available to all the other clients over the network.
Run the Cygwin setup utility, setup.exe, that you saved to the Desktop earlier. At the Cygwin Net Release Setup Program window, click Next. At the Choose A Download Source window, select Install from Local Directory and click Next. Select the directory where you made the Cygwin install files available (probably a local directory if you are still on client 0 or a shared network directory if you are on another client), leave the All Users radio button selected, and click Next. Accept the defaults until the Select Packages window appears. At the top of the list, next to the word All, click once on the word Default. The status of all the packages will change from Default to Install. Click Next. If you see a Resolving Dependencies window, accept the additional packages required and click Next. At the Create Icons window, click Finish.
7 8 9 10 11
Follow these steps to configure the Cygwin environment: 1 2 3 4 5 Open a Cygwin shell (Start Menu > All Programs > Cygwin > Cygwin Bash Shell). Run the following command: cygserver-config When asked Do you want to install cygserver as a service?, answer yes and press <Enter>. The script will install the service, then display Basic Cygserver configuration finished. Have fun!. Close the Cygwin shell. Set the CYGWIN=server and Path environment variables as follows:
VMware, Inc.
73
For Windows Server 2003: a b c d e f g h Open the System Properties window (Start Menu > Control Panel > System). In the System window, select the Advanced tab. Click the Environment Variables button. Below the System variables window, click New. In the Variable name field enter CYGWIN, in the Variable value field enter server, then click OK. In the System variables window, select the Path variable, then click Edit. At the end of the Variable value, append: ;C:\cygwin\bin Click OK three times to exit.
For Windows Server 2008: a b c d e f g h i 6 7 Open the Start menu then select Control Panel. In the Control Panel window, double-click System. In the System window, click Advanced system settings. Click the Environment Variables button. Below the System variables window, click New. In the Variable name field enter CYGWIN, in the Variable value field enter server, then click OK. In the System variables window, select the Path variable, then click Edit. At the end of the Variable value, append: ;C:\cygwin\bin Click OK three times to exit.
Reboot the client system. To verify that Cygwin started automatically, open a Windows command prompt and type: net start If Cygwin is running you will see CYGWIN cygserver within the alphabetical output of this command.
74
VMware, Inc.
Install and Configure the VMmark Harness

This section describes the installation of the VMmark Harness and the software it needs in order to run. For an overview of the VMmark Harness, see VMmark Harness on page 26. In addition to the instructions included in this document, you should also refer to any Readme.txt files included in the VMmark kit.
Install the VMmark Harness Package

To install the VMmark Harness Package 1 If your VMmark Harness package came on a CD, copy the contents (the VMmark2, vclient0, and doc directories) from the distribution CD to the root directory of the C:\ drive. If your VMmark Harness package came as a zip file, uncompress the file and copy the contents (the VMmark2, vclient0, and doc directories) to the root directory of the C:\ drive. 2 For any client other than client 0, change the name of the vclient0 directory to match the tile that this client will be controlling. On client 1 this directory becomes vclient1; on client 2 it becomes vclient2, and so on. NOTE Be careful about the difference between the client names (client0, client1, client2, etc.) and the directory names on those clients (vclient0, vclient1, vclient2, etc.).
Configure the VMmark Harness

The VMMARK2.CONFIG file, located in C:\VMmark2 directory, is used to configure the VMmark benchmark. A sample of the file is included below. This file identifies the client systems and the server virtual machines that have been configured to host specific workloads. The file specifies the number of tiles to run in the test as well as the workloads to run within each tile (with the default being the official VMware VMmark 2.1 workload set). NOTE This file is best edited with a program such as WordPad that correctly handles Unix/Linux style text files, rather than Notepad, that does not. If your test benchmark runs will include multiple simultaneous clone and deploy operations (as calculated in Prepare the vSphere vCenter Server for the Deploy Infrastructure Operations on page 46), you will also need to set the following variables in the VMMARK2.CONFIG file: Deploy/Templates="standby_upgrade" This should be the VC name of the template you create in Prepare the Template for the Deploy Operation on page 124. Deploy/OScustomizations="VMmark2_Standby_Custom" This is the name (or names) of the OS customization specification (or specifications) you created in Prepare the vSphere vCenter Server for the Deploy Infrastructure Operations on page 46. Deploy/DeployLUNs="Shared_Migration_DS" The name of the LUNs on which this operation will be conducted. To perform multiple operations on the same LUN, repeat the name as necessary. Deploy/DeployVMs="DeployVM" The VMname (or VMnames) in the OScustomization (or OScustomizations) created above, which should also be in the prime clients hosts file. Once you have edited this file to reflect your test setup, the VMmark Harness configuration is complete.
# Test Vmmark Config File # Copyright 2006 - 2011 VMware, Inc. All rights reserved. # ------------------------------------------------------------------# Test Configuration Settings # -------------------------------------------------------------------
VMware, Inc.
75
# Number of Tiles to Run TILES=1 # ------------------------------------------------------------------# Infrastructure Configuration Settings # ------------------------------------------------------------------VCServerHOSTNAME="vcserver1" VCServerUSER="Administrator" VCServerPASSWORD="" VCServerCLUSTER="cluster0" Deploy/Templates="StandbyTemplate1" Deploy/OScustomizations="StandbyCust StandbyCust2" Deploy/DeployLUNs="CX3-80-Lun24 CX3-80-Lun23" Deploy/DeployVMs="DeployVM1 DeployVM2" # - Optional Settings - defaults shown # - Warning : changing values may make resulting runs non-compliant # Deploy/DELAYTIME="170" # Deploy/UpgradeTools="1" # Deploy/BurstQueueSize="auto" # Deploy/DeployDelayBetween="300" # # # # # # - Optional Settings - defaults shown - Warning : changing values may make resulting runs non-compliant Vmotion/Workloads="OlioDB" Vmotion/VMotionDelayBetween="180" Vmotion/BurstQueueSize="auto" Vmotion/DELAYTIME="150"
SVmotion/TargetLUNs="CX3-80-Lun21 CX3-80-Lun21" # - Optional Settings - defaults shown # - Warning : changing values may make resulting runs non-compliant # SVmotion/Workloads="Standby" # SVmotion/SleepBetween="300" # SVmotion/BurstQueueSize="auto" # SVmotion/DELAYTIME="160" # - Note : Vmotion and SVmotion infrastructure operations can not operate on the same workload(s) # - at the same time. # ------------------------------------------------------------------# Workload Configuration Settings # ------------------------------------------------------------------MailServer/MailPassword="" MailServer/MailDomains="maildomain0 maildomain1" MailServer/MailQualifier="eng.vmware.com" # - Note : MailQualifier must be 3 parts : For example "eng.vmware.com" # # - Optional Settings - defaults shown # - Warning : changing values may make resulting runs non-compliant # MailServer/SKIPRESTORE="0" # MailServer/DELAYTIME="12"
# # # # # # # # # # # # #
- Optional Settings - defaults shown - Warning : changing values may make resulting runs non-compliant OlioWeb/SKIPRESTORE="0" OlioWeb/DELAYTIME="5" OlioWeb/RAMPUP="300" OlioWeb/RAMPDOWN="300" OlioWeb/SCALE="400" OlioWeb/FINALIZE="10" OlioWeb/OLIORAIN="1" OlioWeb/FinalizeGlassFish="0" OlioDB/SKIPRESTORE="0" OlioDB/SCALE="400"
# - Optional Settings - defaults shown
76
VMware, Inc.
# # # # # # # # # # # # # # #
- Warning : changing values may make resulting runs non-compliant DS2WebA/SKIPRESTORE="0" DS2WebA/DELAYTIME="8" DS2WebB/SKIPRESTORE="0" DS2WebB/DELAYTIME="10" DS2WebB/IterationTimeSlice="240" DS2WebB/DelayTimeSlice="60" DS2WebC/SKIPRESTORE="0" DS2WebC/DELAYTIME="15" DS2WebC/IterationTimeSlice="180" DS2WebC/DelayTimeSlice="120" DS2DB/SKIPRESTORE="0"
# - Optional Settings - defaults shown # - Warning : changing values may make resulting runs non-compliant # Standby/DELAYTIME="2" # ------------------------------------------------------------------# Testbed and Environment Configuration Settings # ------------------------------------------------------------------# Set up the list of client systems - must have # at least as many clients as tiles. CLIENTS="client0 client1" # Workload specific list of Server VM Hostnames MailServer/SERVERS="Mailserver0 Mailserver1" Standby/SERVERS="Standby0 Standby1" OlioWeb/SERVERS="OlioWeb0 OlioWeb1" OlioDB/SERVERS="OlioDB0 OlioDB1" DS2WebA/SERVERS="DS2WebA0 DS2WebA1" DS2WebB/SERVERS="DS2WebB0 DS2WebB1" DS2WebC/SERVERS="DS2WebC0 DS2WebC1" DS2DB/SERVERS="DS2DB0 DS2DB1" # - Note: For best results follow the above naming convention # # # # # # # # # # # Reporter Settings REPORTER=0 REPORTERDIR="/cygdrive/c/vmmark2/tools/" ScriptName="vSphere-get-config.sh" Error handling This will determine how the harness deals with errors during the phases before running. By setting ERRORIMMEDIATE to 1, the harness will exit upon finding the first error (as opposed to continuing through all the remaining work for that phase). ERRORIMMEDIATE=0
################################################################ # Standard Defaults # # All parameters below this line have the standard default # values listed. Normally these values do not require # changing. For the path values, we recommend that you # configure the testbed using the standard path names as # this helps facilitate debugging. There are several numeric # values that have occasionally been shown to be useful for # benchmarkers, but care should be taken when modifying these # values. Uncommenting these parameters will be denoted as # "Overriding Required Default". Changing key parameters # such as RUNTIME will cause the test run to be non-compliant. # ################################################################ # Set the RUNTIME for the test. The Default
VMware, Inc.
77
# for a Fully Compliant test is 10800 secs # (3 hours). Changing this value can make # your test non-compliant ###RUNTIME=10800 # TILEDELAY=seconds # # Delays the start of the tile N by TILEDELAY * N (secs) and also # increments the runtime of the workloads by TILEDELAY * (#TILES - N - 1) # to compensate for the delay. No delay if set to 0. ###TILEDELAY=60 # FLAG to control display/logging of Info/Debug messages # when debugSTAXUtilLogAndMsg is called # if == 0: No messages are displayed/logged # if != 0: Messages are displayed and logged ###DEBUGFLAG=0 # This is a list of the workloads to be invoked # this is to allow a nonstandard list of workloads # to be run with this harness as long as the template # workload functions are in the vmmark/xml directory. # For a fully compliant test all 6 workloads must be # used (default) ###WORKLOADLIST="Standby MailServer OlioWeb OlioDB DS2WebA DS2WebB DS2WebC DS2DB" # This is a list of the infrastructure operations to be invoked # this is to allow a nonstandard list of infrastructure operations # to be run with this harness as long as the template # workload functions are in the vmmark/xml directory. # For a fully compliant test all 3 infrastructure operations must be # used (default) ###INFRASTRUCTURELIST="Deploy Vmotion SVmotion" # Flag to disable the Cluster VM Mapping routines ###DoClusterMapping=1 #!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! # Note: Please observe the special formatting used with # some of the non-numeric character strings in various # parameter settings. Quotes, double backslashes, %u, # and use of path names of the form "C:\Progra~1\... # are required where used in the sample VMMARK2.CONFIG # files to avoid character striping by the shells or # to avoid the use of path names with spaces which are # not supported in this file. #!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! # This directory is the location of the # Harness code ###IMPORTDIR=c:\vmmark2\xml # This is the location of any config files # referenced by the harness code. ###CONFIGDIR=c:\vmmark2 # This is a tmp directory for scratch files # the harness creates ###TMPDIR=c:\vmmark2\tmp # # # # # # # # This is the path on the VMmark clients that contains the individual workload directories. The directory on each client must be numbered by that client's tile number (i.e., vclient0, vclient1, etc.). The VMmark Harness software automatically replaces "%u" in the path below with the appropriate tile number. Note: The extra backslash is to escape the backslash-v that was being treated as a control char when passed
78
VMware, Inc.
# to windows batch files. ###CLIENTROOTDIR=C:\\vclient%u # Parallelize Workload Setup # # Set to 0 to disable ###PARALLELSETUP=1 # FLAG to control cleaning up results files after they've been # copied into the vmmark/results/Results_<datestamp> directory # if set to 0, Cleanup of files will be skipped. ###CLEANUPFLAG=1 # # Post Process Results # Disabled by default except for full runs where: # RUNTIME=10800 and WORKLOADS="<list of all 6>" # Note: %u will be #Tiles in run # The output is sent to: vmmark/results/Results_<timestamp>/Score_#_tile.out # The working directory is vmmark/results/Results_<timestamp> # To post process partial runs, set flag to 1 and modify the post processing # command as needed. # ###POSTPROCESSFLAG=0 ###POSTPROCESSCMD="C:\\cygwin\\bin\\perl -f C:\\vmmark2\\tilescore.pl -t %u" #02112011
Edit the cygserver.conf File

Edit the /etc/cygserver.conf file, uncommenting all values, including shmem and sem. This can be accomplished automatically as follows: 1 2 Open a Cygwin shell (Start Menu > All Programs > Cygwin > Cygwin Bash Shell). Within the Cygwin shell, run the following script: c:/vclient0/scripts/set_cygserver.sh (for clients other than client 0, substitute the proper vclientN directory). Close the Cygwin shell.
Edit Registry Settings in the Client System

The Windows Registry settings described in this section avoid problems with port utilization and improve efficiency during VMmark benchmark runs. This can be done automatically or manually, as described in the following two sections.
Automatically Apply the Registry Settings

1 2 Navigate to C:\vclientN\scripts directory. (Where N is the tile number.) Double click the vmmark_client.reg file. NOTE This file is installed during the VMmark harness installation. 3 4 When asked Are you sure you want to add the information in C:\vclient\scripts\vmmark_client.reg to the registry? click Yes. Repeat step 2 and step 3 for the ProxySettingsPerUser.reg file.
VMware, Inc.
79
Manually Apply the Registry Settings

If you applied the settings using the automatic method above you should skip this section. 1 2 3 Open the Windows Registry editor (Start Menu > Run > Regedit). Navigate to the following registry folder: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters Modify the registry entry listed in Table 4-1: a b c d Right-click the entry. Select Modify. Enter the new data (from Table 4-1). Click OK.
Table 4-1. Registry Entry to Modify

Action Modify Name ReservedPorts Type Multi String Value (REG_MULTI_SZ) Data 1025-5000
In the parameters folder, add the registry entries listed in Table 4-2: a b a b c d e f Create the new entry (Edit Menu > New > DWORD Value). Type the name of the new entry (from Table 4-2). Right-click the new entry. Select Modify. Select the Decimal radio button. Enter the new data (from Table 4-2). Click OK. Repeat for each new entry.
Name MaxFreeTWTcbs MaxUserPort TcpTimedWaitDelay TcpWindowSize Type DWORD Value (REG_DWORD) DWORD Value (REG_DWORD) DWORD Value (REG_DWORD) DWORD Value (REG_DWORD) Data 100000 65534 60 65535
Table 4-2. Registry Entries to Add

Action Add Add Add Add
Navigate to the following registry folder: HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\windows\CurrentVersion\Internet Settings Add the registry entry listed in Table 4-3: a b a b c d Create the new entry (Edit Menu > New > DWORD Value). Type the name of the new entry (from Table 4-3). Right-click the new entry. Select Modify. Select the Decimal radio button. Enter the new data (from Table 4-3).
80
VMware, Inc.
e
Action Modify
Click OK.
Name ProxySettingsPerUser Type DWORD Value (REG_DWORD) Data 0
Table 4-3. Registry Entry to Modify
Close the Registry Editor.
Update the hosts File

The client systems should be able to resolve all the workload virtual machine names in your cluster. These are the names by which the vCenter Server knows the workload virtual machines and is able to perform operations on them. The following is an example hosts file for the clients in a two-tile setup. You'll notice that the fully-qualified name of the mailserver is provided for each tile. Example hosts file:
192.168.7.199 #Tile 0 192.168.7.2 192.168.7.3 192.168.7.4 192.168.7.5 192.168.7.6 192.168.7.7 192.168.7.8 192.168.7.9 #Tile 1 192.168.7.12 192.168.7.13 192.168.7.14 192.168.7.15 192.168.7.16 192.168.7.17 192.168.7.18 192.168.7.19 DeployVM OlioDB0 OlioWeb0 mailserver0 mailserver0.maildomain0.eng.vmware.com maildomain0 maildomain0.eng.vmware.com DS2DB0 DS2WebA0 DS2WebB0 DS2WebC0 standby0 OlioDB1 OlioWeb1 mailserver1 mailserver1.maildomain1.eng.vmware.com maildomain1 maildomain1.eng.vmware.com DS2DB1 DS2WebA1 DS2WebB1 DS2WebC1 standby1
Obtain Exchange Server 2007 and LoadGen

Obtain Exchange Server 2007 as follows: For Windows Server 2003: 1 Download Exchange Server 2007. This must be the same service pack version (if any) that you used for the Exchange mail server in the workload virtual machines (Install Microsoft Exchange Server 2007 on page 121), but in this case must be the 32-bit version. For example, if you used Exchange Server 2007 Service Pack 2 in the mail server workload virtual machines, you would download the 32-bit version (E2K7SP2EN32.exe) from: http://www.microsoft.com/downloads/details.aspx?FamilyID=4c4bd2a3-5e50-42b0-8bbb-2cc9afe3216a 2 Save the Exchange Server 2007 install file to your client system for use in Install Exchange Server 2007 Tools on page 86.
VMware, Inc.
81
For Windows Server 2008: 1 Obtain access to the 64-bit Exchange Server 2007 installation file you used for the Exchange installation in the mailserver workload virtual machine (Install Microsoft Exchange Server 2007 on page 121). NOTE If you download this file from scratch, you must make sure to use the same service pack version (if any) for the Exchange Management Tools in the clients and for the Exchange mail server in the workload virtual machines. If you used Exchange Server 2007 Service Pack 2 in the mail server workload virtual machines, you would download the 64-bit version (E2K7SP2EN64.exe) from: http://www.microsoft.com/downloads/details.aspx?FamilyID=4c4bd2a3-5e50-42b0-8bbb-2cc9afe3216a 2 Save the Exchange Server 2007 install file to your client system for use in Install Exchange Server 2007 Tools on page 86.
Download LoadGen as follows: 1 Download the LoadGen tool (version 08.02.0045 or later) from: 32-bit version:
http://www.microsoft.com/downloads/details.aspx?FamilyId=DDEC1642-F6E3-4D66-A82F-8D3062C6FA98
64-bit version
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=0fdb6f14-1e42-4165-bb17-96c83916c3ec
Save the LoadGen install file to your client system for use in Install and Configure LoadGen on the Client on page 87.
Configure Windows to Not Use a Proxy Server

Configure Windows to not use a proxy server by following these steps: 1 2 Log in as the domain administrator. Open the Control Panel: Windows Server 2003: Open the Start menu then select Settings > Control Panel. Windows Server 2008: Open the Start menu then select Control Panel. In the Control Panel window, double-click Internet Options. In the Internet Properties window, under the Connections tab, click the LAN settings button. At the Local Area Network (LAN) Settings window, make sure none of the boxes contain a checkmark. Close the Local Area Network (LAN) Settings window by clicking OK. Close the Internet Options window by clicking OK. Close the Control Panel.
3 4 5 6 7 8
82
VMware, Inc.
Create Additional Client Systems

If you will be running multiple VMmark tiles, you need additional client systems (one client per tile). You can configure each additional client manually (repeating the above directions for each system), copy the disk images using a commercial third-party disk-imaging tool along with the Windows System Preparation tool (Sysprep), or (if using virtual clients) clone the virtual machine. If you will be copying the disk image or cloning the virtual machine, you should do so before installing and configuring the mail server client.
Copying the Disk Image

If you copy the disk image from the first client to create additional client systems, you must customize each additional system as follows: 1 2 Rename the C:\vclient0 directory to C:\vclientN, where N is the tile number that this client machine will be controlling. Edit the C:\STAF\bin\STAF.cfg file to reflect this client systems short name.
Cloning a Virtual Machine

If you have a virtual client and will be making additional client systems, you can use vSphere to clone the client-template virtual machine. These new clients should be named clientN, where N is the tile number the client will be controlling, and their C:\STAF\bin\STAF.cfg and hosts files should be edited appropriately.
Configuring Additional Clients Manually

To configure each client manually, you would repeat the instructions in this chapter for each desired client system, changing the system name and the vclientN directory to reflect which tile the client will be controlling, and editing the C:\STAF\bin\STAF.cfg and hosts files appropriately.
VMware, Inc.
83
Install and Configure the Workload Client Software

This section describes the installation of the client software. In addition to the instructions included in this document, you should also refer to the various Readme.txt files included in the VMmark kit.
Install the Standby Client Software

The standby workload is monitored by the IdleVMTest client. This client software was installed as part of the VMware Harness installation, described in Install and Configure the VMmark Harness on page 75. No separate installation is necessary.
Install the Olio Client Software

The Olio client software is installed as part of the VMmark Harness installation. No separate installation is necessary.
Install the DVD Store 2 Client Software

The DVD Store 2 client software is installed as part of the VMmark Harness installation. No separate installation is necessary.
Install and Configure the Mail Server Client

The client portion of the mail server consists of the Microsoft Exchange Load Generator, LoadGen. The installation is described in the sections below.
Set the Clients Preferred Domain Name Server

Power-on the client machine, then change its preferred domain name server to its tiles mail server virtual machine, as follows: For Windows Server 2003 1 2 3 4 5 6 7 8 9 On the client system, open the Start menu, then select Control Panel > Network Connections > Local Area Connection. In the Local Area Connection Status window, click the Properties button. Select Internet Protocol (TCP/IP), then click the Properties button. In the Internet Protocol (TCP/IP) Properties window, click the Advanced button. In the Advanced TCP/IP Settings window, select the DNS tab. Below DNS server addresses, in order of use:, click Add. Enter the IP address of this tiles mail server, then click Add. Select the newly-added IP address and use the arrows at the right to move it to the top of the list. Click OK twice, then click Close.
For Windows Server 2008 1 2 3 4 5 On the client system, open the Start menu, then select Control Panel > Network and Sharing Center. In the Network and Sharing Center window, under Network, find the Local Area Connection entry and click View Status. In the Local Area Connection Status window, click the Properties button. Select Internet Protocol Version 4 (TCP/IPv4), then click the Properties button. In the Internet Protocol Version 4 (TCP/IPv4) Properties window, click the Advanced button.
84
VMware, Inc.
6 7 8 9 10 11
In the Advanced TCP/IP Settings window, select the DNS tab. Below DNS server addresses, in order of use:, click Add. Enter the IP address of this tiles mail server, then click Add. Select the newly-added IP address and use the arrows at the right to move it to the top of the list. Click OK twice, then click Close twice. Close the Network and Sharing Center window.
Make the Client System a Domain Member

The client system must be a member of the domain controlled by its tiles mail server virtual machine. This tiles mail server virtual machine must already be powered-on and connected to the network during the installation. NOTE If the client system has more than one network card, it may be necessary to temporarily disable the network cards that will not be using the tiles mail server as a domain name server. These additional cards can then be re-enabled after the client system part of the mail servers domain and the preferred DNS is set. For Windows Server 2003 1 2 3 4 5 On the client system, open the Start menu, and select Control Panel > System. Select the Computer Name tab. Click the Change button. Under Member of select the Domain radio button. Enter the fully-qualified domain name that you used when you created the mail server domain. This should be in the format maildomainN.your.company.com, where N is the tile number that this virtual machine will be part of, and your.company.com is your companys domain name. Click OK. Enter a user name and password (the domain controllers administrator account and password will work). Click OK. When the window appears welcoming you to the new domain, click OK. When the window appears saying You must restart this computer..., click OK.
6 7 8 9 10
For Windows Server 2008 1 2 3 4 5 6 7 On the client system, open the Start menu, and select Control Panel. In the Control Panel double-click System. Select the Advanced System Settings. Select the Computer Name tab. Click the Change button. Under Member of select the Domain radio button. Enter the fully-qualified domain name that you used when you created the mail server domain. This should be in the format maildomainN.your.company.com, where N is the tile number that this virtual machine will be part of, and your.company.com is your companys domain name. Click OK. Enter a user name and password (the domain controllers administrator account and password will work).
8 9
VMware, Inc.
85
10 11
Click OK. When the window appears welcoming you to the new domain, click Restart Computer.
Install Exchange Server 2007 Tools

The mail server client uses Exchange Server 2007 Tools. The following two sections describe the installation for either Windows Server 2003 or Windows Server 2008. For Windows Server 2003: 1 2 Log in as the domain administrator. Run the Exchange Server 2007 install file you obtained in Obtain Exchange Server 2007 and LoadGen on page 81 (i.e., E2K7SP2EN32.exe if youre using Service Pack 2). NOTE Depending on operating system version used for clients, you might need to install Windows Installer 4.5 (WindowsServer2003-KB942288-v4-x86.exe). If you get a warning about this, go ahead and install the new installer. 3 4 5 6 7 At the Extraction Complete message, click OK. Double-click to file setup.exe. Choose Install Microsoft Exchange Server 2007 SP2. (Other Service packs will be slightly different, but choose the relevant install option.) Accept the defaults until you get to the Installation Type menu, where you should choose Custom Exchange Server Installation. At the Server Role Selection window, place a checkmark in the box next to Management Tools and click Next. If you see a warning about Host (A) record, you can safely ignore it. At the Readiness Checks window, after the checks are completed, click Install. Check for updates via Microsoft update.
8 9
For Windows Server 2008: 1 2 3 4 5 6 7 Log in as the domain administrator. Run the Exchange Server 2007 install file you obtained in Obtain Exchange Server 2007 and LoadGen on page 81 (i.e., E2K7SP2EN64.exe if youre using Service Pack 2). At the Extraction Complete message, click OK. Double-click to file setup.com. Choose Install Microsoft Exchange Server 2007 SP2. (Other Service packs will be slightly different, but choose the relevant install option.) Accept the defaults until you get to the Installation Type menu, where you should choose Custom Exchange Server Installation, then click Next. At the Server Role Selection window, place a checkmark in the box next to Management Tools and click Next. If you see a warning about Host (A) record, you can safely ignore it. At the Readiness Checks window, after the checks are completed, click Install. Check for updates via Microsoft update.
8 9
86
VMware, Inc.
Install and Configure LoadGen on the Client

Follow the instructions in the sections below to install and configure LoadGen on the client. NOTE Before installing LoadGen, it is important that all the steps described in Install Microsoft Windows Server on the Client Systems on page 63 be completed. 1 2 Install LoadGen by double-clicking the loadgen.msi file you downloaded in Obtain Exchange Server 2007 and LoadGen on page 81. Follow the prompts to install LoadGen, accepting the EULA and the default options.
Edit the LoadGen Initialization Template 1 2 Use Notepad to open the file vclientN/mailserver/vmmark2initializationtemplate.xml from the VMmark 2.1 kit. Find the ClientAccessServers block, which will look like this: <ClientAccessServers> <string>mailserverN.maildomainN.YOUR_3_PART_DOMAIN_NAME</string> </ClientAccessServers> Within the ClientAccessServers block, replace the Ns in mailserverN and maildomainN with the tile number youre configuring, replace YOUR_3_PART_DOMAIN_NAME with your fully-qualified domain name, e.g. your.company.com, which must contain three parts. (Note that in the VMMARK2.CONFIG file this three-part domain name is referred to as the MailQualifier.) The following line appears in two different places in the XML file:
<LdapBaseDn>OU=Contacts,OU=LoadGen Objects,DC=maildomainN,DC=YOUR_DOMAIN_NAME_PART_1,DC=YOUR_DOMAIN_NAME_PART_2,D C=YOUR_DOMAIN_NAME_PART_3</LdapBaseDn>
In both locations, replace the N in maildomainN with the tile number youre configuring, replace YOUR_DOMAIN_NAME_PART_1, YOUR_DOMAIN_NAME_PART_2, and YOUR_DOMAIN_NAME_PART_3 to match your domain name. For example, if your fully qualified domain name is your.company.com, enter: DC=your,DC=company,DC=com 5 Find the section that begins with the TargetForest line, which will look like this: <TargetForest>maildomainN</TargetForest> <DirectoryUsername>YOUR_MAILSERVER_ADMINISTRATOR_USERNAME<DirectoryUsername> <DirectoryPassword>YOUR_MAILSERVER_ADMINISTRATOR_PASSWORD</DirectoryPassword> <DirectoryDomain>maildomainN</DirectoryDomain> <MasterPassword>YOUR_MAILSERVER_MASTER_PASSWORD</MasterPassword> <MasterDomain>maildomainN</MasterDomain> Replace the N in <TargetForest>maildomainN</TargetForest> with the tile number youre configuring. Insert your mailserver administrator username in place of YOUR_MAILSERVER_ADMINISTRATOR_USERNAME. Insert your mailserver password in place of YOUR_MAILSERVER_ADMINISTRATOR_PASSWORD. Replace the N in <DirectoryDomain>maildomainN</DirectoryDomain> with the tile number youre configuring. Insert your mailserver password in place of YOUR_MAILSERVER_MASTER_PASSWORD. (This will be the same as the YOUR_MAILSERVER_ADMINISTRATOR_PASSWORD from Step 8.) Replace the N in <MasterDomain>maildomainN</MasterDomain> with the tile number youre configuring. Save and close the vmmark2initializationtemplate.xml file.
6 7 8 9 10 11 12
VMware, Inc.
87
Configure LoadGen 1 2 3 4 5 6 7 8 9 10 Start LoadGen (Start Menu > All Programs > Microsoft Exchange > Exchange Load Generator). Click Start a new test. Select Use the following saved configuration file: and browse to C:\vclient0\mailserver\vmmark2initializationtemplate.xml. Click Continue. Click Continue with recipient management. On the User Settings page, leave Mailbox Database set to 0, under the vmmark subheading, set lpriv1 to 500 and lpriv2 to 500, then click Continue. On Advanced recipient settings page, click Continue and wait for recipient creation to complete. On the Specify test user groups page, click Continue. On the Remote configurations page, click Continue. On the Configuration Summary page, click Start the initialization phase (recommended before running the test) and wait for mailbox initialization to complete.
Backup the Microsoft Exchange Server 2007 Database

After LoadGen is fully configured, make a backup of the database in its pre-benchmarking state. To do so, follow the steps below. 1 2 Log on to the mail server virtual machine as user Administrator. Stop the Exchange Services: a b c d e 3 Open the Start menu then select Control Panel. In the Control Panel window, double-click Administrative Tools. In the Administrative Tools window, double-click Services. In the Services window, scroll to Microsoft Exchange Information Store. Right-click Microsoft Exchange Information Store and select Stop. (This operation may take a few minutes.)
After a message appears saying that the Exchange service was stopped successfully, defragment the Exchange database: a b Open a command prompt window (Start > Command Prompt). Enter the following commands: eseutil /d e:\vmmark\lpriv1.edb eseutil /d e:\vmmark\lpriv2.edb Close the command prompt window.
c 4
Copy the database files from E:\vmmark to F:\ (that is, from the vmmark directory of E: to the root directory of F:) as follows: a b c d e Open the database drive (Start Menu > Computer, then double-click on the Database drive (this should be drive E:\). Double click on the vmmark directory. Select all files in the vmmark directory (<Ctrl>-A), and copy them to the clipboard (<Ctrl>-C). Open the backup drive (Start Menu > Computer, then double-click on the Backup drive (this should be drive F:\). Paste the files from the clipboard to the backup drive (<Ctrl>-V).
88
VMware, Inc.
f 5
Close the backup drive.
After the files are copied, restart Exchange Services: a b c d e Open the Start menu then select Control Panel. In the Control Panel window, double-click Administrative Tools. In the Administrative Tools window, double-click Services. In the Services window, scroll to Microsoft Exchange Information Store. Right-click Microsoft Exchange Information Store and select Start. (This operation may take a few minutes.)
VMware, Inc.
89
90
VMware, Inc.
Running the VMmark Benchmark
This chapter describes the process of running the VMmark benchmark. It consists of the following sections: Tile Placement on page 91 Running the VMmark Harness on page 91 Running Multiple-Tile Benchmarks on page 93 VMmark Results Files on page 94 Creating and Submitting a VMmark Full Disclosure Report on page 95 Running VMmark without STAXMonitor on page 98
Tile Placement
For a compliant run, each tile's workload virtual machines should be placed on a single host, rotating through the hosts in the cluster in a round-robin fashion until all tiles are placed. For example in a three-tile, two-host configuration, all of tile 0's and tile 2's workload virtual machines would be on host 0 and all of tile 1's workload virtual machines would be on host 1. If the naming conventions specified in this guide are followed, the VMmark harness will automatically use vMotion to move the workload virtual machines to the correct host at the beginning of every run. If non-standard names are used, the VMmark harness will attempt to place those workload virtual machines it recognizes and will leave the remaining workload virtual machines on their original hosts.
Running the VMmark Harness

Typically, the VMmark Harness is used to control all aspects of a benchmarking run. This section describes its use. After the VMmark Harness has been configured, you can begin running the VMmark Benchmark. The VMmark Harness is initiated from a single designated prime client in the testbed where the STAF & STAX software packages have been installed. To start a test run 1 2 3 4 Make sure that the system clocks on all the clients and the system clock on the server are synchronized to within 10 seconds. Start the STAX Monitor on the prime client: java -jar C:\STAF\services\stax\STAXMon.jar When the STAX monitor window opens, click the Submit New Job button. The Job Info tab is displayed. Under XML Job File, select the local machine radio button (if not already selected).
VMware, Inc.
91
5 6
In the Filename box, enter: C:\VMmark2\xml\vmmark2_main.xml Under Job Options, in the Job Name box, enter an optional suffix to be appended to the name of the results directory. For example, if you enter 30-minutes-3-tile, the results file will be located in a directory called: C:\vmmark\results\Results_<timestamp>_30-minutes-3-tile Click the Function tab. Select the default radio button. Under Arguments, click the Clear button. In the Arguments box enter: { 'CONFIGFILE' : r'C:\vmmark2\VMMARK2.CONFIG' } Click the Submit New Job button.
7 8 9 10 11
The STAX Job Monitor window opens and STAX starts the VMmark Harness. The current status of the running workloads is shown in this window. NOTE After the above information has been entered, you can use the Resubmit Previous Job button. The names entered are remembered across restarts of the Monitor. STAX will locate the latest VMMARK2.CONFIG file and the XML Harness code each time.
92
VMware, Inc.
Chapter 5 Running the VMmark Benchmark
Running Multiple-Tile Benchmarks

To run a multiple-tile benchmark, you must create the appropriate number of tiles. Creating additional tiles is largely a repeat of the creation of the first tile, and is described in Chapter 3, Preparing the Infrastructure for VMmark 2.1 Benchmark Tests. After the additional tiles are created and are in the inventory of the server system, the VMmark Harness needs to be configured to run them, and then a multiple-tile benchmark may be started. These steps are detailed in the sections below.
Configuring the VMmark Harness for Multiple-Tile Benchmarks

The VMmark Harness is configured using the VMMARK2.CONFIG file, located in the C:\vmmark directory on the prime client: The number of tiles to be run is set with the TILES variable. The client system names are set using the CLIENTS variable (these should be set to the systems short names). The individual hostnames are set using the SERVERS variables (these should be set to the systems short names). Make sure that each client has a vclientN directory (where N is that clients tile number). The time and date on all clients must be synchronized to within a few seconds in order to generate valid VMmark test results.
Starting Multiple-Tile Benchmarks

After the VMmark Harness is configured correctly, starting multiple-tile benchmarks is no different than starting single-tile benchmarks, as described in Running the VMmark Harness on page 91. NOTE STAX is only required on the prime client. Its presence on the other clients, however, does not cause problems.
VMware, Inc.
93
VMmark Results Files

After a VMmark benchmark test run has completed, the VMmark harness automatically collects the results data from the client system (or systems) and places those results in the results directory on the prime client, then processes the results to score the test. The benchmark captures the 60-second throughput measurements for each workload and stores them in files with the .wrf (Workload Results File) suffix in the Results_<datestamp> directory. At the end of a compliant test run, the average throughput scores for each workload are recorded in the Score_N_Tile_Test.out file (where N is the number of tiles in the test) along with the normalized scores and the composite VMmark metric. The Score_N_Tile_test.out file contains additional details of the test, such as duration, start time, and end time as well as: TILE_N_Scores: The actual workload scores for each VM during each of the three test phases (p0, p1, and p2). TILE_N_Ratios: The scores normalized against the VMmark reference platform. The composite VMmark score at the bottom of the Score_N_Tile_test.out file is the median of the sums of each phases geometric mean per tile.
94
VMware, Inc.
Creating and Submitting a VMmark Full Disclosure Report

Preparing VMmark results for publication according to the run and reporting rules involves the following tasks: configure the prime client to log in to the virtualization hosts without a password enable reporting in the VMMARK2.CONFIG file and run a compliant test create a name-mapping file generate an HTML-formatted table of results and ratios edit the disclosure.HTML file submit the full disclosure report for review The following sections detail these steps.
Configure the Prime Client to Log Into Virtualization Hosts Without a Password
In order for the VMmark harness to automatically run the reporting script you must generate SSH keys on the prime client and on each virtualization host. This task typically needs to be performed only once. NOTE For a sample script to configure this automatically in ESX (but not ESXi), visit the Documents tab in the VMmark section of the VMware Communities at: http://communities.vmware.com/community/vmtn/general/performance/vmmark To do this manually in VMware ESX (as opposed to ESXi, which is documented in the next section): 1 2 3 Log into the VMware ESX server console. Open the SSH configuration file for editing: vi /etc/ssh/sshd_config Search for PermitRootLogin under the Authentication section. Change PermitRootLogin from no to yes. The resultant section will look like this: # Authentication: #LoginGraceTime 120 PermitRootLogin yes #StrictModes yes Save the /etc/ssh/sshd_config file. Restart SSH services: /etc/init.d/sshd restart If the ~/root/.ssh directory exists, skip to step 7. Otherwise run the following command: mkdir ~/root/.ssh If ~/root/.ssh/id_rsa.pub exists, skip to step a. Otherwise run the following command: ssh-keygen -t rsa (accept the default file name and leave the passphrase empty). If not already done, generate SSH keys on the prime client as follows: a On the prime client, open a Cygwin window. If /home/Administrator/.ssh exists, skip to step b. Otherwise run the following command: mkdir /home/Administrator/.ssh If ~/root/.ssh/id_rsa.pub exists, skip to step 9. Otherwise run the following command: ssh-keygen -t rsa (accept the default file name and leave the passphrase empty).
4 5 6 7
Copy the contents of the /home/Administrator/.ssh/id_rsa.pub file on the prime client to the ~/root/.ssh/authorized_keys file on the virtualization host.
VMware, Inc.
95
To do this manually in VMware ESXi: Run the VMmark2-CreateSSH.sh script by following these steps: 1 2 3 4 Enable ESXi remote tech support mode. Instruction to do this can be found in VMware Knowledge Base article #1017910 (http://kb.vmware.com/kb/1017910). Copy the VMmark2-CreateSSH.sh script from C:\VMmark2\tools to the /root directory of the ESXi system. Using an SSH client, access the ESXi command line. From the ESXi command line, make the script executable then run it: chmod +x VMmark2-CreateSSH.sh sh ./VMmark2-CreateSSH.sh Follow the scripts instructions for enabling passwordless SSH to the ESXi system. If not already done, generate SSH keys on the prime client as follows: a On the prime client, open a Cygwin window. If /home/Administrator/.ssh exists, skip to step b. Otherwise run the following command: mkdir /home/Administrator/.ssh If ~/root/.ssh/id_rsa.pub exists, skip to step 7. Otherwise run the following command: ssh-keygen -t rsa (accept the default file name and leave the passphrase empty).
5 6
7 8
Copy the contents of the /home/Administrator/.ssh/id_rsa.pub file on the prime client to the ~/root/.ssh/authorized_keys file on the virtualization host. Repeat for each ESXi host in the cluster.
Enable Reporting in the VMMARK2.CONFIG File

When you are satisfied with your benchmark run and are ready to submit the results, you need to run a full test with reporting enabled. To do so: 1 2 Edit the VMMARK2.CONFIG file and set the following two variables: REPORTER=1 Rerun a compliant VMmark test.
The harness automatically gathers information about the workload virtual machines and the ESX virtualization host and places that information in the results directory.
Create a Name-Mapping File

Results to be submitted to VMmark for review must include a file showing the mapping of workloads to .vmx file names. A template is provided in the reportingtools directory for this purpose. 1 On the prime client, open a Cygwin window (Start Menu > All Programs > Cygwin > Cygwin Bash Shell) and change to the appropriate results directory: cd /cygdrive/c/vmmark2/results/Results_datestamp (where datestamp refers to the results to be formatted). Copy the template VM-name-map.txt file from the tools directory to the results directory: cp ../../tools/VM-name-map.txt . Edit the VM-name-map.txt file, entering in the right column the path and name of each .vmx file that corresponds to the workload in the left column. The resulting file will look something like this example from a one-tile system:
Workload Name .vmx File Name -----------------------------------------Mailserver0 /vmfs/volumes/494d7f49-4a6b5a5e-5048-001b245bd287/mailserver0/mailserver0.vmx
2 3
96
VMware, Inc.
Standby0 OlioDB0 OlioWeb0 DS2DB0 DS2WebA0 DS2WebB0 DS2WebC0
/vmfs/volumes/494d7f49-4a6b5a5e-5048-001b245bd287/standby0/standby0.vmx /vmfs/volumes/494d7f49-4a6b5a5e-5048-001b245bd287/oliodb0/oliodb0.vmx /vmfs/volumes/494d7f49-4a6b5a5e-5048-001b245bd287/olioweb0/olioweb0.vmx /vmfs/volumes/494d7f49-4a6b5a5e-5048-001b245bd287/ds2db0/db2db0.vmx /vmfs/volumes/494d7f49-4a6b5a5e-5048-001b245bd287/ds2weba0/ds2weba0.vmx /vmfs/volumes/494d7f49-4a6b5a5e-5048-001b245bd287/ds2webb0/ds2webb0.vmx /vmfs/volumes/494d7f49-4a6b5a5e-5048-001b245bd287/ds2webc0/ds2webc0.vmx
Save the edited VM-name-map.txt file.
Generate an HTML-Formatted Table of Results and Ratios

After successfully completing a benchmark run with reporting enabled, the first step in preparing for publication is to generate an HTML-formatted table of results and ratios. This table will later be inserted into the full disclosure report. 1 2 On the prime client, open a Cygwin window (Start Menu > All Programs > Cygwin > Cygwin Bash Shell). Change to the appropriate results directory: cd /cygdrive/c/vmmark2/results/Results_datestamp (where datestamp refers to the results to be formatted). Issue the command: ../../tools/tilescore2html.sh n filename (where n is the number of tiles included in the results to be formatted and filename is the resulting HTML filename).
Edit the disclosure.HTML File

After generating the HTML-formatted table of results and ratios, the next step is to create a full disclosure report. 1 On the prime client, open a Cygwin window (Start Menu > All Programs > Cygwin > Cygwin Bash Shell) and change to the appropriate results directory: cd /cygdrive/c/vmmark2/results/Results_datestamp (where datestamp refers to the results to be formatted). Copy the template disclosure.html file from the tools directory to the results directory: cp ../../tools/disclosure.html . Edit the disclosure.html file. a b Search for instances of ###, and replace ### with the details of your system configuration. In the section titled Performance, insert the contents of the HTML file you generated with tilescore2html.sh as described in Generate an HTML-Formatted Table of Results and Ratios on page 97. Save the disclosure.html file.
2 3
Submit the Full Disclosure Report for Review

Once all the necessary files are in the appropriate results directory on the prime client, you are ready to submit the full disclosure report for review. This involves the following steps: Compress the results folder Generate an MD5 sum on the submission file FTP the file to the submission site Receive an emailed auto-acknowledgment For specific submission instructions send email to benchmark@vmware.com.
VMware, Inc.
97
Running VMmark without STAXMonitor

Using the command-line interface provided by STAF/STAX it is possible to run VMmark without using the STAXMonitor GUI. Once you have the VMmark testbed configured and working, either Windows .bat files or Cygwin shell scripts can be invoked from the command-line interface. This allows you to script the running of multiple tests serially without the need to use the STAXMonitor GUI. To run a one-tile test followed by a two-tile test 1 Create a VMMARK2.CONFIG_T1 and a VMMARK2.CONFIG_T2 that specify TILES=1 and TILES=2 respectively (leaving the remainder of these configuration files containing whatever is appropriate for the specific testbed). Create a multirun.bat or multirun.sh script file that contains the lines:
staf local stax execute file "c:\vmmark\xml\vmmark_main.xml" args "{ 'CONFIGFILE' : r'C:\vmmark\VMMARK2.CONFIG_T1' }" WAIT RETURNRESULT staf local stax execute file "c:\vmmark\xml\vmmark_main.xml" args "{ 'CONFIGFILE' : r'C:\vmmark\VMMARK2.CONFIG_T2' }" WAIT RETURNRESULT
Run the script from the prime client. The script starts the first test, waits for it to complete, then starts the next test. The command window where this is invoked should display text similar to the following as each test completes:
Response -------Job ID: 15 Result: None Response -------Job ID: 16 Result: None
In this example, the results of the tests will be in the last two results/Result_<datestamp> directories on the prime client, and the STAX_Job_<ID>*.log files will match the Job ID list. The Result: None line is the expected output in this case, indicating that STAF successfully ran the command.
For more information on running STAF/STAX from the command line, refer to their user's guides available through http://staf.sourceforge.net/.
98
VMware, Inc.
Creating the Workload Virtual Machines
This appendix describes the steps to create the virtual machines that are used as the workloads for the VMmark Benchmark testing and install the Microsoft Windows or Linux operating systems on those virtual machines. To create the eight workload virtual machines 1 2 3 4 5 Configure the underlying virtual machines (customized for the workload for which the virtual machine will be used). Install the Microsoft Windows or Linux operating system in the virtual machine (partitioning the virtual disk appropriately for the workload for which the virtual machine will be used). Install any customized drivers recommended by the hypervisor vendor for use in virtual machines running the specific operating systems required for VMmark (for example, the VMware Tools package). Install and configure the STAF package in the virtual machine. Configure the guest operating system.
These steps are described in more detail in the following sections, first for the virtual machine running Microsoft Windows Server 2003, then for the virtual machine running Microsoft Windows Server 2008, and lastly for the six virtual machines running Linux. NOTE Many of the steps required to configure the Linux workload virtual machines can be avoided by downloading pre-built Linux virtual machines from VMware, as described in: Create the Olio Database Virtual Machines on page 53, Create the Olio Web Virtual Machines on page 55, Create the DVD Store 2 Database Virtual Machines on page 57, and Create the DVD Store 2 Web Virtual Machines on page 59.
VMware, Inc.
99
Create and Configure the Standby Virtual Machines

The standby virtual machines used in each VMmark benchmark tile run Microsoft Windows Server 2003 Enterprise Edition (32-bit). This section details the creation of those virtual machines.
Create the Standby Virtual Machine

Using the New Virtual Machine wizard in the VMware vCenter Client (Inventory > File > New > Virtual Machine), configure a new virtual machine with the following settings: 1 2 3 4 5 When you are asked to Select the configuration for the virtual machine, select the Custom radio button, then click Next. When you are asked to Specify a Name and Location for this Virtual Machine, enter standby0, then click Next. When you are asked to Select a Datastore..., select your desired datastore, then click Next. When you are asked for a Virtual Machine Version, select Virtual Machine Version: 7, then click Next. When you are asked to Specify the guest operating system..., select the Microsoft Windows radio button and in the Version: window choose Microsoft Windows Server 2003, Enterprise Edition (32-bit), then click Next. When you are asked to specify the Number of virtual processors, select 1, then click Next. When you are asked the Memory Configuration, select 512MB, then click Next. When you are asked to Create Network Connections, set the number of NICs to 1, choose the network to use, choose the Adapter type, make sure Connect at Power On is checked, then click Next. NOTE If you will be using the high-performance VMXNET3 adapter, choosing it at this point will save a few steps later. If you do so, however, you might not have network access from the virtual machine until you install VMware Tools, which includes the custom VMXNET3 driver. 9 10 11 When asked Which SCSI controller type would you like to use?, choose either the LSI Logic Parallel or LSI Logic SAS, then click Next. When asked to Select the type of disk to use, select Create a new virtual disk, then click Next. When asked to Specify the virtual disk size and provisioning policy, enter 4GB, leave the Disk Provisioning options unchecked, select a location (either Store with the virtual machine or Specify a datastore:), then click Next. In the Advanced Options window, leave the options unchanged: Virtual Device Node: SCSI (0:0) Mode: Independent: leave unselected Then click Next. 13 At the Ready to Complete window, click Finish to create the virtual machine.
6 7 8
12
Once the virtual machine is ready, proceed to install Microsoft Windows Server, as described in Install Microsoft Windows Server 2003 Enterprise Edition on page 101.
100
VMware, Inc.
Appendix A Creating the Workload Virtual Machines
Install Microsoft Windows Server 2003 Enterprise Edition

Install Microsoft Windows Server 2003 Enterprise Edition in the virtual machine. The simplest way to install an operating system in a virtual machine is to use an ISO image (or images) of the installation disc (or discs). NOTE The installation instructions in this section are based on the use of Windows Server 2003 Enterprise Edition with Service Pack 2. The installation and configuration of other releases and service packs varies slightly from these instructions. You can start with other releases or service packs, but you must use Enterprise Edition, it must be a 32-bit version, and you must upgrade each standby virtual machine to at least Service Pack 1 before running the benchmark. To install Microsoft Windows Server 2003 Enterprise Edition SP2 1 2 3 4 5 6 Create an ISO image of the operating system installation disc(s). Do this with any third-party CD ripping software tool. Copy the ISO image file(s) to a file system accessible from the system running the VMware vCenter Client. Select the correct virtual machine (created in Create the Standby Virtual Machine on page 100). From the Summary tab in the vCenter Client, under Commands, select Power On. Select the Console tab in the vCenter Client. You will see the virtual machine try to boot, but fail. Click the Connect/disconnect the CD/DVD... button in the menu bar (it looks like a CD with a wrench in front of it), select CD/DVD Drive 1, then select Connect to ISO image... (either on local disk, or on a datastore). Browse to the ISO image of the first operating system installation disc, select it, and click Open. Still under the Console tab in the vCenter Client, click within the center of the console window (you may see no change, or your cursor may disappear). Restart the virtual machine by pressing <Ctrl><Alt><Insert> (this gets passed to the virtual machine as <Ctrl><Alt><Delete>, and causes it to reboot). The virtual machine will boot from the ISO image. NOTE If you have problems getting the system to boot from the ISO image, you can press <Esc> as the virtual machine is booting. This brings up a boot menu, from which you can select CD-ROM Drive. 10 11 12 13 Follow the Microsoft Windows installation prompts until you are given the option to create a partition. Use all available space for the partition. Format the Windows partition using the NTFS file system. Continue to follow the Microsoft Windows installation prompts until you reach the Network Settings window. a b c d e f g 14 Choose the Custom Settings radio button, then click Next. At the Networking Components window, select Internet Protocol (TCP/IP), then click Properties. At the Internet Protocol (TCP/IP) Properties window, select the Use the following IP address radio button, and fill in IP address, subnet mask, and default gateway values appropriate for your network. Still at the Internet Protocol (TCP/IP) Properties window, select the Use the following DNS server addresses radio button, and fill in DNS server addresses appropriate for your network. Click OK. Back at the Networking Components window, click Next. At the Workgroup or Computer Domain window, enter values appropriate for your network, then click Next.
7 8 9
Follow the remaining prompts to install Microsoft Windows.
VMware, Inc.
101
15 16
When the installation is complete, the system reboots. After the reboot, log in as Administrator. If Windows Setup asks for an additional disc you can supply it as follows: a b c d Click the Virtual CDROM(ide0:0) button in the menu bar and select the option that begins with the word Disconnect. Again click the Virtual CDROM(ide0:0) button in the menu bar, this time selecting Connect to ISO image... Browse to the ISO image of the second CD, select it, and click Open. Back within the virtual machine, in the Windows Setup window, click OK and follow the prompts.

For VMware ESXi: 1 2 Power-on the virtual machine. From a web browser on the vCenter Server system, navigate to http://packages.vmware.com/tools/esx, select the ESX version and host operating system you're running, and download its .iso file, saving it to a file system accessible from the system running the VMware vCenter Client. Click the Connect/disconnect the CD/DVD... button in the menu bar (it looks like a CD with a wrench in front of it), select CD/DVD Drive 1, then select Connect to ISO image... (either on local disk, or on a datastore). Browse to the VMware Tools .iso file you just downloaded, select it, and click Open. When a Welcome to the installation wizard for VMware Tools window appears, this time inside the virtual machine console (it may take a few minutes), click Next. When the Setup Type window appears, choose Typical, and click Next. When the Ready to Install the Program window appears, click Install. When the Installation Wizard Completed window appears, click Finish. At the VMware Tools window, click Yes to allow the system to be restarted.
4 5 6 7 8 9
102
VMware, Inc.
Follow these steps to enable time synchronization in VMware Tools 1 2 3 4 Right-click the blue VMware icon in the system tray on the task bar and choose Open VMware Tools. Select the Options tab. Check the box next to Time synchronization between the virtual machine and the host operating system. Click OK to close the dialog box.

Install all available windows updates, both high-priority and optional, by following these steps: 1 2 3 4 5 Run Windows Update (Start Menu > Windows Update). At the Microsoft Windows Update page, click Custom. Install any new Windows Update software, then install all available updates, both high-priority and optional. When the updates are complete, restart the system by clicking Restart now. After the system restarts, check again for updates. If there are more available, install them, restart the system, and repeat until no more updates appear.
Turn off automatic updates by following these steps: 1 2 3 4 Open the Control Panel (Start Menu > Settings > Control Panel). At the Control Panel, double-click Automatic Updates. At the Automatic Updates window, make sure the radio button for Turn off Automatic Updates is selected, then click OK. Close the Control Panel.

Install and configure the STAF package in the Microsoft Windows virtual machine, as described in the following sections. NOTE To avoid communication problems, we recommend that all clients and all workload virtual machines be running the same version of STAF.

To install the STAF package in the virtual machine 1 2 Go to http://staf.sourceforge.net/getcurrent.php. Download the executable (*.exe) version 3.4.1 or later STAF package for either: Win95/98/Me/NT/2000/XP/2003/Vista/WinSrv2008 (32-bit) or: Windows Server 2003/2008, Vista (AMD64) depending on whether this virtual machine is running 32-bit or 64-bit Windows. (The exact name of the file may vary, depending on the current version of the package.) Run the executable setup file, following the prompts to install the software, and using the following inputs: a b c Place the software in the default location of C:\STAF. For installation type, choose Typical. Leave all other options at their defaults.
VMware, Inc.
103

1 Using Notepad (Start Menu > All Programs > Accessories > Notepad), open C:\STAF\bin\STAF.cfg. (To see this file, you have to change Files of type to All Documents (*.*) within Notepad.) NOTE If you have customized a STAF.cfg file for a client or for another virtual machine, start with that one, saving some of the steps below. If, however, you use the STAF.cfg from Client 0, comment out the service stax and service event entries (be sure to get all the lines, including those that are continued with a backslash). 2 In the trust level block add the networking mask for the systems to be trusted and set them to trust level 5. For example: trust machine client0 level 5 Alternatively, to trust all machines in the 192.168 network, add the line: trust machine 192.168.*.* level 5 Save the edited file as C:\STAF\bin\STAF.cfg (change Save as type to All Files so Notepad doesnt append a .txt extension), and close NotePad.

By default STAF starts when a user first logs in. Follow these steps to configure STAF to start when the machine is booted: 1 2 3 4 5 6 7 8 9 10 11 Open the Start menu, then select Control Panel > Scheduled Tasks > Add Scheduled Task. When the Scheduled Task Wizard appears, click Next. Scroll to Start STAF 3.4.1, select it, and click Next. (The exact entry will vary depending on which version of STAF you have installed.) Select the When my computer starts radio button, and click Next. Enter (or confirm) the Administrator user name and password, and click Next. Check the Open Advanced properties for this task when I click Finish box. Click Finish. When the Start STAF properties window opens, select the Settings tab. If the box labeled Stop the task if it runs for [72] hours is checked, uncheck it. Click to OK to close the properties window. Restart windows.
Update the HOSTS File

The standby workload virtual machine should be able to resolve the name used for its tiles client system in the STAF.cfg files trust level block. This might require an entry such as the following in the C:\Windows\System32\drivers\etc\HOSTS file: 192.168.7.1 client0
(replacing 192.168.7.1 with the clients IP address, and client0 with the clients name). NOTE This is not necessary if, in the STAF.cfg file, an IP address range was used instead of a specific machine name.
104
VMware, Inc.
Create and Configure the Mail Server Virtual Machines

The mail server virtual machines used in each VMmark benchmark tile run Microsoft Windows Server 2008 Enterprise Edition (64-bit). This section details the creation of those virtual machine.
Create the Mail Server Virtual Machine

Using the New Virtual Machine wizard in the VMware vCenter Client (Inventory > File > New > Virtual Machine), configure a new virtual machine with the following settings: 1 2 3 4 5 6 7 8 When you are asked to Select the configuration for the virtual machine, select the Custom radio button, then click Next. When you are asked to Select a Name and Location for this Virtual Machine, enter mailserver0, then click Next. When you are asked to Select a Datastore..., select your desired datastore, then click Next. When you are asked for a Virtual Machine Version, select Virtual Machine Version: 7, then click Next. When you are asked to Specify the guest operating system..., select the Microsoft Windows radio button and in the Version: window choose Microsoft Windows Server 2008 (64-bit), then click Next. When you are asked to specify the Number of virtual processors, select 4, then click Next. When you are asked the Memory Configuration, select 8GB, then click Next. When you are asked to Create Network Connections, set the number of NICs to 1, choose the network to use, choose the Adapter type, make sure Connect at Power On is checked, then click Next. NOTE If you will be using the high-performance VMXNET3 adapter, choosing it at this point will save a few steps later. If you do so, however, you might not have network access from the virtual machine until you install VMware Tools, which includes the custom VMXNET3 driver. 9 10 11 When asked Which SCSI controller type would you like to use?, choose either the LSI Logic Parallel or LSI Logic SAS, then click Next. When asked to Select the type of disk to use, select Create a new virtual disk, then click Next. When asked to Specify the virtual disk size and provisioning policy, enter 32GB, leave the Disk Provisioning options unchecked, select a location (either Store with the virtual machine or Specify a datastore:), then click Next. In the Advanced Options window, leave the options unchanged: Virtual Device Node: SCSI (0:0) Mode: Independent: leave unselected Then click Next. 13 At the Ready to Complete window, click Finish to create the virtual machine.
12
VMware, Inc.
105
Install Microsoft Windows Server 2008 Enterprise Edition

Install Microsoft Windows Server 2008 Enterprise Edition in the virtual machine. The simplest way to install an operating system in a virtual machine is to use an ISO image (or images) of the installation disc (or discs). NOTE The installation instructions in this guide are based on the use of Windows Server 2008 Enterprise Edition with Service Pack 2. The installation and configuration of other releases and service packs might vary slightly from these instructions. You can use other releases or service packs, but you must use Enterprise Edition and must use a 64-bit version. NOTE Microsoft Exchange 2007 is supported on Windows Server 2008 versions before R2. However Exchange 2007 is not supported on Windows Server 2008 R2 until Exchange 2007 Service Pack 3. To install Microsoft Windows Server 2008 Enterprise Edition SP2 1 2 3 4 5 6 7 8 9 Create an ISO image of the operating system installation disc(s). Do this with any third-party CD ripping software tool. Copy the ISO image file(s) to a file system accessible from the system running the VMware vCenter Client. Select the correct virtual machine (created in Create the Standby Virtual Machine on page 100). From the Summary tab in the vCenter Client, under Commands, select Power On. Select the Console tab in the vCenter Client. You will see the virtual machine try to boot, but fail. Click the Virtual CDROM(ide0:0) button in the menu bar and select Connect to ISO image... Browse to the ISO image of the first operating system installation disc, select it, and click Open. Still under the Console tab in the vCenter Client, click within the center of the console window (you may see no change, or your cursor may disappear). Restart the virtual machine by pressing <Ctrl><Alt><Insert> (this gets passed to the virtual machine as <Ctrl><Alt><Delete>, and causes it to reboot). The virtual machine will boot from the ISO image. NOTE If you have problems getting the system to boot from the ISO image, you can press <Esc> as the virtual machine is booting. This brings up a boot menu, from which you can select CD-ROM Drive. 10 11 Follow the Microsoft Windows installation prompts until the installation is complete. When the installation is complete and the system reboots, provide a new Administrator password. NOTE This password also needs to be entered in the VMmark2.config file at MailServer/MailPassword=" ". For information about editing this file, see Configure the VMmark Harness on page 75. The password must meet Active Directory requirements for a strong password; for example, VMmark123). Then follow the subsequent Windows prompts, accepting the defaults. 12 13 After the Windows Desktop appears, an Initial Configuration Tasks wizard will come up. Make any configuration changes required, then click Close. Adjust the screen resolution if desired (Start Menu > Control Panel > Personalization > Display Settings).
106
VMware, Inc.

For VMware ESXi: 1 2 Power-on the virtual machine. From a web browser on the vCenter Server system, navigate to http://packages.vmware.com/tools/esx, select the ESX version and host operating system you're running, and download its .iso file, saving it to a file system accessible from the system running the VMware vCenter Client. Click the Connect/disconnect the CD/DVD... button in the menu bar (it looks like a CD with a wrench in front of it), select CD/DVD Drive 1, then select Connect to ISO image... (either on local disk, or on a datastore). Browse to the VMware Tools .iso file you just downloaded, select it, and click Open. When a Welcome to the installation wizard for VMware Tools window appears, this time inside the virtual machine console (it may take a few minutes), click Next. When the Setup Type window appears, choose Typical, and click Next. When the Ready to Install the Program window appears, click Install. When the Installation Wizard Completed window appears, click Finish. At the VMware Tools window, click Yes to allow the system to be restarted.
4 5 6 7 8 9
Follow these steps to enable time synchronization in VMware Tools 1 2 3 4 After the system has restarted, right-click the blue VMware icon in the system tray on the task bar and choose Open VMware Tools. Select the Options tab. Check the box next to Time synchronization between the virtual machine and the host operating system. Click OK to close the dialog box.
VMware, Inc.
107

Install all available windows updates, both high-priority and optional, but dont turn on automatic updating, by following these steps: 1 2 3 4 5 6 7 Run Windows Update (Start Menu > All Programs > Windows Update). In the Windows Update window, click Change Settings. Under Important Updates select Never check for updates (not recommended), then click OK. Click the Check for updates button. Install any new Windows Update software, then install all available updates. When the updates are complete, restart the system by clicking Restart now. After the system restarts, check again for updates. If there are more available, install them, restart the system, and repeat until no more updates appear.
Create the Database and Backup Drives

The mail server virtual machine needs a second virtual disk on which are created two partitions. this section describes the creation of this virtual disk and the creation and formatting of the partitions.
Create a Second Virtual Disk

1 2 3 4 5 6 7 8 Within the vSphere Client Inventory pane, select the mail server virtual machine. Under the Getting Started tab, click Edit virtual machine settings. In the Virtual Machine Properties window, under the Hardware tab, click the Add... button. In the Add Hardware window, within the middle pane, select Hard Disk, then click Next. In the Select a Disk window, select the Create a new virtual disk radio button, then click Next. In the Create a Disk window, enter 40GB, leave the Disk Provisioning options unchecked, select a location (either Store with the virtual machine or Specify a datastore:), then click Next. In the Create a Disk window, select the Create a new virtual disk radio button, then click Next. In the Advanced Options window, leave the options unchanged: Virtual Device Node: SCSI (0:0) Mode: Independent: leave unselected Then click Next. 9 10 At the Ready to Complete window, click Finish to accept settings. Back at the Virtual Machine Properties window, click OK to close the window.
Create and Format the Database Partition

1 1 2 3 4 5 6 Boot the mail server virtual machine into Windows. Open the Server Manager window (Start Menu > Administrative Tools > Server Manager). In the left pane, expand Storage (click the plus sign) and click Disk Management. Right-click the word Disk 1 (it will have a small red arrow) and select Online. Again right-click the word Disk 1 (it will still have a small red arrow) and select Initialize Disk. At the Initialize Disk window, leave MBR selected, then click OK. Right click on the disk space area (toward the right, where it says 40.00 GB Unallocated) and select New Simple Volume.
108
VMware, Inc.
7 8 9 10 11 12
At the Welcome to the New Simple Volume Wizard window, click Next. At the Specify Volume Size window, enter a size of 32768MB, then click Next. At the Assign Drive Letter or Path window, assign drive letter E:, then click Next. At the Format Partition window, set the File system to NTFS, leave the Allocation unit size at Default, and set the Volume label to Database, then click Next. At the Completing the New Simple Volume Wizard window, click Finish. A new window will pop up behind the Server Manager window, saying You need to format the disk.... Switch to this window and click Cancel (the disk should already have been formatted as part of the previous step).
Create and Format the Backup Partition

1 2 3 4 5 6 7 Back at the Server Manager window, right click on the remaining disk space area (where it says 8.00 GB Unallocated) and select New Simple Volume. At the Welcome to the New Simple Volume Wizard window, click Next. At the Specify Volume Size window, enter the maximum disk space of 8189MB, then click Next. At the Assign Drive Letter or Path window, assign drive letter F:, then click Next. At the Format Partition window, set the File system to NTFS, leave the Allocation unit size at Default, and se the Volume label to Backup, then click Next. At the Completing the New Simple Volume Wizard window, click Finish. A new window will pop up behind the Server Manager window, saying You need to format the disk.... Switch to this window and click Cancel (the disk should already have been formatted as part of the previous step).

Install and configure the STAF package in the Microsoft Windows virtual machine, as described in the following sections. NOTE To avoid communication problems, we recommend that all clients and all workload virtual machines be running the same version of STAF.

To install the STAF package in the virtual machine 1 2 Go to: http://staf.sourceforge.net/getcurrent.php Download the executable (*.exe) version 3.4.1 or later STAF package for Windows Server 2003/2008, Vista, Windows 7 (AMD64, aka x64), with a bundled JVM: STAF341-setup-winamd64.exe (The exact name of the file might be different, depending on the current version of the package.) Run the executable setup file, following the prompts to install the software, and leaving all options at their defaults.
VMware, Inc.
109

To create a STAF.cfg file customized for this virtual machine 1 Using Notepad (Start Menu > All Programs > Accessories > Notepad), open C:\STAF\bin\STAF.cfg. (To see this file, you have to change file type (in the lower right corner) to All Files.) NOTE If you have customized a STAF.cfg file for a client or for another virtual machine, start with that one, saving some of the steps below. If, however, you use the STAF.cfg from Client 0, comment out the service stax and service event entries (be sure to get all the lines, including those that are continued with a backslash). 2 In the trust level block add the networking mask for the systems to be trusted and set them to trust level 5. Alternatively, to trust all machines in the 192.168 network, add the line: trust machine 192.168.*.* level 5 Save the edited file as C:\STAF\bin\STAF.cfg (change Save as type to All Files so Notepad doesnt append a .txt extension), and close NotePad.
Configure the Windows Server 2008 Firewall to Allow STAFProc.exe to Pass Through
Windows Server 2008 includes a firewall. Follow these steps to configure the Windows firewall to allow STAF to pass through: 1 2 3 4 5 6 7 8 Open the Start menu then select Control Panel. In the Control Panel window, double-click Windows Firewall. In the Windows Firewall window, click Allow a program through Windows Firewall. In the Windows Firewall Settings window click the Add Program... button. In the Add a Program window, click Browse. In the Browse window, navigate to C:\staf\bin\STAFProc.exe, then click the Open button. Back at the Add a Program window, click OK. Back at the Windows Firewall window click OK.

The mailserver workload virtual machine should be able to resolve the name used for its tiles client system in the STAF.cfg files trust level block. This might require an entry such as the following in the C:\Windows\System32\drivers\etc\hosts file: 192.168.7.1 client0
Install Windows Server Components and Windows Support Tools

The last steps to take before cloning the mail server virtual machines are to install a number of optional server components, and to install Microsoft Windows Support Tools.
Install Windows PowerShell

Install PowerShell as follows: 1 2 Open the Start menu and select Server Manager. In the left pane of the Server Manager window, click Features.
110
VMware, Inc.
3 4 5 6 7 8
In the right pane of the Server Manager window, click Add Features. In the Select one or more features... window, add a checkmark next to Windows PowerShell, then click Next. In the Confirmation window, click Install. After the installation is complete, the Results window appears. Click Close. Dismiss the Windows automatic updating is not enabled... warning. Close the Server Manager window.
Install Windows Server Components

Microsoft Windows Server 2008 requires the installation of a number of components before a system can become a domain controller. 1 2 3 4 5 6 7 8 9 Open the Start menu and select Server Manager. In the left pane of the Server Manager window, click Roles. In the right pane of the Server Manager window, click Add Roles. At the Before You Begin window, click Next. At the Select Server Roles window, add a checkmark next to Web Server (IIS). When the Add features required for Web Server (IIS)? window appears, click the Add Required Features button. Back at the Select Server Roles window, click Next. At the Web Server (IIS) window, click Next. At the Select Role Services window, add checkmarks to the following role services: Basic Authentication Windows Authentication Digest Authentication Dynamic Content Compression IIS 6 Metabase Compatibility IIS 6 Management Console Still in the Select Role Services window, click Next. At the Confirm Installation Selections window, click Install. At the Installation Results window, wait for the installation to complete, then click Close. Close the Server Manager window.
10 11 12 13
Configure Internet Explorer to Not Use a Proxy Server

1 2 3 4 Start Internet Explorer. At the Tools menu, select Internet Options. Select the Connections tab, then click on the LAN settings button. Make sure the box next to Use a proxy server for your LAN is not checked.
VMware, Inc.
111
Create and Configure the SLES Virtual Machines

Six of the eight virtual machines used in each VMmark benchmark tile run SLES 11 Linux (64-bit). This section details the creation and configuration of a single template for these six virtual machines. NOTE This section is only necessary if you choose to configure one or more of the Linux workload virtual machines from scratch. If, as we recommend, you instead download the pre-built virtual machine templates from VMware, as described in: Create the Olio Database Virtual Machines on page 53, Create the Olio Web Virtual Machines on page 55, Create the DVD Store 2 Database Virtual Machines on page 57, and Create the DVD Store 2 Web Virtual Machines on page 59, you can skip this entire section. NOTE The following sections describe procedures using the VMware vCenter Client connected to an ESX host. Other interfaces and other hypervisors may be different.
Create the Virtual Machine

Using the New Virtual Machine wizard in the VMware vCenter Client (Inventory > File > New > Virtual Machine), configure a new virtual machine with the following settings: 1 2 3 4 5 6 7 8 9 When you are asked to Select the configuration for the virtual machine, select the Custom radio button, then click Next. When you are asked to Select a Name and Location for this Virtual Machine, enter SLES11-template, then click Next. If you are asked to On which host or cluster do you want to run this virtual machine, enter the name of the cluster you created in Configure vCenter on page 44, then click Next. When you are asked to Select a Datastore..., select your desired datastore, then click Next. When you are asked for a Virtual Machine Version, select Virtual Machine Version: 7, then click Next. When you are asked to Specify the guest operating system..., select the Linux radio button and in the Version: window choose Suse Linux Enterprise Server 11 (64-bit), then click Next. When you are asked to specify the Number of virtual processors, select 2, then click Next. When you are asked the Memory Configuration, select 2GB, then click Next. When you are asked to Create Network Connections, set the number of NICs to 1, choose the network to use, choose the Adapter type, make sure Connect at Power On is checked, then click Next. NOTE If you will be using the high-performance VMXNET3 adapter, choosing it at this point will save a few steps later. If you do so, however, you might not have network access from the virtual machine until you install VMware Tools, which includes the custom VMXNET3 driver. 10 When asked Which SCSI controller type would you like to use?, choose either the LSI Logic Parallel or LSI Logic SAS, then click Next. 11 12 When asked to Select the type of disk to use, select Create a new virtual disk, then click Next. When asked to Specify the virtual disk size and provisioning policy, enter 10GB, leave the Disk Provisioning options unchecked, select a location (either Store with the virtual machine or Specify a datastore:), then click Next. In the Advanced Options window, leave the options unchanged: Virtual Device Node: SCSI (0:0) Mode: Independent: leave unselected Then click Next. 14 At the Ready to Complete window, click Finish to create the virtual machine.
13
112
VMware, Inc.
15 16
If you receive a warning Creation of the virtual machine (VM) does not include automatic installation of the guest operating system... click OK. At this point, unless you will be configuring a virtual machine from scratch, you should return to the instructions in Chapter 3, Preparing the Infrastructure for VMmark 2.1 Benchmark Tests..
Install and Configure SUSE Linux Enterprise Server 11

Installation of the operating system requires the installation DVD for SUSE Linux Enterprise Server (SLES) 11 (64-bit version). NOTE If you will be using the template virtual machines supplied by VMware, you do not need to use this section. To install SLES 11 1 2 3 4 5 6 Create ISO images of the operating system installation discs. Do this with any third-party CD ripping software tool. Copy the ISO image files to a disk location accessible from the system running the VMware vCenter Client. Select the correct virtual machine (created in Create the Virtual Machine on page 112). From the Summary tab in the vCenter Client, under Commands, select Power On. Select the Console tab in the vCenter Client. You will see the virtual machine try to boot, and fail. Click the Connect/disconnect the CD/DVD... button in the menu bar (it looks like a CD with a wrench in front of it), select CD/DVD Drive 1, then select Connect to ISO image... (either on local disk or on a datastore). Browse to the ISO image of the first operating system installation disc, select it, and click Open. Still under the Console tab in the vCenter Client, click within the center of the console window (you may see no change, or your cursor may disappear). Restart the virtual machine by pressing <Ctrl><Alt><Insert> (this gets passed to the virtual machine as <Ctrl><Alt><Delete>, and causes it to reboot). The virtual machine will boot from the ISO image. NOTE If you have problems getting the system to boot from the ISO image, you can press <Esc> as the virtual machine is booting. This brings up a boot menu, from which you can select CD-ROM Drive. 10 11 12 13 14 15 16 At the SLES Boot Options prompt, scroll down to Installation and press <ENTER> to begin the installation in graphical mode. At the Welcome screen, select a Language and Keyboard Layout, check the I Agree to the License Terms box to indicate your agreement, then click Next. At the Media Check screen, check your media if desired, then click Next. At the Installation Mode screen, leave the New Installation radio button selected, leave the Include Add-On Products from Separate Media unselected, then click Next. At the Clock and Time Zone screen, set your Region, Time Zone, Hardware Clock setting, and Time and Date, then click Next. At the Server Base Scenario screen, select the Physical Machine radio button, then click Next. At the Installation Settings screen, click the Software section heading.
7 8 9
VMware, Inc.
113
17
At the Software Selection and System Tasks screen, make the following changes: Deselect Novell AppArmor Deselect all items under Graphical Environment Deselect all items under Primary Function then click OK. Back at the Installation Settings screen, click Install. At the Confirm Installation screen, click Install. The operating system will install and the system will reboot. At the Password screen, set the root password to vmmark, confirm the password, then select Next (use <Alt><n>). At the Password is too simple screen, select Yes (use <Alt><y>). At the Password is all lower case screen, select Yes (use <Alt><y>). At the Hostname and Domain Name screen, replace the default hostname with SLES11-template, replace the default domain name with the appropriate one for your site (i.e., your.company.com), deselect Change Hostname via DHCP (use <Tab> <Space>), select Write Hostname to /etc/hosts (use <Tab> <Space>), and select Next (use <Alt><n>). At the Network Configuration screen, select Change (use <Alt><c>) and select General Network Settings. At the Network Setup Method screen, unselect Enable IPv6 (use <Tab> <Tab> <Space>), accept the reboot is needed warning, then select OK (use <Alt><o>). Back at the Network Configuration screen, select Change (use <Alt><c>) and select Firewall. At the Firewall Configuration: Start Up screen, select Disable Firewall Automatic Starting (use <Tab> <Tab> <Space>), then select OK (use <Alt><o>).
18 19 20 21 22 23
24 25 26 27
28 Back at the Network Configuration screen, select Next (use <Alt><n>). 29 30 31 32 33 34 35 36 At the Test Internet Connection screen, choose not to test (use <Tab> <Tab> <Space>), then select Next (use <Alt><n>). At the CA Management screen, select Next (use <Alt><n>). At the User Authentication Method screen, leave the defaults, then select Next (use <Alt><n>). At the New Local User screen, leave the fields empty, then select Next (use <Alt><n>). At the Empty User Login screen, leave the fields empty, then select Yes (use <Enter>). At the Release Notes screen, select Next (use <Alt><n>). At the Hardware Configuration screen, select Next (use <Alt><n>). At the Installation Completed screen, select Finish (use <Alt><f>). If a No Running Network warning appears, select Continue.
Install the VMware Tools Package in the Linux Virtual Machine

To install the VMware Tools package in the SLES virtual machine, follow these steps: NOTE These instructions are for Linux virtual machines running in ESX, but not in ESXi. For a pointer to the latest ESXi instructions, see VMware KB article 340 (http://kb.vmware.com/kb/340). 1 2 3 Power on the virtual machine and log in as root. Within the vSphere Client Inventory pane, right-click on the virtual machine and select Guest > Install/Upgrade VMware Tools. At the shell prompt, enter the following commands:
114
VMware, Inc.
For vSphere 4.0: mount /dev/cdrom /media cd /media rpm -ihv VMwareTools* ./vmware-config-tools.pl For vSphere 4.1: mount /dev/cdrom /media mkdir ~/vmware_tools cd ~/vmware_tools tar zxvf VMwareTools* cd vmware-tools-distrib ./vmware-install.pl 4 Accept the default answers to the questions. If errors are reported, refer to VMware product documentation for troubleshooting. NOTE The screen might flicker as the video driver is updated. 5 6 7 When the install is complete, reboot the system: shutdown -r now After the system boots, log in as root. Set a static IP address. NOTE If desired, you can leave it as DHCP until after the virtual machine is cloned. 8 Verify that the VMware Tools package is fully installed by selecting the Summary tab for the virtual machine with which you are working and looking for the OK after VMware Tools: in the General box.
Enable Time Synchronization in VMware Tools NOTE Methods of enabling time synchronization other than the one detailed below might not work. 1 2 3 Within the vSphere Client Inventory pane, right-click on the virtual machine and select Edit Settings. In the Virtual Machine Properties window, under the Options tab, select VMware Tools. In the right pane, under Advanced, add a check mark next to Synchronize guest time with host, then click OK.

Install and configure the STAF package in the Linux operating system virtual machine as detailed in the following two sections. NOTE To avoid communication problems, we recommend that all clients and all workload virtual machines be running the same version of STAF.

1 2 3 From a web browser on another system, go to http://staf.sourceforge.net/getcurrent.php and determine the download URL for the Linux (AMD64) .tar.gz STAF package, version 3.4.1 or later. Back at the SLES virtual machine command line, change to the /tmp directory: cd /tmp Use wget to download the file, using the download URL you determined in Step 1. For example:
wget http://prdownloads.sourceforge.net/staf/STAF343-linux-amd64.tar.gz
(the exact filename might vary, depending on the STAF release and bit version).
VMware, Inc.
115
Execute the following command: tar xvzf STAF341-linux-amd64.tar.gz (again, the exact filename might vary, depending on the STAF release and bit version). Change to the newly-created staf directory. Execute the STAFInst script: ./STAFInst Press <ENTER> repeatedly to read the license, then press <ENTER> to accept it.
5 6 7

1 Using the vi editor, open the STAF.cfg file: vi /usr/local/staf/bin/STAF.cfg NOTE If youve already customized a STAF.cfg file for a client or for another virtual machine, you can start with that one, saving some of the steps below. If you use a STAF.cfg file from a Windows system, however, comment out the line SET DEFAULTSTOPUSING WM_CLOSE, and if you use the STAF.cfg from Client 0, comment out the service stax and service event entries (be sure to get all the lines, including those that are continued with a backslash) 2 3 Use i to enter editing mode, then the cursor keys to move around within the file. In the trust level block, add the fully-qualified domain names of all workload virtual machines and all client systems, and set them to trust level 5. For example, to trust all machines in the 192.168 network, add the line: trust machine 192.168.*.* level 5 4 Save the file and close vi: <ESC> :wq
Set Open File Limits

1 Add the following to the /etc/security/limits.conf file: * soft nofile 65535 * hard nofile 65535 Add the following to the /usr/local/staf/STAFEnv.sh file (at the beginning of the file, just after the #STAF environment variables line): ulimit -n 65535
Disable Persistent Network Adapter Name Rules

1 2 Delete the file /etc/udev/rules.d/70-persistent-net.rules. Delete the file /lib/udev/rules.d/75-persistent-net-generator.rules.
Configure the Linux System to Automatically Start STAFProc

1 2 Open /etc/init.d/boot.local and add the following line to the end of the file: /usr/local/staf/startSTAFProc.sh Reboot the virtual machine.

The workload virtual machines should be able to resolve the name used for their tiles client system in the STAF.cfg files trust level block. This might require an entry such as the following in the /etc/hosts file: 192.168.7.1 client0
116
VMware, Inc.
VMware, Inc.
117
118
VMware, Inc.
Installing Applications in the Workload Virtual Machines
This appendix describes the steps needed to install applications in the six workload virtual machines that make up each tile. You should have created the six virtual machines, installed the operating systems, and performed the other customizations described in Appendix A, Creating the Workload Virtual Machines, on page 99. In addition to the instructions included in this document, you should also refer to the Readme.txt file included in the VMmark kit.
Prepare the Mail Server Virtual Machines

To create the mail server virtual machines, first create the underlying virtual machine, as described in Create and Configure the Mail Server Virtual Machines on page 105, and then follow the instructions in the sections below.
Clone the Mail Server Virtual Machines

Unlike the other virtual machines, the mail server requires custom settings for each tile. Because some of these settings are not easily changed after Microsoft Exchange Server is installed, it is easiest to partially configure a machine to be used as a template, clone it, then complete the configuration and software installation. Make as many clones of the mail server virtual machine template as you will need (one for each tile you expect to run), naming them mailserverN (where N is the tile number to which this workload belongs). The first mail server virtual machine would be mailserver0, the second would be mailserver1, and so on. NOTE If you use vCenter to clone the mail server virtual machines, you can use Guest Customization to configure each one with a hostname and static IP address. If you dont set the hostname and the configure static IP addresses here, youll need to do so manually in the next section.
VMware, Inc.
119
Make the Mail Server Virtual Machine a Domain Controller

After the mail server virtual machines are cloned, follow these steps to promote each one to be a domain controller. 1 If the mail server virtual machine is not yet configured with a static IP address, assign one now. a b c d e From the Start menu, select Settings > Network Connections. Right-click the active network connection (typically Local Area Connection) and select Properties. Under the General tab, scroll to Internet Protocol (TCP/IP), select it, and click the Properties button. Enter a static IP address and other settings appropriate for your location. Click OK to close the Internet Protocol (TCP/IP) Properties window, OK to close the Local Area Connection Properties window (which might have a different name), then close the Network Connections window.
Start the Active Directory Installation Wizard by opening a Command Prompt window (Start > Command Prompt) and typing: dcpromo At the Active Directory Domain Services Installation Wizard click Next. At the Operating System Compatibility window click Next. At the Choose a Deployment Configuration window, select Create a new domain in a new forest, then click Next. At the Name the Forest Root Domain window, enter the DNS name for the new domain, then click Next. This should be the fully-qualified domain name in the format maildomainN.your.company.com, where N is the tile number that this virtual machine will be part of, and your.company.com is your companys domain name. This domain name must be in three parts (i.e., your.company.com). At the Set Forest Functional Level, select Windows Server 2003, then click Next. Dismiss the warning regarding domain controllers that are running Windows Server 2003 or later. At the Set Domain Functional Level, select Windows Server 2003, then click Next. Dismiss the warning regarding domain controllers that are running Windows Server 2003 or later. At the Additional Domain Controller Options window, leave the checkmark next to DNS server, then click Next. If you get a warning window beginning A delegation for this DNS server cannot be created..., click Yes. At the Location for Database, Log Files, and SYSVOL window, leave all folders in their default locations and click Next. At the Directory Services Restore Mode Administrator Password window, enter and confirm a password of your choice, then click Next. At the Summary window, click Next. When the system reports Active Directory Domain Services is now installed on this computer..., click Finish, then click Restart Now.
3 4 5 6
7 8 9 10 11 12 13 14 15 16
120
VMware, Inc.
Appendix B Installing Applications in the Workload Virtual Machines
Install Microsoft Exchange Server 2007

Install Microsoft Exchange Server 2007. To install the Microsoft Exchange Server 2007 package in each virtual machine 1 Obtain Microsoft Exchange Server 2007 Enterprise Edition. (This is available as part of the Microsoft Developers Network (MSDN) Universal subscription.) NOTE Note which Microsoft Exchange Server service pack, if any, is being installed at this time, then make sure that the Exchange Server tools installed in the client systems (described in Install Exchange Server 2007 Tools on page 86) match this service pack (or lack of service pack). 2 3 4 5 6 7 8 9 10 11 12 Insert the DVD or run the downloaded file. Item 1, 2, and 3 should be greyed out (youve already performed these steps). If, for some reason, theyre not completed, click on each one in order and follow the prompts. Click on Step 4: Install Microsoft Exchange. At the Introduction screen, click Next. At the License Agreement screen, select I accept..., then click Next. At the Error Reporting screen, select No, then click Next. At the Installation Type screen, click on Typical Exchange Server Installation, leave the installation path at the default, then click Next. At the Exchange Organization screen, leave it set to First Organization, then click Next. At the Client Settings screen, select No, then click Next. At the Readiness Checks screen, wait for the check to complete, then click Install. If you see a Hub Transport Role Prerequisites warning, you can safely ignore it. At the Completion screen, leave the box next to Finalize Installation using the Exchange Management Console checked, then click Finish.
Configure Microsoft Exchange Server

To configure Exchange Server 2007 1 2 3 4 5 6 7 8 9 10 11 12
VMware, Inc.
In the left pane of the Exchange Management Console, click the plus sign next to Server Configuration and select Mailbox. In the right pane, click on New Storage Group. In the Storage group name: field, enter vmmark, in the Log files path field, enter E:\vmmark, and in the System files path, enter E:\vmmark, then click New. At the Completion window, click Finish. In the lower portion of the middle pane, right click on vmmark and select Properties. In the vmmark Properties window, check the box next to Enable circular logging, then click Apply. In the vmmark Properties window, click OK. In the lower portion of the middle pane, right click on vmmark and select New Mailbox Database. In the Mailbox database name: field, enter lpriv1, then click New. After creating the mailbox database, right click the database, select Properties > Customize.... In the top left of the spreadsheet double-click on the box with a dotted border. This will deselect all the cells (indicated by the cells turning white). At the Completion window, click Finish.
121
13 14 15 16 17 18 19
In the lower portion of the middle pane, right click on vmmark and select New Mailbox Database. In the Mailbox database name: field, enter lpriv2, then click New. At the Completion window, click Finish. After creating the mailbox database, right click the database, select Properties > Customize.... In the top left of the spreadsheet double-click on the box with a dotted border. This will deselect all the cells (indicated by the cells turning white). Close the Exchange Management Console. Disable indexing on the Exchange databases: a b Open the Exchange Management shell (Start > All Programs > Microsoft Exchange Server 2007 > Exchange Management Shell). Enter the following commands: Set-MailboxDatabase lpriv1 -IndexEnabled $false Set-MailboxDatabase lpriv2 -IndexEnabled $false Close the Exchange Management shell.
Update Exchange Server 2007

After installing and configuring Exchange Server, install any critical updates. NOTE Depending on your Exchange version and installation package type, the steps may be different than those shown below. In that case, follow the prompts, dont install any additional roles, and update Exchange manually. NOTE Note which Microsoft Exchange Server service pack, if any, is being installed at this time, then make sure that the Exchange Server tools installed in the client systems (described in Install Exchange Server 2007 Tools on page 86) match this service pack (or lack of service pack). 1 2 3 4 5 6 7 8 9 Back at the Exchange Server 2007 setup window, click on Step 5: Get Critical Updates for Microsoft Exchange. At the Try Microsoft Update web page, click Start Now. At the Review the license agreement page, click Continue. Install the Active X control. Install Microsoft Update software (if requested). Install Windows Update software (if requested). Check for Updates. Click the Custom button. Install all high priority updates.
Increase the Transport Database Cache Size

Increase the transport database cache size as follows: 1 2 3 Go to C:\Program Files\Microsoft\Exchange Server\Bin\ Open the file EdgeTransport.exe.config for editing. Find the line that says: <add key="DatabaseMaxCacheSize" value="134217728" />
122
VMware, Inc.
4 5
Change it to: <add key="DatabaseMaxCacheSize" value="536870912" /> Save and close the file.

By default STAF starts when a user first logs in. Follow these steps to configure STAF to start when the machine is booted: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Open the Start menu, then select Control Panel. Within the Control Panel, double click Administrative Tools. In the Administrative Tools window, double click Task Scheduler. In the Task Scheduler window, under Actions (at the right), click Create Basic Task. In the Create a Basic Task window, enter STAF in the Name: field, then click Next. In the Task Trigger window, select When the computer starts, then click Next. In the Action window, select Start a program, then click Next. In the Start a Program window, click Browse, go to C:\STAF\, select startSTAFProc (the selection identified as a batch file, not the one identified as an Icon), then click Open. Back at the Start a Program window, click Next. At the Summary window, add a check mark next to Open the Properties dialog for this task when I click Finish, then click Finish. At the STAF Properties window, under the General tab, select the Run whether user is logged on or not radio button. Still at the STAF Properties window, go to the Settings tab and remove the check mark next to Stop the task if it runs longer than:, then click OK. At the Task Scheduler window, enter the administrator password, then click OK. Close the Task Scheduler window and the Administrative Tools window. Restart windows.
VMware, Inc.
123
Prepare the Standby Virtual Machines

The standby virtual machine is simply a virtual machine running Microsoft Windows Server 2003 Enterprise Edition with no workload applications installed or running. In order to control and monitor the virtual machine, however, the STAF software package must be installed. To create the standby virtual machine, create a Windows Server 2003 virtual machine, as described in Create and Configure the Standby Virtual Machines on page 100 (note that this includes the required STAF software package, mentioned earlier), then make the following customization: 1 In the C:\ directory of the template VM, create a new file named SignedOff.ini, containing the following text: [Unicode] Unicode=yes [Version] signature="$chicago$" Revision=1 [Registry Values] MACHINE\Software\Microsoft\Driver Signing\Policy=3,0 Save and close the file.
Clone the Standby Virtual Machine

After the configuration is complete, make as many clones of the standby virtual machine as you will need (one for each additional tile you expect to run), naming them standbyN (where N is the tile number to which this workload belongs), then edit their /etc/hosts file to include their tiles client. The first standby virtual machine would be standby0, the second would be standby1, and so on.
Prepare the Template for the Deploy Operation

The deploy operation (one of the infrastructure operation workloads) starts with a virtual machine based on the standby virtual machine, but with an outdated version of VMware Tools. The VMmark Harness clones this template, boots it, performs an operating system customization on it, then upgrades the VMware Tools installation. This section describes creating the template for that operation. 1 2 3 Make a clone of the standby virtual machine you created in Prepare the Standby Virtual Machines on page 124, naming it DeployTemplate. Determine which version of ESX you will use as the source for the out-of-date VMware Tools. For example, if your test systems are all running ESX 4.1, you might use VMware Tools from ESX 4.0. On the vCenter Server system, navigate to http://packages.vmware.com/tools/esx, select the out-of-date ESX version you chose above, and download the Windows 32-bit .iso file for that version, saving it to a VMFS datastore. For example, to use VMware Tools from ESX 4.0, download: http://packages.vmware.com/tools/esx/4.0/windows/x86/VMware-tools-windows-4.0.0-164009.iso 4 5 6 7 Mount the VMware Tools Windows .iso file to the DeployTemplate virtual machine and perform a normal VMware Tools installation. Reboot the DeployTemplate virtual machine to complete the installation. Shut down the DeployTemplate virtual machine. Convert this virtual machine to a vSphere template. a b In vCenter, right-click the virtual machine from Inventory to see its menu. Select Template > Convert to Template.
124
VMware, Inc.
Prepare the Olio Database Virtual Machines

To prepare the Olio database virtual machine, first create a SLES 11 virtual machine template, as described in Create and Configure the SLES Virtual Machines on page 112, then follow the instructions in the following sections. NOTE This section is only necessary if you configure an Olio database workload virtual machine from scratch. If, as we recommend, you instead download the pre-built virtual machine template from VMware, as described in: Create the Olio Database Virtual Machines on page 53, you may skip this section entirely.
Customize the SLES 11 Virtual Machine Template

The Olio database virtual machine starts with the SLES 11 virtual machine template created in Create and Configure the SLES Virtual Machines on page 112. You then customize it, as described in the following two subsections.
Clone the SLES11 Template Virtual Machine

Clone the SLES11-template virtual machine, naming the new virtual machine OlioDB0.
Customize the Virtual Machine for Olio Database

1 2 3 4 5 6 7 8 Within the vSphere Client Inventory pane, select the Olio database virtual machine (OlioDB0). Make sure the virtual machine is powered down. Under the Getting Started tab, click Edit virtual machine settings. In the Virtual Machine Properties window, under the Hardware tab, select Memory. Set the Memory Configuration: to 2GB (it will probably already be set to this amount). In the Virtual Machine Properties window, under the Hardware tab, select CPUs. Set the Number of virtual processors: to 2 (it will probably already be set to this number). Click OK.

1 2 3 4 5 6 7 8 Within the vSphere Client Inventory pane, select the Olio database virtual machine. Under the Getting Started tab, click Edit virtual machine settings. In the Virtual Machine Properties window, under the Hardware tab, click the Add... button. In the Add Hardware window, within the middle pane, select Hard Disk, then click Next. In the Select a Disk window, select the Create a new virtual disk radio button, then click Next. In the Create a Disk window, enter 4GB, leave the Disk Provisioning options unchecked, select a location (either Store with the virtual machine or Specify a datastore:), then click Next. In the Create a Disk window, select the Create a new virtual disk radio button, then click Next. In the Advanced Options window, leave the options unchanged: NOTE Optionally, the data disk can use the VMware PVSCSI device driver. For details, see Using a PVSCSI Virtual Device Driver on page 146. Virtual Device Node: SCSI (0:1) Mode: Independent: leave unselected Then click Next. 9 At the Ready to Complete window, click Finish to accept settings.
VMware, Inc.
125
10
Back at the Virtual Machine Properties window, click OK to close the window.
Install the Software for the Olio Database Virtual Machines

Mount a data disk, then install MySQL and Olio, as described in the following sections.
Partition, Format, and Mount the Olio Database Disk

1 2 3 4 Boot the virtual machine and log in as root. Configure the network settings, as described in Configuring Network Settings in SLES 11 on page 147. Edit the /etc/HOSTNAME file, changing the hostname to OlioDB0. Partition the newly-created disk using fdisk: a b c d e f 5 6 7 8 At a command prompt, enter: fdisk /dev/sdb When asked Command (m for help):, enter: n When asked Command action e extended p primary partition (1-4), enter: p When asked Partition number (1-4):, enter: 1 When asked the First cylinder and the Last cylinder, leave them at their defaults (to use the entire disk) by pressing <Enter> after each question. When again asked Command (m for help):, enter: w
Back at the command prompt, format the disk: mkfs -t ext3 /dev/sdb1 Create the /var/lib/mysql directory: mkdir /var/lib/mysql Mount the disk: mount /dev/sdb1 /var/lib/mysql Open the file /etc/fstab for editing and add the following line: /dev/sdb1 /var/lib/mysql ext3 defaults 1 2
Install MySQL
1 2 3 4 5 6 7 8 9 Connect the SLES 11 installation disk to the virtual machine. At the command prompt, type: yast Using the cursor keys, select Software in the left pane, move to the right pane, select Software Management, then press <Enter>. In the Search Phrase box, type mysql, then press <Enter>. Scroll down to mysql and select it for installation by pressing <Space>. Accept the package list by pressing <Alt><a>. Press <Enter> to accept the Automatic Changes. Press <Alt><q> to quit yast. Start MySQL: /etc/init.d/mysql start
126
VMware, Inc.
10
Configure MySQL to start automatically on future reboots: chkconfig mysql on
Install Tomcat and Java

1 Connect the SLES 11 SDK disk 1 to the virtual machine. NOTE This is not the installation disk, but a separate SDK disk. It can be downloaded from Novell at http://www.novell.com/developer/sle_sdk.html (this requires a Novell account and, due to its size, might take a long time to download). 2 3 4 5 6 7 8 9 10 11 At the command prompt, type: yast Using the cursor keys, select Software in the left pane, move to the right pane, select Software Repositories, then press <Enter>. At the Configured Software Repositories screen select Add (<Alt><a>). At the Media Type screen select DVD (<Alt><d>), then Next (<Alt><x>). At the Insert the add-on product DVD pop-up select Continue (<Alt><o>). At the License Agreement screen select Yes, I Agree to the License Agreement (<Alt><y>), then Next (<Alt><n>). Back at the Configured Software Repositories screen select OK (<Alt><o>). Back at the YaST2 Control Center, using the cursor keys, select Software in the left pane, move to the right pane, select Software Management, then press <Enter>. In the Search Phrase box, type java-1_6_0, then press <Enter>. Scroll to java-1_6_0-ibm and select it for installation by pressing <Space>. NOTE It is important that Java-1_6_0 is selected. Otherwise yast autoselects Java 1.4, which doesnt work with Tomcat. 12 13 14 15 16 In the Search Phrase box, type tomcat, then press <Enter>. Scroll down to tomcat6 and select it for installation by pressing <Space>. Accept the package list by pressing <Alt><a>. Press <Enter> to accept the Automatic Changes. Press <Alt><q> to quit yast.
17 Start Tomcat: /etc/init.d/tomcat6 start 18 19 Configure Tomcat to start automatically on future reboots: chkconfig tomcat6 on Make sure the directory /usr/java/latest exists. If it doesnt, create a directory and a symlink: mkdir -p /usr/java ln -s /usr/lib64/jvm/jre /usr/java/latest
Copy Files
1 2 Copy the file my.cnf from the VMmark kit vserver/oliodb directory to /etc. Copy the file geocoder.war from the VMmark kit /vserver/oliodb directory to /usr/share/tomcat6/webapps/.
VMware, Inc.
127
Configure the Olio Database

1 2 Copy the file schema.sql from the VMmark kit /vserver/oliodb directory to /root. Run the following commands: mysql create user 'olio'@'%' identified by 'olio'; grant all privileges on *.* to 'olio'@'%' identified by 'olio' with grant option; grant all privileges on *.* to 'olio'@'localhost' identified by 'olio' with grant option; create database bpwebapp; use bpwebapp; \. /root/schema.sql quit Make the /usr/local/olio directory: mkdir /usr/local/olio Copy the file dbloader.zip from the VMmark kit /vserver/oliodb directory to /usr/local/olio. Unzip the file dbloader.zip: unzip dbloader.zip Run the following commands: cd /usr/local/olio export JAVA_HOME=/usr/java/latest /usr/local/olio/dbloader.sh localhost 400
3 4 5 6
Clone the Olio Database Virtual Machine

After the configuration is complete, make as many clones of the Olio database virtual machine as you will need (one for each additional tile you expect to run), naming them OlioDBN (where N is the tile number to which this workload belongs). Thus the first Olio database virtual machine would be OlioDB0, the second would be OlioDB1, and so on. After cloning the virtual machines, configure their network settings, as described in Configuring Network Settings in SLES 11 on page 147, edit their /etc/HOSTNAME files, changing the hostname to the appropriate value, and edit their /etc/hosts file to include their tiles client.
128
VMware, Inc.
Prepare the Olio Web Virtual Machines

To prepare the Olio web virtual machine, first create a SLES 11 virtual machine template, as described in Create and Configure the SLES Virtual Machines on page 112, then follow the instructions in the following sections. NOTE This section is only necessary if you configure an Olio web workload virtual machine from scratch. If, as we recommend, you instead download the pre-built virtual machine template from VMware, as described in: Create the Olio Web Virtual Machines on page 55, you may skip this section entirely.

The Olio web virtual machine starts with the SLES 11 virtual machine template created in Create and Configure the SLES Virtual Machines on page 112. You then customize it, as described in the following two subsections.

Clone the SLES11-template virtual machine, naming the new virtual machine OlioWeb0.
Customize the Virtual Machine for Olio Web

1 2 3 4 5 6 7 Within the vSphere Client Inventory pane, select the Olio web virtual machine (OlioWeb0). Under the Getting Started tab, click Edit virtual machine settings. In the Virtual Machine Properties window, under the Hardware tab, select Memory. Set the Memory Configuration: to 6GB. In the Virtual Machine Properties window, under the Hardware tab, select CPUs. Set the Number of virtual processors: to 4. Click OK.

1 2 3 4 5 6 7 Within the vSphere Client Inventory pane, select the Olio web virtual machine. Under the Getting Started tab, click Edit virtual machine settings. In the Virtual Machine Properties window, under the Hardware tab, click the Add... button. In the Add Hardware window, within the middle pane, select Hard Disk, then click Next. In the Select a Disk window, select the Create a new virtual disk radio button, then click Next. In the Create a Disk window, enter 70GB, leave the Disk Provisioning options unchecked, select a location (either Store with the virtual machine or Specify a datastore:), then click Next. In the Advanced Options window, leave the options unchanged: NOTE Optionally, the data disk can use the VMware PVSCSI device driver. For details, see Using a PVSCSI Virtual Device Driver on page 146. Virtual Device Node: SCSI (0:1) Mode: Independent: leave unselected Then click Next. 8 9 At the Ready to Complete window, click Finish to accept settings. Back at the Virtual Machine Properties window, click OK to close the window.
VMware, Inc.
129
Install the Software for the Olio Web Virtual Machines

Mount a data disk, edit the configuration files, and install and configure software, as described in the following sections.
Partition, Format, and Mount the Olio Filestore Disk

1 2 3 Boot the virtual machine and log in as root. Configure the network settings, as described in Configuring Network Settings in SLES 11 on page 147. Partition the newly-created disk using fdisk: a b c d e f 4 5 6 7 At a command prompt, enter: fdisk /dev/sdb When asked Command (m for help):, enter: n When asked Command action e extended p primary partition (1-4), enter: p When asked Partition number (1-4):, enter: 1 When asked the First cylinder and the Last cylinder, leave them at their defaults (to use the entire disk) by pressing <Enter> after each question. When again asked Command (m for help):, enter: w
Back at the command prompt, format the disk: mkfs -t ext2 /dev/sdb1 Create the /OlioFileStore directory: mkdir /OlioFileStore Mount the disk: mount /dev/sdb1 /OlioFileStore Open the file /etc/fstab for editing and add the following line: /dev/sdb1 /OlioFileStore ext2 noatime 1 2
Tune TCP/IP
1 2 Open the file /etc/sysconfig/sysctl for editing and change IP_TCP_SYNCOOKIES to no: IP_TCP_SYNCOOKIES="no" Save and close the file.
Edit the /etc/HOSTNAME and /etc/hosts Files

1 2 Edit the /etc/HOSTNAME file, changing the hostname to OlioWeb0. Edit the /etc/hosts file to include an entry for Olio-DB as an alias for the IP address of its OlioDBN machine.
130
VMware, Inc.
Install Memcached
1 Connect the SLES 11 SDK disk 1 to the virtual machine. NOTE This is not the installation disk, but a separate SDK disk. It can be downloaded from Novell at http://www.novell.com/developer/sle_sdk.html (this requires a Novell account and, due to its size, might take a long time to download). 2 3 4 5 6 7 8 9 10 11 12 13 14 15 At the command prompt, type: yast Using the cursor keys, select Software in the left pane, move to the right pane, select Software Repositories, then press <Enter>. At the Configured Software Repositories screen select Add (<Alt><a>). At the Media Type screen select DVD (<Alt><d>), then Next (<Alt><x>). At the Insert the add-on product DVD pop-up select Continue (<Alt><o>). At the License Agreement screen select Yes, I Agree to the License Agreement (<Alt><y>), then Next (<Alt><n>). Back at the Configured Software Repositories screen select OK (<Alt><o>). Back at the YaST2 Control Center, using the cursor keys, select Software in the left pane, move to the right pane, select Software Management, then press <Enter>. In the Search Phrase box, type memcached, then press <Enter>. Scroll down to memcached and select it for installation by pressing <Space>. Accept the package list by pressing <Alt><a>. Press <Enter> to accept the Automatic Changes. Press <Alt><q> to quit yast. Configure memcached to start automatically on future reboots: chkconfig memcached on NOTE After running this command, you might observe a number of insserv errors. These can be safely ignored.
Copy the Fileloader

1 2 3 Make the /usr/local/olio directory: mkdir /usr/local/olio Copy the file fileloader.tar.gz from the VMmark kit /vserver/olioweb directory to /usr/local/olio. Run the following commands: cd /usr/local/olio tar xvzf fileloader.tar.gz
Install a 64-bit Java JRE

Install a 64bit Java JRE of your choice (as long as it is Java 1.6 or later). NOTE If the RPM version of Sun Java is not used, you will need to make a symbolic link /usr/java/latest to $JAVA_HOME.
VMware, Inc.
131
Install tc Server Application Server

1 Download the Quick Start tc Runtime tar.gz file: a b c 2 3 Go to: http://www.springsource.com/products/tc-server-evaluation Fill out the appropriate information and click the Download Now! button. Download the tar.gz version of the Quick Start tc Runtime file and save it in /root.
Change to the /opt directory. cd /opt Unpack the tar file with the following command: tar zxvf /root/springsource-tc-server-standard-2.0.4.RELEASE.tar.gz (replacing the above filename with the one you just downloaded). Change to the following directory. cd /opt/springsource-tc-server-standard Execute the following: /opt/springsource-tc-server-standard/tcruntime-instance.sh -s olio-web Copy the webapp.war file from the VMmark kit:
cp vserver/olioweb/tc-server/webapp.war /opt/springsource-tc-server-standard/olio-web/webapps/
4 5 6
Install Spring Framework

1 Download the Spring Framework .zip file: a b c 2 Go to: http://www.springsource.com/download/community Fill out the appropriate information and click the Access Download! button. Download latest GA release of Spring Framework (something like spring-framework-3.0.5.RELEASE.zip) to /tmp.
Uncompress the zip file: cd /tmp unzip spring-framework-3.0.5.RELEASE.zip (replacing the above filename with the one you just downloaded). Copy the files as follows (replacing the version numbers in the filenames below with the actual version you just downloaded):
cp /tmp/org.springframework.instrument-3.0.5.RELEASE.jar /opt/springsourcetc-server-standard/olio-web/lib/ cp /tmp/org.springframework.instrument.tomcat-3.0.5.RELEASE.jar /opt/springsourcetc-server-standard/olio-web/lib/
Install EclipseLink
1 Download the EclipseLink installer .zip file: a b 2 Go to: http://www.eclipse.org/eclipselink/downloads/previous_releases.php Select the latest EclipseLink 2.0.x (either Downloads or Patch Set Downloads) and download the Installer zip file, saving it to /tmp.
Uncompress the zip file. cd /tmp unzip eclipselink-2.0.2.v20100323-r6872.zip (replacing the above filename with the one you just downloaded).
132
VMware, Inc.
Copy the files as follows (replacing the version numbers in the filenames below with the actual version you just downloaded):
cp /tmp/eclipselink/jlib/eclipselink.jar /opt/springsource-tc-server-standard/olio-web/lib/ cp /tmp/eclipselink/jlib/jpa/eclipselink-jpa-modelgen_2.0.2.v20100323-r6872.jar /opt/springsource-tc-server-standard/olio-web/lib/ cp /tmp/eclipselink/jlib/jpa/javax.persistence_2.0.0.v201002051058.jar /opt/springsource-tc-server-standard/olio-web/lib/
Install MySQL Connector/J

1 Download the MySQL Connector/J .zip file: a b c 2 Go to: http://dev.mysql.com/downloads/connector/j/5.0.html Select the latest version and Platform Independent. Download the zip archive version, saving it to /tmp.
Uncompress the zip file. cd /tmp unzip mysql-connector-java-5.0.8.zip (replacing the above filename with the one you just downloaded). Copy the files as follows (replacing the version numbers in the filename below with the actual version you just downloaded):
cp /tmp/mysql-connector-java-5.0.8/mysql-connector-java-5.0.8-bin.jar /opt/springsource-tc-server-standard/olio-web/lib/
Install JSTL
1 Download the following files, saving them to /tmp: http://download.java.net/maven/glassfish/javax/servlet/jsp/jstl/jstl-api/1.2/jstl-api-1.2.jar http://download.java.net/maven/glassfish/org/glassfish/web/jstl-impl/1.2/jstl-impl-1.2.jar Copy the files as follows (replacing the version numbers in the filename below with the actual version you just downloaded):
cp /tmp/jstl-api-1.2.jar /opt/springsource-tc-server-standard/olio-web/lib/ cp /tmp/jstl-impl-1.2.jar /opt/springsource-tc-server-standard/olio-web/lib/
Configure tc Server
1 Edit the context.xml file: With an editor of your choice, copy the contents of the file: vserver/olioweb/tc-server/add_to_context.xml to: /opt/springsource-tc-server-standard/olio-web/conf/context.xml 2 Edit the server.xml file: a With an editor of your choice, copy the contents of the file: vserver/olioweb/tc-server/add_to_server.xml to: /opt/springsource-tc-server-standard/olio-web/conf/server.xml Still in the server.xml file: Change tomcatThreadPool to maxThreads="600" Set maxKeepAliveRequests="-1"
VMware, Inc.
133
Edit the setenv.sh file: a b With an editor of your choice, open the file: /opt/springsource-tc-server-standard/olio-web/bin/setenv.sh Set the following:
GENERAL_JVM_OPTS="-Xms2g -Xmx2g -Xss192k -server -XX:+UseParallelGC" APPLICATION_OPTS="-Dwebapp.image.directory=/OlioFileStore/ -DmemcachedInstances=localhost:11211 -DcacheFactoryClass=org.apache.olio.webapp.cache.MemCachedFactory -DgeocoderURL=http://olio-db:8080//geocoder/geocode" JAVA_HOME=/usr/java/latest JRE_HOME=/usr/java/latest
Edit the boot.rc.template file: a b With an editor of your choice, open the file: /opt/springsource-tc-server-standard/boot.rc.template Set the following: TOMCAT_USER=root TC_SERVER_HOME="/opt/springsource-tc-server-standard" INSTANCE_NAME=olio-web JAVA_HOME=/usr/java/latest
Run the following commands: cp /opt/springsource-tc-server-standard/boot.rc.template /etc/init.d/tcserver chmod +x /etc/init.d/tcserver chkconfig tcserver on NOTE After running these commands, you might observe a number of insserv errors. These can be safely ignored.
Clone the Olio Web Virtual Machine

After the configuration is complete, make as many clones of the Olio web virtual machine as you will need (one for each additional tile you expect to run), naming them OlioWebN (where N is the tile number to which this workload belongs). Thus the first Olio web virtual machine would be OlioWeb0, the second would be OlioWeb1, and so on. After cloning the virtual machines, configure their network settings, as described in Configuring Network Settings in SLES 11 on page 147, edit their /etc/HOSTNAME files, changing the hostname to the appropriate value, and edit their /etc/hosts file to include an entry for Olio-DB as an alias for the IP address of their OlioDBN machine and to include their tiles client.
134
VMware, Inc.
Prepare the DVD Store 2 Database Virtual Machines

To prepare the DVD Store 2 database virtual machine, first create a SLES 11 virtual machine template, as described in Create and Configure the SLES Virtual Machines on page 112, then follow the instructions in the following sections. NOTE The majority of this section is only necessary if you configure a DVD Store 2 database workload virtual machine from scratch. If, as we recommend, you instead download the pre-built virtual machine template from VMware, as described in: Create the DVD Store 2 Database Virtual Machines on page 57, you may skip ahead to Prepare the DVD Store 2 Database on page 138.

The DVD Store 2 database virtual machine starts with the SLES 11 virtual machine template created in Create and Configure the SLES Virtual Machines on page 112. You then customize it, as described in the following two subsections.

Clone the SLES11-template virtual machine, naming the new virtual machine DS2db0.
Customize the Virtual Machine for DVD Store 2 Database

1 2 3 4 5 6 7 Within the vSphere Client Inventory pane, select the DVD Store 2 database virtual machine (DS2DB0). Under the Getting Started tab, click Edit virtual machine settings. In the Virtual Machine Properties window, under the Hardware tab, select Memory. Set the Memory Configuration: to 4GB. In the Virtual Machine Properties window, under the Hardware tab, select CPUs. Set the Number of virtual processors: to 4. Click OK.

1 2 3 4 5 6 7 Within the vSphere Client Inventory pane, select the DVD Store 2 database virtual machine. Under the Getting Started tab, click Edit virtual machine settings. In the Virtual Machine Properties window, under the Hardware tab, click the Add... button. In the Add Hardware window, within the middle pane, select Hard Disk, then click Next. In the Select a Disk window, select the Create a new virtual disk radio button, then click Next. In the Create a Disk window, enter 35GB, leave the Disk Provisioning options unchecked, select a location (either Store with the virtual machine or Specify a datastore:), then click Next. In the Advanced Options window, leave the options unchanged: NOTE Optionally, the data disk can use the VMware PVSCSI device driver. For details, see Using a PVSCSI Virtual Device Driver on page 146. Virtual Device Node: SCSI (0:1) Mode: Independent: leave unselected Then click Next. 8 9 At the Ready to Complete window, click Finish to accept settings. Back at the Virtual Machine Properties window, click OK to close the window.
VMware, Inc.
135
Create a Third (Temporary) Virtual Disk

NOTE This 20GB disk is only used during the DVD Store 2 setup and can be removed afterwards. If desired, you can use an NFS datastore instead, in which case you can skip creation of the 20GB partition. NOTE If you have allocated the bare minimum amount of disk space needed for a tile, this additional 20GB might be more than you have available. In this case, powering down all the other virtual machines in the tile (thus automatically deleting their vswap files) will free enough space for this purpose. Once you complete the DVD Store 2 setup and delete the 20GB disk, these other virtual machines can be powered back on. 1 2 3 4 5 6 7 Within the vSphere Client Inventory pane, select the DVD Store 2 database virtual machine. Under the Getting Started tab, click Edit virtual machine settings. In the Virtual Machine Properties window, under the Hardware tab, click the Add... button. In the Add Hardware window, within the middle pane, select Hard Disk, then click Next. In the Select a Disk window, select the Create a new virtual disk radio button, then click Next. In the Create a Disk window, enter 20GB, leave the Disk Provisioning options unchecked, select a location (either Store with the virtual machine or Specify a datastore:), then click Next. In the Advanced Options window, leave the options unchanged: Virtual Device Node: SCSI (0:2) NOTE The virtual device node may be different than shown, but can be left at its default Mode: Independent: leave unselected Then click Next. 8 9 At the Ready to Complete window, click Finish to accept settings. Back at the Virtual Machine Properties window, click OK to close the window.
Install the Software for the DVD Store 2 Database Virtual Machines
The following sections describe configuring the DVD Store 2 database system and installing software for the workload.
Configure the Network and Set the Hostname

1 1 2 Boot the virtual machine and log in as root. Configure the network settings, as described in Configuring Network Settings in SLES 11 on page 147. Edit the /etc/HOSTNAME file, changing the hostname to DS2DB0.
Partition, Format, and Mount the DVD Store 2 Database Disk

1 Partition the newly-created 35GB disk using fdisk: a b c d At a command prompt, enter: fdisk /dev/sdb When asked Command (m for help):, enter: n When asked Command action e extended p primary partition (1-4), enter: p When asked Partition number (1-4):, enter: 1
136
VMware, Inc.
e f 2 3 4 5
When asked the First cylinder and the Last cylinder, leave them at their defaults (to use the entire disk) by pressing <Enter> after each question. When again asked Command (m for help):, enter: w
Back at the command prompt, format the 35GB disk: mkfs -t ext3 /dev/sdb1 Create the /var/lib/mysql directory: mkdir /var/lib/mysql Mount the 35GB disk: mount /dev/sdb1 /var/lib/mysql Open the file /etc/fstab for editing and add the following line: /dev/sdb1 /var/lib/mysql ext3 defaults 1 2
Partition, Format, and Mount the DVD Store 2 Temporary Disk

1 Partition the newly-created 20GB disk using fdisk: a b c d e f 2 3 At a command prompt, enter: fdisk /dev/sdc When asked Command (m for help):, enter: n When asked Command action e extended p primary partition (1-4), enter: p When asked Partition number (1-4):, enter: 1 When asked the First cylinder and the Last cylinder, leave them at their defaults (to use the entire disk) by pressing <Enter> after each question. When again asked Command (m for help):, enter: w
Back at the command prompt, format the 20GB disk: mkfs -t ext3 /dev/sdc1 Mount the 20GB disk: mount /dev/sdc1 /mnt (There is no need to add this disk to the fstab file because we will use it only once. If you reboot before completing the DVD Store 2 database preparation, though, you will need to again mount the drive manually.)
Install MySQL
1 2 3 4 5 6 7 8
VMware, Inc.
Connect the SLES 11 installation disk to the virtual machine. At the command prompt, type: yast Using the cursor keys, select Software in the left pane, move to the right pane, select Software Management, then press <Enter>. In the Search Phrase box, type mysql, then press <Enter>. Scroll down to mysql and select it for installation by pressing <Space>. Accept the package list by pressing <Alt><a>. Press <Enter> to accept the Automatic Changes. Press <Alt><q> to quit yast.
137
9 10 11
Start MySQL: /etc/init.d/mysql start Configure MySQL to start automatically on future reboots: chkconfig --level 3 mysql on Create the web user on mysql by running the following commands: mysql create user 'web'@'localhost' IDENTIFIED BY 'web'; grant all privileges on *.* to 'web'@'localhost' with grant option; grant all privileges on *.* to 'web'@'%' with grant option; set password for 'web'@'%' = password('web'); quit
Prepare the DVD Store 2 Database

1 Copy the my.cnf file from the VMmark kit vserver/ds2db directory to /etc. (This step will already have been completed if you are using pre-built virtual machine template downloaded from VMware, but repeating it will do no harm.) Copy the DVD Store 2 database zip file (ds21.zip) from the VMmark kit vserver/ds2db directory to the /mnt directory (the temporary 20GB drive). (This step will already have been completed if you are using pre-built virtual machine template downloaded from VMware, but repeating it will do no harm.) Unpack the file in /mnt (the temporary 20GB drive): unzip ds21.zip (This step will already have been completed if you are using pre-built virtual machine template downloaded from VMware, but repeating it will do no harm.) Run the following commands: cd /mnt/ds2 perl Install_DVDStore_vmmark.pl Create the database by running the following commands: (Note that the third command (the shell script) can take a long time to complete, during which time it might receive high utilization warnings from vSphere.) cd /mnt/ds2 cd mysqlds2 sh mysqlds2_create_all.sh cd ../data_files rm -f cust/*.csv rm -f orders/*.csv Only if creating the DVD 2 Store virtual machine from scratch (but not if using a template downloaded from VMware), move the entire DS2 directory to /root/ds2: mv /mnt/ds2 /root Unmount the temporary partition used for the DS2 data files: cd / umount /mnt If you used the template virtual machine, remove from /etc/fstab the entry for the temporary disk (its marked Remove after DB setup). Remove the 20GB temporary disk from the virtual machine and delete the corresponding .vmdk file.
8 9
138
VMware, Inc.
Clone the DVD Store 2 Database Virtual Machine

After the configuration is complete, make as many clones of the DVD Store 2 database virtual machine as you will need (one for each additional tile you expect to run), naming them DS2DBN (where N is the tile number to which this workload belongs). The first DVD Store 2 database virtual machine would be DS2DB0, the second would be DS2DB1, and so on. After cloning the virtual machines, configure their network settings, as described in Configuring Network Settings in SLES 11 on page 147, edit their /etc/HOSTNAME files, changing the hostname to the appropriate value, and edit their /etc/hosts file to include their tiles client.
VMware, Inc.
139
Prepare the DVD Store 2 Web Virtual Machines

To prepare the DVD Store 2 web virtual machine, first create a SLES 11 virtual machine template, as described in Create and Configure the SLES Virtual Machines on page 112, then follow the instructions in the following sections. NOTE This section is only necessary if you configure a DVD Store 2 web workload virtual machine from scratch. If, as we recommend, you instead download the pre-built virtual machine template from VMware, as described in: Create the DVD Store 2 Web Virtual Machines on page 59, you may skip this section entirely.

The DVD Store 2 web virtual machine starts with the SLES 11 virtual machine template created in Create and Configure the SLES Virtual Machines on page 112. You then customize it, as described in the following two subsections.

Clone the SLES11-template virtual machine, naming the new virtual machine DS2WebA0.
Customize the Virtual Machine for DVD Store 2 Web

1 2 3 4 5 6 7 Within the vSphere Client Inventory pane, select the DVD Store 2 web virtual machine (DS2WebA0). Under the Getting Started tab, click Edit virtual machine settings. In the Virtual Machine Properties window, under the Hardware tab, select Memory. Set the Memory Configuration: to 2GB. In the Virtual Machine Properties window, under the Hardware tab, select CPUs. Set the Number of virtual processors: to 2 (it will probably already be set to this number). Click OK.
Install the Software for the DVD Store 2 Web Virtual Machines
The following sections describe installing software for DVD Store 2 web systems.
Configure the Network

1 2 3 4 Boot the virtual machine and log in as root. Configure the network settings, as described in Configuring Network Settings in SLES 11 on page 147. Edit the /etc/HOSTNAME file, changing the hostname to DS2WebA0. Edit the /etc/hosts file to include an entry for DS2-DB as an alias for the IP address of its DS2DBN machine.
Install Apache and PHP

1 2 3 4 5 6 Connect the SLES 11 installation disk to the virtual machine. At the command prompt, type: yast Using the cursor keys, select Software in the left pane, move to the right pane, select Software Management, then press <Enter>. In the Search Phrase box, type apache, then press <Enter>. Scroll down to apache2-mod_php5 and select it for installation by pressing <Space>. Accept the package list by pressing <Alt><a>.
140
VMware, Inc.
7 8 9 10 11 12 13
Press <Enter> to accept the Automatic Changes. Press <Enter> again to return to the main yast screen. In the Search Phrase box, type php5-mysql, then press <Enter>. Scroll down to php5-mysql and select it for installation by pressing <Space>. Accept the package list by pressing <Alt><a>. Press <Enter> to accept the Automatic Changes. Press <Alt><q> to quit yast.
Untar, Edit, and Move the DVD Store 2 Files

1 2 3 Copy the DVD Store 2 web tarball (DS2Web.tar.gz) from the VMmark kit vserver/ds2web directory to the /tmp directory. Untar the file: tar xvzf DS2Web.tar.gz Open the /etc/php5/apache2/php.ini file for editing and make the following changes: mysql.allow_persistent = On mysql.default_host = DS2-DB mysql.default_user = web mysql.default_password = web mysqli.default_host = DS2-DB mysqli.default_user = web mysqli.default_pw = web 4 5 Configure Apache to start automatically on future reboots: chkconfig --level 3 apache2 on Move the DVD Store 2 files: cd /srv/www/htdocs mkdir ds2 cd ds2 mv /tmp/ds2/* .
Copy and Clone the DVD Store 2 Web Virtual Machine

After the configuration is complete, make two clones of the DVD Store 2 web virtual machine and name them DS2WebB0 and DS2WebC0. The original (DS2WebA0) and these two clones (DS2WebB0 and DS2WebC0) are now your three DVD Store 2 web workload virtual machines for tile 0. Then make as many clones of these three DVD Store 2 web virtual machine as you will need (one set for each additional tile you expect to run), naming them DS2WebAN, DS2WebBN, and DS2WebCN (where N is the tile number to which this workload belongs). The DVD Store 2 web virtual machines for the first tile would be DS2WebA0, DS2WebB0, and DS2WebC0, for the second tile would be DS2WebA1, DS2WebB1, and DS2WebC1, and so on. After cloning the virtual machines, configure their network settings, as described in Configuring Network Settings in SLES 11 on page 147, edit their /etc/HOSTNAME files, changing the hostname to the appropriate value, and edit their /etc/hosts file to include an entry for DS2-DB as an alias for the IP address of their DS2DBN machine and to include their tiles client
VMware, Inc.
141
142
VMware, Inc.
Optional Configurations and Settings
This appendix provides information about optional configurations for many of the programs and utilities used to perform the VMmark benchmarking tests. It consists of the following section: Running a Subset of the Workloads (a Partial Tile) on page 143. Optional Windows System Settings on page 144
Running a Subset of the Workloads (a Partial Tile)

Any test runs conducted with a subset of the workload suite (that is, with one or more partial tiles) do not count as valid runs for the purposes of calculating a VMmark Benchmark score. There may be times, however, such as debugging, or measuring the effects of various workloads on the overall performance, when it is desirable to perform partial-tile runs. Partial-tile runs may be started manually, by starting only those workloads desired, or may be started using the VMmark Harness. To start a partial-tile run using the VMmark Harness, modify the VMMARK2.CONFIG file, located in the C:\VMmark2 directory on the prime client, changing the WORKLOADLIST and INFRASTRUCTURELIST variables to include only those workloads you want to run. NOTE Partial-tile runs are only for research or diagnostic purposes and must not be used to calculate VMmark Benchmark results.
VMware, Inc.
143
Optional Windows System Settings

There are a number of settings that can be changed on the Windows workload virtual machines to make them easier and faster to use in the VMmark environment. These settings are optional, and do not have a significant effect on the benchmark results. They are mutually independent, thus you can choose to implement all of them, none of them, or any desired subset. When these changes are to be applied to the mail server virtual machine, some of them must be made before the machine is promoted to be a domain controller. NOTE Many of these settings significantly reduce the security level of the workload virtual machines. Make these changes only if your company policies and network environment allow.
Disable the Manage Your Server Window

The Manage Your Server window appears at every reboot until disabled. It can be disabled by checking the Dont display this page at logon box on the bottom left of the window, then closing the window. You can later access the Manage Your Server window from the Control Panel by choosing Administrative Tools > Manage Your Server.
Set the System to Autologon

If your environment allows it, you can set the Windows workload virtual machines to autologon. Instructions to do this can be found at: http://support.microsoft.com/kb/324737
Disable the <Ctrl> <Alt> <Del> Prompt

To avoid the need to press <Ctrl> <Alt> <Del> each time the system requires a login password, follow these steps: NOTE If this setting is to be applied to the mail server virtual machine, it must be done before it is promoted to domain controller. 1 From the Start menu, select Control Panel > Administrative Tools > Local Security Policy.
NOTE In the case of a Windows domain controller, this is under Control Panel > Administrative Tools > Domain Security Policy. 2 3 4 5 6 In the left pane of the Local Security Settings window (or the Default Domain Security Settings window), click the plus sign next to Local Policies, then click Security Options. Within the right pane, scroll to Interactive logon: Do not require CTRL + ALT + DEL, double click it (there may be a somewhat long delay), and in the window that appears select the Enabled radio button. Click OK to close the Interactive logon: Do not require CTRL + ALT + DEL window. Close the Local Security Settings window (again, there may be a somewhat long delay). For these changes to take effect, log out of Windows, then log back in again.
Disable Password Protection on Resume

Windows Server enables password protection on resume. To disable this feature, follow these steps: For Windows Server 2003: 1 2 From the Start menu, select Control Panel > Display. When the Display Properties window appears, select the Screen Saver tab.
144
VMware, Inc.
Appendix C Optional Configurations and Settings
3 4
Uncheck the On resume, password protect box. Click OK to close the Display Properties window.
For Windows Server 2008: 1 2 3 4 5 6 From the Start menu, select Control Panel. In the Control Panel, double-click Personalization. In the Personalization window click Screen Saver. In the Screen Saver Settings window, make sure the box next to On resume, display logon screen does not contain a checkmark (or set Screen saver to None). Click OK to close the Screen Saver Settings window. Close the Personalization window.
Disable Internet Explorer Enhanced Security

Windows Server enables Internet Explorers enhanced security level by default. This feature can be disabled as follows: For Windows Server 2003: 1 2 3 4 5 6 From the Start menu, select Control Panel > Add or Remove Programs > Add/Remove Windows Components. When the Windows Components Wizard window appears, scroll to Internet Explorer Enhanced Security Configuration and uncheck it. Click Next. When you see the message You have successfully completed the Windows Components Wizard, click Finish. Close the Add or Remove Programs window. If desired, open Internet Explorer and change the home page.
For Windows Server 2008: 1 2 3 4 5 Open the Start menu and select Server Manager. In the right pane of the Server Manager window, under Security Information, click Configure IE ESC. In the Internet Explorer Enhanced Security Configuration window, select the Off radio buttons for both Administrators and Users, then click OK. Close the Server Manager window. If desired, open Internet Explorer and change the home page.
Disable the Shutdown Event Tracker

By default, Windows Server 2003 keeps a log of the reasons the system was shut down. Instructions to disable this tracking can be found at the following two URLs: http://support.microsoft.com/kb/555541 http://support.microsoft.com/kb/293814 (you might need to refer to both URLs to disable event tracking).
VMware, Inc.
145
Using a PVSCSI Virtual Device Driver

The high-performance PVSCSI virtual device driver can be used wherever it is supported (though the machines must still meet all requirements in the Run and Reporting Rules, including consistency of workloads). This section describes how to enable the PVSCSI adapters. NOTE PVSCSI support varies depending on ESX version, guest operating system version, and guest operating system service pack. Depending on your combination, PVSCSI might be fully supported, supported only for non-boot disks, or not supported at all. For details, see the VMware Compatibility Guide (http://www.vmware.com/resources/compatibility). When creating a disk for VMmark workloads, you can choose to use a PVSCSI virtual device driver. To do so: 1 Near the end of the creation process, in the Advanced Options window, when you are given the chance to select the virtual device node, instead of leaving the option at its default SCSI (0:1), change it to SCSI (1:0), then click Next. At the Ready to Complete window, click Finish to accept settings. Back at the Virtual Machine Properties window, under the Hardware tab, select New SCSI Controller (adding). In the right pane, under SCSI Controller Type, click the Change Type button. In the Changing the SCSI Controller Type window, select the VMware Paravirtual radio button, then click the OK button. Back at the Virtual Machine Properties window, click OK.
2 3 4 5 6
146
VMware, Inc.
Appendix C Optional Configurations and Settings
Configuring Network Settings in SLES 11

1 2 3 Make sure that the file /etc/udev/rules.d/70-persistent-net.rules has been deleted. From command line, type: yast2 lan At the YaST2 Network Settings screen you might see two NICS listed in the main pane. Under the field IP Address, one will say Not configured while the other will say DHCP. The Not configured NIC is the actual virtual NIC. The other one, if present, incorrectly refers to the template's virtual NIC. Use the <Tab> and cursor keys until the DHCP NIC is highlighted. Type <Alt><t> to delete the DHCP NIC. Use the <Tab> and cursor keys until the Not configured NIC is highlighted. Type <Alt><i> to edit the Not configured NIC. Use the <Tab> key to highlight Statically assigned IP Address, then press <Space> to select it. Use the <Tab> key to move to the IP Address field and enter this virtual machines static IP address, subnet mask, and hostname in their respective fields, then type <Alt><n> to save. Type <Alt><o> to exit.
4 5 6 7 8 9 10
VMware, Inc.
147
148
VMware, Inc.
Troubleshooting
This appendix provides assistance in troubleshooting the VMmark Benchmarking tests.
Workload Troubleshooting
This section contains guidance for troubleshooting problems with specific workloads.
Mailserver Troubleshooting
This section contains guidance for troubleshooting mailserver operations.
Error Validating State of Active Directory

The following error message can be caused by the Exchange Management Tools (in the client systems) and Exchange server (in the workload virtual machines) not having matching service packs.
Error: Setup encountered a problem while validating the state of Active Directory: Setup could not retrieve the schema master domain controller information from Active Directory.
For example, you will see this error message if you installed the SP2 version of the Exchange Management Tools on the client, but you installed Exchange SP1 on the server. Also see Install Exchange Server 2007 Tools on page 86 and Install Microsoft Windows Server 2008 Enterprise Edition on page 106.
Microsoft Exchange Load Generator -- Error 1722

The following error message might be encountered during LoadGen installation:
Product: Microsoft Exchange Load Generator -- Error 1722. There is a problem with this Windows Installer package. A program run as part of the setup did not finish as expected. Contact your support personnel or package vendor. Action START_REMOTE_SERVICE.B08E90EE_3FC7_4509_A7F3_DE16BC275F17, location: net.exe, command: START LoadGenRemote
To get around this issue, disable all network interface cards and retry the installation.
Correcting Mail Server Drive Letters and Names

The mail server wont function properly if its drive letters are not correct. This section details how to change them. (The drive names are just for convenience, and wont actually affect benchmark operation.) 1 2 Within the mail server virtual machine, open the Server Manager window (Start Menu > Administrative Tools > Server Manager). In the left pane, expand Storage (click the plus sign) and click Disk Management.
VMware, Inc.
149
Compare the drive letters and partition names to Table D-1. Table D-1. Mail Server Virtual Machine Disk Partitions
Function Partition size Partition name Partition letter Partition 1 32,768MB OS-and-Apps C: Partition 2 32,768MB Database E: Partition 3 8,192MB Backup F:
If the drive letters dont match the ones listed in Table D-1, correct them as follows: a If one of the disk partitions is using drive letter D:, change it to Z: (as a temporary designation): Right-click on the partition and choose Change Drive Letter and Paths, click the Change button, choose Z from the drop-down menu, then click OK, and Yes. If the CD-ROM is using any drive letter other than D:, change it to D: Right-click on the CD-ROM and choose Change Drive Letter and Paths, click the Change button, choose D from the drop-down menu, then click OK, and Yes. Change the Database and/or Backup drive letters to match Table D-1: Right-click on the partition and choose Change Drive Letter and Paths, click the Change button, choose the appropriate letter from the drop-down menu, then click OK, and Yes.
Optionally, rename the E: and F: drives to correspond with the names in Table D-1: a b c a Right-click on the E: partition and choose Properties. Under the General tab, in the top window, type the partition name from Table D-1. Click OK. Repeat for the F: partition.
When done, close the Server Management window.
LoadGen Log File Errors

LoadGen log file errors similar to the following can be caused by Windows proxy settings:
Outlook2007Online Error: 0 : 6/11/2010 9:44:11 AM -- VMMARKE-081 3D010428-LGU000018:AutoDiscover:Failed. Outlook2007Online Error: 0 : 6/11/2010 9:44:11 AM -- VMMARKE-081 3D010428-LGU000003:AutoDiscover:Error:System.Net.WebException: The operation has timed out
To correct this, turn off automatic proxy settings detection in Windows, as follows: 1 2 3 4 Open the Start menu then select Control Panel. In the Control Panel window, double-click Internet Options. In the Internet Properties window, under the Connections tab, click the LAN settings button. Make sure the box next to Automatically detect settings does not contain a checkmark.
Mailserver Virtual Machines Produce No Results

Entering an incorrect password for the mailserver in the VMMARK2.CONFIG file can result in LoadGen silently failing, resulting in a run with no mailserver results. In the VMMARK2.CONFIG file the mailserver password is entered on a line that looks like this: MailServer/MailPassword="password"
150
VMware, Inc.
Appendix D Troubleshooting
No Exchange Window Visible

If your harness says it's running Exchange, but you don't see the Exchange window, this might be due to the Exchange window is running in a different session. If you are connected to your client over terminal services, usually the first connection will see the window.
LoadGen Cant Connect to Active Directory

If LoadGen says it can't connect to the Active Directory, investigate the following possible causes and some suggested solutions for each: The corresponding mail server virtual machine is powered off. Power on the mail server virtual machine. You are experiencing network problems. Check networking (a ping test should be sufficient). The client system has multiple NICs. If so, run nslookup in a Command Prompt. It should return the IP address of the domain server. If it returns the secondary DNS address, check the order of the NICs as follows: a b c Open Control Panel, right click Network Connections, select Open. In the Network Connections window select the Advanced menu, choose Advanced Settings, then the Adapters and Bindings tab. Make sure that the NIC you are using is first in the list. If its not, use the green arrows to move it.
Repeat the nslookup command. If it now returns the IP address of the domain server, try running LoadGen again. You are experiencing DNS problems. Make sure the client's preferred DNS server is the IP address of the mail server for that tile. The Active Directory service is not running. Make sure that the Active Directory service is running on the mailserver (restart the service if it isnt running).
LoadGen Crashes If Exchange Server Transport Service Stops

Corruption of the Exchange Server transport queue database, or a transport queue database file so large that it is filling the database disk, can cause the Exchange Server Transport service to stop. This, in turn, can cause LoadGen to crash. To fix this problem, try defragmenting the transport queue database as follows: 1 2 Log into the mailserver workload virtual machine. At a Command Prompt enter: net stop "Microsoft Exchange Transport" cd C:\Program Files\Microsoft\Exchange Server\TransportRoles\data\Queue eseutil /d mail.que net start "Microsoft Exchange Transport" NOTE The eseutil command might fail if there is insufficient disk space. It requires about 10MB of free space.
VMware, Inc.
151
Deleting the Microsoft Exchange Database

If the Microsoft Exchange database becomes corrupt, or for some other reason needs to be recreated, you must first delete the existing database. To delete the Microsoft Exchange database 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 From the start menu, select All Programs > Administrative Tools > Active Directory Users and Computers. In the left pane, click on the + next to the domain name for this server. In the left pane right-click LoadGen Objects and select Delete. When asked Are you sure you want to delete the Organizational Unit names 'LoadGen Object'? click Yes. When asked to Confirm Subtree Deletion click Yes. Close the Active Directory Users and Computers window. From the start menu, select All Programs > Microsoft Exchange Server 2007 > Exchange Management Console. In the left pane click on the + next to Server Configuration and select Mailbox. In the right pane, select lpriv1, and delete it by clicking Remove in the right action pane. When a message appears asking Are you sure you want to remove lpriv1?, click Yes. When a message appears stating The store has been removed. You must remove the database files from your computer manually., click OK. Repeat step 9 through step 11 for lpriv2. In the left pane, select the VMmark storage group and delete it by clicking Remove in the right action pane. When a message appears asking Are you sure you want to remove VMmark?, click Yes. When a message appears stating The storage group has been removed. You must remove the log files from your computer manually, click OK. Close the Exchange System Manager window. From the Start menu select Computer. Open the Database drive (typically E:\). Delete all files on the database drive. Recreate the VMmark storage group and the lpriv1 and lpriv2 mailbox stores, as described in Configure Microsoft Exchange Server on page 121.
152
VMware, Inc.
DVD Store 2 Troubleshooting

This section includes guidance for troubleshooting DVD Store 2 operations.
(504) Gateway Timeout Error

If the DVD Store 2 .wrf output file contains the following message, you might have a problem with proxy settings:
Error returned from web server: The remote server returned an error: (504) Gateway Timeout"
To fix this, merge the ProxySettingsPerUser.reg file into your client systems registry, as described in Automatically Apply the Registry Settings on page 79 (or follow the directions in Manually Apply the Registry Settings on page 80) then change the clients proxy settings, as described in Configure Windows to Not Use a Proxy Server on page 82.
VMware, Inc.
153
Miscellaneous Troubleshooting
This section includes guidance for troubleshooting miscellaneous issues.
Out of Bounds Error by Harness During Time Sync

If you observe Out of bounds error by harness during time sync in the STAX messages window, make sure the date and time format used by all Windows-based systems (that is, all clients and the vCenter server) is the USA standard (that is, month/day/year) as opposed to the UK standard (that is, day/month/year).
Popup: Space Allocated for Logging PowerShell Events is Nearly Full

If you see a popup dialog box advising that the space allocated for logging PowerShell events is nearly full, modify the PowerShell properties to overwrite events by following these steps: For Windows Server 2003: 1 2 3 4 5 6 7 8 Open the Start menu and select Settings > Control Panel. In the Control Panel, double-click Administrative Tools. In the Administrative Tools window, double-click Event Viewer. In the left pane of the Event Viewer window, right-click on Windows PowerShell and choose Properties. In the Windows PowerShell Properties window, make sure the Overwrite events as needed radio button is selected, then click OK. Back in the left pane of the Event Viewer window, look for a PowerShell entry (just PowerShell, not Windows PowerShell). If you see such an entry, repeat step 4 and step 5, this time for PowerShell. Close the Event Viewer window. Close the Administrative Tools window.
For Windows Server 2008 1 2 Open the Start menu and select Server Manager. In the left pane of the Server Manager window, expand Diagnostics by clicking the + next to the word, then expand Event Viewer by clicking the + next to the words, then expand Application and Services Logs by clicking the + next to the words. Still in the left pane of the Server Manager window, right-click on Windows PowerShell and choose Properties. In the Log Properties - Windows PowerShell window, make sure the Overwrite events as needed (oldest events first) radio button is selected, then click OK. Back in the left pane of the Server Manager window, look for a PowerShell entry (just PowerShell, not Windows PowerShell). If you see such an entry, repeat step 3 and step 4, this time for PowerShell. Close the Server Manager window.
3 4 5 6
Infrastructure Operations Troubleshooting

If you're having issues with the VMmark infrastructure operation routines, check the following: Make sure your vCenter Server is pingable from the prime client. On the vCenter Server, go to Server Manager > Services, and make sure VMware VirtualCenter Server is Started. It can sometimes take up five minutes for the service to respond to requests after restarting either the vCenter system or the service itself. Before starting the benchmark, verify that the time protocol on the vCenter Server is running and that the vCenter Server time is synchronized with the testing environment.
154 VMware, Inc.
STAF Complains About Trust Level

If STAF complains that systems are at trust level 3, this suggests that the trust level settings in the STAF.cfg file are not being correctly processed. To correct this, make sure each STAF.cfg file contains the following: trust machine 192.168.*.* level 5 (substituting the first three octets of the network to which the machines are connected).
STAF is Unable to Connect to a Particular Server

If STAF is unable to connect to a particular server, this may be due to one of the following causes: A communication or configuration problem. From a command line on the client, type: staf alias ping ping (where alias is the alias in the clients hosts file (for example, DS2WebA0)). The proper response is PONG. Dissimilar versions of STAF are installed. Make sure all clients and all workload virtual machines are running the same version of STAF. Review the hosts files on both systems for accuracy (See Update the hosts File on page 81). Review the firewall status on both systems for the allowance of STAFProc.exe (Configure the Windows Server 2008 Firewall to Allow STAFProc.exe to Pass Through on page 69).
The VMmark Harness Complains That Clients Are Out of Sync

If the VMmark harness complains that clients are out of sync, this is likely due to client clocks not being synchronized to within the allowable threshold. Force clients to sync to their domain controllers immediately by opening a Command Prompt window and typing: net time /domain /set /yes
VMware vSphere PowerCLI Snap-In Loads Slowly

On some prime clients, VMware vSphere PowerCLI snap-in might take a long time to load and display the prompt. This can be caused by either of two different issues, distinguishable by the length of the delay. While it is acceptable for the PowerCLI snap-in to load slowly the first time PowerCLI is run, on subsequent invocations it should load quickly. If it does not, follow the instructions in the appropriate section below.
PowerCLI Snap-In Takes About 20 Seconds to Load

If you observe delays of about 20 seconds for the PowerCLI snap-in to load, perform the following steps: 1 2 3 Make sure you are running PowerCLI version 4.1 update 1 (aka 4.1.1) or later, and that you are running PowerShell version 2 or later. Log in as the domain administrator. Start PowerCLI (Start > Programs > PowerCLI) in an environment that can reach the external web so caching can occur. NOTE If access to the external web requires the use of a proxy server, it should be enabled, PowerCLI should be started (allowing caching), then the proxy server should be disabled before VMmark testing starts.
VMware, Inc.
155
PowerCLI Snap-In Takes About Two Minutes to Load

If you observe delays of about two minutes for the PowerCLI snap-in to load, perform the following steps: 1 2 3 4 5 6 7 Open the Start menu then select Settings > Control Panel. In the Control Panel, double-click Internet Options. In the Internet Options window, under the Advanced tab, in Settings, scroll down to Security. Remove the checkmark from the box next to Check for publisher's certificate revocation. Remove the checkmark from the box next to Check for server certificate revocation. Click OK. Close the Control Panel.
Error: File C:\VMmark2\tools\VMmarkPScli.ps1 cannot be loaded.

In some cases, when running Windows Server 2008 on a prime client, you might observe in your output files the following error: File C:\VMmark2\tools\VMmarkPScli.ps1 cannot be loaded. The file C:\VMmark2\tools\VMmarkPScli.ps1 is not digitally signed. The script will not execute on the system. In this case, perform the following steps: 1 2 3 4 Navigate to the VMmark2\tools directory. Right-click the VMmarkPScli.ps1 file and select Properties. At the VMmarkPScli.ps1 Properties window, under the General tab, click the Unblock button. Close the VMmarkPScli.ps1 Properties window.
156
VMware, Inc.
Networking Issues
This section addresses network issues sometimes encountered while running VMmark.
Setting the Number of Ports on a Virtual Switch in ESX

Virtual network switches in VMware ESX start out with a fixed number of ports by default. If your test uses more than this number of ports, you might experience difficult-to-diagnose errors. Increasing the number of ports should alleviate these issues. Follow these steps to determine the number of ports a virtual switch is configured with and increase that number if required: 1 2 3 4 5 6 Within the vSphere Client Inventory pane, select the ESX host. Select the Configuration tab. In the Hardware window, select Networking. To the right of the virtual switch to be configured choose Properties. Under the Ports tab, the vSwitch Properties section lists the number of ports. If the number of ports listed is fewer than your setup needs, add ports as follows: a b c Click Edit Increase the number of ports. Click OK. NOTE The changes will take effect the next time the ESX host system is rebooted. 7 Click Close.
Setting the Network Name

If a virtual machine is unable to connect to the network, make sure that it is configured to use the correct network (that is, the correct virtual switch), that the virtual switch is configured to the correct physical NIC, and that the NIC is connected to the intended network. For further information, refer to the VMware Infrastructure 3 Documentation set.
Setting Network to Connect at Power-on

If a virtual machine is unable to connect to the network, make sure that its network adapter is configured to connect at power on. This setting can be found in Summary > Commands > Edit Settings > Hardware > Network Adapter 1.
VMware, Inc.
157
Performing a Short Run

When configuring a VMmark system, it can be useful to perform a short VMmark test run to make sure everything is functioning correctly. This can be done by reducing the value of the RUNTIME variable from its default of 10800 seconds (3 hours) to something no less than 900 second (15 minute). One situation in which these short runs are particularly helpful is when setting up the first tile. We recommend creating one complete tile (for example, Tile0) and making sure it is working smoothly, then making backups of this working configuration before cloning the workload virtual machines to create the additional tiles.
158
VMware, Inc.
Testing a Single Failing Client-Workload Pair

When setting up a multi-tile VMmark test, you may find that you are having problems with a particular workload in one of the tiles. To debug the problem, it can be useful to run a test that uses only the specific client system and workload virtual machine pair that exhibits the problem. The VMmark Harness is flexible enough to allow you to perform that type of testing. This section describes how to do so. This example shows a five-tile test and assumes a failure involving OlioWeb2 (that is, the OlioWeb in the third tile) and client2 (that is, the third client). The relevant portions of the VMMARK2.CONFIG file look something like this:
TILES=5 CLIENTS="client0 client1 client2 client3 client4" MailServer/SERVERS="Mailserver0 Mailserver1 Mailserver2 Mailserver3 Mailserver4" Standby/SERVERS="Standby0 Standby1 Standby2 Standby3 Standby4" OlioWeb/SERVERS="OlioWeb0 OlioWeb1 OlioWeb2 OlioWeb3 OlioWeb4" OlioDB/SERVERS="OlioDB0 OlioDB1 OlioDB2 OlioDB3 OlioDB4" DS2WebA/SERVERS="DS2WebA0 DS2WebA1 DS2WebA2 DS2WebA4" DS2WebB/SERVERS="DS2WebB0 DS2WebB1 DS2WebB2 DS2WebB3 DS2WebB4" DS2WebC/SERVERS="DS2WebC0 DS2WebC1 DS2WebC2 DS2WebC3 DS2WebC4" DS2DB/SERVERS="DS2DB0 DS2DB1 DS2DB2 DS2DB3 DS2DB4" ###RUNTIME=10800 ###DEBUGFLAG=0 ###WORKLOADLIST="Standby MailServer OlioWeb OlioDB DS2WebA DS2WebB DS2WebC DS2DB" ###INFRASTRUCTURELIST="Deploy Vmotion SVmotion"
To run just the one workload test between client2 and the Olio workload you can do the following: 1 2 Save your current VMMARK2.CONFIG file. Edit the VMMARK2.CONFIG file, changing the following settings to the values shown:
TILES=1 CLIENTS="client2" OlioWeb/SERVERS="OlioWeb2" OlioDB/SERVERS="OlioDB2" RUNTIME=900 DEBUGFLAG=1 WORKLOADLIST="OlioWeb OlioDB" INFRASTRUCTURELIST=""
3 4
Log on to client2 and copy the C:\vclient2 directory to C:\vclient0. Start the test from your prime client.
The harness will run a one-tile test of just the Olio workload using the client and workload virtual machine that seemed to be causing the problem. OlioWeb and OlioDB comprise the Olio workload, so both need to run to test the failing OlioWeb. In the Results_<datestamp> directory you'll find the files from just that one workload along with the STAX_Job_User.log that contains the same output as was shown in the lower portion of the STAX Monitor window. You can also run a similar test with a functional client and virtual machine workload pair, allowing a side by side comparison of the STAX_Job_User logs. If you suspect the problem may be with the client, you can substitute one of the known good clients for any workload except mailserver (because mailserver is dependent on the domain relationship). In the example above you could use client0, client1, client3, or client4 (just make sure the client you use has a C:\vclient0 directory). If you still don't see how to fix the problem, narrowing the test down to the simplest case where the failure occurs can help expedite the debugging process when reporting the problem to VMware.
VMware, Inc.
159
The VMmark2ConfigChecker Script

The VMmark2ConfigChecker script (VMmark2ConfigChecker.pl) is a Perl script that can be used by VMmark submitters to review their ESX testbed configurations for potential issues. It is not intended to be an exhaustive check, but rather is a framework that can be built upon. The script recursively searches for .vmx files in the directory in which it is run as well as all subdirectories. It then performs a series of checks on each .vmx file and reports its findings. The output of the script does not indicate the presence or absence of errors or noncompliant items in the benchmark configuration. The script simply highlights areas for evaluation for such issues. NOTE If you are running ESXi, see VMware Knowledge Base article #1017910 (http://kb.vmware.com/kb/1017910) for instructions on how to access a command-line interface.
Running the VMmark2ConfigChecker Script

The VMmark2ConfigChecker script is supplied as part of the VMmark kit (in the C:\VMmark2\tools directory). The script can be run in two modes: on a running ESX/ESXi host or on previously-collected reporter results. Both methods are described below.
Running the VMmark2ConfigChecker Script on an ESX/ESXi Host

To run the VMmark2ConfigChecker script on a running ESX/ESXi host, follow these steps: 1 2 3 4 Copy the script to the /root directory of the ESX/ESXi system to be checked. Log in to the ESX/ESXi command line as root. Change to the /vmfs/volumes directory. Run the script: perl /root/VMmark2ConfigChecker.pl -t n (where n is the number of tiles to check.)
The scripts default behavior is to display a summary of its findings to the screen. With the -v option, the script saves more detailed findings to a set of output files. These output files by default are saved to vmmarkconfig-results in the current directory (or /root/vmmarkconfig-results if the current directory is not writable). The location of these output files can also be specified with the -s option.
Running the VMmark2ConfigChecker Script on Previously-Collected Results

To run the VMmark2ConfigChecker script on previously-collected results, follow these steps: NOTE In this mode, the script must be run on results that were generated from a VMmark run made with the REPORTER variable set to 1 (i.e., REPORTER=1). 1 From any shell that has access to a filesystem where workload virtual machine configuration files (i.e., .vmx files) are located, navigate to a folder containing the .vmx files. NOTE If the folder contains configuration-gathering archive files (i.e., .tgz files), but not workload virtual machine configuration files (i.e., .vmx files), expose the .vmx files by unpacking the individual .tgz files using tar xzvf. See Manually Running the vSphere-get-config.sh Script on page 163 for details on the archive file. 2 Run the script: perl /root/VMmark2ConfigChecker.pl -t n (where n is the number of tiles to check.)
160
VMware, Inc.
The scripts default behavior is to display a summary of its findings to the screen. With the -v option, the script saves more detailed findings to a set of output files. These output files by default are saved to vmmarkconfig-results in the current directory (or /root/vmmarkconfig-results if the current directory is not writable). The location of these output files can also be specified with the -s option.
Customizing the VMmark2ConfigChecker Script

To add more checks to the VMmark2ConfigChecker script, change the SearchStrings and the SearchStringModifiers variables near the top of the script. We welcome your suggestions for and examples of custom search strings for future versions of the script. Please send these to vmmark-info@vmware.com.
VMware, Inc.
161
Recovering Data from Abnormal Runs

Even when a VMmark run must be aborted before completion, the partial results files from the run might still be useful for debugging or monitoring purposes. Alternatively, it might be possible to generate usable data from a run that fails to complete normally due to the harness hanging. This may happen, for example, if the STAF process in a workload virtual machine or client system dies during the run.
Recovering Data From An Aborted Run

This section describes how to abort a VMmark run and how it might be possible to recover partial results files. 1 2 3 4 5 To abort a run, in the STAX 3 Job Monitor window, under Active Jobs, right click on the active VMmark 2.0 job and select Terminate Job. In a Cygwin terminal window go to the results directory of the aborted run (C:\VMmark2\results\Results_<datestamp>). Run the canceltest.sh script (included in the \VMmark2\tools directory of the VMmark kit). Follow the scripts directions for recovering results files after a manual STAX shutdown. The recovered files will be moved into the directory in which the script was run. Before beginning another run, reboot all clients and all workload virtual machines.
Recovering Data if the Harness Hangs

This section describes how to generate potentially usable data after some harness hangs. 1 Verify that the total test time has elapsed: Total test time (in minutes) = (RUNTIME + (TILEDELAY * (#TILES - 1)))/60 Default test time (in minutes) = (10800 + (600 * (#TILES - 1)))/60 If at least 20 minutes more than the calculated total test time has elapsed, continue to the next step. In a Cygwin terminal window go to the results directory of the aborted run (C:\VMmark2\results\Results_<datestamp>). Run the canceltest.sh script (included in the \VMmark2\tools directory of the VMmark kit). Follow the scripts directions for recovering results files if the harness is hanging. The recovered files will be moved into the directory in which the script was run. If the STAX Job Monitor Active Job Elements display still shows a process in a tile running, run the canceltest.sh script again as described above until there are no running harness processes. If this was a fully-compliant run, the VMmark harness will output the run result files to the results directory, as described in Creating and Submitting a VMmark Full Disclosure Report on page 95. Before beginning another run, reboot all clients and all workload virtual machines.
2 3 4 5 6 7
162
VMware, Inc.
Manually Running the vSphere-get-config.sh Script

VMmark results for publication must include detailed information about the ESX/ESXi instances in which the test was run. This information is usually obtained automatically, as described in Edit the disclosure.HTML File on page 97. However, if the VMmark harness informs you that the virtualization host reporting script failed to run, you can run it manually, as described in this section. NOTE If you are running ESXi, see VMware Knowledge Base article #1017910 (http://kb.vmware.com/kb/1017910) for instructions on how to access a command-line interface. You also need to change some parameters for ESXi to run the script successfully. Carry out the steps in the comments at the beginning of the vSphere-get-config.sh script before proceeding. 1 2 Copy the file C:\VMmark2\tools\vSphere-get-config.sh to /root on the host. From the ESX/ESXi command line, run the script as root: ./vSphere-get-config.sh After a few minutes, the results of the script will be written to the ESX/ESXi host in a file named hostname-date-time.tgz under the alphanumerically last /vmfs/volumes directory. Copy this .tgz file to the appropriate results directory on the prime client.
Before you submit results, we recommend that you unpack the .tgz file and run the VMmark2ConfigChecker script to review your ESX testbed configurations for potential issues. For further information, see Running the VMmark2ConfigChecker Script on page 160.
VMware, Inc.
163
164
VMware, Inc.

VMmark Bench Marking Guide 2.1-20110310

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

VMmark Bench Marking Guide 2.1-20110310

Uploaded by

Copyright:

Available Formats

VMware VMmark Benchmarking Guide

VMmark 2.1 Revised March 10, 2011

VMware VMmark Benchmarking Guide

VMware, Inc. 3401 Hillview Ave. Palo Alto, CA 94304 www.vmware.com

Preparation Checklists and Worksheets

What is the VMmark Benchmark? 19

VMmark Benchmark Requirements 35

VMware VMmark Benchmarking Guide

Preparing the Infrastructure for VMmark 2.1 Benchmark Tests

Configuring the Client Systems

Running the VMmark Benchmark

Creating the Workload Virtual Machines 99

VMware VMmark Benchmarking Guide

Installing Applications in the Workload Virtual Machines

Optional Configurations and Settings

VMware VMmark Benchmarking Guide

VMware VMmark Benchmarking Guide

VMware VMmark Benchmarking Guide

Virtualization Layer Physical Hardware

Why a Multi-Host Virtual Machine Benchmark?

VMware VMmark Benchmarking Guide

Preparation Checklists and Worksheets

Virtual Machine Preparation

Standby Virtual Machine

Mail Server Virtual Machine

VMware VMmark Benchmarking Guide

Olio Database Virtual Machine

Olio Web Virtual Machine

DVD Store 2 Database Virtual Machine

DVD Store 2 Web Virtual Machine

Deploy Template Virtual Machine

Preparation Checklists and Worksheets

VMware VMmark Benchmarking Guide

Virtual Machine Worksheets

Standby system (32-bit Windows Server 2003)

Table 2. Linux Virtual Machine Worksheet

Olio Web (64-bit SLES 11)

DVD Store 2 Database (64-bit SLES 11)

What is the VMmark Benchmark?

Overview of VMmark 2.1 Benchmark

VMware VMmark Benchmarking Guide

ESX Server One VMmark tile (eight virtual machines)

Chapter 1 What is the VMmark Benchmark?

VMmark Benchmark Workloads

Mail Server Virtual Machine

VMware VMmark Benchmarking Guide

Standby Virtual Machine

Web 2.0 Simulation

Chapter 1 What is the VMmark Benchmark?

Virtual Machine Cloning and Deployment

Dynamic Virtual Machine Relocation

VMware VMmark Benchmarking Guide

Dynamic Storage Relocation

Automated Load Balancing

Chapter 1 What is the VMmark Benchmark?

VMmark Client Systems

VMware VMmark Benchmarking Guide

Chapter 1 What is the VMmark Benchmark?

Contents of VMmark Kit

VMware VMmark Benchmarking Guide

VMmark 2.1 Benchmark Scoring Methodology

Chapter 1 What is the VMmark Benchmark?

Example One: Single-Tile Benchmark Scoring