Appeon Corporation

Page 1

Appeon for PowerBuilder 6.0 Performance Test Report

(Appeon)

[Appeon for PowerBuilder Document]

Appeon for PowerBuilder 6.0 Performance Test Report

Appeon Corporation

Page 2

Content
1 Overview..............................................................................................................................3 1.1 Introduction..................................................................................................................3 1.2 Objectives ....................................................................................................................3 1.3 Test Environment.........................................................................................................3 1.3.1 Application Server and Web Server Configurations ............................................3 1.3.2 Database Server Configuration .............................................................................4 1.3.3 Client Configuration .............................................................................................4 Single User Test...................................................................................................................4 2.1 Test Case 1 - Common Benchmark Test .....................................................................4 2.1.1 Description............................................................................................................4 2.1.2 Test Steps ..............................................................................................................5 2.1.3 Test Result ............................................................................................................5 2.2 Test Case 2 - DataWindow Retrieval...........................................................................5 2.2.1 Description............................................................................................................6 2.2.2 Test Steps ..............................................................................................................6 2.2.3 Test Result ............................................................................................................6 2.3 Test Case 3 Retrieval for DataWindow with Stored Procedure ................................6 2.3.1 Description............................................................................................................6 2.3.2 Test Steps ..............................................................................................................6 2.3.3 Test Result ............................................................................................................6 2.4 Test Case 4 - Dynamic Create DataWindow ...............................................................6 2.4.1 Description............................................................................................................6 2.4.2 Test Steps ..............................................................................................................6 2.4.3 Test Result ............................................................................................................7 2.5 Test Case 5 - Embedded SQL......................................................................................7 2.5.1 Description............................................................................................................7 2.5.2 Test Steps ..............................................................................................................7 2.5.3 Test Result ............................................................................................................7 Stress Test ............................................................................................................................8 3.1 Description...................................................................................................................8 3.2 Test Steps .....................................................................................................................9 3.3 LoadRunner configurations .........................................................................................9 3.4 Test Result .................................................................................................................10 Stability Test ......................................................................................................................13 4.1 Description.................................................................................................................13 4.2 Test Result .................................................................................................................13 Conclusion .........................................................................................................................15

Appeon for PowerBuilder 6.0 Performance Test Report

Appeon Corporation

Page 3

Overview

1.1 Introduction
This document describes the methodology and benchmarks of the APB Web Application Load & Performance testing; captures the observations and results of the Load and Stress Testing. The Load & Performance testing will focus on the capability of handling online users.

1.2 Objectives
Following are the three major objectives of the load testing. Identify the performance of various function points both in PowerBuilder and Appeon for single user. Identify the server memory, processor, and transaction processing time for different online users. Identify the stability of Appeon Server when multiple users run the Appeon Web application for a relatively long time.

1.3 Test Environment

1.3.1 Application Server and Web Server Configurations
We use EAServer work as both Application Server and the Web server. Generally, a separate Web Server will provide better performance and less stress limitation. Type Hardware Item CPU RAM HDD Operating system Software EAServer PowerBuilder Appeon Server Specification Intel(R) Xeon(TM) 2.4GHz Dual CPU 32-bit 2.0GB 150 GB (SCSI 15k RPM) Windows 2003 Enterprise Edition SP2 32-bit EAServer 6.1.0.14 11.2 Build 8407 6.0.0252.00 Enterprise Edition

EAServer is configured as below (Only including key properties): Property Value Http Listener
Maximum Number of Threads Number of Acceptors

1000 500

Appeon for PowerBuilder 6.0 Performance Test Report

Appeon Corporation

Page 4 Ping Pooled Connections

True 200 0 JDK1.5.0.12 -server Xms1024m Xmx1024m

Max Connection Pool Data Source Size Minimum Connection Pool Size Java Version Start Up Options JVM Start Options

1.3.2 Database Server Configuration

Type Hardware Item CPU RAM HDD Operating system Software SQL Server Specification Intel(R) Xeon(TM) 2.4GHz Dual CPU 32-bit 2.0GB 150 GB (SCSI 15k RPM) Windows 2003 Enterprise Edition SP2 32-bit 2000 SP4

1.3.3 Client Configuration

Type Hardware Item CPU RAM HDD Operating system Software Load Runner PowerBuilder Appeon Developer Specification Intel Pentium 2.0GHz 2.0GB 80 GB (IDE 15k RPM) Windows 2003 Standard Edition SP2 8.1.0.0 Build947 11.2 Build 8407 6.0.0252.00 Enterprise Edition

2 Single User Test

2.1 Test Case 1 - Common Benchmark Test
2.1.1 Description
This test case is used to compare the execution efficiency of the following features between PB and Appeon. 1. Script Looping Execute an empty For/Next loop. 2. TAK Benchmark The TAK Benchmark is a widely used algorithm to compare programming tools that make extensive use of function calls. 3. DataStore Sorting Sort the values in a DataStore that is initially sorted in a reverse order. 4. Global Function Overhead
Appeon for PowerBuilder 6.0 Performance Test Report

Appeon Corporation

Page 5

Call an empty global function. 5. Object Function Overhead Call an empty object (window) function. 6. Event Overhead Trigger an empty event. 7. Window Instantiation Open a window with no controls. 8. String Parsing Change all occurrences of one string within another. 9. Global Variable Use a global variable in an assignment statement. 10. Global Constant Use a global constant in an assignment statement. 11. Using DataStores Copy all rows from a DataStore into variables using GetItem..., and then write them to another DataStore using SetItem.

2.1.2 Test Steps

Step1: Click the Run all Tests button. This button will trigger the execution of all testing item in this window. Or you can also click each buttons on the left hand to run one test item one by one. Step2: Click the Set Machine ID button to set the machine hardware information. The input values will be included in the report. Step3: Click the Save Results button to save current test result. Step4: Click the Reports button to review the test result report.

2.1.3 Test Result

These are average values after running 10 times. Test Item Iterations Response Time in PowerBuilder (Second) Script Looping 1000000 0.234 TAK Benchmark 200 0.859 DataStore Sorting 50000 0.141 Global Function Overhead 50000 0.297 Object Function Overhead 50000 0.391 Event Overhead 50000 0.375 Window Instantiation 50 1.391 String Parsing 4 0.422 Global Variable 50000 0.016 Global Constant 50000 0.031 Using DataStores 5 0.328 Response Time in Appeon (Second) 0.187 2.266 0.297 0.250 0.265 0.797 2.547 1.437 0.969 0.969 0.758

2.2 Test Case 2 - DataWindow Retrieval

Appeon for PowerBuilder 6.0 Performance Test Report

Appeon Corporation

Page 6

2.2.1 Description
This test case is used to compare the DataWindow retrieval performance between PB and Appeon. The DataWindow has 9000 rows and 4 columns.

2.2.2 Test Steps

Click Retrieve button to retrieve data from database.

2.2.3 Test Result

These are average values after running 10 times. Response Response Time in Time in Appeon(Second) PowerBu ilder(Sec ond) 0.378 0.730

2.3 Test Case 3 Retrieval for DataWindow with Stored Procedure

2.3.1 Description
This test case is used to compare stored procedure DataWindow retrieval performance between PB and Appeon. It contains three DataWindows whose data sources are all stored procedures.

2.3.2 Test Steps

Step1: Select a customer from DropDownDataWindow. Then all related products information will be retrieved out. Step2: Click any row in the first DataWindow, and then the detailed product information and all relative order are retrieved out.

2.3.3 Test Result

Test Item

1st DW 2nd DW 3rd DW

Response Time in Powerbuilder (Second) 0.061 0.014 0.015

Response Time in Appeon(Second)

0.297 0.032 0.031

2.4 Test Case 4 - Dynamic Create DataWindow

2.4.1

Description

This test case is used to compare the performance of creating a dynamic DataWindow using SyntaxFromSQL between PowerBuiler and Appeon.

2.4.2 Test Steps

Appeon for PowerBuilder 6.0 Performance Test Report

Appeon Corporation

Page 7

Step1: Select one table from the first listbox. Step2 : Select columns from the second one. Then you can see the generated SQLfrom the Syntax area. You can modify this SQL manually. Step3 : Select DataWindow Type Step4 : Select Display Type Step5 : Click Create DW button.
2.4.3

Test Result
Test Item

Select * from t_customers Select * from t_customers, t_groups Select * from t_orders, t_orders_items

Response Time in PowerBuilder (Second) 0.061 0.078 0.084

Response Time in Appeon(Second)

0.063 0.062 0.094

2.5 Test Case 5 - Embedded SQL

2.5.1 Description
This test case is used to compare performance of Embedded SQL between PB and Appeon.

2.5.2 Test Steps

Click those buttons, including Select, Insert, Delete,Update, Commit and Rollback to test the frequently used SQL statement.

2.5.3 Test Result

Test Item

Select Insert Delete Update Commit Rollback

Response Time in PowerBuilder (Second) 0.016 0.094 0.031 0.016 0.016 0.015

Response Time in Appeon(Second)

0.015 0.016 0.012 0.015 0.012 0.015

Appeon for PowerBuilder 6.0 Performance Test Report

Appeon Corporation

Page 8

3 Stress Test
3.1 Description
Creating 20 tables in a SQL server database, each table contains fields with all data types and 10000 rows data. For example, the table contains fields with the following data type: Data, Number, Decimal, Char (255), etc. The types of the column are: two Int, two Char(50), four Char(255), two Char(32253), three Number, Three Decimal(18, 2) and four Date/Datetime.

Develop a PB application including 20 tables.The first twelve tables (from Table 1 to Table 12) are to retrieve 100, 100, 100, 100, 100, 100, 1000, 1000, 1000, 2000, 5000, 10000 rows respectively for each time; and the remaining eight tables (from Table 13 to Table 20) are to update, insert or delete 5, 10, 15, 20, 10, 50, 50, 100 rows respectively for each time. It contains the following 8 function points: No Test Item Operation 1 Function 1 retrieve and update

Remark Use a datawindow to retrieve 100 rows from table t_1 with. Use a datawindow to update 5 rows to table t_13. Use a datawindow update 100 rows to table t_20. Use an esql to retrieve 100 rows from table t_2. Use a datawindow to insert/delete 10 rows to table t_14. Use an esql to update date 50 rows to table t_19 Use a dynamic sql to retrieve 100 rows from table t_3 Use a procedure to insert/delete 15

Function 2

retrieve and update/insert/delete

Function 3

retrieve and update/insert/delete

Appeon for PowerBuilder 6.0 Performance Test Report

Appeon Corporation

Page 9

rows to table t_15 Use a dynamic sql to update 10 rows to table t_17 4 Function 4 retrieve and update/insert/delete Use a procedure to retrieve 100 rows from table t_4 Use a datawidow to insert/delete 20 rows to table t_16 Use a procedure to update 50 rows to table t_18 Use a datawindow to retrieve 100 rows from table t_5 Use a datawindow to retrieve100 rows from table t_6 Use a datawindow to retrieve 5000 rows from table t_11 Use a datawindow to retrieve 10000 rows from table t_12 Use a datawindow to retrieve 1000 rows from table t_7

Function 5 Function 6

retrieve retrieve

6 7 8 Function 7 Function 8 retrieve retrieve

3.2 Test Steps

Step1: Start Appeon Server and login to AEM. Change the Run Mode to the test mode. Step2: Deploy this test case to Web. Step3: Open the LoadRunner in the client machine and record IE operations through Web/Http protocol. Step4: Save LoadRunner script and test it. Step5: Run this script in LoadRunner

3.3 LoadRunner configurations

Think time = 60 seconds LoadRunner Run Time Settings: Run Time Setting Browser/Browser Emulation Simulate browser cache Download non-HTML resources Simulate browser cache General/Log False True False Value

Appeon for PowerBuilder 6.0 Performance Test Report

Appeon Corporation

Page 10

Enable Logging General/Think Time Think Time Options Internal Protocol/Proxy

False

Replay Think Time AS Recorded No Proxy

Load testing scenario: A. Add one virtual users in every 6 seconds until the numbers of the virtual users reach 50. Then keep the online users running for 20 minutes and obtain the test data during this time. And then add more virtual user and obtain data with the same rule. B. Set the virtual user number and function points mapping for every 50 users.
Test Item

Virtual User (Every 50 users) 5 5 5 5 24 1 1 4

Function1 Function2 Function3 Function4 Function5 Function6 Function7 Function8

3.4 Test Result

In our testing, Appeon server is able to handle 440 virtual users without any failure or error. When the virtural users exceed 440, CPU usage will reach 90% sometimes, the response of application server is dramatically slowed down and virtual users operations start to fail. Then the CPU usage will get down a little bit becuase of the increase of virtual userss failure. The following table lists the response time of Appeon server when user number increases from 50 to 450. Average Transaction Response Time(Second)
Function Function Function Function Function Function Function Function Users 1 2 3 4 5 6 7 8

50 100 150 200 250

0.670 0.722 0.872 0.960 1.063

0.338 0.356 0.369 0.451 0.565

0.104 0.121 0.125 0.155 0.217

0.334 0.378 0.386 0.480 0.622

0.018 0.016 0.024 0.027 0.033

1.310 1.329 1.470 1.476 1.513

2.501 2.579 2.786 2.822 2.879

0.110 0.198 0.205 0.220 0.247

Appeon for PowerBuilder 6.0 Performance Test Report

Appeon Corporation

Page 11

300 350 400 440 450

1.529 1.611 1.733 2.200 3.298

0.569 0.729 0.821 1.329 2.734

0.218 0.320 0.479 0.528 2.451

0.765 0.797 0.832 1.336 3.758

0.035 0.043 0.053 0.102 2.088

1.567 1.644 1.655 1.790 3.532

3.093 3.139 3.241 4.473 5.142

0.276 0.302 0.308 0.390 3.332

Ave rage Tra nsaction Res ponse Time(Second) 4.000

Response Tim e(Second)

3.000

2.000

1.000

0.0 00

50
Users

100

150

200

250

300

350

400

440

450

Function 1 Function 4 Function 7

Function 2 Function 5 Function 8

Function 3 Function 6

The following table lists the CPU usage and memory usage of Appeon Server and the database Server when the number of users increases from 50 to 450. Average CPU Usage(%) Users 50 100 150 200 250 300 350 400 Application Server 3.384 7.405 11.922 16.121 20.207 24.154 28.392 32.844 Database Server 1.824 3.480 5.296 7.435 9.272 11.936 14.510 16.651 Average Available Memory(Mbytes) Application Server 1622.645 1618.310 1616.080 1611.170 1606.420 1603.805 1597.725 1590.360 Database Server 1375.765 1327.315 1295.806 1265.060 1226.630 1179.525 1121.425 1053.900

Appeon for PowerBuilder 6.0 Performance Test Report

Appeon Corporation

Page 12

440 450

36.115 30.389

20.036 14.858

1574.790 1570.358

998.740 984.887

40.000 35.000 30.000 CPU Usage(%) 25.000 20.000 15.000 10.000 5.000 0.000 50 100 150

Average CPU Usage(%)

200

250

300

350

400

440

450

Users

Application Server Database Server

Average Available Memory(Mbytes)

1800.000

Available Memory(MB)

1600.000

1400.000

1200.000

1000.000

800.0 00 50 U se rs 100 150 200 250 300 350 400 440 450

Application Server Database Server

Appeon for PowerBuilder 6.0 Performance Test Report

Appeon Corporation

Page 13

4 Stability Test
4.1 Description
During the stability testing, 5 commonly function points in a business application are tested. During the stability test, Appeon set the testing scenario as the following: 1) 100 online users; 2) Think time, 60 seconds; 3) Test for 24 hours.

4.2 Test Result

All five function points listed in the following table passed the stability testing. Each function points were tested for 24 hours and Appeon Server still work correctly and stably. Test NO. Operation Result Step1: Click Scheduling from the menu Module > Scheduling, it 1 Passed will open a window named Select Scheduling Group. Step2: Click Ok Button in the Select Scheduling Group window, then open a window named Scheduling. Step1: Click Scheduling from the menu Module > Scheduling, it 2 Passed will open a window named Select Scheduling Group. Step2: Click Ok Button in the Select Scheduling Group window, then open a window named Scheduling. Step3: Click one row of the Employees list to retrieve. Step4: Modify the schedules and Save. 3 Step1: Click Scheduling from the menu Module > Scheduling, it will open a window named Select Scheduling Group. Step2: Click Ok Button in the Select Scheduling Group window, then open a window named Scheduling. Step3: Click one row of the Employees list to retrieve. Step4: Click Hide/Show Summary Grid from the menu Options > Scheduling Options> Hide/Show Summary Grid. Step1: Click Scheduling from the menu Module > Scheduling, it will open a window named Select Scheduling Group. Step2: Click Ok Button in the Select Scheduling Group window, then open a window named Scheduling. Step3: press ALT+T for combination search. Step1: Click Scheduling from the menu Module > Scheduling, it will open a window named Select Scheduling Group. Step2: Click Ok Button in the Select Scheduling Group window, then open a window named Scheduling. Step3: Click one row of the Employees list to retrieve. Passed

Passed

Passed

Appeon for PowerBuilder 6.0 Performance Test Report

Appeon Corporation

Page 14

Step4: Click Copy Schedule from the menu Options > Scheduling Options> Copy Schedule, then open a window named Create Schedule Step5: Select Copy Schedules by Week and relative week scope and employee in the Create schedule window, then Click Ok.

Appeon for PowerBuilder 6.0 Performance Test Report

Appeon Corporation

Page 15

5 Conclusion
Test results show that the Appeon Web application delivers as fast performance as the PB application does. Some particular functional points even perform a bit faster in APB than PB, and some perform slightly slower in APB than PB. As to why some particular functional points run slightly slower in APB than PB, it has something to do with the different architectures that PB and APB use: part of these functional points' business logic are implemented in JavaScript, as an interpreted language, JavaScript executes slower than PowerScripts which will be complied to the machine code in the realistic environment. As a whole, the Appeon Web application delivers superior performance, though it's a Browser/Server application, its performance is equally good as the client/Server application's. Compared with the Web application on the other architectures, it has obvious advantages. Meanwhile, the stress and the stability tests show that the Appeon Web application can stably work under certain stress, and the Appeon Web application performance fluctuates under stress, but moderately (under extreme stress, the concurrent performance is 2 to 3 times of the single-user performance on the functional point), this indicates the Appeon Web application offers excellent concurrency support and stability. And the test result shows that Appeon Server can support 100-150 online users with high performance in each CPU.

Appeon for PowerBuilder 6.0 Performance Test Report