Professional Documents
Culture Documents
Toyotaro Suzumura, Scott Trent, Michiaki Tatsubori, Akihiko Tozawa and Tamiya Onodera
Tokyo Research Laboratory, IBM Research,
1623-14 Shimotsurusma, Yamato-shi, Kanagawa-ken 242-8502, Japan
{toyo, trent, mich, atozawa,tonodera}@jp.ibm.com
386
that LIBXML2 is much faster than all other for PHP [16] by WSO2 ™ has started to support full set
implementations. of WS-*.
387
server where Axis2 C engine runs as mod_axis2 inside the container, such as JSP script for Tomcat and PHP script
same process as the Apache http server. for Lighttpd and Apache.
Server HTTP Connector SOAP Web 0 500 1000 1500 2000 2500 3000 3500 4000 4500
Configuration Server Engine Services Throughput (# of requests per second)
Label
Axis2 Java Tomcat None Axis2 Java Figure 2 Comparison of dynamic web server without SOAP engine
5.5.26 (thread) Java 1.3
(Java) (Java)
Axis2 C Apache mod_axis2 Axis2 C C 4.4. Comparison with Stock Quote Example
2.2.8 (C) 1.3.0 (C) We conduct a performance comparison between the
PHP+Lighttpd Lighttpd FastCGI PHP5 PHP SOAP engines shown in Table 1, namly Axis2 Java,
1.4.19 (C) 5.2.5 (C)
Axis2 C, PHP+Lighttpd, and PHP+Apache, with a
PHP+Apache Apache mod_php PHP5 PHP
2.0.63 5.2.5 (C) relatively simple web service. The web service used for
our experiment is a stock quote service that returns the
Table 1 Server configuration for target SOAP engines
latest quote of a stock specified in the request SOAP
message. The request message is 277 bytes long and
4.3. Comparing Web Servers without SOAP contains a single element that represents the queried
Engines symbol name. The response is 649 bytes long and contains
In this experiment, to consider the performance not only the latest stock price but also other attributes
characteristics of each web server without SOAP engine, such as last trade information.
we performed two experiments. We compared the web Given the WSDL file for a stock quote service, a service
server performance when serving a static HTML file that skeleton for each SOAP engine is generated for Axis2.
includes the same SOAP message used by stock quote Since PHP currently does not include a WSDL generation
web services as described in the next section. The result is mechanism, we implemented it from scratch. Table 2
shown in Figure 1. Lighttpd performs the best since shows all the code for PHP-based web service, and Table
Lighttpd is designed to provide optimal performance 3 shows the code snippet for Java-based web service and
when serving static files by, for example, using sendfile many other codes generated from WSDL exist.
system call to avoid redundant copying between user and The results are illustrated in Figure 3. The left y-axis
kernel space. Apache 2.2 performs about 7% better than indicates a throughput which corresponds to the number
Apache 2.0 and about 70% better than Tomcat 5.5 . of requests processed in one second calculated by Apache
Figure 2 demonstrates the performance comparison of Bench. The right y-axis shows the performance
running a dynamic web application in each configuration improvement ratio for each SOAP engine against Axis2
without a SOAP engine. This experiment is intended to Java. As shown in the graph, PHP+Lighttpd performance
show the initial overhead for each configuration without a is 86% of Axis2 Java, PHP+Apache performance is 97%
SOAP engine. The web application used in this of Axis2 Java, and Axis2 C outperforms Axis2 Java by
experiment consumes the same request message as the 25%.
Stock Quote web service used in the next section from the Since the performance of a SOAP engine is mostly
Apache Bench client via an HTTP POST method, and determined by the cost of serialization and de-serialization,
then returns the same response message as the web service. and also since this type of web service does not spend that
The dynamic web application for each runtime much time on these operations, this result clearly shows
configuration is written so it can be served in a web that the performance difference comes from other
388
processing. The key difference between the first and the To better understand the performance characteristics, we
second PHP experiments is whether or not the PHP conducted profiling for each SOAP engine in the 10
runtime invokes the SOAP engine extension written in C. structure message echoStruct scenario. The CPU
This result demonstrates the mix of Apache 2.0 and PHP5 breakdown in Figure 9 shows that with PHP+Lighttpd
is less efficient than the mix of Lighttpd and PHP5. In the 84% of CPU is spent in the PHP engine and only 13 % is
Lighttpd configuration, 16 PHP processes run as separate consumed by the HTTP server. To further investigate the
process, whereas in the Apache configuration, the PHP CPU usage within the PHP process, Figure 10 shows that
engine is embedded in hundreds of HTTP processes. XML processing consumed 12% of the CPU., and the C
library consumed 21% of the CPU. A further breakdown
StockQuote Web Service: of functions within the c library shows that many of the
Performance Comparison among SOAP engines functions are related to memory management. Figure
2500 1.4 11shows the CPU breakdown of PHP+Apache. Since
2196 av mod_php PHP engine runs inside Apache HTTP server
re
p 1.2 Ja
2000 1877 process, this graph includes both components. Figure 12
tss 1831 2s
ix
eu 1613 1 A shows the CPU breakdown of Axis2C, and Figure 13 that
qe
r )d 1500 st of Axis2 Java. As shown in Figure 13, garbage collection
fo no 0.8 nia
ce ga
#( accounts for 22% of CPU usage. Since PHP memory
s 1000 0.6 iot
utp aR management uses reference counting, this memory
hg 0.4 pu
uo de management cost is included with the runtime engine CPU
rh 500
T 0.2 ep usage.
S
0 0
Axis2 Java PHP+Lighttpd PHP+Apache Axis2 C
4.5.3. Memory Footprint
In addition to the total throughput, we also need to
Figure 3 Performance Comparison with StockQuote web service
consider memory usage. The memory footprint of PHP in
the PHP+Lighttpd case is proportional to the number of
4.5. Comparison with WSTest PHP processes which are spawned. One PHP process
consumes approximately 5.5 MB, and 16 processes
In the next experiment, we use WSTest [13], a consume around 90.4 MB. On the other hand, in the Axis2
benchmark suite for SOAP processing. We use the Java case, all components including the web server, SOAP
echoStruct scenario in WSTest to echo SOAP messages engine, and web service implementation, run in a single
containing an array of structured elements. process with multiple threads, which consumes 62.1 MB.
4.5.1 Overall Results The virtue of using FastCGI allows PHP processes to be
Figure 4 shows the performance result of an experiment spawned on different nodes by communicating with the
where the number of structures varies from 1 to 10. The http server via FastCGI protocol, but if we use only a
left-hand y-axis indicates the throughput calculated by the single node, Axis2 Java can be more memory-efficient.
same logic as the stock quote example, and the right-hand
y-axis indicates the performance speedup ratio compared
2500 8
with Axis2 Java. As shown in the graph, Apache+PHP
re 7
outperforms Axis2 Java between 1.87 and 5.97 times, and av
p 2000
Ja
st 6
PHP+Lighttpd outperforms it by between 1.65 and 5.23 esu
2s
ix
times. Axis2 C also shows a significant performance qe 5 A
r
fo d)n 1500 re
vo
improvement of 2.29 to 6.99 times. This demonstrates that (# coe 4
tu s iot
1000 ar
the PHP SOAP engine is competitive with Axis2 C as well. ph 3
pu
gu de
Figure 5 shows the result of longer echoStruct messages roh 2 ep
T 500 S
with Axis2 Java and PHP and two different HTTP servers: 1
Lighttpd and Apache. As seen in the shorter message 0 0
example, the PHP+Apache case is slightly better than 1 2 3 4 5 6 7 8 9 10
# of structs
PHP+lighttpd. However, both cases outperform Axis2
Axis2 Java PHP+Lighttpd PHP+Apache
Java by 5.8 to 17 times. Axis2 C PHP+Lighttpd/Java PHP+Apache/Java
Axis2C/Java
Figure 6 shows the same graph as Figure 5 along with
Axis2 C. Axis2 C obtained the expected high score, but Figure 4 Comparison with short messages in echoStruct
with large messages of from 80 to 100 structures, there
are cases where PHP+Apache outperforms Axis2 C.
4.5.2. Profiling
389
1400 18 WSTest echoList: Axis2 vs PHP SOAP
16 1400 8
1200 av
re
p 14 Ja
st 1000 2s )d 1200 7
se 12 x
i no /
uq A ce
er )d 800 st s 1000 6 APO
fo no 10 n ia erp S
#( ce ga tss 5 PH
tu s 600 8 iot eu 800 (P ) Axis2
ar o 2
ph qe 4 ita xisA PHP SOAP
gu 6 pu fro R PHP/Axis2
or 400 de 600 e
hT 4 ep (# 3 can
tu
200
S ph 400 r m
2 gu
or
2 orfe
hT P
0 0 200 1
10 20 30 40 50 60 70 80 90 100
# of structs contained in SOAP message 0 0
Axis2 Java PHP+Lighttpd PHP+Apache
1 2 3 4 5 6 7 8 9 10
PHP+Lighttpd/Axis2 Java PHP+Apache/Axis2 Java
# of list nodes
Figure 5 Comparison with long messages in echoStruct Figure 8 Performance Comparison by echoList
1600 18
16
1 84.5824 13.1556 2.262
re 1400 av
p
st 1200
14 Ja
2s
esu
0% 1 0% 20 % 30% 40% 5 0% 60 % 70% 80% 9 0% 10 0 %
12 ix
qe )d 1000 A php lighttpd others
r r
fo 10 e
onc 800 vo
(# se 8 iot Figure 9 CPU breakdown by each component in PHP+Lighttpd
tu aR
ph 600
6 pu
gu de
or 400 ep
hT 4
S 0%
200 2
1%
00%
0%
% 0%
0 0
10 20 30 40 50 60 70 80 90 100 21%
# of structs contained in SOAP message
php-cgi
Axis2 Java PHP+Lighttpd PHP+Apache libc-2.6.so
Axis2 C PHP+Lighttpd/Java PHP+Apache/Java libxml2.so.2.6.29
C/Java e1000
50% libpthread-2.6.so
aic79xx
Figure 6 Comparison including Axis2C
uhci_hcd
ld-2.6.so
ext3
Throughput over variable number of threads
28%
700
d)n 600
coe
s 500
re
p
st Figure 10 CPU breakdown of echoStruct (10) for PHP+Lighttpd
se 400 PHP SOAP (# of structs = 10)
uq PHP SOAP (# of structs = 50)
ref Axis2 (# of structs=10)
o 300
(# Axis2(# of structs=50)
tu
ph 200 2% 2%1%
gu 6%
or
h
T 100
10%
0 libphp5.so
1 2 3 4 5 6 7 8 9 10 libc-2.6.so
# of threads (pow(2,x) libxml2.so.2.6.22
46%
httpd
libapr-0.so.0.9.5
Figure 7 Throughput over variable number of threads 12%
libaprutil-0.so.0.9.5
e1000
ext3
21%
390
for more system stability and productivity for some expert
1%
1% 0%
0%
0% 00%2%
0% %
libc-2.6.so
libxml2 .so.2.6.29
programmer, Java is surely in a good position.
2%1% libaxis2_axiom.so.0
2% 21%
3% libaxu til.so.0
3% h ttpd
PHP vs. C
libaxis2_e ngin e.so .0
mo d_axis2.so
experiment demonstrates that Axis2 C performs well in
6% e 1000
comparison to other SOAP engines. Axis2 C is
libaprutil-1 .so.0.2.8
libTestServic e.so
libpcre .so.0.0.1
implemented in C, but this performance score might be
7% 19% libpthread-2.6.so expected but since Axis2 is designed to be highly modular.
jbd
e xt3
It is not well known whether Axis2 C still outperforms the
11%
mo d_lo g_config.so
aic79 xx
PHP SOAP engine, which has been designed around with
14%
mo d_lo gio.so
391
7. Conclusions and Future Work [8] Matthias Farwick, Michael Hafner, “XML Parser
Benchmark Part1 and Part2”, http://www.xml.com/
This paper describes the comparison of web services [9] Lighttpd, http://www.lighttpd.net/
engines implemented in PHP with those implemented in [10]Woodstox: StAX (JSR-173) Compliant Parser written
Java and C. Given different architecture and different in Java, http://woodstox.codehaus.org/
programming language, the experimental results show that [11] PHP.NET http://www.php.net/
PHP is reasonably high performing compared to Java and [12] Gnome XML Library, http://www.xmlsoft.org/
C-based implementations while providing users with high [13] Sun Microsystems™, Web Services Performance
software productivity. Our contribution will provide Comparing Java™ 2 Enterprise Edition and .NET
average programmers with more awareness that PHP is a Framework, June 2004
viable option for publishing SOAP/WS-* based web [14] Kenneth Chiu, Web Services Performance: A Survey
services in addition to the currently popular REST-style of Issues and Solutions, In Proceedings of the 7th World
web services. Conference Systems, Cybernetics and Informatics, July
Several topics can be further explored. First we can 2003.
extend a variety of web services such as other scenarios in [15] Christiana Amza, Emmanuel Cecchet, et.al,
WSTest or more real-world scenarios that also involve “Specification and Implementation of Dynamic Web Site
business logic and database access. Moreover, we can Benchmarks”, WWC: IEEE 5th Annual Workshop on
explore a wider variety of server configurations. In this Workload Characterization, 2002
paper we only use Axis2 Java and its default data binding, [16] WSO2 Web Services Framework for PHP,
ADB, but other web service implementations in Java http://wso2.org/projects/wsf/php
could differ from the reported result in this paper. We can [17] Kenneth Chiu, Madhusudhan Govindaraju, and
also compare different dynamic scripting languages such Randall Bramley , “Investigating the Limits of SOAP
as Ruby and Python. Performance for Scientific Computing”, In Proceedings of
Additionally, rather than focusing on SOAP-based the Eleventh IEEE International Symposium on High
services for web services, in future work, we could Performance Distributed Computing (HPDC'02), July
explore performance in terms of RESTful services. XML 2002
<?php
or JSON is used as a data format after accessing web function getStock($symbol) {
services via the HTTP Get operation, but we can compare $stock = new Stock();
$lastTrade = new LastTrade();
PHP and Axis2 since PHP is popular for RESTful web $lastTrade->price = 100.0;
services today, whereas Axis2 is not really used that way $lastTrade->date = 1165997291;
$change = new Change();
even though Axis2 provides both RESTful and SOAP $change->dollar = 98.00000;
functionality for same web service implementation $change->percent = 10.0000;
$stock->lastTrade = $lastTrade;
without extra programming effort. $stock->change = $change;
$stock->symbol = "IBM";
$stock->volume = 1000;
References return $stock;
[1] Alex Ng, Shiping Chen, and Paul Greenfield, “An }
$server = new SoapServer("StockQuoteService.wsdl");
Evaluation of Contemporary Commercial SOAP $server->addFunction("getStockQuote");
Implementations”, AWSA (Australasian Workshop on $server->handle();
?>
Software and System Architecture) 2004
[2] Emmanuel Cecchet, Anupam Chandra, et.al, Table 2 PHP Script for StockQuote Web Service
“Performance Comparison of Middleware Architectures
public GetStockQuoteResponse getStockQuote(GetStockQuote getStockQuote0) {
for Generating Dynamic Web Content”, Middleware 2003 String symbol = getStockQuote0.getSymbol();
[3] Subramanyam S., et al., “Designing a web services GetStockQuoteResponse response = new GetStockQuoteResponse();
’
benchmark”, In ICWS 03. International Conference on
Quote quote = new Quote();
quote.setSymbol("IBM");
Web Services, Las Vegas, USA. quote.setVolume(new BigInteger("1000"));
LastTradeType lastTrade = new LastTradeType();
[4] Toyotaro Suzumura, Satoshi Makino and Naohiko lastTrade.setPrice((float)100.0);
Uramoto, “Optimizing Differential XML Processing by lastTrade.setDate(1165997291);
quote.setLastTrade(lastTrade);
Leveraging Schema and Statistics”, ICSOC 2006, ChangeType changeType = new ChangeType();
Springer, Lecture Notes in Computer Science, pp. 264- changeType.setDollar((float)98.00000);
changeType.setPercent((float)10.0000);
p276 quote.setChange(changeType);
[5] Srinath Perasa, Chathura Herath, Jaliya Ekanayake, et. response.setReturnQuote(quote);
return response;
Al. “Axis2, Middleware for Next Generation Web
}
Services”, ICWS 2006, URL: http://apache.axis.org/axis2/
[7] PHP Usage: http://www.php.net/usage.php Table 3 Java code for Stock Quote web service
392