Web Services Testing Tools Dinakar

Page 1
Symphony Services
Last changed: 12/14/2007
Document Revision Number: Draft Web-Services’ Testing Tools
Web-Services’ Testing Tools
By
Dinakar Adari
Symphony-Lawson GOC Page 1 of 10

Page 2
Symphony Services
Table of Contents
TABLE OF CONTENTS ............................................................................................................................................................ 2
1. ABSTRACT ........................................................................................................................................................................ 3
2. AUDIENCE ......................................................................................................................................................................... 3
3. CURRENT CHALLENGES ............................................................................................................................................... 3
4. PREMISE ............................................................................................................................................................................ 3
5. AVAILABLE TOOLS......................................................................................................................................................... 4
5.1. ALTOVA MISSIONKIT .................................................................................................................. 4
5.2. PARASOFT SOATEST .................................................................................................................. 4
5.3. QUASAR .................................................................................................................................... 5
5.4. XAB TOOLKIT ............................................................................................................................ 5
5.5 ANTEATER ................................................................................................................................. 5
5.6 HTTPCLIENT, JUNIT, XMLUNIT ................................................................................................... 6
5.7 TESTING FROM BROWSER ............................................................................................................ 6
5.7.1 TRADITIONAL APPROACH ............................................................................................................ 6
5.7.2 MODERN APPROACH ................................................................................................................... 7
5.8 SOAPUI ..................................................................................................................................... 7
6. PROPOSED SOLUTION .................................................................................................................................................... 8
7. RESULTS / CONCLUSION ............................................................................................................................................... 9
8. REFERENCES .................................................................................................................................................................. 10

Page 3
Symphony Services
1. Abstract
In this white paper, an attempt is made to discuss various web-services’ testing tools available
and suggest the best possible solution.
2. Audience
Developers/Testers working on web-services’ projects
3. Current Challenges
Testing of web services is time consuming if test cases are executed manually, then comparing
the response with the benchmark responses to ensure the functional correction of the web-
services. The task would worsen exponentially as the number of web services pile up. So, a tool
which would build the service requests, send the requests across to the host server where the
web-service is deployed and give us the response to the request would be helpful.
4. Premise
In the context of SOA(Service Oriented Architecture), following are some of the activities that
happen during lifetime of a service:
a) Service Analysis
b) Service Development
c) Service Testing
d) Service Provisioning
e) Service Operation
f) Service Consumption
g) Service Change Management
h) Service Decommission
After a business process is identified for web service development, development of web service
is done using a web services stack, IDE (Stylus Studio, XMLSpy, Netbeans, etc.) and with a
choice of programming language (Java, .NET, C++, etc.). In the testing phase, unit, smoke,
functional, load testing is done to ensure that all the scenarios envisioned in the analysis phase
are met. Wide variety of software tools is available for testing but few provide with the complete
set for web-service testing.
Testing of web services is time consuming if test cases are executed manually, then comparing
the response with the benchmark responses to ensure the functional correction of the web-
services. The task would further worsen as the number of web services pile up. So, a tool which
would build the service requests, send the requests across to the host server where the web-
service is deployed and give us the response to the request would be helpful.

Page 4
Symphony Services
5. Available tools
Following are some of the testing tools available for web-service testing:
a) Commercial ware:
5.1 Quasar Soap Test Client(Shareware)
5.2 Altova MissionKit
5.3 Parasoft SOAtest
b) Free or Open-Source Software:

5.4 XAB toolkit
5.5 HttpClient, JUnit, XMLUnit
5.6 Anteater
5.7 Testing from browser
5.8 soapUI
With each of the above tools, the underlying communications with the web service hosting server can
be captured from one of the many monitoring tools like TCPmon, ethereal/wireshark, MS Network
Monitor, snoop, Kismet, Netstumbler, etc. Let’s take a deeper look into each of the tools.
5.1. Altova Missionkit

The Altova MissionKit includes complete Web services development capabilities. This is a combination
of two tools – XMLSpy and MapForce. Following are some of its features:
1) Graphical XML Schema editor for embedding schema in WSDL files
2) Visual, drag & drop WSDL design
3) Automatic SOAP message generation from WSDL
4) Universal SOAP client & SOAP debugger
5) Implementing Web services visually
6) Support for XML, database, flat file, EDI, & Web services data sources
7) Mapping of data sources to WSDL operations
8) Consuming existing Web services in data integration mapping projects
5.2. Parasoft SOATest

SOAtest is a comprehensive, collaborative test and analysis tool suite designed specifically for test and
validation of Service Oriented Architectures. It streamlines the process of rapidly constructing robust
regression suites. Some of the features of this tool include:
1) WSDL schema and semantic verification and compliance to WS-I
2) SOAP, PoX (Plain XML) REST, JSON, and BPEL support
3) Asynchronous testing
4) Various WS-* standards support
5) MTOM(XOP)/MIME/DIME Attachment support
6) UDDI support: query verification, validation, and load testing
7) Data-driven testing through data sources
8) Fully reusable functional tests for load testing

Page 5
Symphony Services
9) Apply expected QoS (Quality of Service) metrics to load tests

10) Automate load test execution and track performance metrics throughout the SDLC
11) Security penetration testing
5.3. Quasar
Quasar is a SOA testing tool, allowing for easy monitoring and generation of SOA events. Quasar is a
light weight tool for unit testing a variety of SOA components. The tool comes with an embedded XML
editor. Some of the key features of it include:
1) Make SOAP requests (also available through command line interface)
2) Reply to SOAP requests
3) Unit test your SOAP services
4) Regression test your SOAP services
5) Stress / Performance test your JMS components
*Disadvantage with the commercial ware is

i) Any changes whatever required in future to accommodate new features cannot be done
ii) Bug fixing, maintenance costs extra
iii) Requires client approval before implementing within the project
5.4. XAB toolkit

The XAB program is an application to build 3 tier applications with Mozilla, JavaScript, SOAP, PHP and
MySQL. It can be used to build database applications with a rich user interface around Ajax in Mozilla.
It will automate the building of most or all of an application producing PHP and JavaScript code which
can be installed on a server. It uses Mozilla and DHTML to render its user interface, SOAP as a data
transport method, php for application logic and MySQL as data storage.
It will build a set of PHP classes, JavaScript wrappers, DHTML forms, as well as the SOAP requests.
Disadvantage: The product is not maintained anymore and even with the existing one, in order to use it
you will have to edit the files to fill in default test values and assertions to test. You will also need to
create scripts to call the tests you want to run.
5.5 Anteater
Anteater is a testing framework designed around Ant. It provides an easy way to write tests for
checking the functionality of a Web application or of an XML Web service.
Following is a typical scenario which can be executed using Anteater
• Send a HTTP/HTTPS request to a Web server. When the response comes back, test that it
meets certain criteria. You can check for HTTP headers and response codes, and validate the
response body with regular expression, XPath, Relax NG, or contentEquals tests, plus some
binary formats. New tests can be easily added.
The ability to wait for incoming HTTP messages is something unique to Anteater, which makes it
especially useful when building tests for applications that use high level SOAP-based communication,
like ebXML or BizTalk. Applications written using these protocols usually receive SOAP messages, and

Page 6
Symphony Services
send back a meaningless response. It is only later when they inform the client, using an HTTP request
on the client, about the results of the processing. These are the so-called asynchronous SOAP
messages, and are the heart of many high-level protocols based on SOAP or XML messages.
5.6 HttpClient, JUnit, XMLUnit

This is a combination of tools which needs a bit of homework to be done. The process involves the
following steps -
1) Create a simple Web service
2) Use HttpClient to invoke a Web service
3) Compare the expected response and actual response using XMLUnit
In this process, JUnit can be used to invoke HttpClient and then use XMLUnit to compare the expected
and actual responses.
Disadvantage: Anteater, (HttpClient, JUnit, XMLUnit) combination needs much of the add-on features,
customization needed for a full-fledged web-service testing tool.
5.7 Testing from browser
5.7.1 Traditional approach

With Mozilla 1.0/Netscape 7.0x /Firefox, it is possible for the browser to directly
communicate with web services using its low-level SOAP implementation through
JavaScript.
The underlying Gecko layout engine (second most popular layout engine used by Firefox,
Mozilla, Netscape, Opera, flock, etc) uses JavaScript interface for establishing SOAP calls.
This is a low-level API which requires building the SOAP envelope using several SOAP-
specific JavaScript objects.
However, using JavaScript to talk to web services is subject to the same security policies as
other scripts in terms of cross domain access. Therefore, accessing web services on a
server other than the one where the JavaScript lives will violate the cross-domain policy.
However, this can be circumvented for testing purpose in the following manner:
1) Run the script from your local drive. Save the code locally to your hard drive. The
cross-domain security model does not affect code that is run from the local hard
drive.
2) Enable Cross Domain Access
Cross-domain check can be bypassed by setting a preference in about:config page in
Firefox or in prefs.js file located in the local profile folder in general and in adding a
JavaScript command to request overriding of the cross-domain check(using
UniversalBrowserRead privilege variable).
Testing the web-service from the browser using the low-level API involves following steps:
1) Create a SOAPCall object
2) Encode the function call with list of headers and regular parameters,
3) Invoke the call
a) If the call is successful, the response (SOAPResponse) contains the results
of the service call, output parameters and/or header blocks.

Page 7
Symphony Services
b) If unsuccessful, fault element of the response contains the error.

If the call is invoked asynchronously, then a function is supplied by the caller which receives
the response when it arrives from the remote service.
5.7.2 Modern approach

Using SOAP has been somewhat tedious, requiring manual construction and delivery of the
SOAP envelope the web service expects. SOAP-based response also has need to be
parsed manually for the information required. As of Netscape 7.1/Mozilla 1.4, Gecko
supports WSDL 1.1 proxying. Using the WSDL file, "script" the web services as if it were a
native object, hiding the SOAP and XML aspect. For example, after creating a proxy
instance of a web service using WSDL, call methods on the proxy object like any JavaScript
object
Ex:- proxyWS.getTranslation("en_hi", "Hello World from a Web Service")
The testing process involves the following steps:
a) Create WSDL proxy
b) Use WSDL and fill in the parameters required for the web-service call
c) Make call to the web service and handle the response
Disadvantage: ‘Testing from browser’ can be used only when testing is required on a
small scale and scalability is another big problem with this approach.
5.8 soapUI
soapUI is a free and open source desktop application for inspecting, invoking, developing,
simulating/mocking and functional/load/compliance testing of web services over HTTP. Functional and
Load-Testing can be done interactively or within an automated build/integration process using the
soapUI command-line tools.
Key features:
1) Besides just sending SOAP requests, groovy scripts can also be executed with which state of
the database can be prepared, so that a full and clean test cycle is created.
2) soapUI supports functional testing of Web Services by providing a TestCase metaphor where a
number of TestSteps can be executed in sequence. There are currently six types of TestSteps
available. TestCases are further organized into TestSuites of which an arbitrary number can be
created within each project.
3) Opt for default parameters which are set as parameters for the service. During importing the
WSDL, tick the checkbox asking to create a SOAP request for each operation, if you want one.
If you double click the request you can see the content of the SOAP request in the main window
Results of the call can be seen in the adjoining window. To make a good test tool we need to create a
new test suite in our project and add test cases to it. With these test cases you can send your custom
SOAP requests and check the response with assertions. This can be done by right-clicking the web-
service created and choosing 'Generate Test Suite' option. This generates a test suite with one test
case for every operation. Expand the Test Steps node and double click at the operation to see the
simple SOAP request in the main window. Assertions can be added at web-service or operation level.

Page 8
Symphony Services
This helps in not only checking if the service is reached but also checks if the content of the response is
what you would expect to be returned by the service.
Disadvantage: This product suits most of the needs but a bit of customization in terms of giving inputs
to the tool would make this a perfect one.
6. Proposed Solution
soapUI++
Considering the approaches discussed above, soapUI++ fits the bill perfectly.
soapUI++ is an extension to soapUI making it specific for Lawson CAPI project. As soapUI itself is
developed in java and it’s API publicly available, extra features can be added to the existing tool (which
in itself facilitates for manual testing of web-services).
One of the best things about soapUI is that it provides tools for running test cases. These tools can be
used in association with the APIs provided with the application to create application specific tools.
‘SoapUITestCaseRunner’ is one such tool which runs specified TestCases and reports/exports results
as configured.
soapUI++ runs as a batch of operations. It requires 2 kinds of inputs -
1) A batch file, containing the necessary web-services/individual operation of web-service. The
operations specified in this file are executed in sequence.
2) Input values for each operation is specified in another file which is used in populating the SOAP
request(used in step 2 specified below)
Following steps are automated using soapUI++:
1) Creation of SOAP request
2) Populating the SOAP request
3) Sending the request to the server on which web-services are deployed
4) Receiving the response
5) Comparing the response with that of the benchmarked response
6) Logging the result of comparison, other important events during this process for analysis purpose.
soapUI++ gives the flexibility to run requests at operation/web-service level and in test/benchmark
mode. In benchmark mode, the output is stored, against which the test cases are compared.

Page 9
Symphony Services
7. Results / Conclusion
Benefits:
1) Minimal manual intervention
2) Speedy execution of test cases
3) Saved time
4) Scalable for any number of web-service/operations
5) Reduced effort for future execution
Technical Advantage: This framework is currently used for ‘Functional Testing’ of web services over
HTTP. It’s best used during configuration/environment changes of the server and that when the same
set of web-services are to be checked for new configurations/environments. Ex: Deploying of web-
services developed in older versions in new versions requires the testing of all developed web-services
against the new environment.
Furthermore, the framework’s functionality can be extended for
1) Load, Compliance testing
2) Inspecting, invoking, simulating/mock testing of web services.
Cost Savings:
1) Man days: Though, the exact savings in man days is not captured, ratio of automation to
manual testing is 30:1 per web-service operation.
2) Cost of software: Using free and open source software reduces
a. Cost of procurement of the product
b. As the source code of the final product is available, maintenance costs if any can be
internally borne.

Page 10
Symphony Services
8. References
1) SOA Testing:
https://www6.software.ibm.com/developerworks/education/ws-soa-autotest2/section2.html
2) Web Services in Mozilla:
http://developer.mozilla.org/en/docs/Accessing_Web_Services_in_Mozilla_Using_WSDL_Proxying
3) SOAP scripts in Mozilla:
http://lxr.mozilla.org/mozilla/source/extensions/webservices/docs/Soap_Scripts_in_Mozilla.html
4) soapUI vs. JUnit: http://www.myarch.com/soapui-vs-junit
5) soapUI: http://www.soapui.org/
6) Wireshark: http://wiki.wireshark.org/
7) Gecko engine: http://en.wikipedia.org/wiki/Gecko_%28layout_engine%29
8) Security Restrictions on cross-domain access:
http://developer.mozilla.org/en/docs/Bypassing_Security_Restrictions_and_Signing_Code
9) Calling SOAP Servers from JS in Mozilla: http://www.onlamp.com/lpt/a/5981
10) Signed scripts in Mozilla: http://www.mozilla.org/projects/security/components/signed-scripts.html
11) SOAP in Gecko browsers: http://developer.mozilla.org/en/docs/SOAP_in_Gecko-based_Browsers
12) Using Mozilla SOAP API: http://www.oreillynet.com/lpt/a/2677
13) Quasar:
http://www.integrationcentral.com/product.html?gclid=CLWXmZjX_Y8CFUaPOAodBw0_MA

Web Services Testing Tools Dinakar

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Web Services Testing Tools Dinakar

Uploaded by

Copyright:

Available Formats

Page 1

Web-Services’ Testing Tools

Symphony-Lawson GOC Page 1 of 10

Symphony-Lawson GOC Page 2 of 10

Symphony-Lawson GOC Page 3 of 10

b) Free or Open-Source Software:

5.1. Altova Missionkit

5.2. Parasoft SOATest

Symphony-Lawson GOC Page 4 of 10

9) Apply expected QoS (Quality of Service) metrics to load tests

*Disadvantage with the commercial ware is

5.4. XAB toolkit

Symphony-Lawson GOC Page 5 of 10

5.6 HttpClient, JUnit, XMLUnit

5.7 Testing from browser

5.7.1 Traditional approach

Symphony-Lawson GOC Page 6 of 10

b) If unsuccessful, fault element of the response contains the error.

5.7.2 Modern approach

Symphony-Lawson GOC Page 7 of 10

Symphony-Lawson GOC Page 8 of 10

Symphony-Lawson GOC Page 9 of 10

Symphony-Lawson GOC Page 10 of 10

You might also like