Professional Documents
Culture Documents
AbstractWeb Services are a technology designed to support the invocation of remote elements by
client applications, with the goal of providing interoperable application-to-application interaction while
assuring vendor and platform independence. In business-critical environments, the occurrence of interoperability issues can have disastrous consequences, including direct financial costs, reputation, and
client fidelity losses. Despite this, experience shows that services interoperability is still quite difficult
to achieve. The goal of this paper is to propose a practical testing process to understand the real level of
interoperability provided by web services platforms. An extensible tool, that implements the proposed
approach, has been used to run a large campaign during which we have tested the interoperability of a
large number of web services, comprising both home-implemented and publicly available services, deployed on top of several web service platforms and against 11 client-side web service platforms. Numerous issues have been disclosed, showing the effectiveness of the proposed approach and the importance of having an automatic tool for testing web services for interoperability.
Index Terms Interoperability, Web service, Testing, Testing Tools
1 INTRODUCTION
Web services are being increasingly used to integrate enterprise systems, linking complex applications
in large-scale organizations and supporting critical business-to-business interactions, where the typically heterogeneous environments frequently demand a strict definition of service interfaces and interaction patterns. In such heterogeneous scenarios, systems designers frequently opt for using web services
technology, as it was created with the key goal of supporting the inter-operation between different systems, including systems built using different programming languages (Erl, 2005).
In a typical services environment a service provider announces its interface using a WSDL document
that describes, in a programming language agnostic way, the operations available to the clients. In turn,
clients make use of the interface description to understand how the service operations can be invoked
(Curbera et al., 2002). Nowadays, developers do not need to implement all supporting code for generating services or invoking operations, as most web service frameworks (platforms for creating, deploying,
and invoking web services) provide automatic code generation facilities and runtime support for invoking service operations (Hewitt, 2009). The problem is that, although web services are supported by machine independent protocols and have been designed with interoperability in mind, practice shows that
it still is quite difficult to link distinct applications. This is especially true in complex and more heterogeneous scenarios, such as the ones frequently found in enterprise environments.
The interoperability of web services is an issue in which the Web Services Interoperability Organization (WS-I) (Web Services Interoperabily Organization (WS-I), 2002) has been working for several
years. However, practice shows that developers still struggle to deploy services that can fully interoperate with client applications. In fact, developers many times create and deploy their web services
expecting the underlying framework to provide full interoperability, which is not always the case. Our
tests, presented later in the paper, confirm this scenario, showing that the inter-operation between different frameworks is not yet fully achieved and highlighting many cases where inter-operation is not
possible.
A key problem is that, although efforts have been undertaken towards creating tools to test services
(Eviware, 2011, WS-I, CrossCheck Networks), they appear to be quite limited, when it comes to interoperability testing. Among all tools, those provided by the WS-I Testing Tools Working Group (WSI,) are definitely a step forward, but experience suggests that even WS-I compliant services may show
interoperability issues. Thus, there is a clear need for a specialized approach for interoperability testing
that helps developers conducting broader interoperability assessments, from a practical perspective.
I. A. Elia, N. Laranjeiro, and M. Vieira are with the CISUC, Department of Informatics Engineering University of Coimbra, 3030-290, Portugal.
E-mail: ivanoe@dei.uc.pt, cnl@dei.uc.pt, mvieira@dei.uc.pt
We presented a tool for interoperability testing of web services in (Elia, Laranjeiro, & Vieira, 2014b).
In (Elia, Laranjeiro, & Vieira, 2014c) and (Elia, Laranjeiro, & Vieira, 2014a) we have conducted experiments regarding part of the full inter-operation process (which in the end involves communication). In
this paper we greatly extend the previous work, by having refined the approach and formalized key aspects, extended the approach to now include the exchange of messages between client and server, and
by augmenting the tests and tool implementation to cover the approach extension and thus covering the
whole inter-operation process.
We propose, in this paper, a practical approach to support the evaluation of all the steps of the interoperation process in web services environments. We also present ITWS (Interoperability Testing for
Web Services), an open-source web services testing tool available at (Elia, Laranjeiro, & Vieira) that
supports the execution of the proposed approach. Developers can use ITWS to evaluate the level of interoperability attained by their services (running in the context of a given framework, i.e., a platform
that provides the deployment and runtime support for web services) with regard to one or more clientside frameworks (that offer client-side support for invoking the web services operations). ITWS currently supports testing a set of well-known web service frameworks, but the tool was designed to be extensible and adding more frameworks requires only a simple configuration effort.
The ITWS tool was used to run a massive test campaign involving about 8 thousand web services,
comprising over seven thousand home developed services deployed on top of three major WS frameworks and over one thousand web services publicly available on the Internet and also deployed on top
of multiple frameworks leading to a total of 174547 interoperability tests.
The outline of this paper is as follows. The next section presents the web services inter-operation
problem and discusses existing techniques and tools for interoperability testing. Section 3 describes our
approach for testing interoperability of web services and Section 4 provides an overview of the ITWS
tool. Section 5 presents the experimental evaluation carried out using the tool and Section 6 summarizes
the key findings. Finally, Section 7 concludes this paper.
Fig. 1 represents a typical web services inter-operation scenario using two different frameworks at the
client (on the left-hand side of Fig. 1) and server sides (in Fig. 1, at the right-hand side), which are
placed at each endpoint of the interaction. In the scenario represented, the client uses the client-side
subsystem of framework A, while the server uses the server-side subsystem of framework B (however
these could also be two subsystems of the
same framework). As shown, a set of 5 key
(1)'Service'
steps needs to be performed for a client to inDescrip*on''
Framework'A'
Framework''B'
voke a remote web service operation. These
Genera*on'
steps represent the critical points where platAr*fact'Generator'
form-level interoperability issues can arise.
(2)'Client''
The Service Description Generation step
Ar*fact''
WSDL'
Genera*on'
(1) typically occurs automatically when the
Service'
service is deployed at the server, although it
Client'Ar*facts'
can also be executed manually, for instance by
(4)'Client''
Compiler'
using directly the WS framework tools and the
(5)'Service'
Communica*on'
Processing'
service code as input. The result of this step is
(3)'Client''
a WSDL document that provides the inforSOAP'
Ar*fact''
Client'
mation needed by a client that uses framework
Compila*on'
A to invoke the service provided by framework
B.
Client'side'
Server'side'
During the Client Artifact Generation step
Fig. 1. A typical web services environment.
(2) the clients artifacts are produced using the
artifact generation tool provided by framework A. These artifacts are pieces of code that, at runtime,
will translate application-level calls at the client to SOAP messages that will be delivered to the server.
Some frameworks do not produce source files; instead they generate these artifacts dynamically, at
runtime.
The Client Artifact Compilation step (3) is necessary only for platforms that are implemented in
programming languages that require compilation of the code before execution (e.g., Java, C#), and in
some languages/frameworks (e.g., Python) it may not be required (the client-side artifacts are dynamically generated at runtime). Obviously, the artifacts (that after this step may be in a compiled form) only
provide the methods for a client application to invoke the remote service, but do not actually invoke it.
It is up to the developer to create the client code to invoke the artifact methods that allow communicating with the server.
Communication is represented by the Client Communication step (4) in Fig. 1, which in practice results in sending a SOAP message to the server and ends with the delivery of that message to the service
application. This message is processed at the server during the Service Processing step (5) and a response SOAP message is produced and sent back to the client, carrying the result of the operation. The
entire process ends when this message arrives at the client application.
A set of interoperability issues in the web services technology is analyzed in (Senthil Kumar, Das, &
Padmanabhuni, 2004). These include aspects related with the representation of sequences of elements
and namespaces in WSDL files. The authors also analyze how the WS-I Basic Profile 1.0 (Keith Ballinger, Ehnebuske, Gudgin, Nottingham, & Yendluri, 2004; Web Services Interoperabily Organization
(WS-I), 2002) tries to address the raised issues and highlights the limitations of the WS-I set of interoperability recommendations. Conclusions include the fact that adhering to the WS-I recommendations
helps reducing the web services interoperability issues. However, a more extensive and pragmatic view
is still needed, as the WS-I Basic Profile 1.0 does not cover the practical support for interoperability
provided by current web service frameworks.
The authors of (Ramsokul & Sowmya, 2006) trace back the interoperability issues to the use of a
non-formal language in the specifications, but also recognize that problems can occur much later with
improper implementations of the protocols. A key aspect is that no practical insight is provided on the
interoperability of different frameworks currently being used in the field. Obviously, when deploying an
interoperability-critical system, a practical assessment is of utmost importance.
Interoperability problems can occur when, for instance, native data types or specific language constructs of the service implementation are present at the interface. The problem is that, to the best of our
knowledge, there is no standard indicating which types are adequate to be used in a service interface. In
(Pautasso, Zimmermann, & Leymann, 2008) authors confirm precisely that it is difficult to identify the
right constructs to express the data types that are fully supported by all service frameworks.
The WS-I Testing Tools Working Group has developed tools that are able to check the compliance of
a given web service with various documents, named Profiles (K. Ballinger et al., 2006; Web Services
Interoperabily Organization (WS-I), 2002, 2014). These Profiles are a set of non-proprietary web services specifications, including clarifications, refinements, interpretations and their amplifications,
which promote interoperability. Since our focus is inter-operation, the WS-I Basic Profile and respective tools are closely related to our work (K. Ballinger et al., 2006; Web Services Interoperabily Organization (WS-I), 2014). As previously referred, although the WS-I effort represents a first step, experience
suggests that even WS-I compliant services may present interoperability issues, and obviously a more
practical and realistic insight of a given platforms interoperability is many times preferred to conceptual results. This is especially true in systems where interoperability is critical (e.g., systems supporting
business-to-business interactions).
SOAPSonar (CrossCheck Networks, ) is a web services testing tool, described by its development
company as the global leader in service and API testing. In what concerns interoperability, the tool is
able to check for WS-I Basic Profile 1.1 compliance. SOAPSonars website claims: Adhering to WS-I
Basic Profile 1.1 ensures that your components are interoperable and that your WSDL can be consumed
by any .NET or Java client. In this paper, we show that this is not the case, as our tool outperforms
SOAPSonar in WSDL checking for interoperability, by being able to disclose issues that the referred
WS-I Basic Profile does not cover.
SoapUI (Eviware, 2011) is a well-known open-source tool for testing web services that is able to create various types of tests, including functional, load, and security tests. SoapUI supports compliance
testing by including automated execution of the WS-I Basic Profile 1.1 test suites, for which we already
highlighted the limitations. SoapUI also supports the generation of client-side artifacts with a limited set
of frameworks. However, developers do not have a way of adding a new framework (or in some cases,
not even a new version of an existing one) without changing SoapUIs code, which obviously is not
practical. In ITWS, plugging a new framework has been simplified to the execution of a uniform configuration effort. Also, the analysis of the results can be conducted in a uniform way for all the different
frameworks being used.
As mentioned before, the inter-operation process begins with the Service Description Generation step,
which includes the deployment of a web service and the generation of an interface description (i.e., a
WSDL file) by the server-side framework. After the step concludes, we carry out a classification procedure, which consists of checking the service interface for WS-I Basic Profile compliance (Web Services
Interoperabily Organization (WS-I), 2014). In practice, the classification procedure can result in one of
the following cases:
Error: When the WSDL is absent, i.e., when the framework has failed to produce the WSDL we
classify the event to be an Error as it is not possible to continue to the next inter-operation steps.
Warning: When the WS-I toolset raises at least one interoperability issue we mark the outcome
of the step as a Warning.
Correct: All other cases where no problematic events (fatal or non-fatal) are observed.
Classica&on(
(5)(Service(
Processing(
Execu&on(Phase(Evalua&on(
Fig. 2. Interoperability testing phases.
Classica&on(
(4)(Client(
Communica&on(
Classica&on(
Classica&on(
Classica&on(
The next step is the Client Artifact Generation during which we test the capability of the client-side
framework to produce client-side artifacts. In practice, the step consists of using one or more client-side
artifact generation tools (e.g., Metros wsimport (wsimport), JBossWS wsconsume (jBoss: wsconsume))
to try to generate client-side code from the WSDL produced in the previous step. Again, the step is followed by a classification procedure in which we verify the occurrence of one of the following cases:
Error: The client artifact generation tool failed to produce any output, which might be signaled
by the tool being used, or not.
Warning: The artifact generation tool
produces output but also points out
Genera&on(Phase(Evalua&on(
some issue during its execution. For
(2)(Client(
(3)(Client(
(1)(Service(
instance, a tool can inform that altDescrip&on(
Ar&fact(
Ar&fact(
hough it produced some client-side
Genera&on(
Compila&on(
Genera&on(
code, it was also unable to generate
Client((
code for some operations.
Implementa&on((
Correct: No observation of an Error
or Warning.
the client artifact tools should generate artifacts without any issue. On the other hand, if the WSDL
document is problematic (e.g., which is obvious when the previous step results in a Warning), the artifact generation tool should detect the presence of a problem and provide useful information to the service developer. Erratic artifact generation tools might silently produce the artifacts without signaling a
problem (and this problem may only prevent inter-operation in a later step).
A relevant aspect in this step is that some frameworks do not support generation of the usual static artifacts (which exist in a physical file); instead, they provide support for in-memory artifacts, which are
generated at runtime. In these cases, the classification procedure should verify if the in-memory artifacts, or any other support structure, can be instantiated. It should be possible to create them without
triggering any fatal event (i.e., an Error that prevents further execution of the client application and,
thus prevents further inter-operation, such as an Exception being raised at the client) or non-fatal (i.e., a
Warning, such as an informative warning produced at the client-side). The goal is to verify, only to the
possible extent in this step (i.e., without proceeding to the next steps), if it is possible to generate artifacts without visible issues being observed.
The third step is the Client Artifact Compilation, in which we try to compile the artifacts generated
in the previous step (e.g., using Javas compiler). If the artifacts cannot be compiled, it is a clear sign
that something is not correct and anyway no further inter-operation steps can be taken. The classification procedure carried out after this step consists precisely of evaluating the output of the compilation
tools looking for one of the following cases:
Error: No compiled form of the code is produced.
Warning: The compilation tool is able to compile the generated code, but displays some warning, suggesting some problem (e.g., unsafe operations being used).
Correct: No issue is observed.
This step concludes the Generation Phase. The next phase involves the exchange of SOAP messages
and is described in the next Section.
The client artifacts produced in the previous phase are not a fully functional client, but rather provide
the developer with the support to invoke the operations provided by the service. To effectively test the
two final steps (Client Communication and Service Processing), a complete client must be implemented
at this point (starting with the generated artifacts) and compiled and executed. Executing the client begins the Client Communication step, which finishes with the delivery of a message to the web service
application. Before reaching the service, the message must pass from the client application through the
client-side framework (i.e., in which it is converted to SOAP), the network, then pass through the server-side framework (during which it is converted from SOAP to a programming language level object),
and finally is delivered to the service application. For interoperability testing purposes, the service implementation should be empty of any business logic extraneous to the tests (as we are testing the correctness of the underlying middleware, and not specific service implementations).
The classification procedure in the Client Communication step consists of observing the delivery of
the client message to the service implementation. In practice we analyze the message contents at two
observation points: immediately before leaving the client application (and before being transformed into
SOAP by the client-side framework); and immediately after reaching the service implementation (i.e.,
after passing through the server-side framework). These two points are used for message content comparison and can be achieved, for instance, by using logs. The outcome of the classification is:
Correct: When the message content is the same at the two observation points.
Error: The message is not delivered to the service application (e.g., due to the client failing to
send the message, or due to the server failing to parse it). This is a fatal event, as it prevents the
execution of the last inter-operation step (Service Processing).
Warning: A message is delivered to the service application, but is holding a different content
from the one created at the client (including missing content). This is obviously a problem, yet it
does not prevent the execution of the next step, the Service Processing (i.e., it represents a nonfatal event).
In the final step, Service Processing, the services business logic is executed and the outcome is the
delivery of a response back to the client. This involves having the message going back from the service
application to the client application (and again passing through the server-side and client-side frameworks). Thus, our two message observation points are now the service application (immediately before
the message leaves the service code) and the client application (immediately after the message arrives at
the client application).
In the same manner as before, a classification procedure takes place and the outcome is:
Correct: When the messages content is the same at the observation points.
Error: When no message is delivered to the client application, for instance, due to problems in
building a response at the server-side framework or parsing the response at the client-side.
Warning: A response that is delivered to the client application but is holding incorrect or incomplete information.
Note that, in all steps, we assume correctness from the environment external to the web service frameworks at the server and client sides (e.g., during the Execution Phase we assume there are no network
failures during the SOAP message exchange).
4
In this section we present the ITWS, a web services interoperability testing tool that implements the approach described in the previous section. ITWS provides an easy way to test web services for interoperability and was designed to overcome the limitations of current tools, including the limited extensibility
and reduced interoperability analysis, as previously referred.
ITWS provides support for the inter-operation process presented in Section 3. Currently, the tool automatically handles testing all steps of the Generation Phase. It also provides partially automated support for testing the Execution Phase. The limitation in this phase is that the tool requires client code to
be provided in order to complete the inter-operation process. Although the tool could also generate this
code automatically, we consider this to be a mere technical aspect and not part of the core of our proposal. Anyway, this is an aspect that we intend to develop in future versions of the tool. In the following
sections we describe ITWSs architecture with respect to the approach steps and tool usage.
Fig. 3 presents the application-level architecture of ITWS, also including client framework tools (at the
top of the left-hand side in Fig. 3) and deployed services used (at the bottom left in Fig. 3). The following paragraphs describe the tools main components and their role in the execution of the testing steps.
The Startup component (see Fig. 3) is responsible for initiating (and performing a clean shutdown) of
ITWS. It also sets up the application by instructing the AppConfig component to load application-level
configuration from configuration files. This information includes environment configuration, application-level paths, and a list of service URIs to be tested (refer to Section 4.2 for details on all required
configuration).
The Test Manager is triggered by the Startup component (after the AppConfig finishes its tasks) and
is the core part of the application. The Test Manager supervises a test cycle (each test cycle is executed
once for each service to be tested) that comprises the 5 key inter-operation steps previously referred.
Note that a user may also configure ITWS to execute only part of this cycle (e.g., just the first two
steps). Despite this option, we describe the case for a typical full-cycle execution in the next paragraphs.
For each service to be tested, the Test Manager will start by executing the assessment of the first step
of this testing cycle: the Service Description
Generation. This consists of contacting the WS-I
Client+Tools++
Compliance Tester, which in turn will test the
Axis2.conf+ + Congura8on+
ITWS%Applica,on%
Axis2.conf
Axis2.conf
+
Axis2.conf+ +
Axis2.conf
service for WS-I Basic Profile compliance [16].
Problems++
Tool++
Results+
Results+
The WS-I Basic Profile is the current standard for
Results+
Iden8er++
Cong+
Results+
Results+
understanding the interoperability of a particular
Deployed+Client+Tools+
web service. Many tools provide support for the
Client+ Client+ Client+
Client+ Client+ Client+
Results+
Tool+
+Tool+
+Tool+ Client++
Client++
+Tool+ Client++
Client++
+Tool+
+Tool+ Client++
+Tool+ Client++
older version of Basic Profile (v1.1) (CrossCheck
Tool++
Tool++
Tool++ Tool+
Manager+ handler+
Tool+
Tool+
Networks),(Eviware, 2011). The newer versions
WS@I++
Test+
of Basic Profile (v1.2 used for SOAP 1.1 and v2.0
WS@I+
Test+
compliance+
compliance+
used for SOAP 1.2) appear to be not very comreport+
Manager+
Tester+
Deployed+Services+
mon yet (we found only one application providService+
Service+
Service+
Service+
ing direct support for these tests (yaronn)). DeApp+
Service+
Service+
WSDL+
WSDL+
Service+
Service+
WSDL+
WSDL+
Startup+
Service+
Service+
WSDL+
WSDL+
spite this, for the sake of completeness, ITWS
Cong+
WSDL+
WSDL+
WSDL+
WSDL+
uses both versions of the Basic Profile in our
Applica8on+
tests.
ITWS.conf+
List+of+Services+ Services.conf+
+Congura8on+
In short, ITWS evaluates the interoperability of
the WSDL document produced during service
Fig. 3. ITWS architecture and testing environment.
deployment, according to the WS-I guidelines.
The user always has the option to analyze the documents produced by the WS-I tool, but ITWS clearly
indicates if the WSDL passes WS-I Basic Profile or not (i.e., to mark the presence of a Warning, or not).
This latter task is executed by the Results Handler component, which basically aggregates the WS-I
tool output (i.e., the identification of problems is carried out by the WS-I tool).
After the previous step, ITWS proceeds to the second step of the testing cycle: the Client Artifact
Generation. For this, the Test Manager contacts the Tool Manager, which will instruct the Tool Configuration component to load the configuration of all deployed client-side tools for artifact generation. After configuration finishes, the Test Manager will ask the Tool Manager to execute the next available client artifact generation tool. When the client tool finishes execution, the Tool Manager invokes the Results Handler component.
The Results Handler receives the output produced by each client tool and, when applicable, delegates
the analysis on the Problems Identifier component, which uses regular expressions to identify different
types of inter-operation problems (such as Errors and Warnings) that may have been found during the
tool execution. Obviously, this classification of problems is largely dependent on the client tool being
executed; therefore, this information is also part of the client tool configuration and is provided by the
Tool Manager. The user is also able to tune the default classification procedure (Section 4.2.4 describes
how to customize the configuration of the Problem Identifiers). In the case of frameworks that only
support dynamic generation of artifacts (i.e., no files are created), such as PHP or Python frameworks,
ITWS includes scripts to verify the instantiation of client-side objects.
The tool can now proceed to the third step of the testing cycle - Client Artifact Compilation. This
step is also framework-specific and therefore is executed in the same manner as in the previous step.
The difference now is that a compilation tool is being executed (instead of a client artifact generation
tool) and the identification of problems may be carried out in a different manner (e.g., using different
regular expressions), but in ITWS all of these aspects are simply configuration details. ITWS automatically compiles the client code generated in the previous step. Obviously this step is not executed when
the framework/language does not support compilation. As in the previous step, a default (but configurable) classification procedure based on regular expressions is used to identify Errors and Warnings.
The artifacts generated and compiled up until this point do not compose a fully functional client. The
client creation requires the intervention of the client developer to implement the actual code. This code
will then use the support functions provided by the client artifacts to consume the service. As discussed
in Section 3, to proceed to the two final steps of the testing we need the client code to be placed in the
artifacts directory (see Section 4.2.3 for details on this procedure). After the compilation is carried out
ITWS can be used to execute the two final steps.
The Client Communication step is also initiated by the Test Manager component that orchestrates
the execution of this step by using the Tool Manager component, which in turn starts by recompiling
the full set of the client code to generate an executable client and then executes the actual client. The
identification of problems is performed by configurable Problem Identifiers and assumes the presence
of a log (or redirected console output) produced by the service from which ITWS can read and analyze
the output parameters (which should be the same as the ones sent by the client).
The Service Processing step again assumes the presence of an information log (or redirected console
output) in which is possible to read the response of the service as perceived by the client. This response
should be equal to the one produced by the service and is again classified by the Problems Identifier.
In this section we illustrate how a web services developer can use ITWS to test the inter-operation of a
given set of services against several different client-side frameworks. This section includes: a) how to
setup the testing environment; b) how to use ITWS to test the generation phase of the inter-operation
process; c) how to test the execution phase of the inter-operation process; d) how to extend ITWS to use
more client-side frameworks.
TABLE 1
named testing and will store the generated client artifacts.
CLIENT SIDE FRAMEWORKS
2. Deploy the desired client artifact generTools
ation tools: any client code generation
Framework
Generation
Compilation
Language
tools must be copied to
Tool
Tool
%ITWS_HOME%/tools directory (each
Oracle Metro 2.3 (Oracle,
wsimport
javac / ant
Java
tool under its own subdirectory). Table
2013b)
1 presents the list of pre-configured
Apache Axis1 1.4 (Apache
wsdl2java
javac
Java
Software Foundation, 2008)
tools and describes the name of the
framework, the executable code genera- Apache Axis2 1.6.2 (Apache
wsdl2java
javac / ant
Java
tion tool (for simplicity, code interpret- Axis2 Web Service Framework)
Apache CXF 2.7.6 (Apache
wsdl2java
javac / ant
Java
ers are also included in here), a compiSoftware Foundation, 2013)
lation tool, and the target language of
JBossWS CXF 4.2.3 (Red Hat
wsconsume
javac / ant
Java
the generated code. The user might
Middleware)
need to setup environmental variables
csc
C#
Microsoft WCF .NET Framerequired by specific tools (e.g., setting
work 4.0.30319.17929 (Chapwsdl.exe
vbc
VB .NET
pell, 2009)
JAVA_HOME).
jsc
JScript .NET
gSOAP Toolkit 2.8.16
wsdl2h.exe and
3. Deploy the desired client artifact comgcc
C++
(Engelen, 2013)
soapcpp2.exe
pilation tools: This means installing
Zend Framework 1.9 (Zend
PHP interpreter
N/A
PHP
compilers, such as the ones in Table 1.
Technologies, 2013)
Note that these tools are many times alsuds Python 0.4 (Ortel)
Python interpreter
N/A
Python
ready present in a typical development a) The tool does not compile automatically. A script was added to perform the
environment. As in the previous step,
task; b) Compilation via a tool-generated ant task; c) Compilation is not possible
and depending on the tool, the user
with these langauges; client object instantiation is checked.
might need to do tool-specific configurations.
4. Deploy the services to be tested: as in the client-side, also the server-side platforms are deployed
independently from the ITWS application. The server-side web service framework(s) must be
installed and the services that need to be tested must be deployed. This is a developers task that
should not be managed by ITWS (otherwise we would be interfering with the inter-operation
process, and thus potentially tainting the results). The ITWS user simply needs to provide a
plain-text list of the services URIs that will undergo testing in %ITWS_HOME%/services.conf.
a)
b)
a)
a)
a)
a)
c)
c)
configured tools). If needed, the user can customize the problem identification scheme (see point 3 in
Section 4.2.4). Obviously, in some complex cases the user might need to investigate the nature of the
problems by manually analyzing the produced client artifacts (in %ITWS_HOME%/testing). The artifacts of each test cycle have their own numbered separate subdirectory; this number is the same present
in the results output file. Thus, identifying a specific test is quite easy.
Env
GenScript
GenOptions
GenProblems
CompScript
Comp
Options
Comp
Problems
Description
An identifier of the client tool used
A version for the tool being used
Comma separated environmental variables needed to call the
generation (or compilation) tool.
Example: AXIS2_HOME=%ITWS_HOME%/tools/axis2-1.6.2. Optional,
when the developer has a properly configured system.
Relative path for the location of the artifact generation tool.
Command-line options to be applied to the artifact generation
tools. The %OUTPUT_DIR% and %URI% variables can be
used to dynamically refer to the default output directory and
service to be tested (values will be set by ITWS at runtime and,
as such, will be part of the command that will be executed).
Categories of artifact generation problems to be identified.
Relative path for the location of the compilation tool.
Options for the compilation tool. The %OUTPUT_DIR% variable can be used as in the GenOptions property.
Categories of compilation problems to be identified.
EXPERIMENTAL EVALUATION
10
In this section we present the results of an experimental evaluation using 7239 home developed web
services deployed on top of the three major (server-side) WS frameworks: Oracle Metro 2.3, JBossWS
CXF 4.2.3, Microsoft WCF .NET 4.0.30319.17929 (C#) (Chappell, 2009; Oracle, 2013b; Red Hat Middleware). The services were hosted in major application servers, respectively GlassFish 4, JBoss AS
7.2, and Microsoft IIS 8.0.8418.0 (Express) (GlassFish; Microsoft Corporation, 2013; Red Hat Middleware, 2013).
As mentioned, for this set of services we tested both the Generation Phase and the Execution Phase.
The results for the Generation Phase involve the complete set of server-side resources (i.e., the services
interfaces, generated by the three frameworks on behalf of the services), and the complete set of clientside resources (the eleven client-side frameworks). The results for the Execution Phase, due to the intensive manual work required in creating each client, involve a subset of the services (2489 services
supported by Metro and Glassfish) and one client-side framework (also Metro) supporting each client.
It is important to emphasize that in the tests for the Execution Phase we have the server and client-side
of the same framework, for which we were expecting to see none, or very few interoperability issues,
which was not the case.
11
In the Client Artifact Generation Step, 4763 tests produced at least one Warning and 287 resulted in
Errors (out of the total 152019). The JScript .NET clients are responsible for about 99.8% of all Warnings due to an incompatibility with the Java platforms (Metro and JBossWS) that generates warning
messages at every execution of the tool. The remaining Warnings correspond to other tools trying to
generate artifacts from the WSDL documents that failed the WS-I check. About 97% of the Errors in
this step are produced when using WSDL documents that failed the WS-I check. The remaining 3% are
actually the result of processing WS-I compliant services.
During the Client Artifact Compilation Step the tools produced 14478 Warnings. All of these are due
to compiling the artifacts produced by Axis1 and Axis2 (see also Table 4) and are non-disruptive warnings, which refer to the use of unchecked or unsafe operations. Regarding the Errors in this step, we
detected 1301 tests resulting in compilation Errors that were, in most cases, due to generation of code
carrying naming collisions or missing parameters and even crashes of the compilation tool (the JScript
.NET compilation tool crashed in some tests, with the message 131 INTERNAL COMPILER
CRASH). The compilation of the Axis1 and JScript .NET artifacts account for the vast majority of the
compilation step Errors (approximately 68% and 30%, respectively). The remaining Errors occurred
when compiling the Visual Basic .NET and Axis2 artifacts.
The tests for the Execution Phase identified a quite high number of inter-operation problems. The
Client Communication step failed with an Error in 1303 services (52% of the 2488 Metro services) and
inter-operation was not possible. No Warning was detected in this step. Regarding the Service Processing step, we did not detect any Warning or Error. Thus, once the information arrived at the service
it was able to travel back to the client without any observable issue.
12
The Client Artifact Generation Step is not very prone to Errors, as visible in Table 4. As referred, the
vast majority of the Errors were caused by the unusable WSDL documents mentioned in the previous
paragraph. This is probably due to the fact that, in most cases, the generation of the clients is a very automated task, where each statement is translated to a specific piece of code, and this will obviously fail
when something unexpected is being processed (such as a WSDL without operations). Despite this, Axis1, Apache CXF and JBossWS did not signal any problem when using the WSDL documents without
operations, which is obviously not the right behavior for the users of the tools (since the tools are silently failing to produce usable code), and also shows the silent propagation of a severe issue to the client
side.
Client Artifact Compilation appears to be a more error prone step, where we found the majority of the
errors (refer to Table 4). On the other hand, if we exclude the warnings in the Axis platforms (which are
all of the same type and refer to the use of unchecked operations), the total number of Warnings found
in this step drops to zero. This shows that either the generation process goes perfectly (and also without
warnings), or it fails in result of serious problems in the artifacts code. Most of the compilation failures
detected are due to wrong variables names, naming collisions or duplicate variables in within specific
groups of services (detailed results are available at (Elia et al.)).
We would also like to emphasize the huge behavior differences shown by the different tools. The client artifact generation tools provided with the tested versions of Metro, JBossWS, Apache CXF,
gSOAP, and C# .NET appear to be quite mature as they fail almost only in presence of non WS-I compliant WSDL documents. Moreover, Errors are always in the Client Artifact Generation step and these
tools never produced code that later results in compilation errors or warnings. In addition, the two client
artifact generation tools for PHP and Python show a similar performance, although they lack the compilation step.
Results for the Execution Phase have uncovered a total of 1303 Errors (52% of the 2488 Metro services). All Errors were detected in the Client Communication Step and are essentially of two types: a)
the client-side framework fails to translate the input object into a SOAP message, which results in an
exception on the client application; b) the server-side framework fails to translate the incoming SOAP
message into an application-level object causing the server to respond with 500 Internal Server Error. The former type of Error accounts for 34% (446 Errors) of the total number of Errors, the majority of which related with clients trying to manipulate abstract data types. The latter type of Error accounts for the remaining 66% (857 Errors) and is more common in clients managing data types that
have no placeholders for content (e.g., empty classes).
The high number of issues detected in the Client Communication step is quite surprising, considering
that we are using the same framework on each of the sides and once the framework has published ta
given WSDL, it should offer no later inter-operation issue.
Regarding the Service Processing step, we observed no issues. In fact, once the message arrives at
the service without any kind of issue, it is somehow expectable that it can perform the reverse path. In
future work, we plan to use more combinations of frameworks to understand if this scenario still applies.
As a final note, we have observed of the 1185 cases in which it was possible to complete the interoperation cycle (48% of the 2488 Metro services), 767 cases where client and server are exchanging an
empty message. This can be due to many factors, however what is important is that the developer
should at least be informed with some warning, which was never the case.
For the second set of services we have chosen to analyze a set of 1024 publicly available services. The
idea is to use more complex services that are currently being used in a real environment so to identify
different patterns of behavior with respect to the home-developed services analyzed before.
13
"
No"WS1I"Warnings"
57"
19" WS1I" 38"
WS1I"Warnings"
Warnings"
Errors"(549)"
14
TABLE 5
For these reasons, considering only the compliant or non-compliant nature of a given service proTest
Generation Compilation
vides very little information to early identify inErrors
Assertion
Errors
Errors
V.1.1 V1.2/2 Union
teroperability issues. Thus, we have analyzed the
Total
951
959
967 363/967(37.5%) 224/967 (23.2%) 530/967(54.8%)
result of the (failed) individual test assertions that
BP2402
697
706
713 123/713 (17.3%) 183/713 (25.7%) 281 / 713 (39.4%)
compose the WS-I Basic Profile and tried to unBP2114
706
0
706 122/706 (17.3%) 181/706 (25.6%) 279 / 706 (39.5%)
BP2122
333
0
333 120/333 (36.0%) 48/333 (14.4%) 137 / 333(41.1 %)
derstand how each individual warning relates with
BP2406
236
236
242 242/242 (100%) 34/242 (14.1%)
242/242 (100%)
the interoperability issues found in our tests in subBP2017
1
236
236 236/236 (100%) 29/236 (12.3%)
236/236 (100%)
BP2102
218
0
218
7/218 (3.2%)
17/218 (7.8%)
23/218 (10.6%)
sequent steps.
BP2201
40
0
40
37/40 (92.5%)
19/40 (47.5%)
38 / 40 (95%)
We have observed that some specific test asserBP2108
35
30
37
36/37 (97.3%)
28/37 (75.7%)
37/37 (100 %)
BP2202
0
20
20
8/20 (40%)
15/20 (75%)
17/20 (85%)
tions are good indicators of specific types of genBP2106
0
20
20
8/20 (40%)
15/20 (75%)
17/20 (85%)
eration or compilation errors. In fact, some test asBP2115
14
4
14
4/14 (28.6%)
8/14 (57.1%)
11/14 (78.6 %)
BP2019
14
14
14
3/14 (21.4%)
7/14 (50%)
9/14 (64.3%)
sertions (e.g., BP2402, BP2114, BP2122) are very
BP2703
8
0
8
8/8 (100%)
4/8 (50%)
8/8 (100%)
loosely associated with Errors (at least in the CliBP2417
3
8
8
8/8 (100%)
2/8 (25%)
8/8 (100%)
BP2101
3
7
7
7/7(100%)
5/7(71.4%)
7/7(100%)
ent Artifact Generation and Compilation steps)
BP2113
4
0
4
4/4 (100%)
1/4 (75%)
4/4 (100%)
while other assertions are strongly correlated with
BP2701
3
0
3
3/3 (100%)
1/3 (33.3%)
3/3 (100%)
BP2010
3
0
3
3/3 (100%)
1/3 (33.3%)
3/3 (100%)
generation Errors (e.g., BP2406, BP2017,
BP2208
1
1
1
1/1 (100%)
0/1 (0%)
1/1 (100%)
BP2703) or compilation Errors (e.g. BP2202,
BP2118
1
0
1
1/1 (100%)
1/1 (100%)
1/1 (100%)
BP2106, BP2101). An interesting example is the
case of test assertion BP2406 and its prerequisite BP2017. 100% of the services where these test assertions were signaled resulted in an Error in a later step (with at least one framework). In most of the cases (around 99%) the artifact generation error is caused by the fact that the WSDL requires the use of
RPC/Encoded SOAP messages, which are not supported by some of the mainstream frameworks (e.g.,
Metro, Apache CXF and JBoss) and forbidden by WS-I Basic Profile (BP2406) but appear still quite
common in web services found in production scenarios.
Finally, another aspect is that we could observe is that although the newer versions (BP 1.2/2.0) of
the WS-I tests should be retro-compatible (Basic Profile - Version 1.2 (Final),),(Basic Profile - Version 2.0 (Final),) with the older version (BP1.1), we have verified that this is not the case in many situations. As an example, the Basic Profile Test Assertion BP2122 (which has the same goal in both Basic
Profile versions) fails in 333 services for BP version 1.1 but does not fail in any service in BP 1.2/2.0
(please refer to Table 5 for the details on the remaining test assertions).
In Table 6 we present the detailed results for each of the client-side tools, including Warnings and
Errors. As we can see in Table 6, the most of the errors were disclosed when executing the Client Artifact Generation. We can notice that Metro, Apache CXF and JBoss (widely used and quite popular
frameworks) are responsible for the majority of the generation Errors observed, with around 340 generation errors each.
Most of the detected generation errors (in about 238 services) are related with the WSDL requiring
the use of RPC/encoded messages, which is not supported by these platforms. A large part of the remaining problems found (in 79 services), refer to Undefined Element Declaration errors, which relate to
the use of some XML tags (s:schema, s:lang, any) that the client frameworks are unable to handle. Note
that this can be solved by manually customizing the
data type bindings (Vivek), however the client develTABLE 6
oper has to know precisely which binding to define,
EXPERIMENTAL RESULTS FOR THE PUBLIC SERVICES
which may again result in an interoperability issue.
The remaining detected generation Errors (about
967 out of 1024 services
WS-I Warnings
25) when using Metro, JBoss, and Apache CXF are
Artifact Generation
Artifact Compilation
Client Tools
mostly due to the use of unknown elements in the
Warnings
Errors
Warnings
Errors
XML Schema; missing or duplicate (replicated) defiMetro
717
344
0
0
nitions of types in the WSDL; and collision of attribApache-CXF
1
343
0
1
utes due to problems of case insensitiveness of the
JBoss
0
341
0
1
target platform. This latter is a very interesting case of
Apache Axis1
0
15
1010
17
artifact generation error: it is due to the use of two
Apache Axis2
0
38
1010
9
elements in the WSDL that have the same name, but
gSOAP
71
12
6
6
with different cases (e.g., IP2PROXY and IP2Proxy).
.NET C#:
259
20
0
3
When a client generation tool tries to generate the cli.NET Visual
259
20
1
28
Basic
ent artifacts it needs to create two Java classes with
.NET
JScript
518
20
1
195
the same name, but different case, which results in an
Zend PHP
12
12
error under Windows (which does not support caseSuds Python
0
93
sensitive filenames), but not under Linux. The probWS-I BASIC PROFILE RESULTS FOR THE PUBLIC SERVICES
WS-I BP Warnings
15
lem can be solved by some manual configuration, but still it is a very interesting case where the interoperability issue actually extends from the framework to the Operating System. Obviously, this signals that there is a huge space for improvement. On one hand, when generating, defining, or deploying
the WSDL a developer should be warned that his WSDL file carries potential problems. On the other
hand, some of the client-side tools (in this case, the frameworks showing this problem are mainstream)
could also be improved and be prepared to automatically handle these problematic cases.
Metro is also responsible for a huge number (717) of generation warnings. These occur mostly in
services (703) that offer SOAP 1.1 and 1.2 bindings, where the wsimport tool selects version 1.1 over
version 1.2. The remaining 14 wsimport warnings are related to a basic WS-I check performed by the
tool itself (the .NET framework also performs this type of tests) which reports that the service is not
WS-I Basic Profile 1.0 compliant. Axis1 and Axis2 account respectively for 15 and 38 errors in this
step. The errors are due to missing elements or type definitions (i.e., due to the use of unsupported or
missing schemas), duplicate or missing operations (operations not defined or with the same name).
gSOAP uses two tools to generate artifacts. The first one (wsdl2h) generates a C++ header file from
the service description document and produced 5 generation errors while parsing the WSDL file. The
second one (soapcpp2), which creates the cpp files, accounts for the remaining 7 generation errors (due
to incorrect syntax in the previously generated header file). This framework is also involved in several
generation warnings: non compliance with WS-I Basic Profile version 1.0 (31 services), use of reserved
character sequences in the generated code (27 services), missing bindings or messages (10 services),
duplicate schemas (4 services), unexpected elements (2 services), and undefined QName (3 services).
The three .NET tools fail to generate clients for the same 20 services. Out of these, 15 are due to
binding problems related to missing or not supported element definitions, and 5 represent failures when
processing the WSDL file. In addition, the .NET framework produced a high number of warnings. Most
of these (in 257 services) are due to a preliminary failed check of compliance to WS-I Basic Profile version 1.1 (this check is executed by the client-side framework itself). The other warnings include failed
schema validation (4 services) and optional WSDL extension ignored (2 services). The client generation
tool for JScript.Net produced an additional set of 501 warnings for skipped operations that were not
used in the code generation. In 65% of these 501 cases, the generated code did not have methods that
could be used to invoke the service, which obviously renders the code useless.
The Zend PHP framework produced 12 Generation errors, with 5 occurring when parsing the WSDL
document and 7 when parsing the XML Schema. The tool also produces an internal warning related to
the improper use of system's timezone settings (for the same 12 services).
Finally, the Suds Python platform accounts for 93 generation errors, of which 96% are type not found
exceptions, caused by the inability to process some XML types (mostly s:schema and soapenc:Array).
The remaining errors include internal error messages, which require further analysis by the service or
framework developers. All individual detailed results are available at (Elia et al.).
The Client Artifact Compilation step accounts for a smaller number of Errors when compared to the
previous step (260 compilation Errors against 1256 generation Errors). Metro is the only platform with
no compilation Errors, but also JBoss and CXF performed well, as the only Error reported is related to
the use of an uncommon encoding in one WSDL file (which can be solved, provided that the right encoding is specified when creating the WSDL file or when generating and compiling artifacts).
Problems with unmappable characters cause 3 compilation errors in Axis1 and Axis2. Other compilation problems for these two frameworks include severe issues, such as: package names generated with
invalid characters; variables used without previous instantiation (due to wrong naming); and duplicate
variable names. The Axis frameworks also produce a very high number of compilation warnings (in all
services for which the compilation is possible); however, these are all non-disruptive warnings, which
refer to the use of unchecked or unsafe operations.
The gSoap platform reports a compilation Error related to duplicated namespace definitions for 4
services, while for other 2 it fails to generate the required source files.
Finally, the .NET clients are quite prone to compilation Errors, especially Visual Basic and JScript
clients. The C# client code results in 3 compilation Errors caused by duplicate variables (more than one
variable defined with the same name) and missing type of a variable (which is generated as a generic
Object). The compilation of the VB code results in 25 Errors referring to the presence of duplicate variables. The compilation of the JS .NET client code is responsible for almost 3/4 of the total compilation
errors (190/260). These include 89 errors due to wrong code (e.g., duplicate variables, missing types,
types mismatch, more the one constructor with the same parameters, and code trying to extend nonextendable classes). Moreover, 105 errors are compiler crashes reporting the message 131 INTERNAL
COMPILER CRASH, emphasizing the need for interoperability assessment and improvement.
16
6 KEY FINDINGS
In this section we summarize the main findings and trends obtained from our huge test campaign.
Code generation tools show quite disparate performance. Some appear to be immature (although
they are being used in production environments) and crash during the generation process, others
provide detailed information when in presence of problems, while others provide insufficient information.
Mature generation tools (e.g., Metro, CXF, JBoss, .NET C#) tend to identify issues when generating artifacts. Once the code is produced the likelihood of containing errors is small. Less mature tools (e.g., .NET Jscript and Axis 1) often produce problems that are identified only at compilation time. Most errors identified during compilation are due to wrong naming of namespaces, packages and variables.
Over half of the 1024 public services were involved in some fatal issue and 23% still require
RPC messages, which is prohibited by WS-I Basic Profile and no longer supported by some
mainstream tools. Despite this, each of the 1024 services was able to inter-operate with at least 4
of the client-side frameworks, indicating that there is space for improvements.
Fatal inter-operation errors occur even when the same framework supports client and server.
Interoperability issues are not limited to the framework and programming language domain and
can extend to the operating system. In some cases, a framework was able to generate code in
Linux, but not in Windows due to the case-insensitiveness nature of this operating system.
The WS-I Basic Profile tests do not help much to understand how interoperable a service is.
Their capability of detecting services that in alter steps are involved in inter-operation issues
was very high for our home developed services (where the WSDL documents are quite simple).
On the other hand, this analysis was almost useless for the public services (that can have more
complex WSDL documents), as almost all the services were considered non-compliant (including services that actually created an inter-operation problem and services that did not).
First results from the testing Execution Phase suggest that problematic services might reach the
this phase without showing a single warning, and that in some cases, the server platforms silently publish services, even when the usability of such services is limited.
7 CONCLUSION
Inter-operation of different web services providers and consumers revealed to be a difficult task, resulting in efforts to promote interoperability, such as the ones carried out by the WS-I organization. Despite
this, still developers have limited tools and resources to understand how interoperable their services are.
For this reason we have proposed a well-defined approach for interoperability testing from a practical
perspective, and a tool, named ITWS, for the automatic implementation of such testing. We have also
performed a massive testing campaign providing insights about the interoperability of more than 8
thousands services by detecting a huge number of interoperability issues and being able to disclose
problems even in WS-I compliant services, thus outperforming competitor tools in interoperability assessment.
ACKNOWLEDGMENT
This work has been partially supported by the Project ICIS - Intelligent Computing in the Internet of
Services, number 4774, of the program CENTRO-SCT-2011-02, and by the projects CErtification of
CRItical Systems (CECRIS), Marie Curie Industry-Academia Partnerships and Pathways (IAPP) number 324334; and DEsign, Verification and VAlidation of large-scale, dynamic Service SystEmS (DEVASSES), Marie Curie International Research Staff Exchange Scheme (IRSES) number 612569, both
within the context of the EU Seventh Framework Programme (FP7).
References
17
CrossCheck Networks. (n.d.). SOAPSonar - Advanced Client Simulation for Service Testing. Retrieved January 13, 2014, from
http://www.crosschecknet.com/products/soapsonar.php
Curbera, F., Duftler, M., Khalaf, R., Nagy, W., Mukhi, N., & Weerawarana, S. (2002). Unraveling the Web Services Web: An Introduction to
SOAP, WSDL, and UDDI. IEEE Internet Computing, 6(2), 8693.
Elia, I. A., Laranjeiro, N., & Vieira, M. (2014a). A Field Perspective on the Interoperability of Web Services. In 2014 IEEE International Conference on Services Computing (SCC) (pp. 7582). doi:10.1109/SCC.2014.19
Elia, I. A., Laranjeiro, N., & Vieira, M. (2014b). ITWS: An Extensible Tool for Interoperability Testing of Web Services. In 2014 IEEE International Conference on Web Services (ICWS).
Elia, I. A., Laranjeiro, N., & Vieira, M. (2014c). Understanding Interoperability Issues of Web Service Frameworks. In Dependable Systems and
Networks, 2014. DSN 2014. IEEE International Conference on. Atlanta, Georgia USA.
Elia, I. A., Laranjeiro, N., & Vieira, M. (n.d.). ITWS (Interoperability Testing for Web Services) tool and Experimental Results. Retrieved from
http://eden.dei.uc.pt/~ivanoe/TSC/TSC.zip
Engelen, R. van. (2013). gSOAP: SOAP C++ Web Services. Retrieved December 7, 2013, from http://www.cs.fsu.edu/~engelen/soap.html
Erl, T. (2005). Service-Oriented Architecture: Concepts, Technology, and Design. Upper Saddle River, NJ, USA: Prentice Hall PTR.
Eviware. (2011). soapUI. Retrieved July 4, 2011, from http://www.soapui.org/
GlassFish open-source application server for Java EE. (n.d.). Retrieved from https://glassfish.java.net
Hewitt, E. (2009). Java SOA Cookbook. OReilly Media, Inc.
jBoss: wsconsume. (n.d.). Retrieved from https://community.jboss.org/wiki/JBossWS-wsconsume
Microsoft Corporation. (2013). IIS 8.0 Express. Retrieved December 9, 2013, from http://www.microsoft.com/enus/download/details.aspx?id=34679
.NET Framework Class Library (C#). (n.d.). Retrieved from http://msdn.microsoft.com/en-us/library/gg145045(v=vs.100).aspx
Nezhad, H. R. M., Benatallah, B., Casati, F., & Toumani, F. (2006). Web services interoperability specifications. Computer, 39(5), 2432.
doi:10.1109/MC.2006.181
Oracle. (2013a). Java SE 7 API Specification. Retrieved December 7, 2013, from http://docs.oracle.com/javase/7/docs/api/
Oracle. (2013b). Metro. Retrieved December 7, 2013, from https://metro.java.net/
Ortel, J. (n.d.). Suds - Lightweight SOAP client. Retrieved December 7, 2013, from https://fedorahosted.org/suds
Pautasso, C., Zimmermann, O., & Leymann, F. (2008). Restful Web Services vs. Big Web Services: Making the Right Architectural Decision.
In Proceedings of the 17th International Conference on World Wide Web (pp. 805814). New York, NY, USA: ACM.
doi:10.1145/1367497.1367606
Ramsokul, P., & Sowmya, A. (2006). A Sniffer Based Approach to WS Protocols Conformance Checking. In The Fifth International Symposium
on Parallel and Distributed Computing, 2006. ISPDC 06 (pp. 5865). doi:10.1109/ISPDC.2006.5
Red Hat Middleware. (2013). JBoss Application Server. Retrieved December 8, 2013, from http://www.jboss.org/jbossas/
Red Hat Middleware. (n.d.). JBoss Web Services (JBossWS). Retrieved December 7, 2013, from http://www.jboss.org/jbossws
Senthil Kumar, K. M., Das, A. S., & Padmanabhuni, S. (2004). WS-I Basic Profile: a practitioners view. In IEEE International Conference on Web
Services, 2004. Proceedings (pp. 1724). doi:10.1109/ICWS.2004.1314719
Vivek, P. (n.d.). How to deal with unresolved xs:schema references in WSDL. Retrieved from
https://weblogs.java.net/blog/vivekp/archive/2007/05/how_to_deal_wit.html
Web Services Interoperabily Organization (WS-I). (2002). Web Services Interoperability Organization (WS-I). Retrieved September 16, 2008,
from http://www.ws-i.org/
Web Services Interoperabily Organization (WS-I). (2014). Deliverables - Basic Profile Working Group. Retrieved January 13, 2014, from
http://www.ws-i.org/deliverables/workinggroup.aspx?wg=basicprofile
WSDL XML Schema. (n.d.). Retrieved from http://schemas.xmlsoap.org/wsdl/
WS-I. (n.d.). Retrieved December 5, 2013, from http://www.ws-i.org/deliverables/workinggroup.aspx?wg=testingtools
wsimport. (n.d.). Retrieved from http://docs.oracle.com/javase/6/docs/technotes/tools/share/wsimport.html
yaronn. (n.d.). WsiHero. Retrieved February 13, 2014, from https://github.com/yaronn/WsiHero
Zend Technologies. (2013). Zend Framework. Retrieved December 7, 2013, from http://framework.zend.com/
Ivano Alessandro Elia received the PhD degree from the
University of Naples Parthenope. Currently he is a Postdoc research fellow at the University of Coimbra, Portugal.
His research focuses on web application security as well as
experimental dependability evaluation.
Nuno Laranjeiro is an assistant professor at the University of
Coimbra, Portugal. His research focuses on robust software
services as well as experimental dependability evaluation,
web services interoperability, services security, and enterprise application integration. He is a member of the IEEE
Computer Society.
Marco Vieira is an assistant professor at the University of
Coimbra, Portugal. His interests include dependability and
security benchmarking, experimental dependability evaluation, fault injection, software development processes, and
software quality assurance. Vieira received the PhD degree
from the University of Coimbra. He is a member of the IEEE
Computer Society.