2013 IEEE Seventh International Symposium on Service-Oriented System Engineering

Runtime Monitoring of SOA Applications: Importance, Implementations and Challenges

Farag Zakaria Safy, Mohammad El-Ramly, Akram Salah
Faculty of Computers and Information Cairo University farag.zakaria@tedata.net, m.elramly@fci-cu.edu.eg, akram.salah@fci-cu.edu.eg

Abstract This paper presents the usages, current status and challenges that face monitoring real runtime SOA applications from both research and industry points of view. SOA application monitoring can be done for collecting statistics, guaranteeing quality of service, generating test cases or other purposes. The key challenges that face SOA monitoring are (1) monitoring overhead and performance degradation, (2) the diversity of supported formats and protocols, which is further complicated by the growth in the number of integrated applications that requires complex logic to be able to monitor individual paths across multiple services, and (3) the distribution of services which is further complicated by deployment in the cloud. We cover academic perspectives that are typically proposals for models or architectures for SOA middleware for monitoring. And we cover as well real monitoring techniques supported in SOA frameworks provided by software vendors. Keywords Service Oriented Architectures, SOA, Service Monitoring, Middleware, Interceptor Pattern, OpenESB, BPEL

challenges and making monitoring easier. Finally, section VIII presents the conclusion and future work. II. SIGNIFICANCE OF SOA MONITORING In real runtime environments, such as telecommunication systems, a SOA application may integrate more than 25 applications through different protocols and integration methods. So, monitoring the runtime behaviour and performance of such systems is of great importance. The significance of SOA monitoring is indicated by the many purposes it serves as described herein. A. Service Level Agreement (SLA) Growing cooperation among different integrated software applications raises the need for SLA. SLA is negotiated among companies to address mean time between failures, mean time to repair or recovery and applications' performance. Fulfilling these agreements is crucial for the coherence of integrated SOA applications. [8], [23] B. Quality of Service (QoS) Monitoring is used to guarantee QoS, firstly, in its traditional sense, which relates almost entirely to network performance. And secondly, it is used in its service-centric sense whichrefers to a wide range of non-functional service characteristics. QoS attributes give trustworthy quality information for ranking integrated services. [8], [11], [22] C. Quality of Experience (OoE) or Quality of User Experience This is an overall measure of customer's experience with an existing service. It is an indication of how the service was delivered by the vendor. [22], [24] D. Troubleshooting SOA monitoring may be used to detect fully integrated systems' and services' failures and to analyse and understand applications unexpected behaviours. [6], [8] E. Alert Management These are applications used for analysis, tracking and management of the highest severity alerts in enterprise software applications received from monitoring engines. Handling such alerts leads to more reliable and stable business solutions. [1]

I. INTRODUCTION Service Oriented Architectures (SOA) are mechanisms for improving IT solutions in terms of fulfilling business requirements. [1] A SOA application consists of a set of different collaborating software services that integrate with each other through different enterprise and communication protocols. SOA frameworks are rich software of diverse enterprise integration patterns that address system integration flows and communications through experts' recommended solutions. OpenESB is an example open source SOA framework that is very rich in service engines and binding components. Monitoring applications in such environments is quite complex. SOA monitoring is addressed in various researches but still has challenges in the techniques used as well as runtimeenvironmentconsiderations. This paper discusses these challenges from academic and industrial viewpoints. This paper is structured as follows. The following section, discusses the significance and goals of monitoring SOA applications. Section III explains Enterprise Service Bus (ESB) as one of the known SOA implementations. Section IV presents monitoring techniques proposed in academia. Section V discusses monitoring techniques in industrial environments and their limitations. Section VI addresses the challenges facing real-timeapplication monitoring in SOA runtime environments. Section VII provides a proposal for overcoming these
978-0-7695-4944-6/12 $26.00 2012 IEEE DOI 10.1109/SOSE.2013.61 315

TABLE I. ACADEMIC SOA MONITORING TECHNIQUES

SOA Framework ServiceMix 4.1 Apache Axis 2 Network-Centric applications Electrical applications ServiceMi, OpenESB Application Banking applications Monitoring Technique Interceptor pattern Their own technique Messaging network Functions, feature-based, and model-based monitoring JMX, AOP SITT , server side monitoring, SOAP monitoring SeCSE project ServiceMi, OpenESB ESB SECMOL AOP End-to-end monitoring Ref. [1] [2] [3] [4]

IV. ACADEMIC SOA MONITORING TECHNIQUES Table I summarises the monitoring techniques extracted from referenced papers and research works. Zmuda et al proposed using interceptor pattern in a fourlayered architecture to collect monitored data from eventdriven SOA applications. [1] Their architecture layers are: A. Instrumentation layer It is responsible for discovering the container topology state to affect monitoring of particular services. B. Data collection layer It is the backbone for collecting data from distributed containers. C. Data processing layer It uses complex event processing (CEP) to process monitored data and calculate metrics. D. Data presentation layer It presents monitored data to the end user. Chen et al assume that SOA applications are all about a set of communicated web services only. Based on this, they proposed a technique that defines a service ID, an interface ID, a process ID and a sequence ID to identify the monitored data generated from the web service framework. [2] Terazono et al treat SOA applications as a network of publish/subscribe systems, consisting of a number of producers and consumers of services. So, monitoring can be achieved by logging publisher, subscriber, accounting information, timestamp and the message itself in a message history database. [3] Cachapa et al monitor functions and features offered by production systems as well as models of hardware elements. They capture the signals generated by hardware sensors in XML data format with timestamp and expose that data through the web service in the form of events. [4] Psiuk and Bujok use Aspect Oriented Programming (AOP) to inject cross-cutting aspects between service invocations in order to add support for logging. They use AOP and Java Management Extensions (JMX) to add monitoring and management logic to a Java Business Integration container in such a way that the logic is fully transparent and compliant with the JBI specification and allows to gather, process and expose sophisticated information regarding the qualitative parameters and operational reliability of the JBI container . JMX is used by OpenESB to monitor binding components and service engines start up, termination, input and output. But, JMX does not help in monitoring QoS attributes. [5] Bhuyan et al use service integration testing tools that support monitoring services as well as monitoring communication among SOAP endpoints. [7] Baresi et al present SECMOL (service centric monitoring language) a high level monitoring specification language. It is developed for both stating and monitoring functional requirement and QoS constraints of BPEL processes. [9]

[5] [7]

[9] [10] [22]

F. Software Benchmarking This is the process of comparing applications against each other or against standard applications by running a number of standard business tests and trial procedures. [8] G. Testing SOA Monitoring is used for collecting real-time execution traces to use for generating real test data and synthetic test cases with dummy parameters. [1], [2], [7], [22] III. ENTERPRISE SERVICE BUS (ESB) ESB is the most known software architecture model for SOA runtime environments. Its logical components and their functions, as stated by Psiuk and Bujok, are: [5] A. Rules Engine This is responsible for routing, message transformation, message enhancement and message processing. B. Service Registry This is responsible for mapping a service contract into the corresponding service implementation. C. Service Mediator This is the component that expose services using different types of protocols (Protocol transformation). D. Business Processor This is responsible for service orchestration, for Example, Business Process Execution Language (BPEL). E. Service Container This provides security, transaction support, context propagation, management, logging, and deployment environment. F. Service Manager It provides business monitoring, system monitoring and business auditing through SLA agreements.

316

TABLE II. INDUSTRIAL SOA MONITORING TECHNIQUES

SOA Framework OpenESB Apache ServiceMix Mule ESB JBoss ESB Petals ESB Oracle SOA Suite Monitoring Technique JMX, JConsole JMX, Message Listeners, EIP JMX, MC4J JMX Console, Mule Galaxy, Mule HQ, EIP JMX, Gateway Listeners, ESB Aware Listeners JMX, SNMP API, Fractal framework, Cacti, Nagios JMX, BAM Ref. [12] [13] [14] [15] [16] [17], [19]

Remote Managers Level Adaptors Level Agent Level Instrumentation Level

JMX Manager

Web Browser

SNMP Manager

RMI Adapter

HTTP Adapter MBean Server

SNMP Adapter

MBean

MBean

Psiuk et al propose an end-to-end monitoring model for SOA applications. They propose a generic ESB Metamodel (EMM) that consists of specific entities and defining mechanisms to gather monitoring data related to these entities. EMM includes two plans; topology plan which includes metamodel of ESB functional components and messaging plan that includes messaging interactions. Topology plan consists of these entities; ESB containers, federations, functional components, applications, service endpoints, services and routers. Messaging plan consists of these entities; message transmission (MT), message exchange (ME), message flow (MF), message flow patterns (MFP), ESB process (EP), invocation rate (IR) and error rate (ER). [22] After defining these entities a monitoring goal is defined to describe three monitoring activities for collecting data: 1. 2. 3. Gathering information about ESB topology-constructing the topology model and tracking its changes. Gathering messaging information-constructing messaging model and tracking its changes. Measuring indicators defined monitorable EMM entity. in the the

Figure 1. All the components and layers of JMX [25]

and remote API as shown in Fig. 1. (from Mule ESB [25]) JMX monitoring technique is technology-limited. It only applies to Java technology. All applications functionality needs to be exposed as MBeans. Most of studied SOA runtime expose life cycle and statistics functions only to be monitored through JMX. So, monitored data collected by JMX are not related to each other. So, we cannot have complete scenario data. B. BAM Business Activity Monitoring (BAM) is another effort for standardizing or rather defining monitoring in the context of business and enterprise systems. [5] It is an eventdriven tool for generating the required data in real time. It is divided into 4 parts: 1. 2. Capturing data collected by sensors in BPEL and ESB. Storing the captured data in a normalized form in the active data cache (ADC) with the ability of data correlation. Processing stored data and updating the ADC to generate the required reports. Delivering the data in the required format.

messaging

V. INDUSTRIAL SOA MONITORING TECHNIQUES Table II summarizes the monitoring techniques used in existing SOA runtime environments in industry as OpenESB, Apache ServiceMix, Mule ESB, JBoss ESB, Petals ESB and Oracle SOA suite. We follow by a brief description of the key techniques used for monitoring in industry. A. JMX This Java technology provides tools for managing and monitoring devices, applications and service-driven networks. [18] JMX architecture is divided into three levels: instrumentation level, agent level and distributed services level. Instrumentation level contains the resources exposed on managed beans (MBean). Agent level is the management entity that represents the connection between MBeans and the management application. And finally the distributed services level represents the interfaces required for implementing JMX managers as SNMP, RMI, adapters

3. 4.

BAM technique limitations are the sensors required for each monitored component with its own nature. Thus, collected monitoring data is semi-related and does not form complete coherent scenarios. C. Listeners Many SOA runtime environments may provide various listeners like exchange listeners, endpoint listeners, event listeners and other listeners. These listeners could be used to collect monitoring data from ESB components. Listeners have limitations. First, they must be available for all ESB components for collecting data. Second, an extra application is needed to receive the monitored data. Last, data need to be correlated to each other to make sense.

317

D. Enterprise Integration Patterns (EIP) In their book, Hohpe and Woolf described many different system-management patterns that can be used in SOA applications' monitoring. We will state three patterns that were applied to ServiceMix ESB and Mule ESB. First, in the Wiretap pattern you place a wiretap on certain endpoints to let you see the messages received by those endpoints. A copy of each message is made and sent to a different endpoint so you can look at the messages content without having to pause the processing of the original message or do anything else that might disrupt the running system. Second in the Detour pattern you construct a detour with a context-based router controlled via the control bus pattern. In one state the router routes incoming messages through additional steps while in the other it routes messages directly to the destination channel, so you can examine and optionally alter the message. Third, in the Message Store pattern you capture information about each message in a central location which can be analysed at later times. [20] EIP monitoring limitation is that their application for messaging can be applied to queue messages only. Monitoring data are not related to each other so there is no complete scenario data. E. Fractal framework. Petals ESB is built with and on top of Fractal software component framework provided by the OW2 consortium. [16], [21] This helps to monitor Petals ESB using the monitoring facilities provided by Fractal framework. It uses the separation of concerns principles: namely separation of interface and implementation, component oriented programming, and inversion of control. VI. CHALLENGES FACING SOA MONITORING There are many challenges that face monitoring of SOA applications. These challenges differ depending on the nature of the application and various integrated services. In the following, we summarize these challenges based on our analysis and study and review of relevant resources. A. Performance When considering monitoring, one key factor must be taken into consideration that affects both application and monitoring engine. This is performance degradation. This requires implementing optimization techniques in the monitoring engine. Also, it suggests introducing timebased monitoring that runs under necessity and stops otherwise. [1], [7], [8], [23] B. Variety of supported protocols As an example of various protocols supported by SOA frameworks, OpenESB provides binding components for wrapping these protocols; HTTP, SOAP, FTP, Database, JMS, LDAP, Email, REST, HL7 and TCP/IP and service engines which provide and consume business logic and transformation services to other components like BPEL, XSLT, IEP, POJO and JEE service engines. This requires

complex processing to monitor these applications as each protocol, binding component and service engine has its own monitoring methods and techniques. [5], [9], [22] C. Data monitoring format Most SOA applications generate communicated data in XML, but with different formats. It is a challenge to define wrappers or adapters to handle these various formats and integrate various pieces of data. This requires analysis and decisions on how to represent monitored data, which data type to monitor (message payload or message metadata) and if format conversion is required or not. [2], [3], [4], [5], [22], [23] D. Path monitoring This is one of the hardest problems that face SOA applications monitoring. It is insufficient to monitor components' data individually. So correlation of monitored data is required. Full path monitoring is often necessary for obtaining meaningful scenarios. Enormous and complex integrated services lead to path complexity that requires complex monitoring logic to identify and correlate pieces of data for a certain path. [2], [22] E. Growth of the number of integrated applications As the number of integrated applications grows more processing and sophisticated monitoring techniques are required for processing the mass of collected data. F. Distribution of SOA application itself This adds additional complexity to application monitoring as it requires distribution of the monitoring techniques upon application distribution. It also requires replication of monitoring techniques upon application replication on several servers. [1], [22] G. SOA applications in the cloud Cloud computing (private and public) are picking up as a favourite solution to face on-demand easy-to-scale computing needs. Deploying SOA applications on virtual servers in the cloud adds another dimension of complexity. The virtualization of the platform and the dynamic nature of the cloud (adding and removing servers as needed and building virtual networks) further complicate the monitoring SOA applications. VII. GUIDELINES FOR ADDRESSING CHALLENGES The challenges facing monitoring SOA applications pose open research questions. Briefly, this section provides a draft proposal for addressing some of these challenges. The key idea is adapting SOA frameworks and monitoring middleware to add metadata that are used to make monitoring easier. The following presents five high level ides along this direction. A. Context and application identification The first idea is adding metadata to each application and runtime context and propagating these data to individual SOA components. Thus, we can identify monitored data and relate them to each other and, consequently, address path monitoring and application distribution challenges.

318

B. Data classification The second idea is to equip monitoring facilities to allow defining the required level of data monitoring. This level can be: functional data, non-functional data, application level metadata and runtime context level metadata. This decreases monitoring overhead and performance deterioration. It also allows monitoring engines to focus on monitoring and gathering only a specific level of data. C. Components self- monitoring logs After passing context and application identification data to individual SOA components, the third idea is to adapt runtime environments to force each component to have its own monitoring log. So, monitoring engine can relate these monitored data to a specific application runtime context. D. Adaptive monitoring The fourth idea is building flexible monitoring engines to comply at runtime with pre-defined strategies that start and stop monitoring based on the runtime context. This will reduce monitoring overhead and enhance performance. E. Data format registry In SOA runtime environments, the exchanged messages among different components and applications are XML messages. The fifth idea is building a registry to store the schemas of different XML messages. Relating a message to its schema and developing schema adapters can solve the problem of various formats for the monitored data. VIII. CONCLUSION AND FUTURE WORK In this paper we gave a concise overview of monitoring SOA applications runtime behaviour. We first discussed why it is important and the different possible uses of the collected data. Second, we presented the most important academic works and industrial techniques used for monitoring. Finally, we studied why SOA monitoring is challenging and what the challenges are. We also proposed changing SOA frameworks and monitoring middleware to supply metadata that can be used to tackle some of these challenges. This work is done as part of the M.Sc. thesis of the first author in order to deploy SOA monitoring techniques for automatic generation of test cases for SOA systems, in particular for regression testing. Our future extensions of this work will be in this direction. In particular we will examine closely existing SOA monitoring techniques from the point of view of their suitability and the suitability of the collected data for test case generation. And if necessary, we will tailor existing monitoring techniques and middleware for this purpose and to overcome any challenges. REFERENCES
[1] D. Zmuda, M. Psiuk, K. Zielinski, "Dynamic monitoring framework for the SOA execution environment", International Conference on Computational Science, ICCS 2010. C. Chen, A. Zaidman, H. G. Gross, "A framework-based runtime monitoring approach for Service-Oriented software systems", Delft University of Technology, Software Engineering Research Group, Tech. Rep. 2011-014. J. Terazono, H. Fukuhara, I. Koseda, R. Fujita, T. Miyazaki, S. Saito, T. Hayashi , "Service Oriented Architecture realized by a

[4]

[5]

[6]

[7]

[8]

[9]

[10] [11] [12] [13] [14] [15] [16] [17]

[18]

[19]

[20]

[21] [22]

[23]

[24]

[2]

[25]

[3]

messaging network", 2010 IEEE/IFIP Network Operations and Management Symposium NOMS, 2010. D. Cachapa, R. Harrison, A. Colombo, "Monitoring functions as service composition in a SOA-based industrial environment", 36th Annual Conference on IEEE Industrial Electronics Society - IECON, 2010. M. Psiuk, T. Bujok, "Generic ESB monitoring architecture compliant with JBI", M. Sc. Thesis, Department of Electrical Engineering, AGH University of Science and Technology, Krakow, Poland, 2008. L. Baresi, S. Guinea, Dynamo: Dynamic monitoring of WSBPEL processes", In Proceedings of the International Conference on Service-Oriented Computing (ICSOC05), 2005. P. Bhuyan, C. Prakash, D. Mohapatra, "A survey of regression testing in SOA", International Journal of Computer Applications, vol. 44, no. 19, pp. 0975 8887, April 2012. T. Espinha, C. Chen, A. Zaidman, H. Gross , "Maintenance research in SOA towards a standard case study", Delft University. of Technology, Software Engineering Research Group, Tech. Rep. 2012-001. L. Baresi, S. Guinea, O. Nano, G. Spanoudakis , "Comprehensive monitoring of BPEL processes", Internet Computing, IEEE, vol. 16, Issue 5, 2012. M. Psiuk, "AOP-based monitoring instrumentation of JBIcompliant ESB", World Conference on Services - I, July 2009. Apache ServiceMix website. [Online]. Available: http://servicemix.apache.org/ OpenESB website. [Online]. Available: http://wiki.openesb.java.net/ Sourceforge website. [Online]. Available at: http://digs.sourceforge.net/papers/qos.html Mule ESB website. [Online]. Available at: http://www.mulesoft.org/ JBoss ESB website. [Online] Available at: http://www.jboss.org/jbossesb Petals ESB website. [Online]. Available: http://petals.ow2.org/ Oracle SOA suite website. [Online]. Available at: http://www.oracle.com/technetwork/middleware/soasuite/overv iew/index.html Oracle Technical Network. [Online] Available at: http://www.oracle.com/technetwork/java/javase/tech/javamana gement-140525.html The PACKT website. [Online] Available at: http://www.packtpub.com/article/business-activity-monitoringin-oracle-soa-suite B.W. Gregor Hohpe, Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions. Addison-Wesley Professional, ISBN 0-321-20068-3, 2003. OW2 Consortium website. [Online]. Available at: http://fractal.ow2.org/documentation.html M. Psiuk, T. Bujok, K. Zielinski, "Enterprise Service Bus monitoring framework for SOA Systems", IEEE Transactions on Services Computing, vol. 5, no. 3, 2012. K. Skakowski, J. Sendor, R. Sota, "Application of the ESB architecture for distributed monitoring of the SLA requirements", Parallel and Distributed Computing (ISPDC), Ninth International Symposium on, 2010. SearchCRM website. [Online] Available at: http://searchcrm.techtarget.com/definition/Quality-ofExperience T. Rademakers and J. Dirksen, Open Source ESBs in Action, Example implementations in Mule and ServiceMix. Manning Publications Co., ISBN-13: 978-1933988214, 2009.

319