Its all about closing the loop from conception to execution and back.

82 March 2004 QUEUE

rants: feedback@acmqueue.com

BPM
LAURY VERNER, PROACTIVITY

The Promise and the Challenge

ver the last decade, businesses and governments have been giving increasing attention to business processesto their description, automation, and management. This interest grows out of the need to streamline business operations, consolidate organizations, and save costs, reflecting the fact that the process is the basic unit of business value within an organization. The design and automation of business processes even warrants its own field of study, known as BPM (business process management). A quote from IBM Systems Journal sums it up nicely: BPM technology provides not only the tools and infrastructure to define, simulate, and analyze business process models, but also the tools to implement business processes in such a way that the execution of the resulting software artifacts can be managed from a business process perspective.1 The level of interest and the concomitant marketing hyperbole around BPM has reached a crescendo. As an example, Gartner announced that BPM wins the
more queue: www.acmqueue.com

Triple Crown of saving money, saving time, and adding value. Is this promise being fulfilled? BPM does have the potential to deliver significant value, but there are missing elements that limit its effectiveness. Although BPM technologies are becoming more mature, many software developers and business analysts still find themselves asking the basics: What is it? And why should I care? This article provides a high-level overview of BPM and where it is today and touches on some of the core technologies and standards. The focus is on four specific challenges facing BPM, which are aligned with the four phases of the application development life cycle:2 Discovery refers to having an understanding about the current environment. Design involves the analysis of the current processes and the definition of future state processes. In the development phase the future state is implemented in technology, which may involve process automation using a BPMS (business process management system). The new processes are put into production in the deployment phase. QUEUE March 2004 83

BPM
In each phase we explain the problem and explore its business and technical consequences.

The Promise and the Challenge

HOW DID WE GET HERE?

Business process management is an old discipline that allows you to model the organizational structure, define the business processes, and show the interactions between them. It is traditionally taught in business schools and is put into practice with varying degrees of success. In the 1990s the work of Michael Hammer and James Champy,3 authors of the widely read Reengineering the Corporation, focused attention on business processes, both as a root cause of inefficiency and as the source of potential competitive advantage. They advised a deep and radical redesign of the business to root out waste and increase the focus on the customer. That was 10 years ago. What is different today is the novel use of computing technology to drive the analysis and automation of business processes. In Beyond Engineering, a follow-up to Reengineering the Corporation, Hammer defines a business process as a complete end-to-end set of activities that together create value for the customer.4 The notion of customer in this context refers to the recipient of the value provided, not necessarily a paying customer in a commercial transaction. Innovations in technology such as XML, Web services, component-based development, and message-oriented middleware have fueled the current interest in BPM. Vendors have developed BPMSs that provide the fine-grained integration of systems and data needed to automate business processes. The BPMS links people and systems, manages information access and transformation, handles exceptions, and orchestrates the flow of the process. Organizations are looking to BPM to help solve the following kinds of problems: A computer vendor, to be competitive, needs to cut its costs for fulfilling customer orders for PCs by 70 percent. A pharmaceutical company seeks to extend the patent
84 March 2004 QUEUE

life of its drugs by bringing new products to market one month earlier. A government agency forced to reduce staff by 30 percent must find a way to consolidate and streamline its unemployment benefits services. Company executives, personally accountable under the Sarbanes-Oxley Act of 2002 for their companys financial statements, need to understand and control the processes that result in these statements.

CHALLENGE 1: DISCOVERY

In order to improve business processes, you must begin at the beginning. Typically, business analysts, needing to discover the current state of things, try to visually represent the various processes of the business. The approach taken by many BPM products fails to address this challenge adequately. They employ a drawing metaphor in which an analyst or developer sketches the process using a palette of standard icons. This assumes the existing process is known in advance. Most large organizations, however, simply do not know their end-to-end processes accurately or in detail. Their process knowledge is tacit and decentralizednot explicit and centralized. How can you discover how a business operates? Classical manufacturing processes have been analyzed extensively in quantitative and qualitative terms. Discovering general business processes is somewhat less straightforward. You can adopt either a top-down or bottom-up approach. The top-down discovery approach typically begins with the organization chart. It lists the responsibilities of each department in the organization and identifies the high-level processes that support these responsibilities. These processes are then decomposed into lowerlevel processes, which are decomposed further, until the lowest level is reached. The advantage of this approach is that it provides a broad, organizational perspective. Its disadvantage is a lack of detail and a questionable degree of accuracy. The bottom-up approach begins by interviewing employees about the day-to-day activities they perform
rants: feedback@acmqueue.com

and attempts to integrate this local information into They provide few if any capabilities for process discovery, coherent end-to-end processes. This approach can be defining meta-data, simulation, or analyzing process costs extremely accurate but you can easily get lost in the or cycle times. details, failing to see the forest for the trees. Some hybrid top-down/bottom-up approaches seek to achieve the CHALLENGE 2: DESIGN advantages of both methods. The ultimate purpose of BPM is to improve and optiSince no two organizations are exactly alike in how mize the operations of an organization. Thus, its scope they operate, different discovery methods may be appronecessarily comprises not a single process but all processes priate in different businesses. Further, a single capture of across the organization. Unfortunately, BPMS products business processes is likely to be woefully inadequate. As are generally designed to work on one process at a time. is so often the case, later discoveries inform earlier ones, The developer draws the process, adds implementation and an iterative discovery methodology that continuconstructs, and then executes the process. There is no ally enhances and updates the processes may yield better way in these tools to look across multiple processes, results. Regardless of the methodology followed, there is examine process interconnections, make comparisons, or of course the key requirement that senior management perform analysis. support and drive the business-process discovery. Without It would be helpful to have a process laboratory of this support, the chance of success is minimal. sorts, where business analysts could collaborate, explorBusiness Consequences. As awareness of the imporing the process space, testing ideas, measuring, analyztance of business processes grows, many companies ing, and comparing processes, and generally performing are attempting to capture their current-state business business thought experiments. This laboratory would processes. They form workshops of business users, sketch give analysts the tools to design new processes, view the the processes, and try to achieve consensus within the processes from multiple points of view, extract analytical team. Unfortunately, they have no way to ensure the reports across processes, generate system requirements, accuracy or completeness of the resulting diagrams. and perform simulation and what if experiments. It Often the underlying complexity of existing business would allow them to create centralized reusable processes processes is oversimplified by such workshops. Much of that could be invoked by processes in different operating the important meta-data about the processes, such as units. cost, cycle time, and information flow, cannot be easily fit Obviously, one key element in such a laboratory is a into Visio diagrams. Moreover, the information contained language to express ideas. This process language would in the processes cannot be analyzed effectively in these be used to define the basic entities of business processes diagrams. (processes, participants, activities, links, etc.) and the Technical Consequences. Accurate capture of current rules for their operation and interaction. A standard proprocesses is a prerequisite for defining cess language would allow customers the structure and rules of the desired to use products from different vendors future process. Without this level of for defining and implementing busiunderstanding, the developer may ness processes. Processes defined in today is produce a suboptimal solution with one product could be executed on the BPMS tool or, worse, a solution to another product. the novel use of the wrong problem. BPMS tools were Over the past three years, various computing technology designed to automate processes, not to groups have made numerous attempts to drive the analysis analyze business. The process automato define standards for Web services and automation of tion implemented using a BPMS may and business processes. The relevant therefore fall short of the true goals of organizations are the Workflow Manbusiness processes. the business. Why not ask the busiagement Coalition (WfMC), Business ness analysts to use the BPMS prodProcess Management Initiative (BPMI), ucts to discover and analyze business World Wide Web Consortium (W3C), processes? This would seem to solve and OASIS.5 this problem. Unfortunately, most WfMC promotes the use of workBPMS products have been designed for flow by helping to establish standards developers, not for business analysts. for terminology, interoperability, and

What is different

more queue: www.acmqueue.com

QUEUE March 2004 85

BPM
connectivity between workflow products. It has created xpdl, an XML-based definition for business processes. The mission of BPMI is to promote and develop business process management by establishing standards for process design, deployment, execution, control, and optimization. BPMI has developed three standards: Business Processing Modeling Language, Business Process Modeling Notation, and Business Process Query Language. Founded by Tim Berners-Lee, W3C has published the key standards for XML, HTTP, HTML, SOAP (Simple Object Access Protocol), and Web services. All W3C standards must be royalty free. OASIS (Organization for the Advancement of Structured Information Standards) is a not-for-profit, global consortium that drives the development, convergence, and adoption of e-business standards. OASIS has defined standards for e-business applications (ebXML) and is working on other high-level Web services standards. In contrast to the W3C, standards published by OASIS may have royalties attached to them. An important XML-based language for defining business processes is the Business Process Execution Language for Web Services (BPEL4WS or simply BPEL6), a language created by Microsoft, IBM, and BEA Systems. It grew out of work done by Microsoft (XLANG) and by IBM (Web Services Flow Language). Microsofts approach was inspired by structured programming, whereas IBMs approach viewed a process as a directed graph. BPEL attempts to integrate these two viewpoints. The OASIS Web Services Business Process Execution Language Technical Committee is charted to continue work on

The Promise and the Challenge

partner

message

process engine

the business process language originally published in August 2002. The underlying assumption behind the BPEL specification is that business processes will be composed of a series of interacting Web services. Since WSDL (Web Services Description Language) is the natural language for describing Web services, the BPEL designers chose to make BPEL an extension of WSDL. Each activity in a BPEL process is implemented by a Web service, which is defined by its port types, operations, and messages. In the conceptual scheme of BPEL, a business process is composed of a central process engine (see figure 1) that interacts with a set of business partners. Each Web-service operation is performed either by one of the partners or by the central process engine itself. The process engine communicates with its partners by exchanging messages. The process engine and partner send messages across a communications channel called a service link (see figure 2). To make this more concrete, let us consider a travel agency interacting with an airline. The service link is a bilateral contract between the travel agency process and the airline that defines the services each offers to the other. The process and the partner play different roles across the service link. The process, in its role, exposes a WSDL port typethat is, a set of operations that it agrees to offer. Similarly, the partner, in its role, exposes another set of operations. In our example, when the travel agency process invokes the airlines ticket-order operation, it is actually using a service link, which we have called the buyerLink. The role of the travel agency process in buyerLink is ticket requester and its port type is itinerary. The role of the airline partner is ticket service and message partner its port type is ticket order. Having defined this conceptual framework for partner interaction, BPEL goes on to specify the usual

FIG 1

86 March 2004 QUEUE

rants: feedback@acmqueue.com

building blocks of processes: activities, flows, links, data containers, and assignment operations. These are defined in terms of their XML structure, but without a graphical model for representing them. The influence of Microsoft and IBM is apparent in these definitions. For example, if you want to specify that activity 1 must precede activity 2, you have two ways to do so. One way is to use the structured activity called sequence. The other way is to connect these activities through a link within a flow. The first approach is inherited from Microsofts XLANG, and the second is inherited from IBMs WSDL. The BPEL specification provides no hints as to when to use one technique or the other. BPEL also addresses technical constructs required for proper execution of the business process. These include correlation, fault handling, and compensation. Correlation is the technique used to create associations between process instances by using the data fields as identifiers. For example, a field order number might be used to correlate a purchase order and a purchase-order acknowledgment. Fault handling and compensation specify the procedures to be followed when an error occurs in the process. For long-running processes, the idea is to undo a complex series of activities with a compensating series of activities. In BPEL, each process is an assemblage of Web services, but the process is itself a large-scale Web service. This fractal-like approach enables unlimited composition and decomposition of Web services. BPEL is a significant achievement but has several weaknesses that limit its widespread adoption: Addresses only processes composed exclusively of Web services. Has no graphical rendering. To date, no available products generate process graphics from a BPEL document or vice versa. Does not include a framework for performing top-down designthat is, for creating a process in a series of layers with successive refinement. Provides no capabilities for process analysis. Is a vendor-driven process definitions language that has not yet been reflected in a royalty-free standard published by a recognized standards group. The major competitor to BPEL as a process language based on Web services is BPML (Business Process Management Language). Both BPEL and BPML are ultimately based on the calculus, but the Business Process Management Initiative introduced BPML two years earlier than BPEL. BPEL has actually incorporated many BPML concepts and, with the support of Microsoft and IBM,
more queue: www.acmqueue.com

process

airline

FIG 2

itinerary port type

ticket order port type

it has the advantage of industry momentum. Microsoft, IBM, and BEA Systems will likely introduce BPEL in their products over the next year, perhaps with some proprietary language extensions and some tools to aid in process design. Business Consequences. The goal is to translate the process information gathered in discovery into a standard process language, such as BPEL. But business analysts cant use BPEL, at least not in its current form. Given a BPEL process definition, you will find it nearly impossible to disentangle the business logic from the details of the implementation. The business semantics are obscured by the technical details required for execution. What is lacking is a way to layer BPEL, to filter out such technical constructs as fault handling, correlation, and transaction management. It should be possible for a business analyst to use a process language to design the business logic. Once the business logic is mapped out, a developer can insert the technical constructs. Business analysts generally prefer a visual way to design processes. They are intimidated by code, which tends to obscure the business issues. Also, nondevelopers need to see processes from multiple perspectives, to determine inter-process dependencies, and to extract system requirements from processes. Business analysts would also likely benefit from a way to design processes from the top-down, beginning with high-level processes and refining them to low-level processes. This lies outside the scope of BPEL. Because BPEL has no graphical rendering, the business analyst would have to code XML. This is not likely to happen. Technical Consequences. In many cases the business analysts will design the business process using an analysis tool. Once this is done, how will the process then be automated? It would be desirable to export the process to the BPMS for execution, perhaps using BPEL as the lingua QUEUE March 2004 87

BPM
franca. If BPEL is not used by the business process analysis tool as well as by the BPMS, then a custom mapping will be required to translate between the XML dialects of the two products. In its current form, BPEL will be of limited use since it is designed for processes that are implemented using Web services. It is not usable for legacy processes that comprise package applications, custom applications, or manual activities.7 This excludes 95 percent of all existing business processes. A key goal of business-process design is to define and communicate requirements. For example, suppose a new system will be used in multiple processes, supporting different users and systems performing different functions. Ideally, the business processes are defined in such a way that the functional requirements can simply be extracted from the process definitions. If our new system performs 25 activities across 17 processes, then these activities can be summarized by an appropriate query. This procedure is precisely analogous to extracting data from a relational database. Unfortunately, there is currently no searchable knowledge base of BPEL processes. Moreover, since BPEL has no notion of participant or actor to identify the proposed system, it will not be able to support this important goal of the process laboratory.

The Promise and the Challenge

CHALLENGE 3: DEVELOPMENT

A successful development project is the result of many favorable conditions, one of the most important being close collaboration between business analysts, who define the needs of the business, and developers, who implement systems that meet these needs. Business analysts and technical developers, however, are driven by different goals, speak different languages, and tend to work at different levels of precision. In the domain of BPM this gap is manifested by automated business processes whose execution does not match the original business requirements. Business analysts typically communicate business needs in the form of requirements documents to the developers, who may interpret the needs rather differ88 March 2004 QUEUE

ently. Using a BPMS tool, the developers implement the automated solution as they understand it. Why not have the business analysts define the business process using the BPMS tools? This is not realistic since BPMS products are generally intended for use by developers, not by business analysts. They enable developers to create technical constructs, not business requirements. Tools based on UML (Unified Modeling Language) are sometimes suggested as an alternative. UML is well suited for developers who need to design class diagrams and lay out the interactions between method calls. The UML suite of diagrams is, however, not so well suited for business analysts working with business processes. Business Consequences. The business consequences of the gap between business analysts and developers are increased risk of failure and longer lead times for development. In its research into the success rate of application development projects, the Standish Group discovered that most software development projects are cancelled before they are completed.8 It also found that fewer than 20 percent of projects are completed on time and on budget. Of those projects that do deliver on time and budget, half fail to deliver the projected functionality. The Standish Group notes that a key cause of such failures is the lack of clarity in capturing and communicating user requirements. Technical Consequences. The classical requirements document leaves considerable room for interpretation. It rarely provides the IT level of precision needed by developers, who need to know each activity that must be performed by the system, the step-by-step control logic of the enveloping business process, the specific data required at each step of the process, and the business rules that govern changes to the data. This lack of precision may lead to misunderstandings between the analyst and the development team. Software misses the mark, making scrap and rework inevitable. Redefining requirements, redesigning, and recoding midstream are expensive and time-consuming. What is missing is a seamless way to integrate design
rants: feedback@acmqueue.com

BPM
and development. Business analysts should create process designs at the business level within their process laboratory and then export these designs in XML form to the BPMS. Developers will then refine the design, adding the technical constructs needed for implementation. This eliminates any ambiguity about the requirements and business need.

The Promise and the Challenge

CHALLENGE 4: DEPLOYMENT

The whole point of automating business processes is to improve operationsin cost, time, or quality. Once a process has been developed and deployed, how can we know if it is meeting the intended goals? We know how to instrument IT systems and monitor them with a high degree of precision. These statistics, however, do not generally provide a business-process context around this information. The challenge is to aggregate and present execution data at the business-process level. Gartner coined the term business activity monitoring (BAM) for this capability. It defines BAM as providing realtime access to critical business performance indicators to improve the

analysis

discovery

design

FIG 3
90 March 2004 QUEUE

execution

speed and effectiveness of business operations.9 Business Consequences. Without BAM, operational managers have no way of determining whether the processes for which they are responsible are meeting their objectives. For example, they will not be aware that the cost of the order fulfillment process in the Cincinnati region has increased 20 percent above average, or that the time required for handling new benefits claims declined by 10 percent in the second quarter, or that the outage optimization process for steam turbine generators is in trouble. Lacking this information, executives have no way to determine which action to take. A way to aggregate execution statistics in process context would help the business manager better manage these types of exceptions. Over the past year, several companies have developed BAM products, but in many cases they are discrete event monitors that lack overall process context. To achieve true process context, you must link individual activities into a process to provide information on what is done in that process and by whom. For example, it is necessary to evaluate the cost of each process instance. Moreover, it is necessary to aggregate process instance information based on various criteria. For example, you can group process instances by geography, customer, or organization. Finally you need to chain processes that logically belong together, such as an order process and an invoice process. All this information should be summarized and presented on an executive dashboard on the enterprise portal. Technical Consequences. At some level, most companies recognize the importance of BAM but have no effective way to collect, aggregate, and analyze execution statistics. Often it is done in an ad hoc manner, in which reports from legacy systems are combined into a data warehouse from which summary reports can be generated. It is possible to determine quite precisely the utilization of each disk drive, server, and network component in the IT environment. From such statistics, however, you will not know which resources need to be expanded, consolidated, or upgraded to support the business objectives. For example, you simply may not know whether or
rants: feedback@acmqueue.com

not increasing the capacity of a specific disk will affect the order-fulfillment-cycle time. With BAM, we come full cycle. The results of process monitoring will enable the rediscovery and redesign of business processes. Executives will know about hotspots that demand their immediate attention. In the longer term, the execution will keep pace with the business needs and the process designs. Figure 3 summarizes this life-cycle concept.

CONCLUSION

against plan. With this information, they can then redesign or optimize the processes. Gradually, the technologies and techniques described here will change the way businesses and governmental agencies apply technology. In the words of Howard Smith, Third-wave business process management methods and systems will utterly transform the way companies conceive, build, and operate automated systems.10 Q

Any enterprisecorporation, government, or nonprofit organizationcan be viewed as the sum of its business processes. Each process delivers value to customers, suppliers, employees, or other stakeholders. BPM, the discipline for enabling and automating business processes, is in a period of rapid growth and will fundamentally change the way computing power is applied in organizations. Whereas BPM has already delivered considerable value in many companies, the components of the full BPM solution are still evolving and are the subject of ongoing research and development. One noteworthy advance has been BPEL, an XML-based language for describing business processes composed of Web services. This article has focused on four immediate challenges: 1. Process discovery is the beginning of any BPM solution and is necessary to ensure that the solution matches the real business needs. 2. Business analysts lack a process laboratory in which to design, analyze, and simulate business processes. Process languages, such as BPEL, while key elements in a process laboratory, need to be enhanced with graphical capabilities that link them with process discovery. 3. Integration is missing between the tools used for business process design and the tools used for execution. BPEL may play a key role in this integration. 4. BPMs generate valuable performance statistics from executing business processes. Businesses need to monitor these execution statistics, organize them into their process context, and present them in the form of alarms, reports, and executive dashboards. Weve made significant progress in BPM over the past three years. We expect many of these challenges to be addressed in the next three years. Others may prove less tractable and will take a little longer to solve. Once this happens, for the first time we will have a complete, closed-loop approach to business processes: from conception to execution and back. This will give executives the ability to design their business processes, automate them, and determine quantitatively how well they are doing
more queue: www.acmqueue.com

REFERENCES

1. Leyman, F., Roller, D., and Schmidt, M.-T. Web services and business process management. IBM Systems Journal 41, 2 (2002) 198. 2. CSCs Catalyst 4D development methodology. 3. Hammer, M., and Champy, J. Reengineering the Corporation. New York: HarperCollins, 1993. 4. Hammer, M. Beyond Reengineering. New York: HarperBusiness, 1996. 5. Workflow Management Coalition: see http:// www.wfmc.org; Business Process Management Initiative: see http://www.bpmi.org; World Wide Web Consortium: see http://www.w3.org; OASIS: see http: //www.oasis-open.org. 6. Business Process Execution Language for Web Services, Version 1.0. IBM: see http://www.ibm.com/ developerworks/library/ws-bpel/. 7. Unless they have been previously wrapped as Web services. 8. The Standish Group. The Chaos Report (1994), http: //www.standishgroup.com/sample_research/chaos_ 1994_1.php. 9. McCoy, D. Business activity monitoring (BAM)deeper meanings, Business Integration Journal (Aug. 2003), http: //www.bijonline.com/Article.asp?ArticleID=755. 10. Smith, H., and Fingar, P. Business Process Management: The Third Wave. Tampa: Meghan-Kiffer Press, 2002. LOVE IT, HATE IT? LET US KNOW feedback@acmqueue.com or www.acmqueue.com/forums LAURY VERNER (lauryv@proactivityinc.com) is chief technology officer of ProActivity, a Boston-based company that develops software for business process analysis and design. Prior to that, Verner was a partner at Computer Sciences Corporation Consulting, specializing in application architecture. Verner has a Ph.D. from the Johns Hopkins University and a B.A. from the University of Pennsylvania.
2004 ACM 1542-7730/04/0200 $5.00

QUEUE March 2004 91