As many people correctly predicted, Web services and their corresponding service-oriented architecture (SOA) have proven their promise. Companies and businesses are starting to not only integrate new components and technologies, but are also opening up back-end systems and legacy processes. The former is helping to end compatibility and integration issues of the past while the latter is working to foster growth and more forward-thinking ideas, thus exempting the need to waste time on costly rewrites of existing technology.

Web services do not solely add benefits to individual enterprises; they also help to speed partnerships through quicker integration across businesses. This is a direct result of the abstract way a Web service describes itself, as it is almost completely agnostic of both medium and platform. So it is no longer a requirement for companies that wish to employ business to business (B2B) services to agree on a common language, infrastructure, or tool anymore. In this way, it is clear that Web services have been a primary player in a large paradigm shift, moving the emphasis from a technology-centric world to one that can focus purely on the business problem at hand.

The success of Web services and SOA means that companies want to build more complex systems, and need to construct blueprints detailing how the different systems link together.

A Map, Not a Route
Web services and SOA, as with any sophisticated architecture, add benefits as well as new complexities. You must realize that architectures are analogous to procedures. They are methods and processes for doing things. They describe how elements are to be bound together. Therefore, these technologies in and of themselves are not going to solve the problems of business integration overnight. They provide the pathways or the map, but not the specific direction in which to head nor the specific route to take.

Thinking About The Big Picture
Bear in mind, new technology is great, but it must be used with caution, care, a thorough understanding of its pros and cons, and knowledge of how it fits into "the big picture" ­ anything less is just asking for trouble. Businesses must understand that in order to produce a well-oiled overall product, each of its constituent parts must work well before the oil is applied. If each were constructed with care and detail, they will all come together nicely and operate cleanly and consistently with one another. However, if you lapse on even a single item, the smallest component, or a solitary procedure, it could spell trouble down the line when that piece is incorporated into a larger plan.

Furthermore, no matter how well architected a system is, if the parts that produce the whole have been neglected, the deficiencies are guaranteed to show up in the final product. A Web service doesn't hide implementation issues; it magnifies them and opens up the problem for all the world to see. In other words, no tool today is going to make something happen magically, or turn a poorly architected system into a wonderfully coherent and understandable one. To reiterate, Web services provide the pathways, but not the specific direction.

Making the Blueprint a Reality
Designing the blueprint for a complex, integrated architecture is not a simple task. However, there is a tool today that can significantly simplify the process. It's called WebSphere Business Integration (WebSphere BI).

WebSphere BI is actually a suite of products that can model and analyze scenarios at the business process layer, not solely at the application layer as the seasoned developer is used to. Through something known as the business process engine, applications are loosely coupled with one another in tremendously flexible ways. Now, instead of statically coding the order in which applications or processes are called, instead of writing custom wrappers or communications connectors, instead of arduously and awkwardly writing exception-handling routines at varying levels of granularity, WebSphere BI brings everything together. It is an easily understood and quickly adaptable environment that provides the business analysts and developers with a tool of immeasurable value.

Think of all of the factors that go into writing even small applications, which may or may not be part of a larger architecture. You need to worry about design issues, reusability and maintainability, exception handling and user-input validation, support for transactions, authorization and authentication, encryption, chokepoints that require optimizations, hidden and/or missed business rules, changing business rules, etc. The list goes on and on. The point here is that with such a diverse and complex set of issues that must be addressed, there is a high probability that mistakes will be made along the way. Moreover, because so many companies are finding ways to reuse each other's services to cut costs or to share data because of mergers or partnerships, there has been an increasingly pressing necessity to find a solution that quickly adapts to these highly dynamic business models. It cannot be stressed enough just how important understanding the big picture is.

Look No Further
Thankfully, the solution to process management comes in the form of a standard ­ BPEL4WS (Business Process Execution Language for Web Services ­ BPEL for short). A mouthful to say, it introduces an array of features that can help control the seeming chaos at the business level. BPEL does not introduce anything particularly new or unthought of. It does, however, introduce a new way of approaching old problems, with a reusable framework that makes describing intricate dependencies, complex error handling, and difficult synchronization a snap. BPEL still supports capabilities from the application layer ­ all different sorts of control flow mechanisms, timer activities, a publish-subscribe architecture, and fault handling ­ but what it adds on top of that layer is the important part. Below I will highlight what I think are the most important concepts of BPEL and IBM's extensions to the language the have simplified the choreography of traditional back-end processes.

Parallelism of Tasks
First and foremost is parallelism of tasks. Even in a highly object-oriented environment, this is much more easily said than done. In an ideal situation, all tasks to be executed would be asynchronous. That is, each process would be called and immediately return control to the main processing thread. Then, after all tasks of interests are started, a separate module could sit and listen or poll for responses from each of the individual tasks that were fired (see Listing 1).

However, what happens if one or more of the processes were synchronous? If a call into a method didn't immediately return, and was blocked until its completion? Essentially, it then requires the use of separate threads. After encasing each synchronous method in a lightweight thread wrapper (see Listing 2), they could be started and forgotten about just as in the asynchronous scenario (see Listing 3).

Any way you look at it, there is still going to be a level of indirection. A process will have to complete and then either explicitly notify the original caller of its results, or perhaps somehow write the result to a well-known location that the caller can later poll. This may not sound too bad for the case of a simple business process where the control flow is relatively uncomplicated. However, what happens in situations where a parallel series of tasks is nested within a sequential series of tasks, which is in turn just a subtask of another series of parallel tasks? For a very complex process this could be a veritable nightmare to program by hand at the application level.

There are two tools from the WebSphere BI suite that make dealing with parallel and complex nesting of tasks simple. A business analyst is going to tend to want to use the WebSphere BI Modeler tool while a developer would lean towards Process Choreographer, which is built right into WebSphere Studio Application Developer Integration Edition. Modeler is a high-level tool and does not attempt to map a process flow to any implementation, whereas Process Choreographer does. Aside from that, each tool provides a similar graphical front end, which makes working with parallel tasks a simple matter of visualizing the task and dragging and dropping components onto a palette.

Figure 1 shows a WebSphere BI Process Choreographer rendering of a complex business model including both parallel and sequential processing and nested tasks, some of which require manual intervention. It was written using Process Choreographer in less than three minutes! With just a few clicks and drags, a process can be structured with highly complex control flow, composed of both synchronous and asynchronous processes. Horizontal blue lines with a shaded gray background define the parallel processing parts, whereas those grouped within a solid white background define tasks to be executed in series.

When looking at it, one can clearly see that a great deal of the power and flexibility comes from the fact that links can be drawn explicitly from the output parameters of one service to the input parameters of another. In other words, the feedback mechanisms through polling and/or writing to a globally well-known location are all handled as part of the business process itself thanks to BPEL. The business process engine (the infrastructure that will eventually be executing the model) seamlessly keeps track of all appropriately labeled, contextually relevant data that needs to be communicated between two or more cooperating services, regardless of whether these services are local or hosted by external business partners.

Compensation Handler
Going back to the possibility of a highly complex business model, what happens if something goes wrong? Without going into as much detail as I did for parallel tasks, just imagine how difficult it would be to correctly write code to handle exceptions at the various levels of nesting for a larger process. If something fails along the way in one branch, have you adequately written a framework for notifying other branches that they should stop executing and begin the rollback process? Furthermore, have you been recording what you have done cumulatively up until that point so you can perform a logical rollback? Perhaps. I'm not saying it's impossible, and the experienced programmer might be bold enough to think it is not too difficult to handle such a case.

The BPEL language, however, has again come to the rescue with the compensation activity. Here a "reverse" action can be programmed for any given scope within a process. If some business rule fails or a fault is generated, a series of compensation activities can automatically take place, rolling back the process to some previous, stable state. The process engine ensures that the activities are called in the correct reverse order to completely undo all uncommitted or unfinished work up until that point, regardless of whether this happened in a process that spans seconds, hours, days, weeks, or even months. The duration of the process is not really an issue anymore; the tools focus on the interaction. You do not directly see the compensation handlers in Figure 1, but the handlers can be and are associated with some task or scope. And no, these compensation activities are not exactly the same as your good old exception handlers. As I said above, they are associated with long-running business processes, not just segments of code. They go on to further dwarf their historic counterparts with their ability to span disparate services boundaries.

It cannot be denied that time is saved through the ability to construct these models and all their levels of complex flow control through an intuitive, graphical environment, which can also explain why problems with maintenance would be dramatically decreased too. If a part of the model changes down the line and requires slightly different compensation activities, or needs more or less communication to or from another process, trying to look at the hundreds if not thousands of lines of code that it would take to accurately represent the entire model (had this been written by hand) should be frightening. Again, the fact that everything can be configured and quickly changed through this graphical representation not only helps to make it possible to write and maintain complex business processes, but also helps to make it simpler for fewer technical personnel, such as the business analyst, to validate and verify that each model correctly represents what happens or what needs to happen in real life. Remember how long it took for me to create the process in Figure 1? That is not necessarily uncharacteristic of the time it would take to update the model in the future. Did someone say "on demand"?

Forward Thinking
BPEL was never meant to solve every conceivable issue that you might encounter while writing the definition for a business process. It was only meant to catch most of them in a standards-based way. To that end, there would not be much reason to go with any one vendor rather than another unless one of them were able to differentiate itself in the marketplace and provide unique value to the product and an enhancement to the standard. IBM has done just that by introducing a series of powerful and necessary extensions to BPEL.

Staff Activity
In my opinion the most important extension is the staff activity. In standard BPEL, there is no notion of manual intervention in a process, nothing that mediates a human interacting with the system. However, this requirement will tend to occur time and time again and must be dealt with. For instance, consider a system that needs a final "yes" or "no" on a loan application before the funds can be reserved, or another system that requires a human analysis of the results of a background check before considering someone for employment. Using this extension, operators at various stages of a process can conveniently retrieve work from a queue, where it can be handled in absolutely any order.

Figure 2 shows you exactly how this queue is used. An employee could log into the system and first "claim" the task; in other words, that he or she is assigned the manual work and promises to perform the required user intervention. Once the line item is claimed, the person can choose to perform the work needed, view additional information about the current item, return it to the pool for a different employee to pick up, or even transfer it to someone else who might need to do more work on it. From a modeling perspective, Figure 2 shows just how easy this is to set up. The staff activity was created just like any other design element, and so is effortlessly dropped onto the palette and quickly incorporated into the complex process.

Furthermore, by integrating these manual activities into long-running processes, there is no concept of blocking in the traditional sense. The next process or service in the flow of the model is no longer waiting indefinitely to receive user data, wasting precious CPU cycles and forcing it to repeatedly switch contexts back and forth between services that have no data yet to receive. This common notion in complex models has been developed into its own feature with the full capabilities of any synchronous process (one that executes on its own, without user interaction) ­ the matter of synchronicity is now delegated to the staff personnel interacting with the system.

Java Snippet
Finally, the Java snippet cleans up where standard BPEL and staff activities cannot. Realize that the BPEL standard did not necessarily leave anything out. No standard, regardless of what technological idea it characterizes, should ever try to be too emcompassing. BPEL did its job of defining the core competencies of the solution and left the details up to the implementers to customize certain parts of the technology for what they believe would benefit their customers most, or what niches of the market they wanted to delve into. (Thus, the staff activity was born, but it is so ubiquitous and clearly needed that it really seems to have been accidentally left out of the standard, as opposed to strategically chosen to not be put in.)

For any other shortcomings of the BPEL standard, IBM solved the problem with the Java snippet. Each snippet is compiled as a method within a Java class that is attached to the actual business process itself. Since RMI isn't actually used in this case, a Java snippet can be a form of optimization. Yes, this entire article does revolve around the fact that Web services are great for reuse, integration, asynchronous modeling, etc. However, it cannot be ignored that Web services projects come at a particular cost. Deserialization and parsing of incoming messages, as well as serialization of outgoing messages (the three major steps in Web services communications) come at a strikingly high cost, compared to other existing methodologies. Sometimes, if the cost cannot be justified, or simply cannot be ignored because of some critical functionality or chokepoint in the system, IBM's BPEL engine seizes the day and lets you refactor that slow running process into a piece of Java code, which is executed within the same container as the processing engine, has direct access to any variables in scope in the process, and can still make calls out to external resources such as databases, Web sites, or EJBs, etc. There is complete flexibility.

Skyscrapers, not Mazes
This software has far-reaching implications and can dramatically reduce the amount of time needed to implement business models, regardless of their complexity. The most important thing to realize is that Modeler and Process Choreographer are tools that, in addition to building Web services, focus on their interaction. Building Web services in an ad hoc fashion will produce a product that might, more or less, resemble a maze. Each of the services represents a wall, and in this bottom-up fashion, hadn't been thoroughly thought about in terms of their position in the big picture. The WebSphere BI tools force the analysts and developers to create systems using a top-down approach, where you can gain fine control of the blueprint and build a truly impressive, finely polished structure ­ a skyscraper, not a maze.

Where to Proceed
As this technology is relatively new, there isn't an abundant amount of material on it. However, since I am a big advocate of Redbooks for self-paced study (and if you want to learn more about the product, its related products, and their full capabilities and features), I suggest you begin by heading to www.ibm.com/redbooks and downloading "sg246381 - BPEL4WS Business Processes with WebSphere Business Integration: Understanding, Modeling, Migrating." As of this writing it was still in draft form, but it gives an extensive overview of BPEL, its history, the standard, and of course, the IBM extensions. From the same site, if you search on topics such as "business integration" or "server foundation," you will find more documents describing how process choreography ­ as a general term, not the product ­ can be leveraged to produce systems capable of quickly adapting to ever-changing business models.