Integration

Digg This!

There are a handful of enterprise application integration (EAI) products aimed at helping small, medium, and large businesses integrate enterprise assets to reduce costs and improve communications. This article is the first of a two-part series introducing IBM's EAI solution, WebSphere Business Integration Server, and its benefits. It is first essential to understand the advantages of a hub-and-spoke architecture, particularly in comparison to point-to-point integrations.

With years of expertise conducting EAI training sessions and implementing customer integration solutions, we have seen a gradual increase in the awareness of the financial and technical benefits of replacing point-to-point programs with a hub-and-spoke architecture. While many readers do not need to be convinced of the benefits of EAI, more often than not we find ourselves drawing a hub-and-spoke architecture on whiteboards for company decision makers. This tells us that the benefits of using a hub-and-spoke solution are not yet well understood throughout corporations worldwide.

Eventually, an understanding of hub-and-spoke will be widespread and its advantages over point-to-point interfaces will be obvious to all serious interface engineers. This article is an attempt to bring us closer to that day.

This first article focuses on the benefits of a hub-and-spoke architecture. The second article focuses on WebSphere Business Integration Server and discusses how the principles of a hub-and-spoke architecture are consistently reflected in several different aspects of the product. It is surprising how frequently hub-and-spoke is the right answer to common integration problems.

The overall goal of this series is to describe how WebSphere Business Integration Server is a simple, clean, modular, and easy-to-understand tool for obtaining a hub-and-spoke integration solution.

Hub-and-Spoke vs. Point-to-Point

In this section, we will detail some of the benefits of a hub-and-spoke architecture.

FEWER INTEGRATION PROGRAMS
A visual comparison of hub-and-spoke and point-to-point should make the most obvious benefit of hub-and-spoke apparent: there are fewer dependencies to maintain.

In Figure 1, there are five applications at CompanyX: appA, appB, appC, appD, appE. These applications could be specified as actual products such as SAP for manufacturing, Siebel for sales, PeopleSoft for HR, Clarify for support, and an internal application, developed in-house for CompanyX's niche market. For the purposes of this article, however, we will name the five applications appA, appB, appC, appD, and appE.

QUESTION: If every application needed to communicate with every other application using a specialized point-to-point program, how many total point-to-point programs are needed? For example, one program is needed to move data from appA to appB, and a second program is needed to move data from appA to appC.
ANSWER: All 5 need to talk to the other 4, so 5x4=20.

Figure 2 demonstrates this model. Note that only 10 lines are shown, but each line can actually be thought of as being two programs. The program that transfers data from appA into appB could be vastly different from the program that transfers data from appB into appA.

In general, if n is the number of applications, n*(n-1) is the total number of programs required for a complete integration. Depending on the company and its objectives, it is not likely that every application must talk with every other application, yet some require multiple inputs. For example, the inventory received and inventory shipped input might come from the manufacturing system. For this reason, the upper bound of n*(n-1) remains a good estimator of the approximate number of interfaces for a well-integrated company that is leveraging its data.

A hub-and-spoke design for integrating these five applications will look much simpler. As diagrammed in Figure 3, each of the five applications, appA, appB, appC, appD, and appE, is a spoke and each connects with only one other application, the hub.

Instead of 20 programs to integrate all of the applications, there are 5. To be consistent, because each line was counted twice in the point-to-point diagram, the hub-and-spoke solution should be thought of as having 10 connections rather than 5. For the purpose of comparison, moving data from appA into the hub and moving data from the hub into appA should be thought of as two separate programs.

Perhaps comparing 20 programs with 10 programs does not look like a huge savings, only cutting the total number of integration programs in half? More to the point, this ratio might not look attractive enough for a company to replace a "working" integration solution with new software that implements a hub-and-spoke solution. However, as enterprises grow, it is important to consider the incremental costs of future enhancements to your integrated environment.

LOWER COSTS OF MAINTENANCE
The ratio of 2 to 1 changes quickly in favor of a hub-and-spoke solution when adding a sixth application. With the point-to-point solution, the total number of programs grows from 20 to 30 (6x5 = 30). For the hub-and-spoke solution, another connection into the hub counts as 2, making a total of 12 (10+2 = 12). Adding just one application, the ratio has gone from 20:10 to 30:12 (that is, from 2 : 1 to 2 _ : 1). Adding a seventh application should complete the justification from the mathematical viewpoint, as the point-to-point solution requires 3 times as many programs as the hub-and-spoke solution. Consider how often your organization adds new applications or replaces existing systems.

DECOUPLING OF APPLICATIONS
One general reason to adopt a hub-and-spoke design instead of numerous point-to-point interfaces is the decoupling of applications. Instead of applications relying on communication with the other applications, the only dependency is the connection with the hub. The freedom provided by such an anonymous system results in applications maintaining their independence.

Besides providing anonymity, the decoupling of applications also allows for asynchronous communication. From the point of view of an application, once the data is sent to the hub the effort is complete; it's the hub's job to make sure integration is successful. This enables each application to continue executing what it was deployed to do instead of spending cycles verifying the success of the data integration. A simple example of asynchronous communication is the process of sending an e-mail. The e-mail is sent without knowing or needing to know if the receivers are currently available. Without this feature, it would be practically impossible to use e-mail to correspond with a group on a project.

ACCOMPLISHING MORE THAN DATA INTEGRATION
In the hub-and-spoke design, all data flows through one place - the hub. It provides an opportunity to do more than just integration, for example automating a business process. Consider attempting to determine how many customers - who have paid for unlimited support - are contacting a company early Monday morning compared to late Friday afternoon. Depending on the applications, this could be quite a challenge. In fact, corporations are likely to limit their ideas to match the boundaries of their software design. With the hub-and-spoke solution, due to its centralized core of information gathering, this request is implemented with little effort, easily accomplished by a nondeveloper in a single afternoon.

To assist in such tasks, the WebSphere Business Integration family of products includes tools to model, monitor, and manage these business processes. The tools enable you to design, simulate, and justify changes to your current business processes, as well as measure and improve the new processes once they are in a production environment.

EASE OF GENERAL ADMINISTRATION OF INTEGRATED APPLICATIONS
Depending on the environment, the costs of handling system failures could outweigh the costs of maintaining the integration software. It might be quite difficult to detect and recover from errors that occur while extracting or inserting data to and from software applications that run on various computers.

For example, consider the situation if the computer that hosts appC goes down at some point during the execution of the point-to-point integration programs. It could be before, after, or during the integration. This disastrous situation results in either corrupt data or in the difficult chore of attempting to figure out exactly which data from the other applications (appA, appB, appD, and appE) made it completely or partially into appC, as well as from appC into the other applications.

A hub-based environment centralizes all of the data and processes concentrating on the handling of the hardware and software failures.

FLEXIBILITY TO UPGRADE OR REPLACE APPLICATIONS
Considering the number of interdependencies that exist for any one of the five applications in our example, simply upgrading to a new version of the application could require modifications to four different point-to-point interfaces. With the hub-and-spoke design, the task of updating or replacing any application involves only the interface between it and the hub. As was detailed earlier, even more evident is the ease and flexibility gained when adding and integrating a new application into the environment.

Having explained some of the benefits of hub-and-spoke over point-to-point, you might wonder if using point-to-point interfaces is actually a working solution. Obviously, point-to-point works, and has for dozens of years at many companies. The difference between today and previous decades (when many enterprise applications such as PeopleSoft were introduced), is the reality of the coexistence of numerous enterprise applications along with the "latest-and-greatest" applications such as Internet-savvy, custom e-Business software.

One scenario in which it is beneficial to use a point-to-point program is if there are only two applications. In that case, point-to-point requires fewer integration steps than hub-and-spoke.

Point-to-Point: A Closer Look

Figure 2 is actually a simplistic view of point-to-point programs - a single line is drawn from appA to appB. The program to move data from appA to appB is in itself quite complex. As shown in Figure 4, from the highest level the program to integrate data from appA to appB involves three major pieces: extracting data, transporting data, and inserting data.

This is once again quite oversimplified, as the task of extracting and inserting data could be challenging. Issues such as security and navigating firewalls could make the transporting task far from trivial. Finally, overlooked in the diagram is the task of converting the data from appA into the format expected by appB.

To explain the task of converting data, consider simply one common piece of data, a customer's name. One application might store the first name and last name in two separate fields, while another application might store the full name in one field, using a space to separate the first name from the last name. Yet another application might store full name in one field as last name, space, first name. This provides us with enough background for a discussion of converting data, but it is worth pointing out that "Dr. Jane J. Doe III" and similar names provide an opportunity for a far more involved discussion than would fit in this article.

Data Conversion

Returning now to the task of converting data from appA's format to appB's format, let's assume appA stores first names and last names in two separate fields and appB stores the full names in one field as last name, first name. As questioned in Figure 5, between which of the three major steps does it make the most sense to do this conversion?

One might think the answer is obvious, but major EAI vendors do not agree on exactly where and when to do the conversion(s). For this reason, they have different architectures and different abilities.

Remembering that beyond appA and appB are appC, appD, and appE, the complexities of using point-to-point to integrate applications again become completely evident. An appA-to-appB conversion program is needed as well as numerous similar-yet-different conversion programs from appA into appC, appD, and appE.

In a hub-and-spoke solution, the first realization is that conversion will never be directly from one application to another: appA data is not converted to appB data. Instead of converting data directly from appA to appB, appA's data is converted into the hub's format. The next step is to convert from the hub's format to appB's format. It might appear wasteful to write two programs instead of one to convert data from appA to appB. However, it is important to consider the reduced effort when appC, appD, and appE are integrated into the picture.

Data Representation in the Hub

The final decision to make is to figure out what format to use for the hub. Returning to the discussion of representing first and last names as separate fields, or as one field using either a space or a comma and a space to separate first names from last names, Table 1 lists example formats for appA, appB, appC, appD, and appE.

Given this setup, which would be the ideal representation for storing the customer name in the hub? Because there is a conversion of data to the hub format, the hub format must have a definition, so the question is which format should be chosen for this example?

When implementing a hub-and-spoke solution, it is essential to focus on the overall design of the hub. Often, too much attention is placed on the integration of the data, reducing the role and effectiveness the hub can offer in the hub-and-spoke design.

The hub-and-spoke design is valuable not only to reduce the total number of applications that must be maintained, but also to provide a single, centralized place to which all data flows. The benefits of this could be enormous. Companies become better equipped to gain access to corporate data to analyze and improve their current business processes. Organizations can also add business processes between the applications, not just pass data, such that the applications work together as if they were indeed one unified application. For example, sales from early Monday morning can be compared with sales from late Friday afternoon. Even better, if sales on Fridays are above projections, the process can take action immediately and allocate appropriate resources (for example, e-mailing an executive or manager, or automatically sending purchase orders to appropriate trading partners).

Keeping an eye on the bigger picture reveals an opportunity to greatly improve and evolve yesterday's business processes. Returning now to the question of which data representation to use in the hub, appA, B, C, D, or E - but keeping an eye on the opportunity to use the hub as the owner of business processes - it becomes clear who drives the definition of the hub's data. The designers of the business processes use the hub's data definition, keeping the business process application independent.

In this example, the business processes are owned by CompanyX, so the representation of a customer's name should be determined by CompanyX's business processes, completely independent of the appA, B, C, D, E specifics.

It would seem that we are asking a lot from CompanyX. Not only will they phase out numerous point-to-point programs and replace them with a new hub-and-spoke system, they must also define the company's representation of a customer as well as specify their business processes that use this new data representation. This is where the WebSphere Business Integration family of products provides an exceptional offering. Together with the software and tools required to obtain a hub-and-spoke design, it provides out-of-the-box business processes and the corresponding data representations.

WebSphere Business Integration Family

The WebSphere Business Integration family of products enables you to optimize your current business processes and integrate your disparate applications.

MODEL, MONITOR, AND MANAGE
The WebSphere Business Integration Modeler and Monitor provide you with the tools required to design and simulate business functions and processes, and then measure and improve your performance dynamically against your business objectives.

The modeler provides a graphical interface to detail current processes and associated costs, and design new processes, simulating various optimization choices in order to project which provide the best business benefits. A great benefit of modeler is its automatic flow from the business analysts to the integration programmers, generating code that is imported into IBM's workflow engine.

The monitor provides customizable dashboards and reports that reflect how the modeled business processes are working in the production environment. These processes can then be changed dynamically to improve effectiveness and provide for real-time business resource reallocation.

Integrate and Connect

WebSphere Business Integration Connect is an ideal solution for B2B requirements, enabling you to connect with your business partners to reduce costs and improve productivity. Internal business processes can be reused with your trading partners and customers.

At the heart of the enterprise and at the heart of this article is WebSphere Business Integration Server. The design of WebSphere Business Integration Server revolves around prebuilt out-of-the-box business processes, referred to as "collaborations." These collaborations are based on years of experience helping customers solve real problems and noticing commonality and patterns among various solutions. As shown in Figure 6, the WebSphere Business Integration Server hub becomes a container for a company's business logic.

IBM also recently released WebSphere Business Integration Server Express, which is designed to integrate and automate business processes for small and medium businesses.

Conclusion

Replacing expensive point-to-point programs with a hub-and-spoke architecture enables a company to compete using the power of today's software technology. A hub-and-spoke architecture provides much more than reduced long-term software maintenance expenses; it is an opportunity to benefit from an overall application design that introduces a centerpiece for managing all business processes.

The foundation has been set for a better understanding of the benefits of implementing a hub-and-spoke solution using WebSphere Business Integration Server. The prebuilt collaborations, designed to automate common business processes, are at the root of the overall design. The second part of this series of articles explains collaborations in greater detail and introduces the other major components of WebSphere Business Integration Server.

For more information about WebSphere Business Integration solutions, attend our Prolifics WebSphere Business Integration Server Webinar and Workshop series at www.prolifics.com/new/events.html.