A lot has been said about the opportunities presented by service-oriented architectures (SOAs), especially their ability to enable business flexibility in an interoperable, technology-agnostic manner. But little has been said about verifying the functional quality of these SOA applications. As many organizations start delivering their first SOA applications, they realize that the flexibility they're gaining dramatically impacts the way software quality has to be addressed. The days of well-known test configurations are gone. A successful SOA is always in flux, and flux is the enemy of quality assurance engineers!

SOA is about making the IT fabric adapt quickly to business needs. This agility impacts the way organizations tackle how they verify their applications' quality. We will discuss the dynamic nature of service-oriented development and its implications for ensuring SOA applications' quality. We will highlight and describe the key activities needed to verify and test SOA applications. And we will stress the importance of addressing quality early in the lifecycle and explain how test automation plays a key role in this new operating environment as well as the need to do on-going tests once applications are deployed.

Testing SOA applications is not a trivial variation of traditional testing. SOA is blurring the line between development time and runtime, changing the very nature of what we used to call an application. An SOA application is constructed by composing and orchestrating loosely coupled services. A key SOA architectural principle is that service consumers have no knowledge of their service providers' implementation, making it virtually impossible to identify the closure of an application. In other words, trying to "box" an application is like pulling strings in the dark: You don't know when you'll get to the end of it.

Figure 1 shows a simplified view of an SOA application with its different layers. At the bottom, application interfaces representing business functions are exposed as services. These services get composed into higher-level business services that get choreographed to deliver the business value of an SOA application right at the top.

Of course, the reality of an SOA is not a clean, layered stack of services, but rather a network of services. Compared to traditional component-based apps, this makes incremental integration testing more challenging, and calls for thorough unit testing of individual services. As for any component-based application, an SOA application quality is as good as its weakest link. Figure 2 exemplifies the potential impact of poor-quality services as they get integrated into higher-level services. The higher in the stack, the more likely services will fail due to the poor quality of an underlying service; the more difficult it becomes to analyze the problem. Testing services as units is not an option.

Thoroughly Testing Services Is Not an Option
Unit testing services is obviously a critical activity in ensuring the functional integrity of an SOA application. To a great extent the unit testing of services is similar to testing regular software components. But one needs to address some specifics of an SOA application when it comes to testing services.

First, interoperability testing is an important step in the unit testing process. As SOA applications comprise a number of services developed in an organization or provided by third-party vendors, a number of integration issues may arise from incompatible interpretation of services messages. Taking the example of SOAP-based Web Services, it's highly recommended to verify services implementations against pre-defined profiles.

Tools exist to facilitate this task. For instance, the Web Services Validation Tooling is an Eclipse technology project (see www.eclipse.org/wsvt/index.html), which delivers capabilities to verify Web Services against profiles defined by the WS-I Organization (www.ws-i.org/). Specifically, service specifications captured in WSDL, as well as SOAP messages, can be verified, preventing a number of interoperability issues that could arise during integration.

Another important difference between services and regular software components lies in the various usage models services can expose for their consumers: from the simple Remote Procedure Call usage model to Request-Response to Fire and Forget, etc.

When dealing with Remote Procedure Calls, the testing approach can be very similar to regular unit testing. Proxies generated by development tools such as Rational Application Developer can be leveraged to test a service much like a regular class, using simple testing frameworks such as JUnit.

But even with RPC services, the nature of the unit testing needs to be a little different because services and regular Java classes are different:

1. Basic services are often stateless.
2. Services are designed to promote loose coupling, hence they expose a limited number of operations (verbs) with fairly granular data

Equivalence partitioning is a suitable technique for creating data-driven tests. Glenford Myers described it for the first time in the "The Art of Software Testing." The technique is simple enough to be applied manually, but Rational Application Developer simplifies it by providing dedicated tools to edit and verify the test data. Figure 4 shows a screenshot of a test data table.

Besides RPCs, request response is a very common usage model for services. It enables simple short-lived asynchronous execution of services. The challenge with such model is that the service response comes in the form of an invocation of a service operation with some meaningful data. Meaning that the test driver needs to stub this operation to do the appropriate data verifications. Figure 5 shows a test configuration for a service using a Request-Response model.

Similarly, fire and forget presents unique challenges. The invocation of a service generally won't have any visible effect on the consumer side, but instead will perform some action on the back-end such as a database update or a service invocation. In some cases the test driver could directly verify these actions by accessing a resource that was modified by the service under test. But very often one needs to stub out external services as shown in Figure 6 to verify the interactions between the service under test and some other components or services.

In reality, most usage models require stubbing of services or regular components, but not simply dummy stubs. They need to be smart enough to simulate behaviors, to check data, etc. A number of Open Source frameworks can be leveraged for this kind of stubbing. The most popular ones are EasyMock and MockObjects. Rational Application Developer adds a number of capabilities to these frameworks by enabling a quick definition of stubs behavior based on data partitions. Very much like business rules get defined in Websphere Business Integration Modeler, one can simply define the behavior of a stub based on the data values used by the consumer and associate a response to this input.

Independent of these usage models, testing services doesn't stop with testing the operations a service exposes. An SOA application will likely rely on services developed by other development teams as well as third parties. To facilitate the integration between the services developed by a given team with externally provided services, robustness testing must be done on service consumers. Figure 7 shows how stubbing can be leveraged to do service consumer testing.

Stubbing definitely plays an important role in the process of testing services. But its usage doesn't stop there.

• Figure 3