Think Async

Sep. 26, 2003 10:45 AM

Within the IT industry, we're seeing a steady shift from an RPC (remote procedure call) integration style to asynchronous, document-based integration. More and more developers are realizing that asynchronous services are core to a new, loosely coupled and message-driven architecture. In practice, while most developers and architects I work with are sold on the value of an asynchronous architecture, it introduces a steep learning curve that makes it more difficult for them to take the next step. Having seen the good, the bad, and the ugly sides of many integration projects, I'll try here to pass on some experiences on the benefits, common hurdles, and best practices of implementing asynchronous services and the systems that support them.

The main benefits that are enabled by asynchronous services and the associated loosely coupled, message-based architecture are:

Reliability: A synchronous integration style results in brittle applications where the weakest link in a chain impacts an entire application or business process. Reliable integration requires asynchronous interactions.

Scalability: Message-oriented middleware vendors and developers have shown that queues provide for maximum throughput and efficiency when connecting systems with different processing and availability capacities. The same approach can be taken with all integration projects by leveraging asynchronous services.

Long-running services: Integration projects frequently bring the greatest value when automating business processes that involve both manual processes and automated systems. Any service that supports or requires manual intervention must have an asynchronous interface due to the time it will likely need to complete.

Taking a real-world example, a developer needs to build a bidding service and wants the benefits of an asynchronous, message-driven architecture. What are the problems he or she might run into?

Designing an asynchronous interface: The goal is to think about the interface to the service as a message-driven/document style interface. The first step is to use XML Schema to describe the documents that will be passed into and out of the service. Similarly, the output of the service should not be thought of as a return value and exception(s) but rather as callback messages. In the XML Web services world, WSDL and PartnerLinkTypes (which specify conversational callback messages) are particularly helpful.

Coordinating reliable/asynchronous conversations: Asynchronous, conversational architectures with a service can be highly reliable but introduce some new sproblems that are typically not issues for synchronous, RPC-style integration. For example, when it is time to send a callback message to the client of a service, what address should it be called back at, and how should the callback message be correlated with the initiation message? The WS-Addressing specification, promoted by the same software vendors behind the BPEL specification, provides a standard answer to these questions, or a custom solution could be crafted using content-based correlation. Another problem surrounds the reliable delivery of messages over less fault-tolerant protocols like HTTP. A well-behaved asynchronous service should both support a reliable message delivery standard (such as WS-ReliableMessaging) to enable once-and-only-once guaranteed delivery of messages as well as idempotence (the characteristic that sending the same message multiple times has the same effect as sending it once).

Managing transactional integrity: In an asynchronous world, services will need to support compensating transactions (also known as long-running transactions) rather than the traditional XA model of locking transactions. This means that services should implement pairs of operations (doXXX and undoXXX) so that a client of the service, or a transaction manager, as standards like WS-Transaction take hold, will be able to cancel a fully "committed" operation if necessary. A related facet of this problem is that of exception management and task services. In an asynchronous environment, people and manual tasks can be inserted in the workflow of a service or business process when appropriate. Frequently this enables support of existing manual processes for exception cases when automating a flow, and speeds adoption (and thus ROI) of the automated system.

As you can see, there are many standards (BPEL4WS, SOAP, WSDL, WS-Addressing, WS-Transaction, etc.) that assist in implementing asynchronous architectures. These standards provide a framework for an asynchronous architecture and bring significant value in both the design and implementation phases of an integration project by supporting asynchronous interactions as first-class citizens. But all of these standards and technology are the second step - the first step is for developers and architects to learn to be more comfortable designing asynchronous, loosely coupled services and systems.

About Edwin Khodabakchian
Edwin Khodabakchian is the CEO of Collaxa. Previously, he was the CTO of the AOL eCommerce divison where he took an active part in the Merchant Integration, Shopping Search, Wallet and AOL 6.0 products. Before the acquisition of Netscape by AOL, Edwin worked at Netscape as the chief architect of the Netscape/iPlanet Business Process Manager, a product he successfully drove from inception to release.

YOUR FEEDBACK

Think Async

Edwin Khodabakchian wrote: Mike, You are making a good point: there are problems that are better addressed using a tighly-coupled, synchronous architecture/design patterns. And those classes of applications are not going away. It seems to me though, that J2EE and JCA are well equiped to address those requirements. Where J2EE/JCA fail short and the reason why XML web services are emerging is that there is a class of integration application that would be very brittle if designed with a synchronous, tightly-coupled glue. I do not think that one single architecture will address the need of all classes of applications. Different patterns result in different trade-offs.