Really Simple Asynchronous Web Services

Really Simple Asynchronous Web Services

(http://www.computerworld.com/developmenttopics/development/story/0,10801,79698,00.html)

 

March 25, 2003 (Computerworld) -- The first word in SOAP (Simple Object Access Protocol) is "simple," but the world of Web services has become very complicated. In early 2000 the SOAP specification was published, and it was quickly followed by the SOAP with Attachments; Universal Description, Discovery and Interoperability (UDDI); and Web Services Description Language (WSDL) specifications. These core technologies have been very widely adopted, implemented and endorsed. Then came a plethora of proposals for additional functionality, positioned as extensions to the core technologies, such as WS-Security, WS-Transactions, WS-Routing, BPEL, WS-Coordination, HTTP-R and WS-Reliability.

It's not yet clear exactly what applications Web services are really going to be used for, and therefore it's not really clear whether or not all these additional specifications and technologies are really necessary. Most business applications have fundamental requirements around security and reliability that represent a kind of minimum criterion for adoption. Many, if not all, of the proposed extensions fall into these two major categories. But these basic requirements can actually be met using a very simple approach in which a lot of these complicated extensions aren't necessary.

Most current Web services applications are used in a connection-oriented manner. That is, in order to use the applications over the Internet, a connection is required between the service requester and service provider. But for the mobile traveler, and for the emerging world of wireless networks, always being connected may not be possible or practical.

In the world of high-speed, wireless networking and mobile Internet-enabled devices such as laptops, personal digital assistants (PDA), cellular telephones and automobiles, it is unlikely that a connection will always and constantly be available. A really simple solution works well in this environment, insulating users from worrying about connectivity loss. If a connection is present, the message is immediately transferred. If a connection is not present, the message is queued up for later transfer.

Connections also pose security risks because they represent a hole in the firewall. Many Web sites impose security checks through log-on popups and encrypted protocols to help prevent unwanted access to sensitive or valuable data, and to guard against intrusions. But this is a constant battle, and many viruses and other security breaches often occur despite the best precautions. A simple solution improves the situation through the use of file and directory protection, and by allowing an intermediary data store to be placed outside the firewall, thereby avoiding the need for direct connections.

The most popular transport protocol for Web services, HTTP, doesn't provide the type of message delivery guarantee business applications depend upon. A really simple alternative solution provides message delivery guarantees by using the file transfer protocol (FTP) to implement the type of reliable store-and-forward techniques found in message queuing middleware.

Figure 1 illustrates a simple solution to Web services connectivity, reliability and security issues by mapping SOAP messages to files and using FTP to transfer them over the Internet. By treating SOAP as files rather than on the wire streams, the file system can be used to implement a store-and-forward mechanism that improves reliability and allows applications to work equally well whether an Internet connection is present or not.

The solution is completely compatible with all Web services standards: WSDL can still be used to provide the SOAP message description, and headers can be added for any of the newly proposed specifications. The only difference is that the resulting SOAP message (which is by definition an XML file) is written to a file instead of to a network transport.

Figure 1, The Simple Solution Maps SOAP Messages to Files

File-based security protection can be added, including read- or write-only directories, access control and embedded digital signatures. Furthermore, an FTP directory server can be placed outside the firewall, or in the demilitarized zone (DMZ), eliminating the need to allow direct HTTP connections for Web services that access sensitive corporate documents and data.

Applications built using this approach can take advantage of richer client features for the end user, since the user is completely decoupled from the need to know whether or not a connection to the Internet is present at a given moment. Web services-based applications can take full advantage of features and functions in client-side technologies, such as Flash, Swing and Visual Basic graphical user interfaces (GUI), and interact with large files and graphic images more easily and with better performance.

Why Complexity Exists

The core specifications SOAP, WSDL and UDDI are widely adopted and implemented. They provide basic interoperability across widely disparate software domains and networks, and are sufficient for many applications. SOAP and WSDL are also defined to encompass the two major distributed computing communications paradigms: remote procedure call (RPC) and document or message-oriented interactions (the terms document and message are used synonymously in this article). But most developers are more familiar with the RPC-oriented interaction model of distributed computing, or prefer it, and the tendency to view Web services as RPCs creates most of the complications.

Figure 2 illustrates the difference between RPC-oriented and document-oriented Web services. RPC-oriented Web services implement the request/response communication paradigm, which is typically used for short-lived requests. Document-oriented Web services typically are used to implement longer-running interactions such as process flows. Technically, however, document-oriented interactions can be used for short-lived requests, and vice versa.

The request/response communication paradigm is a major feature of modern distributed computing environments, including CORBA, COM and Java 2 Enterprise Edition. Most new development of customized applications is carried out using one or more of these environments, and the RPC-oriented paradigm is easy to use and familiar to programmers. To a developer, an RPC is easy to use because it makes calls to remote programs and objects using the same basic language syntax used to perform local procedure and object calls.

Figure 2, Contrast Between RPC-Oriented and Document-Oriented Web Services

Document-oriented Web services, on the other hand, are much better matched to the store-and-forward communication paradigm common to message-oriented middleware systems such as MQSeries, Sonic MQ and MSMQ. Developers are not used to thinking of these systems as mechanisms for calling remote procedures and objects, although in practice they often are. Developers are also unused to dealing with data in the form of documents or messages (procedure or method arguments are the more typical form).

The majority of Web services tool kits, books, articles and products on the market today focus primarily on RPC-oriented Web services for these and other reasons. But the fact is that SOAP and WSDL are just not very well suited to RPC-style interactions.

RPC-oriented technology assumes the continual presence of a network connection, while message-oriented middleware works reliably whether the network connection is present or not. On the Internet today, connections are neither reliable nor guaranteed. And as the world moves more and more toward mobile connections, the reliability service guarantees are more likely to resemble a wireless telephone network than an office LAN.

RPC-oriented technology also typically depends upon a sharing state between the service requester and provider using a mechanism called a session or a conversation. Web services do not support this mechanism, meaning that typical add-on services to RPCs such as security, transactions and guaranteed delivery don't work as they do in existing RPC-oriented technologies and have to be reinvented for Web services. The misperception that Web services are well suited to RPC-oriented interactions leads to a lot of frustration, and a lot of effort in the Web services community to date has focused on filling the gaps between SOAP and "real" RPC-oriented technologies.

An equally valid approach, however, is to focus instead on the document-oriented, store-and-forward technologies for which Web services are much better suited. Instead of adding features and functionality to improve the fit for Web services as RPC-oriented technology, it is actually much easier and more productive to add features and functionality that improve the usage of Web services for message-oriented applications. SOAP messages, after all, are just XML documents, and are more simply transported as files than on the wire streams.

The Document-oriented Solution

Because Web services are better suited to the document-oriented style than that RPC-oriented style, it makes more sense to compare them to message-oriented middleware systems MQSeries, Sonic MQ and MSMQ. The document-oriented style of communications solves more of the barriers to adoption with the core standards than the RPC-oriented style does.

It's easy to think about entire SOAP messages as files, since they are fundamentally instances of XML documents, possibly with attachments. And therefore it's easy to think of using file transfer mechanisms as a very simple transport for SOAP messages. For this reason, most vendors have already implemented mappings of SOAP to message-oriented middleware such as the various Java Messaging Service (JMS) providers, MQSeries and MSMQ. However, these solutions are proprietary in the sense that they require the vendor's product in order to work.

A very simple, inexpensive and open mechanism for file transfer already exists, however, because every operating system has a file system, and the Internet has FTP, which altogether provides a worldwide, virtual file system for transporting SOAP messages.

All computers, whether laptops, desktops, specialized devices, mobile phones or PDAs, are capable of dealing with files. A very simple mechanism for transporting Web services requests, therefore, is to use ubiquitous file management capabilities to transport the messages from the source to the destination, and then execute the request once the destination is reached. Copying files is much simpler, more secure, and more reliable than dealing with a stream of bytes over the network.

From the point of view of the client application, or Web services requester, all that has to be done is write a file to a directory, something almost any application can do. A network connection is not required. Clients that can write XML files simplify things even more. Once a file is available in a local directory, intelligent Web services software can import the file, add a SOAP envelope and headers (if not already present), and transfer the XML file to a remote FTP site where it can be picked up by the service provider, parsed and executed. If everyone is connected and online, it happens immediately. If a connection is not available, it happens later, when a connection is available.

A Web services request can be forwarded to one or more additional intermediate directories, typically remote from the original client, before the request is ultimately executed. Responses typically flow back through the system following the reverse path: The response message is placed in a local reply directory and forwarded to the remote system for processing.

Treating SOAP messages as files and using a message-oriented architecture, reliability can be introduced at the transport level very easily. A file is not deleted from its source directory until it has successfully been transferred to its target directory, and so on.

File protection security can be added to the messages very easily. Directory protection can ensure that files are transferred in one direction only, target directories can be write-only, and source or input directories can be set up to be read-only. Furthermore, file transfer greatly reduces the need for logging on required by HTTP connections. A file protection password can be included in the file header, or in an FTP operation. Digital signature can easily be added to protect the contents of the XML file itself. A simple additional security precaution can be accomplished by placing the FTP server in a DMZ, or by using a trusted third party to host the FTP server completely outside the corporate firewall.

An Example of Product Architecture

An example of the simple architecture for handling asynchronous, document-oriented Web services interactions is described below.

Figure 3, Simple Asynchronous Web Services Architecture

In Figure 3, the left side of the dotted line illustrates the occasionally connected client components (that is, client applications that are tolerant of intermittent Internet connection), and the right side illustrates the server components (including Web services choreographies, content management systems and integrations with various back-end applications).

The client and server sides are connected through the gray box at the bottom labeled "Asynchronous SOAP Messaging (FTP, HTTP, JMS . . .)." The functionality represented by the gray box allows SOAP messages to be stored in files locally on the desktop if a connection is not available to the server side. In this architecture, the occasionally connected client can reliably send a Web service request to the server even though there may not be a connection available between the client and server when the Web service call is made.

In this scenario, the desktop application generates a service request to execute some service on the server side. The Intelligent Web Service Agent picks up the service request, which is responsible for generating a self-contained SOAP message and storing the message in a persistent store if no connection is available. When a connection is available later on, this SOAP message can be transferred immediately to the server side and executed.

The asynchronous SOAP message transport layer handles the transportation of this SOAP message reliably no matter whether the connection is available or not. The benefit of this architecture is that the client-side applications can continue to work whether they are connected to the Internet or not. Client-side applications therefore don't have to worry about being connected and can take advantage of rich local tools, easily work with large files and take part in long-running business process orchestrations.

Another benefit of this loosely coupled architecture is that the asynchronous SOAP messaging layer can be based on different transports, such as FTP, JMS and HTTP, and easily replaced depending on different application requirements. The architecture is kept as generic as possible so that the asynchronous transport layer can be easily replaced by different transports as required. The HTTP transport (or a specific synchronous transport such as Java Remote Method Invocation) can be used instead of the FTP transport, for example, if an application needs it.

In this loosely coupled approach, the interface between the desktop application and the client uses "incoming" and "outgoing" file folders. When a Web service request or document is dropped into the outgoing folder, the Intelligent Web Service Agent picks it up and does the necessary processing to generate a valid SOAP message with header information. The SOAP message is persisted to the persistent store by the agent in order to reliably make the Web service call later on when there is a connection available.

On the other side of the persistent store, the FTP transport agent, which is responsible for detecting the network connection, uploads the SOAP message to the FTP intermediate persistent store once a connection is available. Then the Web service message is downloaded from the FTP site to the server side by the server application and processed.

In the reverse case, the same FTP transport agent can download any SOAP response message from the FTP site whenever a connection is available and persist the server's response message to the local persistent store. The Intelligent Web Service Agent picks up the SOAP response from the persistent store, unwraps the SOAP response message and processes the content by placing the response message in the Incoming folder of the desktop client application.

Summary

File-based transfer mechanisms provide a really simple way to implement asynchronous Web services, resolving common concerns such as occasional connectivity, security and reliable messaging without adding the complexity of extended standards. File-based transfer mechanisms are consistent with current and emerging Web services standards.

Considerable frustration with current Web services technologies arises when comparing them with RPC-oriented middleware since by definition Web services are not well suited to the RPC-oriented interaction style. Web services are much better suited for the message-oriented interaction style. Extending Web services toward message-oriented middleware architectures provides a better way to improve short-term value than proposing complex RPC-oriented extensions.

Client-side applications benefit from the simple asynchronous architecture since they do not have to worry about whether or not they are connected to the Internet; and they can use better GUI tools, work with large documents and participate in long-running business process orchestrations.

Eric Newcomer is chief technology officer at IONA Technologies.