Web Services Directions

It's been a long time since the last XML in Transit column. Did you miss my musing on Web services? I doubt it. More likely, you were busy keeping up with all the new initiatives in the Web services space. Those of you with corporate responsibilities were probably wondering how to get some real ROI out of Web services. The more entrepreneurial of you were thinking how to make money in this new world.

I too have been trying to organize my thoughts. This is going be the first in a series of articles that go beyond the basic SOAP/WSDL/UDDI Web services stack to cover the innovation and market dynamics in adjacent areas that are of key significance to the evolution and ultimate success or failure of Web services as the new new thing in distributed application development, assembly, and integration.

The Basic Technology Stack
Figure 1 shows the basic Web services interoperability stack the way most people would think of it up to last year. Going from the bottom up, in a typical Web services scenario we have HTTP as the communication protocol, XML for data representation, XML Schema for data format specification, SOAP for service invocation, WSDL for service description, and UDDI for service discovery. This technology stack enabled the basic Web services workflow described in Figure 2. We've covered these technologies and the publish-find-bind processes in detail in past issues of XML in Transit.

What we get from the combination of these technologies is the ability to connect disparate systems. Okay, I'm simplifying things a bit, but I think the basic message holds true. The potential is there for doing much more than simply connecting applications. We could expose enterprise-quality Web services. We could orchestrate atomic processes into meaningful B2B applications. We could assemble entire new applications (front end to back end) from Web services.
We could, but as a whole our industry is quite far from this better world. We're held back by the combination of three factors: technological reality, standards friction, and market dynamics. I'll cover some of the technologies we need to put in place in this issue. In the next installment I'll finish off the discussion with a look at standards friction and market dynamics.

Technological Reality
Let's do a quick check of the various technological capabilities we'd need to have to achieve what's promised by the Web services hype machine: enterprise-quality seamless dynamic assembly and orchestration of Web services.

We can start by looking at what it means to have enterprise-quality Web services. Since enterprise requirements do vary, we'll focus on a relatively stringent requirement set.

Probably the most basic thing to think about is reliable transport. The overwhelming majority of Web services nowadays run on top of HTTP, a protocol with pretty poor reliability. The delivery of requests and responses isn't guaranteed. Certain exceptional conditions in the HTTP stack can leave a system in an undetermined state. We can use SOAP over any protocol; however, the industry as a whole hasn't agreed on how to do this. Therefore, if you want to use reliable HTTP (HTTPR) or guaranteed messaging, you can't count on interoperability. This is the main problem with the current state of Web services - pretty much anything is possible, but little beyond the basic technology stack is common practice.

In addition to reliable transport, enterprise-quality Web services often need to be transactional. This means that the systems exposed through Web services need to be able to participate in a transaction under the coordination of a transaction manager (TM, also known as a distributed transaction coordinator, or DTC). Distributed transactions used to be rocket science. With the proliferation of transactable DBMSs and managed components such as EJBs, we're getting into a world that can deal reasonably well with transactions. However, transactions pose at least two problems for Web services:

1. SOAP messages need to carry transaction identifiers (TIDs) that associate the message with a currently executing transaction.

2. Distributed transaction management requires that the systems participating in transactions have a back channel of communication about the transaction itself, independent of the actual Web service operation.

There are some approaches for Web services transaction management. The leader is probably the Transaction Authority Markup Language (XAML) but the industry buy-in is not overwhelming. There also are hybrid approaches that use the divide and conquer pattern: put TIDs in SOAP messages but do transaction coordination out-of-band without the burden of Web services. The Transaction Internet Protocol (TIP) could enable this approach.

In environments where Web services are used for automating business processes, system implementers face another problem - long-running transactions. Without going into implementation details, suffice it to say that the basic approaches for managing transactions that take on the order of a few seconds to a few minutes do not scale to handle transactions that take days or weeks to complete. Long-running transactions have more complicated management cycles. This affects Web services by requiring agreement and standardization of additional APIs, SOAP headers, and WSDL extensions. OASIS has done some interesting work on the Business Transaction Protocol (BTP), which could be a good base for broad standardization in this space because it covers both cross-enterprise transactions and long-lived transactions.

Another requirement for flexible business process management is the customizable routing of messages. Therefore, Web services routing is another area that needs standardization. Microsoft's WS-Routing is a step in the right direction, but it lacks industry support.

Before I get off the topic of business process automation, I have to point out that if business processes are exposed via Web services, then we need a set of standards defining the required APIs of these services together with a complete description of a process in an XML format so that composite business processes can themselves be exposed as Web services.

The good news is that much work has been done in this area. Microsoft developed XLang for BizTalk Server. IBM developed Web Services Flow Language (WSFL). The bad news is that the merger of the two specifications into one (something that IBM and MS did well in the case of NASSL and SCL/SDL that became WSDL) is not in sight.

Wait, I haven't had the chance to talk about security and privacy. At a basic level we need at least three capabilities:

1. Associate credentials (username/ password, X.509 certificate, a Kerberos ticket, an authenticated session identifier) with Web services messages.

2. Maintain message integrity to guarantee that nobody has tampered with the message content.

3. Maintain message confidentiality by preventing anyone other than the author and the target recipient from seeing the contents of the message.

These basic capabilities are well understood in the security space, and we've built (over and over and over again) infrastructure to provide them. Web services pose two challenges: (1) the need for widely adopted standards and (2) the complications of intermediaries that may need to see some parts of SOAP messages but not others. All the pieces are in place for the industry to agree on a solution to this problem; we just haven't gotten around to doing it.

In fact, the various companies and industry groups have produced a number of specs and are busy working on others. The W3C has XML Signature, XML Encryption, and XML Key Management Specification (XKMS, consisting of X-KRSS [XML Key Registration Service Specification] and X-KISS [XML Key Information Service Specification]). Microsoft has combined some of these into WS-Security, a reasonably complete approach to Web services security covering credentials association as well as message integrity and confidentiality.

Meanwhile, OASIS is working on a security assertions markup language (SAML) that will help describe trust assertions for user identities and authorizations, something that becomes important in assembling aggregate services and in automating business processes via Web services. Needless to say, the fact that so many specifications are in development and that they are split between different companies and standards organizations will delay adoption of more universal standards.

Another huge topic of interest is quality of service (QoS) and the requisite monitoring and management capabilities that enable QoS metrics and tuning. This is a large area with unfortunately too little coordinated action related to the Web services space. There are two types of QoS that we need to be concerned with:

1. Technical QoS, which encompasses concepts such as availability, accessibility, integrity, performance, and reliability
2. Business-level QoS (called by some quality of business [QoB]), which focuses on metrics related to the business processes supporting a service - for example, inventory levels, shipping guarantees, and insurance

Providing targeted QoS significantly complicates the architectures and workflows of using Web services. Consider the following that need to be in place:

• Common vocabularies for defining service-level agreements (SLAs) as collections of QoS assertions.
• Extensions to UDDI to include QoS parameters in service discovery.
• Agreed-upon patterns for QoS negotiation between service requesters and service providers.
• Pervasive monitoring of all components (technology- or business-level) taking part in a transaction. This requires common APIs and message formats that will likely be exposed via Web services.
• Management infrastructure that can alert interested parties when QoS parameters change.

This is a hot area where we can expect significant innovation and much competition for market dominance between the established leaders in the Web services space.

The Future Technology Stack
Putting all of this together, it seems that the Web services technology stack of the future looks much more like what is shown in Figure 3. It's somewhat difficult to represent the dynamics of the Web services space in two dimensions, which is why I resorted to the use of color as a third dimension. The green zones denote calmer areas where the standards are already well established and/or vendors and standards bodies are collaborating reasonably well. The yellow zones are areas where standards are still being fleshed out and there's more uncertainty. The red zones define areas where interoperability is a long way away because standards are incomplete and there is high likelihood of platform battles. The blue zones are areas yet to be explored in an organized way.

If history offers any lessons from the software space, we must expect at least a couple of turbulent years full of standards flux. While the basic message of Web services is that of interoperability across disparate systems, if you're building enterprise-quality Web services, you should prepare to fund a lot of custom integration work between various types of Web services. We're starting to see the beginnings of this period: Web services produced by some of the major vendors' enterprise systems are not interoperable. You know, on some gloomier days I think that God is a systems integration consultant.