Simeon Simeonov

Subscribe to Simeon Simeonov: eMailAlertsEmail Alerts
Get Simeon Simeonov via: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn

Related Topics: XML Magazine

XML: Article

The Interoperability Stack

The Interoperability Stack

For the past six months we've looked in detail at the nuts and bolts of XML protocols, Web services, and XML data encoding. These are the foundation technologies of next-generation Internet distributed applications. In the next several months, I'll focus on another, no less important area - higher-level description, discovery, and integration services.

These technologies are catalysts for the adoption of Web services. Without them, businesses will never be able to leverage Web services to their true potential.

The Need for Higher-Level Interoperability Services
Even the simplest example of Web service use shows the need for higher-level interoperability services. Consider the example in Figure 1 in which a user application uses a SOAP-based Web service back end to retrieve the latest stock prices of various companies. From a pure SOAP perspective, the interaction is trivial. However, things get much more complicated when we consider how much common understanding must exist between the user application and the Web service back end.

For the interaction to succeed, at a minimum, the user application needs to know:

  • The address of the web service is www.webservicebackend.com.
  • It should make requests and receive responses over HTTP.
  • It should use SOAP.
  • Requests and responses should use a specific schema.

Throw in security, payments, error handling, and whatever else, and the need for shared knowledge expands even further.... How can the user application acquire this information? How is this done in practice? Traditionally web service capabilities have been advertised in some human readable form. Application engineers have read these descriptions and configured user applications to communicate with particular web services.

While this approach works in principle, it's not scalable for many reasons:

  • It requires costly (time and money) manual configuration by highly skilled, therefore scarce, personnel.
  • It's error prone because it doesn't utilize formalized service specifications.
  • It precludes automatic discovery and engagement of web services; a priori knowledge is required for configuration of the user application.
  • There's no built-in mechanism for change notifications and/or failure recovery; every time a web service evolves, there's a risk that existing user applications will fail.
These are some of the reasons why industry leaders are looking at developing the specifications for, and implementing, higher-level interoperability services. The best way to analyze these is to look at the interoperability stack for Web services.

The Interoperability Stack
One of the key attributes of the Internet is the heterogeneity of approaches it enables. For better or worse, companies everywhere are taking the basic building blocks of Internet applications - TCP/IP/HTTP/SMTP, XML, SOAP - and coming up with distinctly different approaches to similar types of problems. This diversity, fueled primarily by customer and competitive pressures to innovate rapidly, is paramount to the success of the Internet as a new communication and commerce platform.

At the same time, however, diversity creates an integration challenge that has limited our ability to fully leverage network effects on the Net. The key question is how to enable interoperability without stifling innovation and diversity. The lessons learned from the Web so far have been that businesses and users are willing to rally around simple, extensible, and well-targeted specifications. The idea of the interoperability stack emerges out of this realization.

Figure 2 shows my personal view of the interoperability stack. On the left side are the seven core interoperability layers. On the right I've listed some relevant specifications/efforts; the bold-faced ones are getting the most attention right now either because of particular industry dynamics or because they've superceded some of the other specifications on the list.

In previous columns, we've mostly looked at the first four layers of the stack. At the foundation are the native Internet communication protocols. TCP/IP is a foundation technology in all of them. HTTP supports request-response interaction scenarios best. SMTP is best for basic messaging. Believe it or not, a lot of B2B gets done by moving documents over FTP.

With the exception of pure binary content, structured data is typically represented using XML. With the introduction of XML namespaces, we achieved the ability to mix data from different formats in the same document. The format for data encoding is described in some type of a schema language such as DTDs or XML Schema. RDF has been successfully applied to relationship-oriented data structures. Other specifications, such as XMI, for example, are used to map preexisting data models, described using UML to XML.

The web service layer focuses on infrastructure for XML distributed computing. It addresses different patterns of interaction, intermediaries, envelope and extensibility frameworks, error-handling models, and higher-level capabilities such as security, transaction management, and audit trails. A lot of early XML protocol efforts have been superceded by second-generation XML protocols such as SOAP. W3C's standards effort in this space - XML Protocol (XP) - takes SOAP as a starting point but will look at a slightly broader scope than the original SOAP specification. The ebXML activity has the lofty goal of addressing key features of enterprise-level web service scenarios, a very exciting effort indeed.

The top three layers - service description, discovery, and integration - will be the topic of the next several columns of "XML in Transit". This month, I'll outline the space.

Service Description
As we all know by now SOAP (and the upcoming XP) offers simple yet extensible mechanisms for packaging information and bindings for various protocols, so the information can be moved between systems. SOAP provides a basic wire format for web services; it allows requests and responses to be encoded for transport and moved across system boundaries. SOAP, however, offers no capabilities to describe the shared knowledge that must exist between a web service user and a provider of that service for a successful interaction to take place.

From our simple example in the beginning we can see that there are at least four fundamental pieces of information that are needed to know how to interact with a web service:

  1. The types of data that are changing hands: In our case this would be the symbol element for the request and the price and currency elements of the response.
  2. The operation that needs to be performed: In our example that was GetLastTradePrice.
  3. The rules for accessing the service: In our case, the service was accessible via HTTP at the root-level URL.
  4. The exact location of the service provider: In the example we used www.webservicebackend.com.

Furthermore, since it's likely that the same service may be exposed over various protocols, it's important that (1) and (2) be specified abstractly, then bound in a particular way for any given set of access rules (3).

The specification that's meant to address these issues is Web Services Description Language (WSDL). At the time of this writing WSDL is in release 1.0. However, the technology is reasonably mature because it came out of a merger of several revisions of thinking between Ariba, IBM, and Microsoft (Service Control Language [SCL], Service Description Language [SDL], and Network Accessible Service Specification Language [NASSL]).

Service Discovery
If we have the ability to describe Web services we'll be better off than we are now, but we'll have solved only a part of the web service integration problem. Service descriptions tell us how to access web services, but where and how do we get hold of a service description? We need some form of a service discovery mechanism, the target of our discovery being a service description. A directory or search engine for web services will probably do.

There are many different ways to search for web services. We could do simple searches by name (think white pages) or by the type of services (think yellow pages). For types of services we can use various category hierarchies of business types (e.g., the NAICS U.S. government industry codes or other properties such as geographic location). A new type of search involves business-level attributes such as availability and speed of service. It's likely that digital marketplaces will provide these types of search capabilities, because they're likely to have this type of detailed information about the business entities providing the service.

Service providers will need some sort of a registration mechanism so they can provide information about the web services they're offering and make changes as their web services evolve. Further, for scalability reasons, service providers need to have some advertising capabilities for their web services. It's likely that there will be many web service registries and, with new registries coming online, it will be foolish to put the entire registration load on web service providers. Web service registries should have mechanisms to automatically discover the presence of web services in the same way that search engine web crawlers can automatically discover new content on web sites.

The main industry effort in this space is Universal Description, Dis- covery, and Integration (UDDI) managed via UDDI.org. UDDI provides specifications for associating service descriptions with white- and yellow-page-type information about the business entity providing the service, and the service itself. The members of UDDI.org have implemented several registries and defined SOAP-based APIs for searching and modifying the information in them. UDDI doesn't address advertising of services. Currently there are two specifications - Advertisement and Discovery of Service (ADS) by IBM and Discovery of Web Services (DISCO) by Microsoft - that take rather different approaches to this problem. Both companies are working on a common way to solve the problem within future revisions of the UDDI framework.

Service Integration
At the top of the interoperability stack is the elusive promise of seamless, automatic service integration. The orchestration of web services poses significant challenges from both technical and business perspectives.

On the technical side, seamless service integration requires a significant technological foundation. Most important is the description of service behavior, defined by the sequence and rules for sending and receiving messages. Next is the problem of composing services into process-based interactions. The problem is made harder by the requirement that some composition bindings must happen at run time. Without this capability, the technology can't map well to well-established business processes such as representation, referral, and brokering.

On the business side, the problems are no less significant. From a business perspective, service integration is a workflow problem and as such, may introduce dependencies on aspects of companies' core business models. Particularly difficult in this perspective is the potentially most valuable type of service integration - the one that spans enterprise boundaries.

System integration is still a field where large consulting companies command multimillion dollar contracts to help their clients deal with a mess of applications that were never meant to interoperate. Through Web services the complexity of these systems can be encapsulated behind industry-standard XML protocols. One-to-one integrations are being replaced with one-to-many interactions based on Web services. The promise of higher-level interoperability initiatives is that over time we'll be able to develop the set of standards, technologies, and tools that will enable businesses small and large all over the world to easily engage in meaningful B2B interactions. We're only halfway through the interoperability stack. There's a lot of work to be done, but the future is bright.

More Stories By Simeon Simeonov

Simeon Simeonov is CEO of FastIgnite, where he invests in and advises startups. He was chief architect or CTO at companies such as Allaire, Macromedia, Better Advertising and Thing Labs. He blogs at blog.simeonov.com, tweets as @simeons and lives in the Greater Boston area with his wife, son and an adopted dog named Tye.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.