Welcome!

Simeon Simeonov

Subscribe to Simeon Simeonov: eMailAlertsEmail Alerts
Get Simeon Simeonov via: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn


Related Topics: Apache Web Server Journal, XML Magazine

Apache Web Server: Article

SOAP Part 2

SOAP Part 2

The last edition of the XML in Transit column (XML-J, Vol. 1, issue 4) introduced the Simple Object Access Protocol (SOAP). Instead of dwelling on technical issues, it focused on the driving forces behind the technology.

To put SOAP into context we looked at its history, parsed the buzzword-compliant phrase ubiquitous XML distributed computing infrastructure and scoped the SOAP specification within the broader set of standards likely to emerge in this area. One possible layering of SOAP and related specifications is shown in Figure 1. The areas already covered by SOAP are grayed out. As far as protocol support is concerned, the latest SOAP specification (SOAP 1.1) provides bindings only for HTTP.

In this column we'll go deep into the SOAP specification. Let's start our dive into the technical details with a look at SOAP's extensibility framework.

Extensibility Framework
XML has become the lingua franca for structured data exchange on the Internet. In particular, with the wider adoption of XML Namespaces and XML Schemas we now have the ability to define and extend XML protocols in a distributed environment. In other words, it's now possible for a single organization or standards body to create a specification and publish it for broad use while also guaranteeing that it can be safely extended at a later date by independent third parties. Safely in this context means that (1) optional extensions shouldn't break existing applications, (2) mandatory extensions will work only with applications that understand them and (3) multiple extensions can coexist peacefully.

Don't get me wrong - I said it was possible, not trivial, to do this. It's not the case that schemas and namespaces automatically enable this type of interoperability. However, they make it possible for mere mortals to develop specifications and extensions with these properties, and this is important enough. Think about the last time you tried to extend some flat data model or even tried to add a couple of elements to your XML DTD. Could your existing software safely skip over the optional extensions? Or did you have to change your code? Be honest. No one's looking.

Simplicity and extensibility are major design goals for SOAP. They're achieved through modularity and layering. SOAP is designed to maintain different aspects of the specification as completely independent or "orthogonal."

Here's how several SOAP facets are treated:

  • Communication protocols: Generally speaking, the choice of communication protocol has no impact on the structure and content of SOAP messages. SOAP is protocol-independent. However, the SOAP specification details how SOAP messages can be sent over HTTP because it's likely that HTTP will be the most common delivery mechanism for SOAP messages. We can also be sure that SMTP will be a common mechanism for simple message passing and that pure sockets are going to be used frequently inside firewalls for performance.

  • Communication patterns: The choice of communication patterns (request-response, point-to-point, publish-subscribe) may change the structure of a SOAP message, but the schemas identified by the SOAP-specific namespaces don't need to be changed. SOAP messages are a mixture of elements from SOAP-specific namespaces and any other namespaces. The choice of communication pattern may impact the schema design within these other namespaces. In other words, "application-level" schemas might change, but the core SOAP schemas stay intact. SOAP defines one possible way to do RPC, but others can be provided.
  • SOAP Envelope: According to the SOAP 1.1 specification, "the SOAP Envelope construct defines an overall framework for expressing what's in a message, who should deal with it, and whether it's optional or mandatory." All this is done with just a few XML elements, as we'll see later in this column. The schema is kept in a separate namespace (http://schemas.xmsoap.org/soap/envelope aliased as SOAP-ENV in the specification) so it doesn't affect other parts of SOAP. The envelope framework allows any "application-level" XML to be packaged as headers or the body of a SOAP message. It's the only SOAP schema that can't be replaced.
  • Data encoding: SOAP defines one particular set of data encoding rules. Again, they're kept in their own namespace (http://schemas.xmlsoap.org/ soap/encoding aliased as SOAP-ENC in the specification). The SOAP implementation by the Apache XML Project already has an alternative encoding based on XMI. Any other data encoding rules can be provided - for example, Web Distributed Data Exchange (WDDX). Any portion of a SOAP message can have its own encoding rules, which can be intermixed freely. The SOAP-ENV:encodingStyle attribute binds encoding rules to an element and its content. The value of the attribute is the namespace for the encoding.
Now that you understand the "pretty much anything goes" policy of SOAP, let's look at a detailed example. We'll pick it apart to learn more about the envelope framework.

SOAP Message Structure
Listing 1 is an example request to look up the name "Big Boss" using a SOAP, server managed by Lookup Central. In this case we're using SOAP to implement an RPC over HTTP mechanism.

This is a fairly complex SOAP example. To get a better handle on it, I've broken it down structurally in Figure 2 and added some comments.

The first thing that's apparent is the layering of logical on top of physical protocols. The XML of the logical SOAP message is contained within a physical HTTP request. (Yes, I do agree that the notion of logical versus physical protocols gets quickly confused when one thinks about a thick protocol stack, but please bear with me here.) There are a couple of important things to note about this:

  • The location of the SOAP server (www.lookupcentral.com/LookupCentral) is identified within the HTTP request; it's not part of the SOAP message. By being addressing-agnostic SOAP can be protocol-independent.
  • The optional SOAPAction header of the HTTP request identifies the intent of the SOAP request. Its value can be any URI. For example, the value can be used by firewalls to appropriately filter SOAP messages.

When we look at the beginning of the SOAP message, we see the SOAP Envelope element. It's the mandatory top element of the XML document representing the message, and because it's uniquely identified by its namespace, it allows processing tools to immediately determine whether a given XML document is a SOAP message.

This certainly is convenient, but what do we trade off for this capability? The biggest thing we have to sacrifice is the ability to send arbitrary XML documents and perform simple schema validation on them. True, we can embed arbitrary XML inside the SOAP Body element, but na•ve validation will fail when it encounters the Envelope element at the top of the document instead of the top document element of our schema. The lesson is that for seamless validation of arbitrary XML inside SOAP messages, we need smarter tools.

A final note about the Envelope element: it doesn't expose any protocol versioning information (such as a major and minor version number). Instead, the notion of versioning is tied to the namespace URI. As the protocol evolves, the URI is likely to change.

Going further with the example message, under the Envelope element we see a Header element. Headers, which are optional in SOAP, are the primary mechanism by which additional facets can be added to SOAP-based protocols. They provide an elegant yet simple mechanism to extend SOAP messages in a decentralized manner. Typical areas where headers get involved are authentication and authorization, transaction management, payment processing, tracing and auditing. Another way to think about this is that you'd use headers to pass any information orthogonal to the specific information needed to execute a request.

For example, a stock quote service only needs a stock symbol to generate a quote. In real-world scenarios, however, a stock quote request is likely to contain much more information (e.g., the identity of the person making the request, account/payment information). This additional information is usually handled by infrastructure services (login and security, transaction coordination, billing) outside the main stock quote service. Encoding this information as part of the body of a SOAP message will only complicate matters. That's why it will get passed in as headers.

All immediate children of the Header element are header entries. In Figure 2 we have two. The first one has something to do with the authorization level of the entity making the request. The second passes what looks like a transaction identifier to the SOAP server. In general, header entries can have arbitrary XML in them. Note how the unrelated header entries use separate namespaces.

Also, notice the SOAP mustUnderstand attribute with value "1" that decorates the Transaction element. This attribute indicates that the recipient of the SOAP message must process the Transaction header entry. If a recipient doesn't know how to process a header tagged with mustUnderstand="1" it must abort processing with a well-defined error. This rule allows for robust evolution of SOAP-based protocols. It ensures that a recipient that might be unaware of certain important protocol extensions doesn't ignore them.

Because the AuthorizationLevel header entry isn't tagged with mustUnderstand="1", it can be ignored during processing. Presumably this will be okay because a server that doesn't know how to process authorization levels will operate under a safely low authorization level, one that doesn't allow intruders to gain access to confidential information (in this case, perhaps the salary of the Big Boss).

I'd like to finish the discussion of headers with a reminder that the two headers in the example are fully fictitious. There has been no broad consensus in the SOAP community about how to handle authentication, authorization or transactions. We need these to enable enterprise-quality SOAP-based systems.

The SOAP Body element immediately surrounds the information that's intended for the final recipient of the SOAP message. Truth be told, you can pass the same information using a header entry tagged with mustUnderstand="1", but using a mandatory Body element just makes things easier to understand and process.

All immediate children of the Body element are body entries. In the example, we want to perform an RPC request to the LookupCentral server and look up the name "Big Boss" in it. Although arbitrary XML can be passed via a body entry, by convention, in many cases of RPC over SOAP the name of the first body entry element identifies the method you're interested in accessing. In this case, the LookupPerson element identifies a method with the same name. Note the use of yet another namespace to keep method-related information separate from everything else.

The content of LookupPerson is the set of parameters we need to pass to the method to do its job. In the example we're passing two strings - the first and last names of the person we're looking up. These are simple values, but keep in mind that SOAP can encode pretty much any data structure.

That's most of what there is to a SOAP message structure. Let's look at a possible response message.

The SOAP Response
As you can see in Listing 2, the structure of the SOAP response message is identical to that of the SOAP request message.

The Transaction header entry is returned to indicate that the response is part of the same transaction as the request. The AuthorizationLevel header entry is absent because it makes sense only during the request. The Body element contains a LookupPersonResponse element. Adding "Response" to the request element name is a convention suggested (but not required) by the SOAP specification. The response data is encoded via the SOAP encoding style used by the request. We can see that Big Bad Boss is CEO of UberSoft Corporation and that she has a nice round salary of $10,000,000. That's all there is to it.

SOAP Message Exchanges
On the surface SOAP message processing seems really simple. A SOAP server will get a message, check that it knows how to process all elements tagged with mustUnderstand="1" and then invoke whatever back-end functionality the message requests. Actually, there's a lot more going on. The fact that SOAP is independent of the choice of protocol and communication pattern's is great from the perspective of simplicity and flexibility; however, it means that SOAP clients and servers must maintain separate conventions on how to perform message exchanges and that takes some effort. Let's look at the example in a little more detail.

The SOAP message arrives as an HTTP request. The SOAP server that's the recipient of the message is identified at the HTTP protocol level (www.look- upcentral.com/LookupCentral). The SOAP client needs to know (1) that there's a SOAP server at that URL and (2) how to send a message to it. This means sending an HTTP request to the URL with ContentType text/xml, with the message as the body of the request.

The SOAP server needs to be listening for messages at that URL and, although SOAP messages can be thought of as one-way transmissions from a sender to a receiver, in this particular example both the SOAP client and server must know that the response message will be part of the HTTP response. Successful message processing should return HTTP status 200 OK. The bottom line is that there needs to be a lot of shared knowledge about the protocol over which the client and the server must communicate.

The same is true at the logical protocol level. Both client and server need to know that they are engaging in an RPC communication pattern. Both must use the convention that the name of the first body entry element specifies the operation that needs to be performed. Both must understand the encoding style used to represent information.

Other semantics need to be agreed on. For example, the client needs to encode transaction information into a header entry. The server needs to be able to process this transaction information (the Transaction element is tagged with mustUnderstand="1"). And the server needs to know that the response message should include the same transaction ID as the request message.

As you can see, for meaningful message exchanges to occur over SOAP, clients and servers need to agree on a lot of things that are outside the scope of the SOAP specification. That's why it's so important for the industry to quickly address the broad set of problems in the area of XML-based distributed computing as shown in Figure 1. I'll spend more time looking at these issues in future installments of this series.

I'll complete the SOAP trilogy with SOAP Part 3, which will address aspects of the latest specification we haven't covered so far: data encoding, error handling and the really cool concept of intermediaries. Once again, SOAP will prove simple, elegant and flexible, and far short of providing the full solution. Stay tuned for details.

More Stories By Simeon Simeonov

Simeon Simeonov is CEO of FastIgnite, where he invests in and advises startups. He was chief architect or CTO at companies such as Allaire, Macromedia, Better Advertising and Thing Labs. He blogs at blog.simeonov.com, tweets as @simeons and lives in the Greater Boston area with his wife, son and an adopted dog named Tye.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.