Has it really been almost a year since I've posted? Not sure why I stopped... not sure why I'm starting.

I'm no longer working on web services. Here is a clue as to what I am working on.

With Games of Havoc, Men Will Be Boys. The best-selling and most violent video game in America is Grand Theft Auto 3, in which players are rewarded for having sex with a prostitute, killing her and stealing her money. [New York Times: Technology] Interesting.  And I thought sneaking up on a thug and stabbing him in the back was realistic (Solider of Fortune).  I might have to rent this and see what all the fuss is about... [Justin Rudd's Radio Weblog]

I can't help but to compare such games to the virtual child p**rn issue (avoiding undersirable links from search engines). It seems that all the arguments, pro and con, could be applied equally in both cases.

Thomas A. Edison. "Hell, there are no rules here-- we're trying to accomplish something." [Quotes of the Day] A philosophy that many engineering companies should adhere to... [Justin Rudd's Radio Weblog]

Some engineering companies do, then have to conform to a concent decree for it. :-)

Via Haughey, a great quote from Pascal inventor Niklaus Wirth: "People seem to misinterpret complexity as sophistication." Tatoo that on your forehead, post-it on your monitor, send it to the editors of XML.Com, and to the leaders of the W3C.   [Scripting News]

A mantra for our time.

I think I need to say this in an unambiguous way as possible:

• I think that every SOAP and WSDL implementation should support every feature of the SOAP and WSDL specifications. That is the only way to insure true universal interop.
   
• SOAP and WSDL are complex specifications with lots of surface area (including all of XML Schema) with lots of special edge cases where they bump up against each other and other specifications. It is practically impossible for every implementation to implement every feature of the SOAP and WSDL specifications. It certainly can't happen "all at once," but only though a continual process.
   
• The SOAP 1.1 and WSDL 1.1 specifications are essentially drafts. They had not received the careful consideration from a diverse working group that is necessary to insure that they are complete and appropriate. In the process of implementing them, a number of instances of where they are incomplete or inappropriate may be encountered. At these points, a developer is forced to make a guess as to what the intent of the specifications may be.
   
• We (meaning the SOAP community as embodied on soapbuilders) have done a damn fine job working around and within these specifications to deliver an amazingly interoperable cross platform messaging infrastructure. It is an accomplishment that ranks very high on the all time list of truly wonderful things that have happened with computers. We should be very proud of this.

[Simon Fell] Keith & Mike both seem happy to subset SOAP and WSDL. Wave bye-bye to the chance that SOAP interop means anything more than interops with MSFT. Where are all the MSFT guys who were up in arms 12-18 months ago about subsets?

Oh come on!

Choosing not to support a given feature of a specification is very different from publishing a subset of a specification and implementing only that. The first case merely reflects the the limitations of a platform, the specific needs of a given application domain, or the scarcity of resources available for implementation. The latter is more akin to trying to take control of a technology from a standards body and forcing other vendors to come along.

Which category do you honestly think Microsoft's SOAP and WSDL implementations fall into? What about the other vendors that do not implement everything in SOAP and WSDL? Is there any reason Microsoft should be held to a different standard then these other vendors?

My point here, which you seem to have completely missed (which must be my fault), was that vendors necessarily prioritize the features they think are important to their customers. They get the important ones done and never quite find the time or need to do the others. If a vendor gets feedback on missing features from their customers, the vendor changes their priorities and gets those features done. If every vendor ends up getting the same set of features done, it most likely means that those features are the core of a technology.

As the specifications for that technology evolve, they should reflect this reality by specifically identifying the less desirable features as being adjuncts or optional. If the standards don't change in this way, then you really do end up with a situation where interoperability is defined by the vendor with the largest market share. That is what I do not want to happen!

Peter Drayton points to A Comparison of Alternative Encoding Mechanisms for Web Services which has a comparison of XMill and other compression techniques used in a web service context. I've tended to discount XMill, and many other compression oriented solutions, in this context because there doesn't seem to be a reasonable way to implement them over a stream of XML. Having to buffer every message puts a lot more memory pressure on a server and that should impact throughput.

However, I didn't see evidence of this in my first (brief) reading of the cited paper. This may be because using just two computers (a client and a server) is not a configuration where memory pressure would be significant. If the server were handling 1000 simultaneous requests from 1000 different clients, I expect it would be quite noticeable.

Also, many current SOAP implementations buffer anyway, mostly because of the interaction between SOAP Faults and HTTP status codes, but this will not be the case as A) SOAP directly over TCP/IP becomes the norm (at least I think it should become the norm); and B) intermediaries become part of many web service designs.

Simon Fell: If you're going to provide section 5 support, then do it, if you don't like section 5, then don't support it, but providing a subset just confuses the issue.

While I won't encourage selective implemenation of standards, I do think there is an interesting point to be made here. To my knowledge, no implementation really supports everything in SOAP and WSDL. You get a venn diagram of features. I assert that the features in the center are the only ones that are really useful. I don't mean they are useful only because everyone implements them, but that everyone implements them because they are useful. It is technological evolution in practice.

(Can you belive there is a http://www.venndiagram.com!)

Simon Fell says "in my experience the MSTK is significantly more compliant with the specs than ASP.NET web services."

I have to agree with Keith. Interoperability tests show that the STK and ASP.Net have more or less the same level of interoperability. To my knowledge, no SOAP/WSDL implementation is 100% complete.

Of the features listed by Keith and Simon, the STK doesn't support sparse arrays or partial arrays. Multi-dinensional arrays are supported only if you know how to edit the WSDL we generate (meaning that about 2 people on earth can do it reliably). The root attribute and hexBinary were just added in the 3.0 release. (What is the point of hexBinary anyway?). We do ignore mustUnderstand="1" when the actor is specified and has a value of any URI other then "http://schemas.xmlsoap.org/soap/actor/next" (but the SOAP 1.1 specification is ambiguous on this issue). I'm not sure what Simon ment by "certain kinds of Generic Compound Types."

I want to shed some light on why the STK supports DIME and not SwA. This topic has come up on soapbuilders. The purpose of this post isn't to defend the party line nor to dispute it. It is just an attempt to explain how we ended up where we did.

Many people may not know that the STK is a product of the Webdata XML team at Microsoft. We do System.Xml, MSXML, and various other pieces of XML infrastructure. A different team is responsible for Microsoft's SOAP and XML Web Services vision and the implementation of that vision in the .Net Framework. How the Webdata XML team ended up doing the STK is an accident of history.

The STK is a tactical product. It was concieved as a solid standards compliant and interoperable SOAP implementation to replace the SOAP Toolkit 1.0. STK 1.0 was an MSDN sample SOAP implemenation that wasn't very standards compliant. The .Net Framework is Microsoft's strategic platform for SOAP and XML Web Services.

The STK turned out to be a bit more useful then some people thought it would be. A number of customers used it for real life applications. The STK team got excited about it and wanted to improve on it. One of the more common customer requests was for better performance when sending blobs and large XML documents. The obvious solution was to add support for SwA.

While we were working on SwA, the web services team decided that DIME was the strategic direction for encapsulation. DIME is a better technical solution. It can be implemented much more efficiently and with simpler code then SwA. They also wanted the WSDL Extension for SOAP in DIME to be simpler then WSDL's MIME binding. There is also the potential for confusion between the infrastructure that can be provided by MIME and the infrastructure that will be provided by GXA. DIME solves one simple problem while MIME solves many problems, many of which are addressed in more appropriate (XMLish) ways by GXA.

Our customers needed SwA for interop, but if we released support for it in the STK, it may have made SwA a permanent part of the web services landscape. The STK team was forced to make some hard decisions. After much discussion, we decided to support the web services team's long term strategic direction. We replaced the MIME support we had partially completed with DIME support.

Was providing only DIME support in the STK the right thing to do for web services in the long term? I hope so. Was it the right thing to do for the STK's customers in the short term? No. I'm sorry we could not provide the SwA support they needed.

In response to threads started on xml-dev, I wrote up the following on the topic of SOAP, HTTP, and REST.

I take the view that SOAP and HTTP are at the same level of the network protocol stack.

The more I understand about HTTP and REST the more it seems that using HTTP as a transport for SOAP is a mistake. It is convenient for SOAP, but I can see how it is at odds with HTTP and why that legitimately bothers some people. To SOAP's credit, it is not tied to HTTP, but can be easily layered on TCP/IP for use on the Internet.

SOAP has the advantage of having XML at its very core. I believe this is having profound effect on how people think about and use SOAP vs. HTTP. Some of the churn around how SOAP fits into the web may be because people intuitively want an XML based evolution of HTTP. From some perspectives (mine for example), SOAP satisfies this desire. From others, it is apparently a completely different (and horrible) thing.

SOAP's extensibility model is very different from HTTP's. The mustUnderstand attribute on headers is a vital part of that difference. So is the SOAP processing model that dictates what you do when you receive a message with content that you don't understand.

I think the SOAP extensibility model is an improvement over HTTP's. SOAP's extensibility model allows for application protocols to be layered on top of SOAP in clean, composeable, and well defined ways. In effect, SOAP is an application protocol platform.

Like HTTP, SOAP can be used to build a system in the REST style. More accurately, you would apply the interaction constraints that, when applied to distributed hypermedia systems, lead to REST to the more general distributed systems allowed by SOAP. Once the resulting architectural style is defined, SOAP headers that convey the necessary information to support that style could then be easily defined. As with HTTP, doing REST with SOAP it is a matter of discipline and understanding.

Another issue is how URLs are used to identify resources on the web. WSDL requires that a single URL identify an entire web service. Instead, a service could have an absolute base URI and each port type and operation could specify additional relative URI pieces that are composed with the base URI to construct the actual URI associated with a message. There are a number of open issues in the WSDL working group that could be addressed by such a mechanism. It could easily be used in the next version of WSDL.

Are these things enough to allow SOAP to be considered an XML based evolution on the concepts of HTTP? This is not to propose that SOAP replaces HTTP, but that a new breed of application can be built using SOAP in parallel with the use of HTTP on the web.

[simonstl] Developers who work primarily with XML focus on document structure, while Web Services developers focus on application structure. It's kind of like the difference between building a house designed for a particular group of occupants (XML) or building a set of houses stamped out identically regardless of who will live there or why (Web Services).

I always thought of web services as a bridge beween being document (and data) structure focused and application structure focused. The problem with CORBA and DCOM is that they are purly application structure focused. The problem with the web (as an application platform) is that is purly document (and data) strcuture focused. Web services tie both together.

We released Beta 1 of the Microsoft SOAP Toolkit version 3.0 today.

[Simon Fell] Mark Baker emailed me to point out what the issue with RPC part of SOAP and REST is [thanks Mark]. So are people using SOAP in the messaging style ?, I don't know of a single toolkit that defaults to messaging style SOAP [I discount ASP.NET in this case, as although it defaults to doc/literal SOAP, the messages it defines have RPC semantics]

In the SOAP Toolkit, I pushed to get the SOAP Messaging Object (SMO) Framework feature added. SMO provides a message based alternative to the RPC programming model. I believe strongly that the programming model is one of the things matters a lot when tricking* people into building a loosely coupled distributed systems. You are right, a doc/lit wire format isn't the same thing at all. (The other thing that matters is baked in support for versioning, and doc/lit is a useful pre-condition for that.)

Unfortunately, I have been unable to get the all the resources needed to do SMO right. For one thing, a key part of it is a wizard that reads XDR schemas and generates VB classes that act as an typed programming model over an XML document. That wizard should be updated to read XSD schema (and WSDL files), but that isn't ever going to happen (the STK walks a fine line between providing good SOAP support for COM programmers and "encouraging" developers to move to .Net). In the 3.0 release, the core SMO Framework classes are included as sample code and the generator is gone.

* SOAP implementations should help a programmer build a well design distributed system without them having to think about it.

james hong's Radio Weblog: Dave is like the BigCo of weblogs.

Write a clever phrase, get a link.