Coping with Change
By Sam Ruby, March 15, 2002.
This document describes the considerations which go into making an extensible wire level protocol that can gracefully evolve over time. No prior understanding of SOAP is required, but some knowledge of basic programming concepts and familiarity with XML is presumed.
Getting one's own house in order 
Local procedure calls (LPC's) are pretty easy. Define a function or a subroutine in one place. Call it from many places, passing a few parameters. Everything pretty much "just works".
Now what happens when needs change? Well, the most frequent change required is the addition of a new parameter. Accomplishing this may be tedious, but generally is straightforward. One simply needs to find each of the callers (tools like grep may be useful for this purpose), make the change, and then change the function/subroutine. If you are dealing with a statically compiled language, then the compiler can also help in the identification process.
Object Oriented Programming Languages such as Java and C# enable multiple classes to implement a common interface. This means that a change to an interface may not only affect multiple callers, but also each implementation of the interface. However, the process is essentially the same.
As long as you have all of the source, and the resulting program is deployed as a single entity, and the interfaces are all internal, then versioning problems such as these are bounded by the size of the program.
When you introduce multiple components with different release cycles, problems become increasingly more difficult to address. One does not even need to introduce Remote Procedure Calls (RPCs) to manifest the problem, the issue can occur on the same machine, within the scope of a single process, and even with software all from one vendor.
One manifestation of this problem is popularly known as DLL hell. Installing or uninstalling one program may suddenly cause a seemingly unrelated program to fail. Inevitably, the root cause is a change to an interface. The traditional solution generally involve some sort of change management.
Distribute this same software over the scope of the internet and the problem rapidly becomes intractable. One simply can't rely on the ability to simultaneously upgrade all servers and clients with a snap of the fingers. To deal with this, old versions of clients need to work with upgraded servers. Depending on the topology of the software involved, the reverse may need to be true too.
The COM approach (upon which DCOM is based) is to essentially deny that there is a problem. A fundamental tenet of COM is that interfaces, once published, are immutable. If you want to make a change, you are required to create a new interface. This may sound impractical, and after a few releases, it actually proves to be. If you release frequently, you often end up with interfaces that look like this (compliments of Simon Fell)
coclass CoSerializerFactory
{
[default] interface ISerializerFactory;
interface ISerializerFactoryConfig;
interface ISerializerFactoryConfig2;
interface ISerializerFactory2 ;
interface ISerializerFactoryPool ;
interface ISerializerFactoryPool2 ;
interface ISerializerFactoryEx ;
};
The "2" and "Ex" suffixes are commonplace in COM definitions that have been around a while. The solution is a bit tedious and generally requires your software to query to see if a given interface is supported before calling it, but in general is workable for a period of time in local environments. In fact, many who have done this for years simply get used to it and assume that it is required.
However, to effectively scale in remote procedure call environments, something less brittle and more loosely coupled is called for.
If you have a protocol which is designed to support named parameter associations, adding a new parameter is easy. One merely requires the recipient to match the actual parameters to the arguments that they were expecting, to ignore anything they do not understand, and provide reasonable defaults for anything that they were expecting that they do not receive. In my opinion, similar rules are the fundamental reason for the huge success of the HTTP protocol over which the entire web is based.
SOAP is another such a protocol which supports this principle in theory, though not all implementations apply these principles in practice.
Remember how in Local Procedure Calls there was only one implementation of a function with multiple callers, whereas in Object Oriented Programming, there was the possibility of multiple implmentations? The same is true in general of RPC. When you develop a popular wire level protocol, multiple servers will inevitably want to implement it. This works well if one person controls the protocol, but such an approach stifles innovation. The solution? Require everybody who adds a parameter to define it in a namespace that they own. Since there are a virtually inexhaustible supply of namespaces, multiple extensions can coexist peacefully.
What you say is often as important as how you say
it
Once you accept that interfaces may change over time, you will soon see the need for some level of metadata in order to aid discovery. XML schema can be embedded in the message to make the content self defining. Or it can be contained in a separate document (e.g.., WSDL).
Schema information also helps languages and environment cope with differences in data type models (e.g., static, compiled languages vs dynamic, scripting languages) and parameter passing modes (positional vs. named). More on this topic is covered in Dealing with Diversity.
As a general rule, the more metadata the better. Other things to look out for are passing strings when the data involved has more meaningful structure. A sure indicator of this is when an interface requires a string to be XML encoded in order to be passed as a string.
Two things required to make a protocol which can withstand the test of time: namespaces and named parameter associations. These may not seem important initially when you are designing an interface, but they inevitably will become important for the same reason that there is the "X" in XML: extensibility. Taken together they mean that I can piggyback some data on the payload that you don't understand but can deal with. It also means that I can add parameters to an interface and default them meaningfully when you don't send them.
Once these basic needs are taken care of, optional metadata can increase the robustness and interoperability of the solution.