Nick Gall's Weblog

Saturday, August 28, 2004

Please use my new RSS feed. For those of you reading my blog via a feedreader (aka RSS aggregator), please take a moment to redirect to my new and improved RSS feed provided by Feedburner. Feedburner is a cool RSS "intermediary" that will translate my basic RSS feed into multiple RSS formats and other cool things. To get the URL for my latest feed, simply use this chicklet ( ), or the one in my "masthead" on the right side of my blog under the Radio Mug. Thanks. 7:10:07 AM

Yet more network vs. edge debate. Cross posting my comment to Radovan here: Radovan: I am talking about *THE SAME ISSUES* being dealt with on different levels. For the SAME REASONS that transparent intermediaries exist at BOTH the Internet IP level, AND exist at the HTTP level (think WebSSO, Akamai, and web load balancers), they WILL exist at the SOAP and BUSINESS LEVEL. Let me give you an example roughly based on a composite of several client discussions I've recently had. Global bank needs to comply with the illegal money laundering (ILM) provisions of the Patriot Act. So it needs to match every "open account" message going over its global network against a DB of "bad guys", because the ILM forbids banks from doing business (offering bank accounts) with bad guys. So, its plan is to transparently route all "account open" messages past this new "bad guy matching service" without having to touch any of the dozens of account management services this bank has built around the globe. (To be more accurate, this bank didn't build these account management apps, they acquired them via M&A.) So here's a perfect example of a service provider having its messages rerouted without its knowledge in order to add new functionality to the overall system without having to make changes to the provider. This is exactly how a Web Services Network should be designed. Does this reduce the autonomy of the account management "service providers"? You bet. They no longer unilaterally decide who can open an account, because this matching intermediary now imposes its decisions on them. Big deal. As for your FedEx/DHL example, the way this should work at a business document level is that I, as the provider, should indicate that a business report must be delivered to the recipient the next day. (Note that I have the authority, ie autonomy, to make this timing decision.) I put the document in my outbox on my desk with a routing slip attached and my corporation's mail room decides what overnight carrier we use for which destinations, given the discounting deals we've arranged. Does this reduce my absolute autonomy? You bet. I've lost the precious right to decide which overnight delivery service I prefer. Big deal. As for your blog "service" example, check out Feedburner . This intermediate service is exactly the kind of ["intelligence" in the Web Services Network] I am talking about. Rather than having my blog rss feed provider have to know all the different feed formats or having the ability to track feed pings, etc., I let this rss feed service intermediary, feedburner, do it for me. I tell feedburner my simple rss feed address for my blog, I put feedburner's rss feed icons on my blog home page and suddenly, every new rss subscriber is getting MY rss feed VIA feedburner. What I get (and what I really want that Radio Userland does not supply) is the ability to see who's reading my blog via rss. Pretty slick. This is exactly the kind of management in the middle you seem to be arguing against. Why? Does this reduce the autonomy of my blog service? You bet. Now its dependent on feedburner being up and running. Big deal. The autonomy of web service providers with regard to the web services network will be more like the "relative autonomy" of state governments with regard to the US federal government, not the "absolute autonomy" of a sovereign nation (which these days isn't really so sovereign itself, what with UN, WTO, etc.). Some decision making rights will be in the hands of the service providers (the states) and some decision rights will in the hands of the "network" (the federation). The network is the federation. Autonomy will be federated autonomy. That's why I've been writing about SOA as a federated architecture for the past several years. (For publicly available material see http://radio.weblogs.com/0126951/2004/02/18.html#a98 and http://techupdate.zdnet.com/techupdate/stories/main/Service_Oriented_Architecture_Enables_Global_ITO_print.html and http://www3.ca.com/Files/IndustryAnalystReports/SOA.pdf) Autonomy and interdependency (or conformity) are very relative and very intertwined concepts. This subtle interplay has been endlessly discussed in various fields from philosophy to economics to sociology to physics (see the book "Six Degrees"). In sociology, this discussion goes under the title "Structure and Agency" (See http://en.wikipedia.org/wiki/Structure_and_agency and http://human.ntu.ac.uk/pgcert/structureandagency.html). In the IT realm, the structure/agency discussion often goes under the title "network vs. endpoint" or "network vs. edge". Given how subtle the thinking and passionate the debate has been in other disciplines over the duality of structure and agency, I expect no less in SOA discussions. But lets at least learn something from these prior debates in these other disciplines and quickly move beyond the shallow debate of absolute autonomy vs. its absolute interdependency, into a deeper discussion of what aspects of autonomy should a service provider have and not have under what circumstances. Or even better, how I can dynamically shift responsibility from the edge into the network and back again based on various collaborative policy decisions. 6:16:44 AM

Sunday, August 22, 2004

Dissolving the paradox of decades long disruptions. [This is an excerpt of an email discussion we are having regarding the pace and effect on incumbent carriers of the migration to VOIP. In a previous post I had stated that, "While I agree that the disruption of and eventual replacement of ALL forms of wired communications with wireless digital communications, as all good disruptians know, "disruptive technology" can sometimes take place over decades." Of course, this generated the standard objection: how can a technology evolution that takes decades be called "disruptive." Since this happens so frequently, I'm posting this answer to this FAQ on my blog so I can just link to it in the future.] Both "disruptive innovation" and "sustaining innovation" (Christensen's terms) are kinds of evolution. And both may happen relatively quickly or relatively slowly. What distinguishes disruptive innovation from sustaining innovation is clearly laid out in Christensen's book. I highly recommend EVERYONE read it. There are many aspects that distinguish the two; most of which I highlight in my Disruptive SOA Trends pitch--an homage to Christensen, which I'd be happy to send to anyone. But the easiest way to tell if an innovation is disruptive (and hence the reason Christensen calls it by that name) is to examine its impact on the market, i.e., whether the dominant companies in a market remain the dominant companies after the transition to the new technology. If they are, then it was "sustaining" technology because it "sustained" the common business model of the dominant incumbents. But if most of the dominant companies are wiped out by the new technology, then it was disruptive technology because it disrupted the common business model of the dominant incumbents. That is why Christensen's research on the excavator market is much more interesting than his research on the hard drive market. In the excavator market, the dominant incumbent cable-driven excavator manufacturers had thirty years to adapt to the new hydraulic excavator technology and they still failed to do so. Thus virtually all of them disappeared when the cable excavator disappeared from the market. Christensen's fundamental question was to understand why seemingly well run companies that dominated a technology market would often be unable to adapt to new market conditions and as a result, cease to exist, or at least lose their dominance. His theory of the two different types of technical innovation and their differing impact on dominant business models was his answer. And one of his fundamental insights, often overlooked, is that adaptiveness, is NOT about being able to change quickly; it's about being able to change at all. Some companies have plenty of time--decades--to see a new technology growing in the market, and yet they fail so do what is necessary to adapt their business model to it! Thus, the adjective "disruptive" is not being applied to the technological innovation itself, but to the impact it has on the business models of the dominant firms in a market. When you hear "disruptive technology" think "disruptive-to-the-business-models-of-market-leaders technology". If you look at it this way then the answer to your question ("If a change in technology occur [sic] over a period of say 30 years can it really be called disruptive") is a definite yes. And vice versa: a technology change occurring in five years can be sustaining. So as to Telstra, the question is whether or not after the decades it takes to migrate the market to VOIP, Telstra will end up dominating that market, or cease to exist because of it. 4:36:22 AM

Wednesday, August 18, 2004

ESB vs. ESN: Yet another end-to-end debate. Good post by Radovan Janecek, CTO of Systinet, re Enterprise Service Bus and Web Services Management. We both think "bus" should be changed to "network". So Radovan uses the term ESN: Enterprise Service Network. If I were king, would also change "enterprise" to "web", because Web services are used just as much inter-enterprise (or more the point, "a web of enterprises") as intra-enterprise. Thus, we should be talking about a WSN ("Web Services Network") not an ESB. (BTW, I could live with "Business Service Network", since business is inter- and intra- neutral, and fits the small-to-medium business market better.) However, the ESB term has traction and META Group has decided to adopt it, despite its flaws. What I disagree with in many debates about Web services and SOA is the implicit assumption that the choice between "dumb endpoints and smart network" vs. "smart endpoints and dumb bus network" is a binary one. This "end to end" argument has been debated since the dawn of the Internet, and the reality (as opposed to the theory) is that BOTH the endpoints and the network need to have some smarts. I THINK Radovan leans towards this point of view given the distinction he makes between an ESB and an ESN: Indeed, the problems of monitoring, state management, or debugging would be simpler to address with a shared infrastructure like ESB. ESB can monitor, route, enforce, transform, report, secure, and orchestrate everything because every single message exchanged among web services goes via ESB! Unfortunately, you cannot fulfill this requirement in the real world – so you loose these advantages anyway. So, yes, in the "ESN", every service (or its hosting environment like WASP, .NET, etc.) is responsible for many things: it should report several metrics, it is secured, it does transformations it needs, it routes, it enforces policies, and so on. Of course, as I mentioned previously, these services can use other services for specific tasks – registry for finding other services or policies or other metadata, single sing-on, or even content router service. Important factor is that services decide to use other ones on their own. There is nothing like – use the bus otherwise you cannot play the SOA game. The first two sections I highlighted suggest that he agrees that "BOTH the endpoints and the network need to have some smarts." What concerns me however is the third highlighted section. I don't believe that the service providers "decide", I think it is a negotiated decision by the service consumer, the service provider, and the ESN. For example, the service consumer may want to use Kerberos tokens for service interactions, but the provider only supports X.509 certificates. The ESN (or more precisely a Token Security Service intermediary) could enable a federated interaction without the service provider ever knowing that the intermediary was there. On the other hand, if both the consumer and provider can use Kerberos, then there is no intermediary for the service interactions. So it's not a consumer-makes-right model, or a provider-makes-right model, its a network-makes-right model. Radovan is correct to criticize a model that conceptually requires an intermediary in all cases; but I would criticize his alternative model the conceptually requires that "services [providers] decide to use other ones on their own." (I'm assuming that Radovan is referring to service providers in the section I've quoted.) The right model is one that allows optional intermediaries, which may be transparent to the consumer, the provider, or both. This is why I like the SOAP processing model so much. Its concept of roles and nodes enables one to transcend this ancient (and false) dichotomy between end-to-end and in-the-network intelligence. SOAP enables one to define absolute (e.g., ultimateReceiver), relative (e.g., next), or even functional roles. A functional role is completely independent of node location. That is, either endpoint (initial sender or ultimate receiver in SOAP terminology) or any intermediary can "play" a particular functional role. Gudge uses the example of a "logger" role, soap:role='http://example.org/loggers', for a log header. Any node assigned that role may process the header. The sender could play the role, or an intermediary, or the receiver. In fact, the role could be dynamically reassigned to different nodes by the ESB transparently. Even better, depending on how the log header is defined, multiple nodes can play a single role, each processing the header and passing it along. In fact, this is exactly how WS-Security (I'm citing a slightly out of date version) works: As stated, a message MAY have multiple <wsse:Security> header blocks if they are targeted for separate recipients. However, only one <wsse:Security> header block MAY omit the S:role attribute and no two <wsse:Security> header blocks may have the same value for S:role. Message security information targeted for different recipients MUST appear in different <wsse:Security> header blocks. The <wsse:Security> header block without a specified S:role MAY be consumed by anyone, but MUST NOT be removed prior to the final destination or endpoint. "Consumed" in this context means processed. Thus multiple nodes (including intermediaries) could change the security header as it is routed through the network. For example, as discussed above, a Token Security Service between enterprises may change from Kerberos to PKI encryption of the message WITHOUT either endpoint ever knowing such transformation was done on the fly "in the network". The concept of functional role completely demolishes the useless debates about where network processing should be done. Message SOAP headers can now specify WHO should do the processing regardless of WHERE they are located. In fact, with SOAP roles and the SOAP processing model, one can begin thinking in terms of Aspect-Oriented Networking (a term I just coined), where each aspect of a message exchange is defined by a role. One might also call this Concern-Based Protocols, which Radovan has even mentioned in his blog. So in a properly designed ESB, just as in a properly designed IP network, some messages will go point-to-point between endpoints without going through any intermediaries (e.g., in an IP network, two PCs on the same physical Ethernet LAN) and some will go through multiple intermediaries (e.g., in an IP network, a PC in a Microsoft campus building in China communicating with a PC in a Sun campus building in Massachusetts). And the programs on both PCs should never know the difference. This is what makes ESB potentially so radically different from most EAI architectures as implemented: the endpoints should never know whether or not a ESB product is being used, and should certainly never know WHICH ESB product is being used, anymore than two network endpoints know whether routers are being used or which vendor's router is being used. 4:53:19 PM

Sunday, August 08, 2004

Cards surpass cash and credit. Apparently in 2003, for the first time, checks and cash were used for less than 50% of all consumer in-store payments, according to 2003/2004 Study of Consumer Payment Preferences sponsored by the American Bankers Association. They were displaced by card-based payments, with debit cards being the fastest growing type of payment card. I think this is a watershed event, yet I only read about it in a recent Boston Globe article on layoffs at the Federal Reserve check processing centers: "Check volume at the Fed has been falling faster in recent years. In 2003, check volume dropped nearly 5 percent, compared to 2 percent in 2002 and a half percent in 2001. The Fed handles about 40 percent of the nation's checks; the rest are processed by big banks and private clearinghouses." What's also intriguing is that the number of consumer payments has doubled in the past thirty years. Obviously, some of the growth is due to population growth, but it would be fascinating to know what caused the doubling: more frequent payments of less value? did the value of consumer payments double as well (doubtful)? The report on the study did not go into this statistic. The study also had interesting statistics on Internet payments and bill payments (the only two other types of consumer-to-business or consumer-to-government payments?). 5:05:58 AM

Saturday, August 07, 2004

Captology. Just came across a neat new term captology: "The word was coined by Professor Fogg in 1996 as a partial acronym—from the initial letters of Computers As Persuasive Technology—together with the ending -ology for a field of study. Someone engaged in the field is a captologist. Captology is defined as "the study of computers as persuasive technologies. This includes the design, research, and analysis of interactive computing products created for the purpose of changing people's attitudes or behaviors." An important subissue of captology (or any study of persuasion) is credibility. And the folks at the The Stanford Persuasive Technology Lab, where the term was coined, are spending a lot of time on the study of web credibility. Their survey on web credibility and material on web credibility guidelines seem especially useful. This seems very related to my recent entry on trustworthy opinion discussing slashdot's comment ranking system. The Stanford researchers cite a similar system, eBay's rating system, as an example of microsuasion ("elements of persuasion built into the user experience"). So I guess the proper academic name for trustworthy opinion is something more like web opinion credibility. 6:39:09 AM

Thursday, August 05, 2004

A lesson learned. A while back, I made the connection between Neurath’s Boat and the Ship of Theseus. I thought the lack of connection elsewhere on the Web was interesting. Only a site by M.R.M. Parrott (MP) mentioned both terms. So I emailed him  (see below), hoping he might be a kindred spirit. MP did not find it interesting and his reply seemed a bit snarky, especially a comment about trolling, and what appeared to be a sig block with a quote about stupid (see below). So I originally posted his full reply on my blog so I could comment on it. However, a few days ago, MP discovered that I had quoted his email and emailed me to delete it. Guess what, this email was snarky too. Obviously, I was pissed off. I felt like telling MP to f-off. Instead, I made myself think about his request for awhile. Certainly, given that he had sent the original email to a total stranger, he had no legitimate privacy concern. As for copyright, clearly my quotation would be within the realm of fair use, since I was commenting on it and deriving no commercial value from it. So I decided to visit his site to see if it would reveal something about his personality or attitude that would help me decide what to do. While there, I read his article on wikis, thinking this would give me insight as to how MP felt about posting etiquette, including copyright issues. The article contained the obligatory reference to Wikipedia, but also mentioned a wiki I'd not yet heard of Wikinfo. To make a long story short, it appears that MP posted a Wikipedia entry about himself, which was voted for deletion because it was self-promoting. MP then joined Wikinfo, partially in response to this, where he could post an entry about himself, given Wikinfo's looser point-of-view rules (compare SPOV to NPOV). Though I do think MP's pseudonymous self-promotion is unbecoming (others may disagree), I do admire his independence, talent, tenacity, and industriousness. Helping to build a spin-off encyclopedia with the posting rules of one's choosing, rather than merely submitting to the authority of others, is a great example of the freedom the open source and open content world is enabling. I especially value the XML import innovation of Wikinfo, which is a great step towards better wiki federation (e.g., I can post MY views on subjects in which I have a POV in my wiki, but link to other wikis for subjects in which I have not [yet] expressed a different POV). This might come in handy for a wiki project I am considering. In the end, given the sensitivity he displayed in the thread re deleting his entry, I decided to honor MP's request. I post this story here only to demonstrate that if one takes the time to get to know someone else better, instead to returning slight for perceived slight, one can learn something interesting and useful, and perhaps avoid an escalating conflict. From: nick.gall@metagroup.com To: "pixel" Sent: Thursday, January 29, 2004 6:56 AM Subject: "Neurath's Boat" "Ship of Theseus" I recently discovered that essentially (no pun intended) the same paradox has two different names, both quoted in my subject line. Of course I googled the two phrases in quotes and your "Generation of 'X'" was the only hit. (There's a googlism for a search that returns exactly one hit, but I can't remember what it is.) What struck me as curious, and hence motivated this email, was that you did not seem to notice the connection. The two phrases are used in distant parts of your book. Did you think they are the same paradox? If so, why no connection in your book? If not, why not? Given that both are fairly well known paradoxes, I'm surprised that no one has made the connection. I guess I will have to do so on Wikipedia and my blog. -- Nick MRM's Signature Block: "Stupid so stupid that it goes way beyond the stupid we know into a whole different dimension of stupid." - http://users.gurulink.com/drk/humor/flame.html [BTW, this flame is indeed the best flame I have EVER seen. -- NLG] 1:08:08 PM

Tuesday, August 03, 2004

Trustworthy Opinions. The Review of Reviews. In the New Your Times Editorial Page for Tuesday, August 3, 2004, there is an editorial about Amazon.com's Real Names program, which is "designed to prevent reviewer fraud, your reputation depends on what others say about what you say." This is another example of the emerging trend towards mechanisms for ensuring what I am calling "trustworthy opinions". (I'm sure there's already some other name for it out there, I just haven't seen it.) Another example is the very sophisticated system used by Slashdot, which has arguably one of the most challenging signal/noise ratio problems of any site, given the volume of postings and its empassioned and technically clever posting (i.e., gaming the system is built into their genes). It uses a combination of automatically selected distributed moderators, meta-moderation (i.e., moderating the moderators), karma, and friends/foes tagging (i.e., trusted/untrusted) to enable very fine-grained filtering of useful, trusted content from useless, untrustworthy content. For an interesting history of how Slashdot's moderation system evolved see How did the moderation system develop? 12:11:02 PM

Sunday, August 01, 2004

Prions and the Windows monoculture debate. My friend Keith recently sent me an email with the subject Self Organizing containing a link to the following New York Times article  Scientists are reporting that, for the first time, they have made an artificial prion, or misfolded protein, that can, by itself, produce a deadly infectious disease in mice and may help explain the roots of mad cow disease." The findings, being reported today in the journal Science, are strong evidence for the "protein-only hypothesis," the controversial idea that a protein, acting alone without the help of DNA or RNA, a cousin of DNA, can cause certain kinds of infectious diseases. http://www.nytimes.com/2004/07/30/national/30protein.html?hp I wonder if the Science article mentions the concept of enzyme in discussing prions. Prions (technically PrP-sc prions) strike me as being protein catalysts (enzymes) since they shape a chemical reaction (formation of a new protein) without being consumed by the reaction. In fact, they seem to be a unique form of self-transforming enzyme. A prion doesn't replicate (compose copies of itself from parts), it transforms the shape of an already existing (normal) prion (PrP-c) in a new (abnormal) prion (PrP-sc). This new shape can then reshape other normal proteins until all of them have been reshaped. This ability to transform from whole vs. assemble from parts is why a prion, unlike all other infectious agents, does not need genetic material. I think of a prion as an autocatalytic enzyme, self-catalyzing enzyme, or self-copying enzyme. This leads me to think of my blog entry on standards as templates. If the template has a flaw, it can prove catastrophic after millions of copies have been made from the template. Which reminds me of an email debate on the Microsoft monoculture I intended to blog but never did. Here is an excerpt that summarizes my opinion: I've resisted the urge to jump into this fray up til now because I think the issues around monoculture vs. biodiversity are so hotly debated and politically loaded in the life sciences, why would we look to them to gain insight into technological diversity? For example, if monoculture is such a bad thing, then life on earth is in trouble, because we are all based on the same set of four DNA nucleotide bases (A, G, C, T)! And ohmygod, we're all based on the same 64 codons of the "genetic code" that maps DNA to amino acids. If any hostile entity were able to inflict damage due to this fundamental "DNA monoculture" shared by all life on earth could be used to devastate the planet! Nooooo! If only evolution had been wiser and had evolved life on earth with diverse genetic architectures based on different nucleotides or different codons, we'd be at less risk out complete annihilation from one threat. I'm NOT saying diversity provides or doesn't provide benefits, inherent or otherwise. I simply point out that it is an open and interesting research question in the life sciences as to the benefits of varying degrees of biological diversity. At certain layers of the ecosystem we see massive diversity (orchids, insects), at other layers, none at all (all life uses the same four nucleotide bases; homo sapien is the only extant species descended from homo habilias-sp?), at others we see something in between (typically there are fewer "top predator" species than prey species in an ecosystem). I have my own pet theory about some of the factors that appear to govern diversity (e.g., when one layer spans another, diversity decreases in the spanned layer, and increases in the layer above the spanning layer), which I'll write down one of these days. Bottom Line: No one knows the general laws of equilibrium or optimization of diversity vs. homogeneity in biological ecosystems, so why go around spouting dubious monoculture analogies to software ecosystems. Talk about useless FUD. It now strikes me that the existence of deadly prions is a perfect biological analogy to the Windows monoculture risk. Just as the whole world is at risk from a single Windows virus because we all use Windows, we are all at risk from a single prion (PrP-sc) because we all use the same protein shape in our brains. Is having massive numbers of identical PrP-c's (ie, a prion monoculture) in our brains good or bad? Its good because it makes the brain possible, its bad because it makes it vulnerable. 5:10:08 AM