Holistic Web Services

It's been really interesting to watch the continuing efforts of the Internet industry to define and deliver platforms for web services. I've been involved in helping to define XML protocols for distributed computing for a long time, and Macromedia has put together an extremely powerful yet simple set of software to help people build web services.

Last week's Web Services II conference, held by InfoWorld, pulled together the latest thinking and technology in this crucial emerging space. You can read extensive coverage of the event here.

But I've continued to be annoyed by the narrow-minded thinking that has dominated discussions of web services to date. Like many IT technologies, the central thrust of the web services worldview has emerged from the classic middleware infrastructure providers, which has in turn colored our thinking on web services significantly. The essential problem is that web services as defined by a core collection of XML protocols (SOAP, WSDL, UDDI) focuses almost entirely on API-level application integration, rather than a broader view of how software will be created, distributed and consumed in the future. Most importantly, it lacks any real notion of what the user experience of web services will be (though there are nascent standards efforts such as the Portlet specs, and WSUL).

The over-focus on protocols and middleware reflects the political economy of the IT industry, where control of programming languages, APIs and runtimes draw the most attention because the stakes are so high.

There are many people who have actively considered a broader view of web services, one that encompasses the client-side user experience as well as the back-end plumbing that enables transparent use of logic and data in the network. But those people have been few and far between, and certainly not very visible in the broader industry discourse on web services.

It was with this perspective that I was really pleased to see this article on Marc Benioff's keynote at the conference. Marc runs salesforce.com, one of a new breed of software companies that delivers its software as a hosted service; what the industry hyped a couple of years ago as Application Service Providers (ASPs).

It's really important that we move beyond the back-end and out to the user experience and realize that the broader vision of "software as service" requires a reset in how we create and deliver desktop software over the Internet. And, of course, Marc is right on with his vision and execution.

The original idea of "software as service" emerged before the current hype around web services. It was the notion that applications would run in the network. That the user interface could be downloaded and used on the fly on any Internet-connected device (most likely a PC), and that substantial portions of the application --- in particular those that were focused on logic and data --- could be exposed and used by other applications easily.

I love the idea of being able to use a high-quality, desktop-like, media-rich software application over the Internet --- to have a means to have that application cached on my local computer; to work offline; to synchronize new versions when needed, and the ability to easily consume and integrate logic and data in other applications on the network. It's a big part of what makes Macromedia MX so special.

Lots of folks have dismissed the idea of "hosted applications" and ASPs (apparently some of the audience at Marc's talk did too). What do hosted applications have to do with web services, they asked? EVERYTHING! It's precisely the ability to deliver an application over the network, and integrate that across all releavant cooperating nodes and applications, that makes web services a vision worth fighting for.

So what went wrong, and will we see a renaissance of ASPs that are the true torchbearers of web services?

It seems to me that there were three missing pieces to the vision of ASPs and software as service when the idea first emerged:

1. Rich user interfaces. End-users expect their productivity applications to be as rich and responsive as any desktkop application. While we can tolerate web applications for discrete tasks (filling out an order form, searching a database, getting our web mail), using them everyday as a core of our business is painful. The model of the HTML-based web was not meant for the kinds of software experiences that PC users require. With the advent of rich clients like the Macromedia Flash Player, it's now possible to deliver software experiences that rival and even surpass what we experience on the pure desktop. I'd use a Flash-based mail client all day if there was one out there!
2. Easy application integration. This, I think, was and is a huge issue for ASPs. If a corporation is willing to use an outsourced or hosted application, they will more than likely need to integrate aspects of that application with their own internal systems. For example, a hosted commerce application needs to communicate with customer management and inventory management systems. An HR application may need to communicate with finance and accounting applications. Several years ago when ASPs emerged, XML protocols such as SOAP were just ramping up, and there really didn't exist any mainstream commercial development and deployment platforms that made such integration easy. Now with what the industry is calling "web services", a hosted application can participate in a network of applications, enabling API level integration between internal and external systems. While there remain security issues at this level, enormous progress has been made.
3. Massive scale and distribution. Initially, many early ASPs sought to deliver applications in their own hosted environment. As these services have grown in popularity (such as WebEx), the capital investments needed to deliver massive scale, host-based software have grown incredibly. At one level, commodity clustering (the approach used by nearly every large-scale website on the planet) is solving this, but this can't scale. Small, medium and large software service providers will need a distribution platform that can scale but which doesn't require them to themselves make the enormous capital investments and ongoing operating expenses. This to me is the missing piece in fully realizing the software as service vision --- a 'utility-based', distributed execution environment that ISVs and service providers can plug-into. Kevin Werbach is exploring some of these issues too; I intend to as well.

Let's turn the debate around and focus again on the broader vision of software as service rather then keep ourselves down in the plumbing.....