15 July 2004

Tonsillectomy and Open Source My youngest daughters, twins, and I have been in hospital for the last two days for their Tonsillectomy and I have taken the opportunity to re-read the Cathedral and the Bazaar, and to start reviewing my past experiences with Open Source with the model it describes.  I have also started to review its progress against Eric Raymond’s predictions and some of the challenges it faces.  As a result I now have an Open Source category in my blog and two daughters without Tonsils.    4:02:52 PM      comment [] Looking to the future My previous two posts gave two examples of projects that would have benefited from having been Open Sourced.  They happened a long time ago, when I worked for a different community and both have disappeared so there’s no problem with discussing them.  When these products were developed Open Source was just an emerging concept.  I am not going to discuss the history of Open Source, that’s been well documented already, and I am not qualified.  However I am going to start to build up a series of articles that describe some of concerns and the challenges I think the Open Source model faces in the future.  I am going to assess some of the established beliefs as documented in Open Source bibles like the Cathedral and the Bazaar and I am going to do my bit to try and help. 4:00:20 PM      comment [] Lost Opportunity 2 I thought I would pick two previous projects and look at their potential as Open Source projects.  The second I picked is a bit whacky.  It was written almost entirely in DCL and provided an automated help desk job logging, analysis and reporting tool and knowledge base.  To write this in DCL is a testament to the flexibility and power of the VMS scripting environment and indexed files and the creativity of one of its developers, (not me).  But the concept was mine and its ‘conceptual integrity’ was maintained for many years.  What value could this have had in an Open Source context:   The application itself had little value, although it was trivial to deploy and could have proved very popular for other small support teams. The tools developed to manage a complex system constructed from DCL would have been very valuable The library of DCL routines would have been very useful to the community The concepts used in the system which allowed 4GL like application development may have inspired other developers to rapid application prototyping of similar applications. This system lived for well over a decade and died because the team that used it were disbanded,  had that not happened it may have continued to spiral out of control through entropy like neglect which Open Source enthusiasm and stewardship would/might have prevented. 3:58:32 PM      comment [] Lost Opportunity 1 In a previous post I described a simple networked or standalone, (depending on data definitions), document imaging processing project I led.  This system would have been perfect as an Open Source project for the following reasons:   The need for such capability to be provided as infrastructure was universal.  The main constraint at the time to this happening being the high cost and complexity of the current applications.  So adoption would probably have been quite rapid The target user community tended to be very network centric, as they were often charged with distributed access to large central collections of information. My company had no interest in the software,  its interest was in an efficient way to capture and distribute its image data as efficiently as possible The system was very extensible, allowing additional storage drivers, scanner drivers, printer drivers, viewers and databases to be added by other developers.  The automatic maintenance of these components in line with new hardware advances and the increased deployment reach would have been very valuable There were many areas of potential improvement.  Some of the concepts were not fully realised, and many functional/feature improvements such as OCR were missing  It’s also likely that the community would have seen the natural evolution of the product to a web application, and created that UI and made the necessary database access module changes, saving my company the need for that re-write.  Finally vendors of specialist image viewers, scanners and printers/plotters may have sponsored development as it would have enabled them to provide a working system, rather than a component.  In addition to the raw savings in development cost for my company the potential benefits for the community were greatest. The provision of document imaging capability, a truly universal need on every device and in every business, has never happened at the infrastructure level, but rather in an ad-hoc way at department level in most cases. This system not because it was the best, but because it was very simple, made few assumptions about its data, and few assumptions about its environment could have been the start of making this happen. 3:47:03 PM      comment [] Open Source – even then As I gradually migrated towards infrastructure and away from line of business applications, the reality of having to deliver applications to Windows and manage them on Windows began to dawn on me.  To a developer used to centralised computing, with remote access through X Windows or terminal clients this was a considerable shock.  However my first real Windows development project showed one of my most valuable character traits, I don’t give up easily!  Without going into the gory details, here are some of the attributes of that first application, a system for capturing, storing, accessing and viewing large image collections:   Very easy deployment Self maintaining code, i.e. a minimal system start-up application compared what was installed with what should be installed according to a central manifest, and updated itself accordingly. A very flexible storage model built on the concept of logical storage units, (a bit like VMS logical names on steroids), which handled the fact that images could be on removable CD’s, local disks, media libraries, networked disks etc and in different combinations. Data driven.  The whole system was configured through simple text files and meta data definitions that defined the actual data structures in the SQL database. Globally unique naming of files, to avoid cache conflicts between image collections from different sources accessed at the same client The result was an incredibly cheap system to purchase and deploy, that could run on a laptop or on 1000 networked desktops with the differences all defined in its data definitions.  Data collections could be distributed on CD, with updates on floppy disk, or networked but transparently cached for fast WAN access.   My team, (a very small one), built this system making extensive use of freely available software libraries, (for Visual Basic), in a very short period of time, and by having a small team with a leader with a clear vision we kept the concept intact from design through to initial deployment.  Although like all systems entropy resulted in some degradation of that concept over time, resolved only when a friend of mine reinvented the system as a web application with similar clarity of vision. 3:44:15 PM      comment [] Open Source – the real beginning In this article I use Open Source in its broadest sense, (i.e. not consistent with the specific licensing defined by opensource.org), but meaning my experience in using other peoples source or developing source for others to use.   In particular I wanted to give examples of where the concept of Open Source had it been so visible then, would have suited some of my projects.   My computing true home has, and always will, be VMS.  As a VMS systems integrator I learned rapidly to admire the power, elegance and consistency of the system, its ‘conceptual integrity’ if you will.   It’s this history of VMS that made it so difficult to admire Unix, which by contrast has always seemed lacking in that same ‘conceptual integrity’, to stretch a point, always seeming to have been assembled rather than architected.   It’s also that history of VMS that first attracted me to Windows NT, (the development of VMS was led by a team from Digital who worked extensively on VMS).  Having been repelled by DOS and Windows, I was excited by the prospect of NT but ended up being left gaping at the lost opportunity, (still not realised today).  I was truly shocked to read the early history of NT and discover the neglect of that sacred VMS like architectural vision for the platform as a whole.   This neglect seems to have been caused by two key issues   the focus that Dave Cutler and his team had on the kernel at the expense of everything else the need to achieve compatibility with the DOS/Windows/OS2 legacy.  Had Bill Gates recruited a different, or at least more balanced, part of the Digital VMS team the world might be a very different place.  Of course, pre-Linux,  I realise that the Unix community saw a similar lost opportunity seeing the progressive dominance of the inferior NT and MAC/OS, (now corrected), whilst the superior Unix was increasingly marginalised.   It’s through Digital that I first experienced the web and deployed by first Intranet and obtained my first CD full of Open Source, (actually GNU), software tools and utilities, (for VMS). 3:40:42 PM      comment [] Open Source – the beginning This is the first real post in the Open Source category of my blog, but one of many in the blog as a whole, and one of many to come on Open Source.  If you have read my blog from the beginning you will know that much of my recent experience is with Enterprise Infrastructures, and that has to a large extent involved software from IBM/LOTUS and Microsoft.  Integration with Unix systems being through well defined and mature interfaces like NFS, X-Windows and DNS.  This means that I have a lot of familiarity with Microsoft and I am open about the fact that there are things to admire about Microsoft, (and many things to not admire of course), however I have a long held low tolerance for Zealotry which I have talked about previously, but want to expand on here.   When I say I dislike Zealotry, I am not talking about passion, I am talking about taking a stance for or against something that can not be defended by rational argument.  This causes me a problem, because as an enterprise architect, I spend my life having to defend the decisions I take with rational argument, and for me, ‘I hate Microsoft’ or ‘I hate Unix’, just does not pass the test.  Second as an architect I have to recognise real world constraints such as legacy applications, investment prioritisation, application compatibility, cultural aversion to change or risk etc in my solutions.  These two factors have shaped most of my thinking around the Linux/Windows and previously Unix/Windows debate.   However that is changing, and this blog will chart that change.  However before I dive into that I want to provide two of the main reasons why Open Source has become important to me personally and also professionally.   Reason 1: Credibility I have noticed an increasing number of well respected Microsoft employees or expert system integrators either migrating to Linux or becoming expert in both Windows and Linux.  These individuals seem to have maintained a rational view of the world and are able to develop solutions recognising the strengths and weaknesses of both platforms.  It’s also interesting to see this happening in other areas, a good example being the .NET/Mono community who talk of the elegance of the .NET framework, whilst at the same time criticising Microsoft for lacking the vision to provide cross platform support.   I want to ensure that I am more than credible as an architect, open to, and conversant with the benefits of Open and Closed source alternatives and their respective merits.   Reason 2: Concern I have become increasingly concerned about the prospect of the worlds infrastructure being run by the proprietary technology of a single supplier.  I increasingly feel that the world’s IT infrastructure needs to be owned by the world.  That’s not to say that I am confident that the mechanisms are in place for this to happen naturally with Open Source as it is delivered today, but more of that in later articles.   Professionally I find myself increasingly being asked to provide advice on Open Source alternatives and to develop solutions based on them. 3:35:59 PM      comment [] New category - Open Source I have just created a new category on Open Source in this blog.  Some old posts have been re-categorised as relevant to this topic, the reason for the relevance may not always be immediately obvious.  However I have a plan for articles I want to write on this topic, and some of these old background posts fit into that plan. 3:21:13 PM      comment []

© Copyright 2004 Steve Richards. Last update: 05/08/2004; 08:57:56. This theme is based on the SoundWaves (blue) Manila theme.