David Black: Java

James Pasley's Just Enough XML to Survive

Thu, 08 Feb 2007 12:56:35 GMT

Excellent video presentation from James Pasley, Cape Clear's CTO, on XML, XML Schema, SOAP and WSDL. 15 minutes that could just change your (professional) life for the better!

An approach to monitoring and managing clustered servers as a single entity

Tue, 23 Jan 2007 14:01:00 GMT

Often we need to be able to monitor and manage clustered servers as a single entity. This is an outline of an approach I’ve used before to monitor and manage a clusterable enterprise server product which had very demanding RASP and throughput requirements. It is based on JMX, but allows for an aggregated view of the mbeans (monitoring attributes and management operations) in the cluster i.e. it allows for things to be examined in a meaningful way at the cluster (or service) level, while at the same time being able to view an individual nodes value for that attribute.

It also enables an operation to be executed once for the whole cluster e.g. flush a cache, rather than having to manually do that for each server instance in turn. This can be achieved using a standard single server aware mbean console (a limitation of most off the shelf open source ones).

The kind of aggregation appropriate for an individual attribute can be easily configured, and a view can be presented at the cluster level and at the individual servers (node) level. Different attributes require different kinds of aggregation. For example, the aggregation of the number of items in a given cache across the cluster would be the sum of all the individual cache sizes, but at the same time we also want to be able to see the value for an individual node. In the case of message throughput the aggregation might be an average of all the individual servers throughputs, and we might also want a count across the cluster of messages processed to date, as well as those statistics on an individual node basis. For another attribute it might not be sensible to aggregate it, so we might just want the individual values for each node to be shown together. The point is that a sensible “cluster” view of a given attribute differs.

The basic idea is to instrument the classes that house the stuff you want to expose as normal using an MBean interface. So each node in the cluster exposes all its stuff as an individual server. Additional metadata is added to the MBean interface using variables (static final Strings), which describe what is to be done with the individual attributes and operations. I say static final Strings because this annotation is Java 1.4.x compatible, but if you're using Java 5 then of course you can use "real" annotations.

The mbeans are registered with the mbean server via a façade – lets call it MBeanServerFacade. The facades job is to interrogate the mbeans and extract metadata using introspection. A simple convention is followed for the variable names in order to associate them with the monitoring attributes and management operations. The values of the strings then describe how the attribute is aggregated (and other useful things). The metadata could be described somewhere else, but the advantage of putting it on the mbean interface is that its easy to develope, and it lives with the attributes, so when the mbean changes the metadata can be easily changed with it. Also, it is really simple to add these annotated mbeans so there is no barrier to adding instrumentation to the server (and for clustered servers setups you can never know enough about what is going on under the covers). MBeanServerFacade writes the metadata to a simple database table, if that particular type of mbean hasn’t already been registered. It also writes the host address of each node in the cluster to another table.

The next element in the architecture is a broker process. This should (but does not have to) live separately to the cluster, so on a standalone server instance. The MBeanServerBroker reads (and periodically reloads) the descriptions in the tables populated by MBeanServerFacade. So the broker has a full description of all the relevant mbeans available and all the servers where they can be found. MBeanServerBroker is used as an intermediary between the various MBeanServers and the calling application. Using the metadata MBeanServerBroker creates JMX dynamic mbeans which expose an appropriate aggregate view of the individual mbeans across the cluster. Because it uses dynamic mbeans, the aggregate mbeans can also expose the individual attribute values (and which server they belong to). When an aggregate operation is invoked from the console, MBeanServerBroker runs that operation on each node in the cluster. The other important job of the broker is caching the returned mbean information. This is because JMX requests are expensive both in terms of the caller (waiting/blocking) and the component exposing the mbeans (doing the work of gathering the mbean info etc.). The console remains responsive because its hitting a cache, and the broker updates the cached values frequently. It’s the dynamic mbeans exposed by MBeanServerBroker that the console looks at.

Here is an example of an annotated mbean interface:

interface MyStatsMBean {

/** attributes */

int getCacheSize();
int getAvgJmsReadsPerSecond();
int getMaxInsertMessageTime();
int getMinInsertMessageTime();
int getUniqueValue();

/** annotations */

String getCacheSize_AM = AggregateMethod.SUM;
String getAvgJmsReadsPerSecond_AM = AggregateMethod.AVG;
String getMaxInsertMessageTime_AM = AggregateMethod.MAX;
String getMinInsertMessageTime_AM = AggregateMethod.MIN;
String getUniqueValue_AM = AggregateMethod.LIST;
}

In this example we want to see the sum of cache sizes across the cluster, the average JMS reads per second across all nodes, the slowest message insert into the database, as well as the quickest, and a list of individual values for an attribute that it does not make sense to aggregate.

James Gosling and Groovy

Fri, 01 Sep 2006 10:00:23 GMT

According to Guillame LaForge, who is the Groovy project lead, James Gosling has been using Groovy in some projects at Sun, and he likes it very much. How cool is that?! I am really looking forward to the publication of the first Groovy book, and the release of the first Groovy JSR release candidate, and hopefully before too long the big 1.0 ...

Cape Clear ESB keeps on winning with IT Week Magazine Editors Choice Award for 2006

Wed, 09 Aug 2006 09:50:07 GMT

"One of the best ESB products available and it wins our Editor's Choice Award for functionality and performance." - IT Week, July 31, 2006

"Clear's Cape Clear ESB 6.6 provides an intuitive and capable platform for creating, managing and deploying services, processes and data for an SOA infrastructure."

And Cape Clear will just keep getting stronger - we have a host of goodies in store for later this year. I can't wait! :)

SCA v0.95 - some much need simplification

Fri, 04 Aug 2006 16:16:13 GMT

I am so glad to see that in the latest SCA draft specification the assembly composition has been greatly simplified using a recursive model. I thought the whole module, fragment, sub-system and system thing was hopelessly confusing (how many times did I read the 0.9 spec?!). Now we have Composities (which are just the same as Components) that include other Components and so on. So its basically composite services and services.

OSGi tutorial at ApacheCon Dublin

Wed, 28 Jun 2006 21:37:47 GMT

ApacheCon Europe 2006 is being held in Dublin this week. I couldn't attend the whole week, but I did get to go to a half-day OSGi tutorial given by longtime OSGi evangelist Peter Kriens. Peter, who is the current editor of the OSGi specification, says:

"OSGi is kind of SOA"

"OSGi is kind of a Java Operating System" or at least a "mini-kernel"

OSGi is "the Dynamic Modularity Layer for Java"

OSGi is a Java framework for developing remotely deployed service applications. OSGi started life in 1998 with the original use case of home automation. OSGi applications can run on different software and hardware architectures. There is a lot of buzz around it right now, particularily in the enterprise space, partially because of Eclipse Equinox, which is OSGi R4 based and the technology underlying Eclipse bundles (actually Equinox goes beyond pure OSGi). But there are a bunch of other implementations, including the Apache Felix project. The specification is freely available and royalty free.

OSGi applications adhere to a basic SOA, the contract between client and service is based on a Java interface and discovering and binding available implementations is done through the framework. Client and service are softly coupled. So it provides an in-VM service model. Declarative services, introduced in R4, are akin to having dependency injected components. Applications declare their service dependencies in an XML file, and are notified by the framework at runtime as services becomes available and go away. The last is an important point: services can go away at runtime, just as they can be dynamically discovered. The next major release of Spring is adding support for OSGi. One of the tutorial attendees made the point that OSGi is a generic event notification framework.

OSGi applications are packaged as bundles, handled by the OSGi Module Layer. The Module Layer addresses packaging, class loading modularization, protection and versioning (multiple versions of the same package) - things Java does not do well, or at all, by itself. In addition, OSGi provides basic Life Cycle Management. An API for managing bundles allows them to be installed, resolve their dependencies, started, stopped, refreshed, updated and uninstalled. So basically, bundles register one or more services with the framework service directory, and other bundles can use those services. Service contracts are just Java interfaces, so service implementations are just POJOs and no special classes need be extended nor interfaces implemented.

Its a pity OSGi wasn't somehow used as the basis of JBI, there are so many similarities.

Its also a pity that there are currently two rival JSRs attempting to establish a dynamic modularity layer for Java SE. JSR-291 is the OSGi Alliance backed one that aims to "define a dynamic component framework including component lifecycle for existing Java SE platforms". Peter is on the expert group. JSR-277 aims to do more or less the same thing, expect not as well according to Peter - who is is not on the expert group despite his best efforts! Richard Hall was at the tutorial, he is on both groups. According to Richard he is trying to steer the 277 folks in the right direction. It will be interesting to see what happens. Richard was giving a couple of OSGi related talks in Dublin, I wish I had been able to attend them.

And I think the Eclipse Rich Server Platform proposal deserves some serious attention. There is so much happening in the module/component/service packaging, assembling and deployment space (the McSPAD space) (ok, I guess thats several kind of overlapping spaces). We've got JBI, SCA (though don't hold your breath) and now (yes it has been around for quite some time) OSGi. What we really need to do is get everyone in an aircraft hanger (big enough?), seal the doors and tell them they are not getting out until there is only one. One McSPAD to rule them all.

Farewell PaceMetrics

Sun, 23 Apr 2006 17:43:52 GMT

In a few days time I'm finishing up at PaceMetrics, the company I've been with for the last three years. Leaving Pace was a tough enough decision for me. I joined on an up curve just as the company was starting to build its fourth generation BAM (Business Activity Monitor), and winning a huge deal with a tier one investment bank. To deliver that project and productise it, I was lucky enough to quickly assemble one of the best software development teams I've ever had. This team was world class, all experienced, seasoned, innovative, agile engineers. We had a really stellar project manager and business architect in Colm Toolan, and a super professional services team, and the result was PaceMaker 4:

- fully Java, J2EE (POJOs of course!) and JMS based, leveraging best-of-breed open source components and tools
- loosely coupled components communicating asynchronously using XML-based messagess
- a simple maven based build
- a super-scalable, fully reliable, and architecturally innovative BAM engine - thanks James :-)
- a scalable web front end built on some pretty cool caching technology
- enterprise ready monitoring and management through JMX from our own custom web based admin console
- Eclipse based tooling (courtesy of yours truely)

Its been a great few years, but in the end I've been tempted away by the shimmering lights of SOA and ESBs. I'll post more on that in a couple of days.

Agila & Groovy

Tue, 16 Aug 2005 15:11:40 GMT

The BPM engine in Apache Agila now supports Groovy nodes, which is really neat as you can use Groovy to develop custom tasks for your workflows. You get all the usual benefits of Groovy of course, and in particular you can use Groovy markup builders to generate the form/page for the task that the HTTPRenderer (returned by the TaskActivity) generates.

In the process definition XML, the following syntax clues Agila into the fact that the node is a Groovy script:

<node id="3" type="activity" class="groovy:node/ExampleGroovyTask.groovy" display_name="Example Groovy Task">

There is an example Groovy task at

src/test-resources/node/ExampleGroovyTask.groovy

under the bpm module that is both Task and Renderer etc. to get you started.

Thanks to Chris Lim for taking a look at this contribution and commiting it so quickly.

IoC and dependency injection

Wed, 16 Jun 2004 14:56:04 GMT

I'm hearing and reading more and more about Inversion of Control (IoC) and dependency injection. The latter is basically the idea of programming only to interfaces, and not knarling up your code with factories to get concrete implementations of those, rather letting something outside of an object "inject" the implementations, prior to them being required. This injection can happen either through setter methods, or constructors. So its inverting control, hence IoC. The container controls which implementations an object gets, the object (class) is developed against interfaces. Wiring the whole lot up is an exercise for the container, based on some sort of (usually non-code based) configuration, like an XML file.

IoC containers that are on my radar include Spring (not just a container btw), PicoContainer and HiveMind.

Its a bit weird at first, but once you get used to it, the benefits become clear: code that is easier to evolve, everything is pluggable, and unit testing is a doddle - you can isolate units of code at a very granular level, stub everything else with test implementations (like your junit TestCase can implement all of a test targets dependencies and feed it exactly what it needs to prove some bit of functionality).

Eclipse RCP

Mon, 26 Jan 2004 19:12:33 GMT

I understand there's going to be a new Eclipse project, carved out from the current IDE, for building rich client applications using SWT and JFace.

I've spent a lot of my development career building thick and creamy client apps for trading rooms (InVision Advanced Workstation, +Arena) and (not so successfully) development tools for next (read: never) generation interactive A/V content. I'm a big fan of Eclipse largely because of SWT.

The idea of being able to develop a native Java application that is at home on the windows desktop is great. Add to this an OSGi based core, if the Eclipse Equinox subproject works out, and you've got dynamic module loading.

I can see those next generation financial workstations now, and the traders screaming blue murder as the thing locks up as the garbage collection kicks in :) Just kidding guys, go for it!

Weblogic Workshop? I don't think so

Thu, 18 Dec 2003 22:02:18 GMT

Have you seen the new Weblogic Workshop released with Weblogic 8.0? Are you impressed by the flash demo, cool graphics and seamless BPM workflow visual editor based development?

I've just had quite a discussion with some of my companies product management people. It was suggested we ditch the open source and open standards based stuff we're using to build a major web application for a big customer we have, in favour of this new workshop and BEAs webflow technology. So right now we're using Eclipse for development, Struts, Tiles, JSTL and EL. Hmmm, let me think about that for just a nanosecond - no way dude! First of all, why didn't they just build on Eclipse? Why reinvent an inferior, Swing-based (barf) IDE wheel? And all this jpd stuff is totally proprietary. Even if BEA push to standardize some of this it'll be years before there are other implementations (and it'll be different). I just can't see too many people ditching Eclipse and best use open source for this. I guess BEA have deep pockets, but I think this is a mistake.