Jeff Potts: Documentum

Tue, 04 Oct 2005 19:02:33 GMT

EMC to Resell WoodWing's Smart Connection Enterprise. WoodWing Software announced that EMC Corporation will resell its Smart Connection Enterprise software as part of the EMC Documentum Enterprise Publishing Solution (EPS), a new content management solution that provides editorial design and layout capabilities. WoodWing's Smart Connection Enterprise is an Adobe InDesign and InCopy integration tool that enables workflow flexibility through management, organization and document security standards for editorial production. The integration of WoodWing Smart Connection in EMC Documentum EPS allows publishers to manage their editorial production workflows, using the same print content and related ancillary content for publishing to the Web or other emerging channels such as wireless devices. Documentum EPS is built on the unified EMC Documentum enterprise content management (ECM) architecture, enabling users to utilize the full ECM capabilities such as digital asset management, web publishing, XML support, workflows, security, content rendering and localization. Smart Connection Enterprise serves complex workflows and environments, where searching files and/or the content of a story or specific keywords or metadata is required, and comes with support for MySQL, Oracle, Microsoft SQL Server, and Sybase. The server runs on Windows, Mac OS X, Linux and Solaris. http://www.woodwing.com [Gilbane Report News]

Mon, 26 Sep 2005 04:41:36 GMT

There seemed to be a lot of interest in load-testing WDK applications with something other than Mercury LoadRunner. Here is an article I wrote that gives an introduction on using Apache JMeter to perform load-tests against Documentum WDK applications.

Thu, 15 Sep 2005 23:13:10 GMT

Although steeply-priced at $2495, the annual EMC Documentum Developer Conference offered in-depth looks into the forthcoming 5.3 sp1 release (due out tomorrow) and access to Documentum product managers and engineers. I'd estimate about 300 developers from around the world attended the four day conference in Berkeley.

Release 5.3 sp1 promises to be pretty major, as far as service packs go. Many of the features that didn't make it in to 5.3 due to time constraints are included in the service pack. That's contrary to the normal EMC Documentum practice of limiting service packs to bug fixes with only minor changes or additions to functionality.

From my perspective, one of the most anticipated features is the next generation of the Business Objects Framework, BOF 2.0. The ability to hot-deploy and sandbox BOF Type Based Objects (TBO's) and Service Based Objects (SBO's) is high on my list. There's also a new package of Search interfaces that look interesting.

This release will be the first in which Linux is supported across the stack. Release 5.3 installed without issue on my Red Hat Linux Enterprise 3 VMWare image. Customers already familiar with content server installations on other flavors of Unix should not have a problem.

Mon, 12 Sep 2005 06:04:13 GMT

I'm in Berkeley for the Documentum Developer Conference. Spent a wonderful evening over at my aunt and uncle's house in El Cerrito. Great dinner and lots of fun catching up. The Claremont Hotel & Resort, the venue for the conference, is pretty sweet.

One way to tell your Documentum Java Method Server how to find your BOF objects

Thu, 16 Jun 2005 17:56:49 GMT

When you create custom Java methods for Documentum's Java Method Server you can add the class or JAR files to $DOCUMENTUM/dba/java_methods and they'll get added to the Java Method Server's classpath automatically. However, if those methods leverage BOF classes, you may need to do some tweaks so that the Java Method Server can load those classes.

Currently, our preferred approach for telling the content server where our custom classes live is to create a directory off of $DM_HOME, let's call it nav/lib for this discussion, in which our JARs live. We then create a JAR that only contains a manifest, we'll name it nav.jar, in $DM_HOME. The manifest points to the JARs in nav/lib. This enables us to have a single file in the classpath, nav.jar, with as many additional JARs in nav/lib as we need and we only have to update the manifest file in nav.jar when we make changes rather than changing the classpath.

Documentum does this as well. That's what dctm.jar is all about. It contains no classes. Only a manifest. Here's what's in the dctm.jar's manifest.mf:

Manifest-Version: 1.0
Created-By: Documentum Installer Component Library
Class-Path: Shared/dfcbase.jar Shared/dfc.jar Shared/bsf.jar Shared/lo
g4j.jar Shared/xalan.jar Shared/xercesImpl.jar Shared/xmlParserAPIs.j
ar Shared/xml-apis.jar Shared/All-MB.jar Shared/ldapjdk.jar Shared/ld
apfilt.jar Shared/jss311.jar Shared/certj.jar Shared/sslj.jar Shared/
jsafe.jar Shared/jnet.jar Shared/ldap.jar Shared/ldapbp.jar Shared/jn
di.jar Shared/bpmutil.jar Shared/ci.jar Shared/subscription.jar Share
d/workflow.jar Shared/xforms.jar Shared/dam_services.jar Shared/tar.j
ar Shared/wcm.jar Shared/WcmMethods.jar

An easy way to create a JAR with only a manifest is to use the Ant jar and netsted manifest tasks, like this:

On UNIX servers, the Java Method Server is started manually using Tomcat's startup.sh script. There are many ways to tell Tomcat about classes you want to share across webapps. The way Documentum does it (and therefore, the technique we've copied) is to update setenv.sh to include dctm.jar (or in our case, nav.jar) in Tomcat's classpath.

When your content server runs on a Windows host, however, the Java Method Server is started as a service which calls tomcat.exe. If you want to use a similar approach as described above, you can use regedit to update the registry so that the right arguments are set when tomcat.exe is run. In this case, the classpath needs to be updated to include nav.jar.

The key you need to update is HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesDmJavaMethodServerParameters. On my content server the classpath argument was in the string named JVM Option Number 0. I updated mine to include nav.jar, like this:

-Djava.class.path=C:Program FilesDocumentumtomcat4.1.27binbootstrap.jar;C:Program FilesDocumentumjdk131_04libtools.jar;C:Program FilesDocumentumdctm.jar;C:DocumentumConfig;c:program filesDocumentumnav.jar

After you make the change, restart the Documentum Java Method Server service and your methods will be able to find your BOF classes without a problem.

Wed, 11 May 2005 18:32:34 GMT

I originally wrote a shorter version of this Documentum WCM Overview for the Southwest Documentum User Group. I've made this version a little friendlier for people who might not be as familiar with Documentum.

Documentum admin tip: Stubborn jobs

Thu, 05 May 2005 23:03:02 GMT

From time-to-time, you may find that dm_jobs fail to run according to their schedule. Sometimes they can be "encouraged" to run by logging in to Documentum Administrator, opening the properties for the job, changing the next invocation date to a time shortly after the present, and saving the changes by clicking OK. Or you can simply open the object, select "Run After Update" and click OK.

When those attempts fail, and assuming you've already checked $DM_HOME/tomcat/logs/catalina.out and $DOCUMENTUM/dba/log/<docbase_name>.log for obvious error messages, your agent_exec process might be hung. If none of the jobs are running when they are supposed to this is another indicator of a hung agent_exec process.

One way to clear this problem is to simply kill the agent_exec process. On UNIX you would grep for the process (ps -ef|grep agent_exec ought to do the trick) and then kill it (kill <process id>). The content server will notice that the agent exec process is dead and will start a new one. This should free up those stuck jobs.

Wed, 20 Apr 2005 14:13:10 GMT

EMC Posts Higher Net; Stock Up 10 Pct.. EMC Corp. reported its profit nearly doubled, as it shook off a weak global economy and reaffirmed its outlook for the year, sending its shares up 10 percent. [eWEEK Technology News]

Wed, 13 Apr 2005 16:19:40 GMT

WDK 5 Reporting Component. This WDK 5 component allows you to run reports from within Webtop. The reports are specified using an XML file which contains a DQL query and formatting instructions. The Developer Program team will make use of this component to create administrative reports which will be released in the Component Exchange. UPDATED April 11, 2005: Added a new feature to export results to a tab separated file that can be opened in Microsoft Excel. [EMC Documentum Developer Program - New Content]

Fri, 08 Apr 2005 03:01:32 GMT

My post on general ECM skills reminded me of an internal post on Documentum consulting skills. I thought I'd cross post it here in case anyone found it helpful. I've made minor edits to expand acronyms, clarify product names, or clear up other ambiguities but the post is pretty much intact.

What makes a great Documentum consultant?

So, what makes a great Documentum consultant? I think there are desert island skills that every one must have or that you'd really want to have if you were alone on a desert island facing some sort of Documentum project. There are also fringe skills that add value and could be critical depending on the project.

A consistent "why do you like working with DCTM" answer from the people we've interviewed is that people get to work with a wide variety of technologies. Looking at this list shows why. There aren't many people that can fit this bill. It's also important to note that someone broad enough to score well against these categories could actually be a great fit for non-Documentum projects.

Desert Island Documentum Skills (In no particular order)

Core consulting skills (project management, written & verbal communication, client management, selling, teamwork, confidence)
People-centric application experience (Collaboration, process, workflow)
Document-centric application experience
Documentum fundamentals: Workflow, Security, Object Model, Documentum Foundation Classes (DFC)
Basic Documentum administrative tasks (repository care-and-feeding, best practices, installing content server, creating a repository)
Documentum Query Language, Basic SQL
Basic XML/XSLT
Basic operating system (starting/stopping processes, navigating the Documentum installation folder hierarchy, running programs/scripts, editing files, changing permissions/owners of files, setting environment variables, using XServer (for UNIX), administering users and groups)
Basic Relational Database (relational concepts, minimal SQL, ability to speak somewhat intelligently with a DBA)
Basic Java (knows what a classpath is, can write and compile a class, can leverage Javadocs)
Basic BASIC
Basic Web Development Kit (WDK)
Basic application server
HTML, JavaScript, CSS
Ability to negotiate IT processes and human resources
Troubleshooting and debugging

Fringe/Value-add Documentum Skills (In no particular order)

Intermediate to Advanced WDK, Business Objects Framework (BOF)
Java Server Faces, Struts, other frameworks
JSP, servlets, JDBC, EJB
Web services
All other Documentum products such as Web Publisher, InputAccel, Content Rendition Services, WebCache, Site Delivery Services, Content Intelligence Services, Digital Asset Manager/Rich Media Services, Business Process, Reporting Gateway, JDBC Services, Manager/FormsBuilder
Documentum federations, replication
High availability/high performance, load testing
Enterprise architecture
Identity management (Netegrity, Oblix)
Portals in general, Documentum WDK for Portals
Imaging, COLD, fixed asset management
Structured authoring tools (Epic, Framemaker, XMetal)
Web Services
Advanced XSLT, FOSI, SGML, Schema/DTD
Apache FOP
Enterprise Integration
Industry-specific or horizontal solutions (Collaboration/eRoom, Records Management, SarbOx, Aerospace, Pharma, Oil&Gas, Manufacturing)

So, a "great" Documentum consultant would have all of the "Desert Island Documentum Skills" nailed as well as the "Fringe/Value-Add Documentum Skills" applicable to the project at-hand.

Wed, 06 Apr 2005 17:00:39 GMT

Customizing XML Application Behavior with Java and Type based Business Objects (TBOs). Sometimes it's necessary to customize the operations supported by Documentum XML Applications to apply business logic and enforce object-specific business rules. Often this can be implemented only by developing additional code. This article discusses writing Java code that can be called from within an XML configuration file and implement a TBO to satisfy these needs. [EMC Documentum Developer Program - New Content]

Wed, 06 Apr 2005 17:00:20 GMT

Transformation Service (BOF 1.0 Service-based Object). This service provides an interface to apply an XSL stylesheet to an XML document. It uses the DFC Transformation APIs to achieve its purpose and provides the result of the transformation as raw data that can be used by presentation layer for display. In the example provided, the output is saved to an HTML file. UPDATED 31 March, 2005 [EMC Documentum Developer Program - New Content]

Wed, 06 Apr 2005 14:06:12 GMT

Documentum has announced that their new major release, 5.3, has shipped.

The most noteworthy bullets from the new release:

- Linux support across the stack
- Content Rendition Services re-architected and rolled up with other "transformation" products into Content Transformation Services
- Integration of eRoom into the Content Server including the ability to associate a discussion with any object
- Integration of records management retention policies into the core content server
- WDK usability changes (Note: They call them "enhancements" but I'll wait until I've seen them for myself before I use that word!)

Thu, 31 Mar 2005 22:01:39 GMT

WebSphereNetworkDeploymentConfigurationGuide_525SP3. This document contains limitations, Websphere set up steps, WDK set up steps and a troubleshooting guide to aid in configuring a WebSphere network deployment. The information in the document relates to all 5.2.5 SP3 WDK-based applications. [EMC Documentum Developer Program - New Content]

Thu, 31 Mar 2005 21:58:51 GMT

Devprog Samson-like Eclipse Plugin. Some of you may be familiar with an unsupported repository utility called Samson. This simple Eclipse plug-in attempts to recreate some of the Samson functionality. The plug-in provides a repository tree navigation view, type tree view, properties dump view and the ability to execute DQL and XDQL queries. [EMC Documentum Developer Program - New Content]

Thu, 31 Mar 2005 21:55:53 GMT

Tips for Developing WDK Benchmarks. Developing automated test scripts for WDK based applications such as Webtop,DCM, Webpublisher and your custom applications can be quite challenging, due to the complexity of the WDK. This document outlines the best practices, top tips and common pitfalls as learned by the engineers in Documentums Performance and Capacity Planning group during the development of our own internal benchmark scripts and utilities. [EMC Documentum Developer Program - New Content]

Tue, 28 Dec 2004 23:14:13 GMT

Got charting working. The XMLDB pieces I noted in Step 3 and Step 4 of this post were actually very easy. The syntax for getting XML into Xindice is simple as is the querying. Once I got that going it was just a matter of hooking of the pieces of my pipeline to do what I wanted to do. I did have to tweak the XSLT that produces the SVG. I didn't build it to handle enough data points (bars were too wide, not enough graph area, etc.).

The cron and xmldb samples were really helpful in getting this working, both from a code perspective and from a functional perspective. As I stored XML in Xindice, I'd pop over to the xmldb browser sample and browse my collection to verify that it worked as expected. I used the cron sample OOTB to set up a task to run the DQL queries against Documentum on a schedule. Going forward, I'll need to incorporate an admin/config interface into my app for creating the cron task and browsing the xmldb collections.

Mon, 27 Dec 2004 06:31:31 GMT

When I dusted off my Documentum-Cocoon integration stuff I had to do a bit of a fix up. It seems that my WDK install had either rearranged some classpath entries (maybe different versions of JARs Cocoon dependend on behind its own) or made the classpath too long. In any case, I had to update the catalina.bat file to remove the WDK entries as a temporary fix.

I then noticed that when I ran any pipelines that used my Documentum-Cocoon components, they didn't seem to be getting called. My loggers weren't showing any entries and the page was just coming up blank. It turned out I had taken a little too much out of my classpath. Obviously, Tomcat needs to be able to find the Documentum DFC classes because my components rely on those. It was frustrating that no one was returning a helpful message to alert me to my blunder.

Something helpful in this situation is the Cocoon Status page in the Samples area. On that page you can show the classpath. If it doesn't see the DFC JAR and the Documentum config directory, you could be in trouble.

Mon, 27 Dec 2004 06:19:37 GMT

Had a look at the Documentum Business Process Management tools (BPS, BPM, Forms Builder, etc.) last week and they looked pretty good. I'm thinking if you are doing anything modestly serious with workflows you could really save yourself some time by essentially upgrading your workflow to the BPM toolset. And, obviously, if you are doing any sort of BPM like integrating with other apps via JMS (or an SOA) or pulling external enterprises into workflow processes, you really need it.

Mon, 27 Dec 2004 06:14:57 GMT

Made some progress on the Documentum-Cocoon stuff over the Christmas break. I'm working on a piece that will allow me to dynamically build charts and graphs based on data stored in the docbase. Here's what I've done and what's left to do:

Step 1. Figure out how to build a bar chart in SVG

First, I needed to figure out how to build a chart using SVG. I found some Java code that builds charts using SVG but what I was looking to do was build the SVG programmatically using XSLT to transform my source data which will be XML coming from my Documentum Transformer.

I needed something I could use to graphically draw the chart to get me started on the SVG. I downloaded a drawing tool called Sodipodi for this. It worked great. It stores everything as SVG. So all I had to do was draw my chart and save it to get a nice XML file to start with.

Step 2: Create the SVG with XSLT

Now that I had a static SVG file I needed to create XSLT that would take source data XML and transform it into the bar chart SVG. First, I copied my static SVG file into my Cocoon install to make sure it was happy rendering it into JPEG and PNG. It was. Next, I downloaded the Batik binaries. I wanted a tool I could use to quickly view my SVG file as I tweaked it. Batik comes with an SVG viewer called Squiggle. In WSAD, I created a test XML source data file, copied my static SVG file from Step 1 into a new XSLT file, tweaked, and then viewed the results in Squiggle.

The first thing I did was add comments so I could figure out which part of the SVG did what. In Sodipodi I had set every object's "id" attribute, so correlating the XML to the object it was describing was pretty easy. Adding comments made it easy to rearrange blocks of XML to work how I needed it to in the stylesheet.

The next thing I did was clean up the SVG by rounding off all of the numbers Sodipodi had used for the coordinates of my shapes. Obviously with a design tool everything has got to be very precise, but for what I needed, whole numbers worked just fine and it made the XML easier to look at.

I then figured out the algorithms I'd need to use to figure out how to dynamically size and position the bars and bar legends in the chart.

Finally, I replaced the hardcoded values with XSLT variables. I then updated my source XML data with new values, transformed, and voila: Dynamically generated bar chart.

Step 3: Grab the data from Documentum on a schedule

The data I need to chart is in Documentum. It changes over time. Because it's historical data I need to get a snapshot from Documentum on a schedule and store the result somewhere. Then, my chart will be built on-demand via a Cocoon pipeline using the data that's been captured for a given time period.

I already have a pipeline that gets the snapshot from Documentum. All I need to do is execute that pipeline on a schedule and store the result somewhere. I've tested out the "cron" feature of Cocoon and it works great. It allows you to execute a pipeline on a schedule. A perfect fit.

Now what I need is to put my snapshot data somewhere. I'm planning on using Xindice. It's an XML database embedded in Cocoon (you can use a separate Xindice install if you want). My pipeline will pull data from Documentum, transform it (I really only need to store a subset/summarized cut of the data), and store it in Xindice.

To complete my work for this step, I need to look at the XML:DB examples in Cocoon to learn how to stick data in Xindice from a pipeline.

Step 4: Build the chart based on the historical data

Now that I've got my historical data summarized and sitting in Xindice, I need to create a pipeline that will query Xindice for the data and transform the query result using the SVG I build in Step 2.

To complete my work for this step, I need to look at the XML:DB examples in Cocoon to learn how to query XML data from Xindice using XPath. The rest should be a piece of cake.

The result

Once this is in place, I'll have an approach and some reusable code I can use to capture and chart source data from Documentum.

Fri, 12 Nov 2004 02:25:01 GMT

Here's a new minor release of the Documentum-Cocoon integration components. It includes the reader and generator components I referenced a while back and a minor tweak to how I'm handling namespaces. (I'm not convinced I'm handling them the way I should, BTW). I've mentioned needing to develop a Documentum-specific protocol but now that I have a reader and a generator, that doesn't seem as important--readers and generators could be used as "internal" resources within a pipeline to retrieve things like XSL from the docbase to perform transformations.

I'm a little torn on what to work on next regarding this integration. I'd like a more elegant way to handle sessions. I can add single sign-on fairly easily so that when called from a WDK application the credentials would be passed to Cocoon and then used to create a session as the currently logged in Documentum user.

I'd also like to migrate the administrator dashboard prototype I've developed to use the Cocoon Portal UI framework.

Another idea is to use Cocoon scheduler to wake up periodically, execute a pipeline that reads data from Documentum, and persist that data either back to Documentum or into the Xindice XML database. This could be used to run historical reports, for example.

At Momentum I ran across a presentation by a fellow doing XML pipelining with Cocoon and Documentum. We exchanged information. If he posts his components for public consumption I'll link to them.

Sun, 24 Oct 2004 23:00:18 GMT

I'm in Montreal this week for the Documentum conference, Momentum Live. I actually came up a few days early with my wife. A few pics and stories from that piece of the trip is at Day 1, Montreal. If there are any noteworthy announcements or other ECM insights at the conference, I'll blog them here.

Tue, 24 Aug 2004 02:15:13 GMT

Got a Reader and a Generator added to my Cocoon-Documentum integration package last week. I also tested out the sendmail transformer that comes with Cocoon. Now, I can grab any file from Documentum from Cocoon using the Reader and then stream it to the browser or email it. If I want to transform an object stored in Documentum, I can use the Generator to snag it and then use it as I normally would in any other pipeline. For both I'm using the same protocol Documentum uses to reference documents through their "virtual link" support which is /<docbase>:<full path to content>.

I haven't made the Reader or the Generator available yet. I'll add it to the Transformer package when I get a chance.

What's left to do is create a protocol. That will allow me to use any Documentum object anywhere in a pipeline rather than just being restricted to Readers and Generators. That means I could store my XSL stylesheets as objects in the Documentum repository instead of on the file system if I wanted/needed to, although that would degrade pipeline performance.

Fri, 23 Jul 2004 22:48:28 GMT

Finally got the Documentum Cocoon Transformer cleaned up in WSAD to make it easier for me to share with others. Here is version 0.1. I've released it under the Apache License.

It isn't anything fancy. Very little code. There are probably other Cocoon-Documentum integration points that would be useful but I haven't knocked any out yet.

Tue, 15 Jun 2004 04:50:53 GMT

Finally got my Cocoon Documentum Transformer fully functional this evening thanks to the good folks at the Cocoon Users Mailing List. My problem was that the Documentum xDQL query was returning a full XML document. When I tried to parse that, the extra startDocument and endDocument calls were causing the runtime exception (archived mailing list thread).

Now that I have my transformer working, I can configure any number of XML docs I want, each with any number of xDQL queries against any docbase my server can see. This is real handy because there are times when you want to run the same query against multiple docbases. I've currently got the pipeline configured to style the query results using a generic xDQL-to-HTML stylesheet, but now that I've got everything working, the sky is the limit. Here are some ideas...

Documentum admin portal

Go to one page and see very quickly what the status is of key metrics across all Documentum servers, docbases, and environments (Dev, QA, Prod). This is easily done using Cocoon's aggregation of my styled query results. If I really wanted to get fancy I could use the Cocoon portal framework and create an honest-to-goodness portal with multiple users and profiles (different users might want to monitor different things on different docbases) as well as security.

But, at some point, this thing starts to approach the out-of-the-box Documentum Administrator client and I'm definitely not up for reinventing that wheel. So, this one is a lower priority.

Documentum RSS alerts

Create one or more RSS feeds that will syndicate alerts for things in the environment that need attention. To do this, I'll add another transformation in my pipeline that will apply business logic to the query results and format "alerts" as XML. I will then add an alert-XML-to-RSS stylesheet. (I'm separating the two because I might want to format alerts in HTML). Once that is in place, I can use a news aggregator like Amphetadesk or NewsMonster to poll my Documentum RSS alerts periodically for things that need attention.

This one is nice because it does not force the admins to go to a web page--they just leave their aggregator running on their desktop.

Return Documentum Query Results in a Spreadsheet

Sometimes you'd like to be able to query multiple docbases and have the results in a spreadsheet so you can then do some analysis or whatever. Today you have to query each docbase and then cut-and-paste the results. With my transformer in place, I ought to be able to spit my query results into an Excel spreadsheet, PDF, SVG, whatever.