Friday, April 24, 2009

Blurblet: BFG Tutorial next month

Another blurblet posted on my new blog.  We're doing a BFG tutorial at the Plone Symposium East next month.  Click through the burb-o-link for details.

3:05:32 PM   comment []   

New Blog: BFG site launched, Shane on BFG

Almost a year since a blog post. Sigh.

Well, it's time to make a cut-over to a modern blog. I'll do some writing in my new blog, and post little blurbs over here during a transition.

Today's newblog-pointer-blurbs are both about BFG, the small web framework with the big fun name:

2:02:37 PM   comment []   

  Friday, June 6, 2008

PSPS: The Work Cycle of Theming

Deliverance, DVNG, pagetypes, themesets, compiled themesets, many words, so many details.  Let's step back and look at how an ecosystem (gag) of theming might work.

This proposal describes a system where themers make themeset packages and share them at themeset sites.  Site managers can then browse, install, and update these themeset packages with a friendly user interface.

Big Goals

We want to make it easy for a themer to:

  • Make a theme
  • Share the theme with others

We want to make it easy for a site manager to:

  • Find and use themes

It's easy to get mired in tons of details:

  • Who is this "themer"?
  • What is and isn't a "sitetheme"?
  • What are the benefits of this approach to sitetheming?

For now, let's focus just on how we could reinvent the process of making themes, providing something substantially easier from end-to-end.

How It Could Work

Let's break the world of theming into 3 parts:

  • Make.  The work to produce and package a "theme".

  • Distribute.  Put the theme in an organized place where people can find it.  This place is specifically designed to maximize convenience.
  • Use.  Find available themes, use them, and get updates.

What are some crazy things we could do to really change the game on"easy"?  How might we make this a significantly competitive part of our story?


Deliverance helps themers avoid learning a framework's architecture and APIs.  What if they could develop and test a theme without installing any unusual software?

For example, imagine the first, simplest way to develop a theme:

1) Create an HTML file for the theme, some CSS and images, and make sure they look alright.  This just HTML as files, so "alright" means"Open File" in a browser.

2) Next, make the Deliverance "themeset" file and some sample content pages.

3) Run "zip" to package it all up.

4) Use "curl" to POST the zip file to a web service URL.  This webservice returns a packaged, compiled themeset as a zip file.

5) Unpack the resulting zip file and view the sample content, themed as appropriate under the themeset rules.

We could foolproof this step by providing a zip file that had a sample themeset, ready to go.  You just run a "" or "run.bat" in the zip file's unpacked directory.  We could even include the curl and wget binaries for Windows.

This approach is a nice, small-step for the not-yet-committed.  How could we, though, make this process more productive for people that regularly theme?

We could write a Firefox extension that put a GUI on the operations and options, hiding the upload/download as background tasks.  This extension could start small: point at a themeset, a button to view the results, and links to the themed sample content.

Over time, the extension could do more: show log messages about rule matches, provide a GUI to edit and validate the themeset file, make thumbnails, and upload to distribution servers.

Finally, those who are committed to Deliverance could use easy_installto grab the Deliverance themeset "compiler" itself, and not rely on a web service.  This would be a command-line tool that used lxml/libxml2 to generate the compiled themeset and package it up for distribution.We could also include a Paste-based local HTTP server that did theming on-the-fly.


A "compiled themeset" is a very, very simple package that can apply one or more look-and-feels to content based on some criteria.  It uses technology that is completely safe to distribute: no programming language, APIs, access to the filesystem, access to the web application, etc.

Because they are safe and commodity, it makes sense to share them and pass them around.  We can have sites that provide hosting of themes,using a somewhat-smart themset server providing the following:

1) List packages.

2) Compile themesets.  This serves the purpose above, allowing themers to make themes without local software.  It also, though, provides some security check: rather than trust the themer's compiled result, make it on the server.

3) Host versioned URLs to static theme resources.  Google recentlyannounced that they would provide stable URLs for important JavaScript resources.  Themeset servers could do the same for the images and CSS that go with a particular version of a theme.

With this, using a theme becomes even easier, as the static resources point to a stable, versioned URL that sets very long expires headers. In fact, we could leverage a free CDN such as Coral, without themers really having to understand it.  (And if the site manager didn't like it, they could run local versions.)

By doing so by default, we make such things easy.

It would be tempting to have themeset servers that are just dumb static-file sites.  However, we should preserve the idea that the themeset server generates the compiled themeset, rather than trusting the theme makers.  Note that, rather than a full web application, you could still use Subversion with hooks that execute scripts.

In the workflow for a consulting project, you really don't think of putting the artifacts on a public site.  Thus, it should be easy and normal to run a private themeset server for a consulting project. (Alternatively, produce the themes and just put them in the buildout for the project, if the customer will never want to select new themes.)

Why zip files that aren't eggs?  Foremost, a themeset package isn't "executable".  The process of installing a themeset shouldn't accidentally run some Python code.  We could leverage eggs, but that inherits quite a few contracts that might be stifling for the goal of"easy".


We now have lots of themeset packages available at various public and private themeset servers.  How can we radically improve how site managers leverage all this?

First and foremost, make sure this part of the equation is dirt-simpleand reliable.  We can do so by moving most of the software complexity to the "Make" step: a compiled themeset is just an XSLT file plus some static resources and packaging artifacts.  All the work for parsing the theme(s) and the rules was already done in the compilation process.

Next, we provide a web GUI for configuration.  This allows operations such as:

  • Browse themeset servers to find and install new/updated themeset packages
  • List the currently installed themeset packages
  • Change the active themeset to a different one
  • Remove an installed themeset

Where should this GUI be located?  It's natural for Plone to put thisin the 'site setup" as a configlet.  There are reasons, though, thatthis might not be the only place to consider:

  • Deliverance is about corporate identity across multiple sites and multiple systems.  It makes sense to centrally manage your themes from outside a particular Plone site.  (OTOH, the people doing this might not want a GUI.)
  • It is possible to have themeset configuration on a subsite basis.

There are tons of opportunities here for fresh thinking about theme configuration, and most will prove easy to implement.  We should be open to new ideas on this aspect.


Making, distributing, and using "themes" is an important cycle to look at for Plone's strategic planning.  Doing a superlative job at this part would make a deep impact on Plone's initial perception.  Doing this job in a strategic way, where we think more about how Plone fits into the larger world for themers, integrators, and site managers, is worth discussion.

11:18:38 AM   comment []   

Deliverance PSPS Update

I've worked the last couple of days full-time on Deliverance:

Themeswitching Demo

I worked with Martin Aspeli, showing to use the new themeswitching features on the Deliverance trunk.  For this I made a small tutorial. (I plan to re-do some of that tutorial, as I now understand more about WSGI filters.)


I've been talking on the Deliverance list about an experimental rewrite of Delivernace called DVNG.

After the PSPS we solicited input on what was needed in Deliverance for Plone theming and Ian (with Wichert and others) came up with a proposed spec. This is an experiment in implementing those capabilities, along with an emphasis on: speed, reliability, debuggability, and extensibility.


Since the PSPS is doing a follow-up at the symposium this week (e.g. today), I wanted to make a prototype of ideas on how my ticket might work.  Thus, I'm pretty close on one approach to doing so and have a writeup that contemplates the big picture for theming.

I'm working on a writeup for this which I'll post later.

10:12:50 AM   comment []   

  Monday, March 10, 2008

Deliverance and Repoze talks at Plone Symposium

Tres and I are going to the Plone Symposium at Penn State this week.

I'm giving a talk on Deliverance, aimed at a designer and integrator (versus core developer) audience. Tres is giving a talk on how our Zope2/Plone world matches up to the Python web frameworks world.
11:31:28 AM   comment []   

Virtual hosting and the ghetto

Chris slays the VHM dragon. The people contributing to Repoze are helping to show the upside of WSGI and the benefits of getting out of our ghetto.

Lotta stuff going on in Repoze land these days.
11:28:01 AM   comment []   

Joel gets insanely great review

I just read a review of Joel Burton's Plone training.

Despite its absurdly low cost, this course was the best open-source training I[base ']ve ever taken, and among the best classes I[base ']ve attended overall. The instructor possesses not only a mastery of the material, but he is also an excellent teacher, with a knack for drawing analogies to real-life experience, and interacting with and captivating the attention of large groups of students. I would not hesitate to recommend this class or others with this instructor. In fact, I would take the same class again, especially since I missed part of the five-day course. In addition, I would certainly be interested in more advanced courses taught by this instructor.

While those sound like strong words, they sound right to me. Joel is, indeed, insanely great at teaching.
11:20:13 AM   comment []   

  Tuesday, February 5, 2008

Plone-the-product vs. Plone-the-platform, Joel-on-software edition

This week Plone has a summit to discuss strategy. For me, "strategy" means high-level kinds of questions like:

- What should Plone do well (vs. not worry about doing well)

- Who is it for (vs. not necessarily for)

- What makes it unique (vs. what things are commodity)

- Who are your natural competitors

Using summit time to assess features for future releases is a good idea. But there should also be time to tackle strategy decisions that inform feature planning. I believe Jon is helping make that happen.

In trying to help sort out such strategy questions, it's useful to make a distinction between "Plone-the-product" and "Plone-the-platform" (aka the framework).

The former is an out-of-the-box application with specific functionality for certain users. That is, a specific user experience. The latter is UI-less set of capabilities from which you can go in various directions to make your own products. And each has a totally different set of answers to the questions above. Stated differently, a totally different "strategy".

In trying to bring this up, I get either strong support or strong pushback. The latter goes along the line of "we're both", "why limit ourselves", and foremost, "why does it matter".

In this case, Joel Spolsky does a very admirable job discussing platforms.

It's really, really important to figure out if your product is a platform or not, because platforms need to be marketed in a very different way to be successful. That's because a platform needs to appeal to developers first and foremost, not end users.

I just had the good fortune to read an advance copy of Rick Chapman's excellent new book on stupidity in the software industry...One of the biggest themes in software industry failures is a platform vendor that didn't understand that they were a platform vendor, so they alienated their key constituency: the developers.

Over the course of Zope's history, I've watched this issue go through several permutations. The original Zope was a product that tried to hide, of all things, Python. It then grew an extension concept (Products) for developers.

The extension part was so popular that Python developers became the core constituency. Zope 3 mostly-abandoned the "product" side, while still sorta implying it would come up with backwards-compatibility to keep those interests in the fold. Instead, Zope 3 was a self-contained application server and platform. And now, having failed to get traction as a self-contained universe, there is a final push to make it assume its most natural place as a system of awesome, high-quality Python components. (See Jens's response on "The Zope Confusion".)

I also think Mozilla is an example of not knowing if you're a product or a tool. Three years ago, Mozilla was trying to make the best browser on the planet. Oh, and a great mail client. And a great calendar. And (of all freaking things) chat client.

Oh, and it was also a platform of technologies (XUL, etc.) to write extensions for said suite of stuff that went along with said browser. In fact, forget our products entirely...Mozilla is a platform for building *anything*.

People got woefully confused. What exactly was Mozilla trying to succeed at? And so Mozilla re-branded: the browser became Firefox, the suite was sent to Siberia, the mail client was spun out, and the chat client was "set free". The word "Mozilla" meant platform.

And that still wasn't enough focus. The Mozilla Foundation finally, FINALLY made it clear whether their raison d'etre was to make a killer browser or a generic internet technology. They say that in the ham-and-egg breakfast, the chicken is interested but the pig is committed. MoFoCo said the browser *would* be excellent and the plaform *should* be excellent. Big distinction.

Using the word "Plone" to mean both the product and the platform is, IMO, the least attractive way forward. It will never be clear to people what, exactly, Plone is and isn't for. What is it committed to excellence on. Who is its primary audience. And finally, the reason for not choosing product-or-platform will become the ultimate "house divided against itself" outcome: people will think Plone components developed for the Plone framework will only run in Plone.

The good news is that Plone already runs on a framework. In fact, several frameworks (Zope 2, CMF, Archetypes, Zope 3.) Anoint one (or help invent one) as that which people should pay attention to, then invest our activities under that flag, and make Plone a gorgeous assembly atop that other framework's genius.
3:10:18 PM   comment []   

  Wednesday, December 5, 2007

Nice meetup for last night's talk

I gave a talk last night at the ZPUG in DC.  In a nutshell, the talk was a gentle introduction to Paste and Deliverance.  Basically, I gave the kind of how-to-get-started that I would want to get, as a non-hardcore developer.

Turnout was good, but the atmosphere was even better.  Lots of discussions and questions at each step, frequently with the people discussing with each other.  There's a lot of interest in the topics of WSGI/eggs/Paste/middleware, and virtualenv/buildout/PyPI, and then how all the relates to particular frameworks like Zope and Plone.  My guess is that there is a pent-up demand for something like Repoze, which tries to take all those things people think they should get started on, and make it Just Work in the Zope/Plone world.

People really had some cool angles on how all this could shake out.  I really enjoyed the meeting.  Thanks to Alex Clark for organizing and Eric Coffman (and TNC) for hosting.

Regarding the material, I spent a bunch of time making progress on how to give a presentation and then have the material ready for good review afterwards.  Unfortunately, I still haven't found exactly the process that I'm looking for, but got a lot closer.

Plus, I know a lot more about Paste now. [wink]

Also in Repoze news, Chris and Tres had a talk last night in Chapel Hill, NC and one tonight in Charlotte, NC.

12:28:15 PM   comment []   

  Friday, November 30, 2007

Talking about Deliverance next week in DC

Next week I'm giving a talk in Washington DC on Tuesday, Dec 4 to the Zope/Python Users Group (ZPUG) DC.

I'll be talking about Repoze from a middleware perspective. Specifically, I'll talk about using Deliverance to apply a single corporate identity to all Python web applications. I'll also try to give a beginner's guide to using Paste as a quick way to get a sandbox going.

If you're interested in Python web technologies, but aren't a guru, this would probably be a decent meeting to attend. Hopefully we can talk a little bit about the people that put user interfaces, navigation, and other non-core-development value into a project, and find out more about how to fit their needs.
3:52:47 PM   comment []   

  Wednesday, November 7, 2007

New ideas regarding page composition

The prodigious Carlos de la Guardia wrote an intriguing article about Plone Performance and the Future of ZPT. He poses a question in the article: "I wonder if Deliverance could be useful here also?"

Short answer: yes. Longer answer: sorta, and I'd like to find out what people think. So read on, and if you have an interest in alternative thinking on how HTML UIs get generated, ping me.

As background, before I used the word "Deliverance", I was focusing only on Zope Site Themes. I view theming as distinct from skinning. A theme is only about corporate identity. There isn't really any computation, iteration, or any programmer-oriented stuff. Instead, you're just re-arranging HTML. You are probably leveraging existing artifacts.

I then got interested in splitting content management from content delivery and wanted a project that did both theming and composition. And I thought Deliverance was a hilarious name.

I keep vacillating, though, on whether one facility should do both. I still change my mind constantly on this. On one hand, I think they are very distinct audiences (site designers and templating integrators.) In the new, Repoze-oriented world of middleware, that would mean:
  • A WSGI middleware step takes the CMS output (likely semantic markup) and does "templating" to generate an HTML UI
  • The next middleware step (Deliverance) takes the mostly-done HTML and applies a corporate ID.
On the other hand, people might want it to be conceptually part of one step. For example, Ian said I should start by using the XSLT version of Deliverance and just having some custom rules that do the first step. It certainly would shave off some performance penalty you'd get by converting a tree to a string and back into a tree between middleware steps.

My instinct says that isolation, though, is better. I'll probably start on that path. (Tres gave me pointers on how to approach it.)

As Alan mentioned in a link from Carlos's article:

Enfold is working on xslt and doing page assembly outside of zope (in apache or IIS) using enfold.lxml (in our public repository).

I worked with Enfold on this for two very large projects, one of which (Open Society Institute's KARL system) was the keynote at the Plone Conference. In this, Plone generates an XML string for the information needed on a page. Something outside Plone takes that and generates a UI.

This has some interesting and positive performance implications. More importantly, it had a big consulting process impact. We were able to model every screen in the system as an XML file on disk. We then refined the UI with the customer, using templates (in our case XSLT) that didn't require a server or framework. Meaning, the customer could hack on it without screwing up "the server".

Since these same templates were used in production, there wasn't really a mockup stage. As long as the "back end" generated the same XML structure our dummy files expected, everything Just Worked.

Forget for a moment whether this uses XML/XSLT or JSON/Genshi or EBCDIC/Cobal.  Let's isolate the concept itself (moving HTML UI generation out of the framework) and see if there is interest.

I'd like to discuss using WSGI middleware for page composition and theming to achieve the following goals:
  • Separate the generation of the information (Plone) from the generation of a custom HTML UI
  • Increase performance by (insert a few reasons here)
  • Allow integrators to make more meaningful customizations without learning a component framework
  • Use schemas to increase quality by bringing a unit-test-like approach to the UI stage
If you're interested, let me know. I can start with a series of screencasts (and yes, Geir, writeups) that try to show what's been done and why it might be worth discussing, if there is interest.

11:52:01 AM   comment []   

Repoze at ZPUG last night

Last night Chris McDonough gave a presentation to the Zope/Python User Group of Washington, D.C.  I went along for the festivities and quite enjoyed myself.  Pretty good turnout.

As preparation, Chris wrote a detailed article about getting started with Repoze:

We've just created and released Developing With repoze.zope2 which is a short 10-page PDF that explains the benefits of and differences between developing Zope 2 applications using Repoze and "stock" Zope 2. It might be helpful to folks developing under Plone and CMF.

I believe we're slated to give another talk at the next ZPUG meeting in December.  I might give a walkthrough on using Deliverance to theme your Plone site.  I'll probably just play my screencast and claim I put effort into preparation. [wink]

We're also interested in getting Repoze more out and into the wild.  First, we're talking with Plone/Zope/Python user groups on the east coast, seeing if we can get our travel covered to come give more talks on WSGI and Repoze.  If you're interested and can cover our travel, drop us a line.

We also want to get more folks into Repoze, so it's bigger than us.  Both people that want to contribute, but also, people that want to start using it for real.  Drop by #repoze on and hang out with us.

10:52:28 AM   comment []   

Might we be returning to a One True Zope?

When you say Zope to someone, they may or may not hear the same thing you said.  There's Zope 2.  There's a completely different thing called Zope 3.  There's also a company, but that's a different topic.

It's thus with great relief this morning to read Baiju's At last Zope 3 releases coming to an end ! post.  It's somewhat startling, and exciting, to hear this said in public:

I won't be surprised if the term "Zope 3" itself vanish in the future.

I realize there is a small contingent that wants their Zope 3 isolated from Zope 2, never to even touch each other.  Honestly, if that was the outcome you wanted, why even call the rewrite Zope in the first place?

Next discussion, hopefully, will be whether that one Zope is an application server or a framework, or some other jargon that doesn't mean "closed world platform". [wink]

10:40:43 AM   comment []