Updated: 12/6/06; 8:32:01 AM.
Fluid Flow
Info about Antidunes, San Jose Neighborhoods, plus some Frontier/Radio scripting.

Friday, January 16, 2004

This is the second installment of my discussion on Manila best practices. In the first, I discussed server hardware. Today I want to look at server maintenance.

Server Maintenance

The chief source of problems for Manila servers is the loss of speed in accessing the databases that back the Manila application. Over time as data moves in and out of the databases, they get larger and it takes longer to find and process data needed to server Manila pages. Compacting the databases using Frontier's "Save A Copy..." command will reduce the size of the database, speed data acquisition, which in-turn speeds the servers response.

Compacting Frontier's databases is a processor intensive task and you will not be able to server files during compaction. Also, you will need to compact multiple databases for optimal speed improvements. For this reason, you should turn off the webserver before you start compacting databases. Otherwise, you could receive a request between "Save A Copy" that alters a database that you have already compacted.

You cannot "Save A Copy" over an existing database while Frontier is running. So you must save the compacted database to a new location, then quit Frontier and right over the existing database with the compacted version. As you can see compacting your databases can be a long and arduous process.

Depending on the size of your databases and the frequency that you compact them, it can take your webserver offline for several to tens of minutes. With a decent maintenance schedule and reasonably-sized databases you should be able to limit your servers downtime to under 5 minutes.

So which databases should you compact. At a minimum you should compact config.root. and manilaWebsites.root (and any other website databases). These are the databases that will change most frequently and are most likely impacting server performance. If you are running the news aggregator, aggregatorData.root is also a good candidate. The application databases, Frontier.root, mainresponder.root, and manila.root don't seem to change that much overtime, but it won't hurt to compact them. Any tool or plugin that frequently writes and deletes data, should be routinely compacted.

My current approach is to compact all of Frontier's open databases. This is overkill, but it is easier to script than a solution that compacts some open databases and not others.

From the discussion above, you should see that while it is a necessary maintenance step, it is a pain to compact databases. It is definitely one that calls for a scripted solution.

I am currently testing a pair of scripts to handle this maintenance. The first script is in Frontier, it turns off the webserver, compacts all of the open databases to a set location (a directory within the Frontier directory), and then calls an external script. The external script (written in AppleScript), quits Frontier, moves the compacted databases to their correct locations within the Frontier directory, then restarts Frontier.

These scripts are currently running nightly on my development server. My plan is to run them weekly on my production Manila server. I should be able to specify a weekly downtime to my clients.

I will report back as I get more data.
3:22:13 PM    
Comment []

I feel the earth move under my feet.... Brent Simmons dug up an excellent bit of news: an RSS feed for earthquake alerts from the USGS. [Mac Net Journal]

This is cool, I had been tracking the earthquake feed used by the folks who make worldKit. But they use the >2.5M for their feed. The signal to noise ratio is too high to be useful. It is much more useful to track >5 magnitude quakes.

I am pleased to see that the USGS produces multiple feeds including a >5M feed.
12:31:43 PM    
Comment []

I am seeing a strange problem with the comments link. They don't work when I click on them from my primary computer, but they do when I click on them from my development box (which sits two feet away). Strange.

Anyway I was wondering if anyone else was having problems with the comments link on Fluid Flow.
11:52:38 AM    
Comment []

I have released a new Manila plugin for my ES Designs hosting clients. The stencilEditor plugin allows Managing Editors modify the appearance of many of my custom macros output.

I have mentioned stencils here before and how they make my life easier, since I don't have to dive in the macro code to make changes to the appearance of the macros output. Through the stencilEditor plugin, I can give Managing Editors the same ability to modify the appearance.
9:52:31 AM    
Comment []

© Copyright 2002-2006 Tom Clifton.
January 2004
Sun Mon Tue Wed Thu Fri Sat
        1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31
Dec   Feb

Click here to visit the Radio UserLand website.

Subscribe to "Fluid Flow" in Radio UserLand.
Click to see the XML version of this web page.