Radio UserLand, RSS, Weblog Tools and Design
Monday, March 15, 2004EContent: Can RSS Relieve Information Overload?
EContent: Can RSS Relieve Information Overload?. UserLand's CEO, Scott Young, and Lead Developer, Jake Savin, discuss the potential and application of RSS news readers within business settings - allowing the convenient delivery of information to those who need it. [UserLand Product News]
Totally Offline Radio UserLand Weblog
http://radio.userland.com/userGuide/advanced/privateWeblog [Radio UserLand Messages]
Topix.net Launches
Still More RSS Feeds
TechRepublic and Builder.Com both have new RSS 2.0 feeds. [Scripting News]
Personal Web-based RSS News Reader
Personal Web-based RSS News Reader.
Rocketinfo.com is a free personal Web-based RSS news reader.
By meryl@lockergnome.com (Meryl). [Lockergnome’s RSS Resource]
SmartManila Beta 2 is Ready
SmartManila Beta 2 is ready. Read the release notes here for details about new features. This one really rocks. [Blogging Alone]
Upstreaming in Radio UserLand
Upstreaming in Radio UserLand. The most common source of confusion for new Radio UserLand users is upstreaming, the process that turns a bunch of text files in Radio's www folder into Web pages on UserLand's Web server.
If you can spare a couple minutes, I'll show you a simple exercise from Radio UserLand Kick Start that makes upstreaming easier to understand:
- Open Radio's www folder and create a new subfolder.
- Copy the file #template.txt from www into the new folder.
- In the new folder, create a text file called countdown.txt just like mine. Don't leave out the blank line after the #title line.
- Save the file, then check Radio's Events Log to see what you did.
Radio upstreams countdown.txt as countdown.html, using the file #template.txt as a template for the Web page.
You can use your new folder to experiment with page creation and template editing without messing up your weblog. Changes to the template and the text file will cause the page to be upstreamed again automatically. [Workbench]
RSS: Doing The Robot
More on RSS and web analytics from Chad Dickerson, CTO at InfoWorld
Via Feedster, I picked up Sean Gallagher's response ("The RSS ego bubble") to my recent post about InfoWorld's RSS request trends. While the robot-like activity of aggregators is certainly a factor in looking at the numbers (as I noted in my original post), I think some of the information in Sean's post was misleading and deserves further discussion.
I've been involved in hands-on analysis of web server logs at some extremely highly-trafficked web sites over the years, so I dutifully noted the robot-like behavior of news aggregators in my original post:
I realize that the characteristics of RSS aggregators' requests are different than those initiated by regular users browsing your site -- aggregators behave more like robots or spiders. But I still think this is significant.
Sean responded:
Sure, it's significant. But does Chad really understand the difference?
Yep -- I wouldn't have mentioned the request characteristics of news aggregators in my initial post otherwise, but it's certainly worth a deeper discussion with more useful and defensible data points.
In his post, Sean extrapolates the importance of InfoWorld's RSS request trend based on the 63 users he estimates visit his site daily (50 who use a browser and 13 who use news aggregators). InfoWorld's sample is a few orders of magnitude larger, and includes a broader mix of regular browsers, aggregators, search engine spiders, etc. so I think a closer look at our usage patterns might be more useful to the larger community.
As anyone who has analyzed lots of web server logs knows, the overall web measurement picture is a bit complicated by robots, spiders, proxies, etc., and RSS measurement is no different. (The issues in counting RSS subscribers were summarized quite well by Tim Bray back in May of last year, for those who are interested in digging deeper.)
However -- I think Sean's emphasis on the robotic behavior of news aggregators is a bit overblown and depends too heavily on a scenario where the average aggregator is updating every 15 minutes, an assumption that is not borne out in InfoWorld's server logs. In InfoWorld's case, the most popular news aggregator among our users is Radio Userland (check this out), and you can only configure it to fetch feeds once per hour. Considering that fact, the 100 potential requests per day that Sean suggests is off by a factor of at least 4 for our largest body of aggregator users.
The whole world doesn't use Radio Userland, so to be fair, I picked 5 random IPs of NetNewsWire users to get a rough estimation of how often they request our Top News feed (the subject of my original post) in a 24-hour period: 49, 48, 6, 48, and 48. I'm not a NetNewsWire user myself, so I downloaded and installed it only to discover that the only choices for update frequency are: 1) manually, 2) every 30 minutes, 3) every hour, and 4) every four hours. Sean's 100-requests-a-day scenario depends on a NetNewsWire client updating every 15 minutes. Hmmm. I guess you could manually update every fifteen minutes via the "News->Refresh All News" menu, but my random tests suggest that most users update every 30 minutes. 100 requests per day for a feed would seem to exaggerate NetNewsWire's behavior by at least a factor of two.
Requests from RSS clients certainly exaggerate requests to some debateable degree; however, there are some notable corrections in the other direction. In the case of web-based aggregators (Bloglines, Feedster), you have the opposite of robotic behavior -- the RSS aggregator acts as a proxy making a single request for a pool of users. If you don't have substantial numbers of subscribers using the web-based aggregator, this won't matter so much, but we do. As I investigated the effect of services like Bloglines on our Top News RSS feed numbers, I was able to determine our subscriber numbers from the User-Agent string available in requests from Bloglines' server:
Bloglines/2.0 (http://www.bloglines.com; 981 subscribers)I'll admit, I hadn't realized before that Bloglines included subscriber numbers in their User-Agent string (others already knew), but how cool is that? In any case, I also checked to see how often Bloglines' server requested our Tops News RSS feed last Monday -- 23 times. The ratio of subscribers to requests is about 43:1 right now, and the gap is widening every day.
(Another interesting technical aside -- as our RSS requests have grown quickly, we have noticed increased server loads at the top of the hour as aggregators "wake up" to pull feeds. Not a huge problem for us right now, but the surge has roughly the same characteristics as a distributed DoS attack and could eventually present trouble for really huge web sites unless aggregators become a bit smarter. I was working at CNN.com when IE4 and its Active Desktop with various CDF "channels" was released, and boy was it active. CNN.com and CNNSI.com were default channels in the new browser. All the newly-downloaded IE4 clients absolutely pounded our servers with requests for CDF files. It was a big pain, and I wish I could remember how we dealt with it.)
Finally, the discussions about print and online publishing business model disruptions created by RSS are nothing new to us at InfoWorld (see here, particularly comments from Matt McAlister, our online GM). We're experimenting with various business models around RSS like everyone else who needs to pay the bills, but ultimately we're focused on giving users valuable content in the format they want, and the growth in RSS requests is an indication that we're getting it at least partially right.
[Chad Dickerson]
New Radio Tool: Workbench.root
New Radio tool: Workbench.root. I have released the first beta version of Workbench.root, a Radio UserLand tool that contains scripts that extend the functionality of the software:
My goal is to offer simple macros that Radio UserLand users of all skill levels can put on their Web pages and Web site template files to offer content in new ways.
The first release offers two scripts:
- Workbench.viewCategoryLinks, a script that displays a list of links for all of your weblog's public categories.
- Workbench.viewPostIndexes, a script that displays a list of your weblog's entries in reverse chronological order, like the one I offer on Workbench.
Users of my old workspace.viewPostIndexes script will find that this version has been improved. It updates all of your post index pages each time you publish a new weblog entry instead of updating them once per day. [Workbench]
The Future of Blog Tools
Lisa Williams has been going through all the notes of ideas that people left on Dave Winer's blog about "the future of blog tools." She's written up this awesome summary. Thanks to Amy Wohl for pointing to this.
[Scobleizer: Microsoft Geek Blogger]
A New, Cool Looking, RSS Aggregator
A new, cool looking, RSS Aggregator.
Ian Hanschen: "Presenting BlogNavigator. The ultimate in RSS experience." Very cool looking. Anyone try this yet? Ian's stuff always looks so cool.
[Scobleizer: Microsoft Geek Blogger]
Radio UserLand Static Sites Tool
Re: Directives. What is the Static Site Tool really intended for, if Radio can basically do all the things you say? It seems redundant in that case.
It is redundant -- Radio also can be used to publish Web sites that have nothing to do with weblogs.
To see what I mean, try this quick exercise:
http://www.cadenhead.org/workbench/2004/03/11.html#a1528
Your new folder is a Web site created with Radio that isn't a weblog.
The Static Sites tool is a tool for publishing Web sites using files and data stored in Radio's object database. Radio can publish Web sites using files stored outside of Radio's object database (in Radio's www folder and its subfolders).
The functionality now known as the Static Sites tool was the killer app of Frontier 5. You can find docs on how to use it here:
http://www.scripting.com/frontier5/tutorials/web/
http://www.scripting.com/frontier5/tutorials/scripting/ [Radio UserLand Messages]
Andrew Integrates RSS and BitTorrent In Radio UserLand
I've finished an initial version of a RSS+BitTorrent integration tool for Radio Userland's news aggregator. This is beta software, Windows only. If you've been following the buzz you might find this interesting. I'll be writing more about the idea over the next several days. Please leave your comments, bug reports and fixes here.
[Andrew Grumet's Weblog]
RSS + BitTorrent = Next Big Thing
RSS + BitTorrent = Next Big Thing.
Andrew Grumet and Dave Winer team up to lead the way.
Wired News "Is there a way to publish videos, games and other enormous files online without breaking the bank on bandwidth fees or driving downloaders mad with delays? A demo publishing system launched Friday by a popular programmer and blogger merges two of this season's hottest tech fads -- RSS news syndication and BitTorrent file sharing -- to create a cheap publishing system for what its author calls "big media objects." The hybrid system is meant to eliminate both the publisher's need for fat bandwidth, and the consumer's need to wait through a grueling download.
Both technologies have considerable buzz. The New York Times reported last month that BitTorrent traffic accounted for roughly one-tenth of the packets on the Internet2 research network. Last week, editors at InfoWorld magazine said InfoWorld's RSS readership had surpassed its website in daily traffic. According to Grumet, an RSS feed of one's favorite bands, actors, shows, games or other "big media objects" could be kept up-to-date on a computer desktop for daily perusal, rather than forcing music and movie fans to suffer the "click and wait" delay associated with downloading huge files on demand."
[Scott Shuda: Scott's Radio Userland Weblog]