...by the inmates...for the inmates...
Monday, March 15, 2004Topix.net Launches
More on Conversion Rates
Without Conversion Rates, You Don’t Know if You’re Mickey Mouse or Mickey Mantle. by Steve Jackson The conversion rate on a Web site is easy to measure. Unfortunately, businesses too busy concentrating on their bottom line most often overlook it. [Current Articles from MarketingProfs.com]
Literary Aspects of Blogging
Jj commented on literary aspects of blogging: "If you are interested in the literary and scholarly aspects of blogging, check out the work of Jill Walker and Torill Mortensen. For example, they have written a paper titled "Blogging thoughts: personal publication as an online research tool". They also co-author Blogonblog, a blog about blogs which contains some interesting pointers." -- Thanks for these insights! [Universal Rule]
RSS: Doing The Robot
More on RSS and web analytics from Chad Dickerson, CTO at InfoWorld
Via Feedster, I picked up Sean Gallagher's response ("The RSS ego bubble") to my recent post about InfoWorld's RSS request trends. While the robot-like activity of aggregators is certainly a factor in looking at the numbers (as I noted in my original post), I think some of the information in Sean's post was misleading and deserves further discussion.
I've been involved in hands-on analysis of web server logs at some extremely highly-trafficked web sites over the years, so I dutifully noted the robot-like behavior of news aggregators in my original post:
I realize that the characteristics of RSS aggregators' requests are different than those initiated by regular users browsing your site -- aggregators behave more like robots or spiders. But I still think this is significant.
Sean responded:
Sure, it's significant. But does Chad really understand the difference?
Yep -- I wouldn't have mentioned the request characteristics of news aggregators in my initial post otherwise, but it's certainly worth a deeper discussion with more useful and defensible data points.
In his post, Sean extrapolates the importance of InfoWorld's RSS request trend based on the 63 users he estimates visit his site daily (50 who use a browser and 13 who use news aggregators). InfoWorld's sample is a few orders of magnitude larger, and includes a broader mix of regular browsers, aggregators, search engine spiders, etc. so I think a closer look at our usage patterns might be more useful to the larger community.
As anyone who has analyzed lots of web server logs knows, the overall web measurement picture is a bit complicated by robots, spiders, proxies, etc., and RSS measurement is no different. (The issues in counting RSS subscribers were summarized quite well by Tim Bray back in May of last year, for those who are interested in digging deeper.)
However -- I think Sean's emphasis on the robotic behavior of news aggregators is a bit overblown and depends too heavily on a scenario where the average aggregator is updating every 15 minutes, an assumption that is not borne out in InfoWorld's server logs. In InfoWorld's case, the most popular news aggregator among our users is Radio Userland (check this out), and you can only configure it to fetch feeds once per hour. Considering that fact, the 100 potential requests per day that Sean suggests is off by a factor of at least 4 for our largest body of aggregator users.
The whole world doesn't use Radio Userland, so to be fair, I picked 5 random IPs of NetNewsWire users to get a rough estimation of how often they request our Top News feed (the subject of my original post) in a 24-hour period: 49, 48, 6, 48, and 48. I'm not a NetNewsWire user myself, so I downloaded and installed it only to discover that the only choices for update frequency are: 1) manually, 2) every 30 minutes, 3) every hour, and 4) every four hours. Sean's 100-requests-a-day scenario depends on a NetNewsWire client updating every 15 minutes. Hmmm. I guess you could manually update every fifteen minutes via the "News->Refresh All News" menu, but my random tests suggest that most users update every 30 minutes. 100 requests per day for a feed would seem to exaggerate NetNewsWire's behavior by at least a factor of two.
Requests from RSS clients certainly exaggerate requests to some debateable degree; however, there are some notable corrections in the other direction. In the case of web-based aggregators (Bloglines, Feedster), you have the opposite of robotic behavior -- the RSS aggregator acts as a proxy making a single request for a pool of users. If you don't have substantial numbers of subscribers using the web-based aggregator, this won't matter so much, but we do. As I investigated the effect of services like Bloglines on our Top News RSS feed numbers, I was able to determine our subscriber numbers from the User-Agent string available in requests from Bloglines' server:
Bloglines/2.0 (http://www.bloglines.com; 981 subscribers)I'll admit, I hadn't realized before that Bloglines included subscriber numbers in their User-Agent string (others already knew), but how cool is that? In any case, I also checked to see how often Bloglines' server requested our Tops News RSS feed last Monday -- 23 times. The ratio of subscribers to requests is about 43:1 right now, and the gap is widening every day.
(Another interesting technical aside -- as our RSS requests have grown quickly, we have noticed increased server loads at the top of the hour as aggregators "wake up" to pull feeds. Not a huge problem for us right now, but the surge has roughly the same characteristics as a distributed DoS attack and could eventually present trouble for really huge web sites unless aggregators become a bit smarter. I was working at CNN.com when IE4 and its Active Desktop with various CDF "channels" was released, and boy was it active. CNN.com and CNNSI.com were default channels in the new browser. All the newly-downloaded IE4 clients absolutely pounded our servers with requests for CDF files. It was a big pain, and I wish I could remember how we dealt with it.)
Finally, the discussions about print and online publishing business model disruptions created by RSS are nothing new to us at InfoWorld (see here, particularly comments from Matt McAlister, our online GM). We're experimenting with various business models around RSS like everyone else who needs to pay the bills, but ultimately we're focused on giving users valuable content in the format they want, and the growth in RSS requests is an indication that we're getting it at least partially right.
[Chad Dickerson]
The Future of Blog Tools
Lisa Williams has been going through all the notes of ideas that people left on Dave Winer's blog about "the future of blog tools." She's written up this awesome summary. Thanks to Amy Wohl for pointing to this.
[Scobleizer: Microsoft Geek Blogger]
Onfolio: Software start-up mines Explorer niche
Software start-up mines Explorer niche. New company Onfolio launches with a package of Internet Explorer-based tools designed to help people store, search and publish information found online. [CNET News.com]
Is Onfolio A Feature Or A Product?
Is Onfolio A Feature Or A Product?. J.J. Allaire apparently knows how to get publicity. His latest company, Onfolio, launched today, and it's tough to find a tech news publication that isn't covering the story (just ask Google news). So, what does Onfolio do? It appears to be a serious improvement on the way people use "bookmarks" or "favorites" in their browser - letting them more easily stores pages, organize them, take notes on them and share the results. I don't doubt those they've seeded the program to who claim that it's very useful. However, is it actually worth $30? First of all, we've pointed out that many people have completely given up on using their bookmarks/favorites, and it's difficult to convince them to go back. However, even if they did want to go back, why would they want to pay $30 to do so? Especially when there are free services out there like del.icio.us and my new favorite Furl? As far as I can tell, Furl lets you do just about everything Onfolio does with the added benefit of it being able to build on the community aspect of everyone contributing links that others can see. If anything, Onfolio looks to be a feature. If it really started to catch on, why wouldn't Microsoft or the team at Mozilla just build in similar functionality to their browsers? [Techdirt]