Permalink Sunday, March 16, 2003
RSS search engines
Feedster is the latest RSS search service. As a friend of mine once said (when talking about girlfriends as it happens), this is only the latest and not the last. There were RSS feed search engines around some time ago. But the permanence of relationships aside, what makes a good RSS search engine, or any search engine for that matter? For me it's immediacy.

Google has a pretty quick turn around such that when a new web page is available Google will index it pretty quickly. But quick in this context is a few days (at least for most sites - Google probably scans some sources much more frequently e.g Google News sites). This is fine for many web pages but not for current news, or news as it happens. For example, a community of weblogs in Iraq could be a vital source of information providing news as it happens. Couple that to a picture or video weblog and you've got a powerful voice. I want to search for these kinds of events now, not in a day or so. Ok, I know I could bookmark a weblog and view it often if I wanted up-to-the-minute news but what if I didn't know a site existed and wanted to find it? Particularly just after an event has happened. Sometimes, even with more mundane searches, you want to find something that's just happened minutes ago.

Weblogs are a good source of up-to-the-minute news as well as RSS feeds. An advantage of searching an aggregation of RSS feeds is that prior art has created a system whereby when a weblog is updated it can ping a central server or servers to notify it that it has changed. A search engine that knows to index a weblog (or any other site) that has just changed will always be right up to date. Google uses other tricks to increase the relevance of a search result, for example by looking at how many links point to that page. An RSS search engine will need to give some thought about how to create a relevance ranking in the absence of a rich set of inbound links (maybe provenance of a site or weblog will increase ranking?). If Feedster uses the same ping protocols as weblogs.com then it's got the jump on other RSS search engines and search engines in general as it'll always have access to the most recent information, in theory information only minutes old. If it hasn't then maybe Scott should look into this because Google has bought Blogger so will likely be looking at something like this (and if not they should be!).

Good luck Feedster!